The signal path and the EEVBlog
The Signal Path is one of the most fascinating corners of the internet if you are curious about the complex interplay between high frequency radio engineering, electronics and hardware repair.
On two of Shariar’s videos, he does a teardown and repair of an Anritsu MS2721 spectrum analyzer. Both A and B model series are meticulously opened and carefully explained from a RF engineering perspective. Since I do not have any test equipment of that sort and I’m interested in learning more about RF, I went and bought a non-working unit on ebay.
After some time poking around with it and documenting my thinking process as described below, I recently found an EEVBlog users post where I can hopefully cross-check notes with other folks in similar situations. That’s what this blogpost is about: summarising my findings so far.
Welcome to my hobby repair and reverse engineering (for fair use) journey of a SuperH4 based spectrum analyzer.
Application running! (null)
As the users in the EEVBlog forum comment, the unit comes with no internal compact flash. Luckily there’s a firmware upgrade process available through the “Master Software Tools”. With one of the programs from the suite, USB Loader, one can copy the files to a USB drive that will then be used to prime the internal CF card.
The USB upgrade mechanism does not work from the instrument, so copying the file structure directly into the CF card seems to go somewhere. Unfortunately, here is a screenshot of the device as it boots right after inserting a CF card:
Before the optimistic “Application running!” message, loading cst_gui.out and cst_base.out fly by as the progress bar goes on and on.
That being said, there’s also an “Emergency recovery” 3-key-pressing mechanism which will come in handy in the future:
When hardware is not really the main issue
The nonstop progress bar symptom is fairly similar to the signal path video. On Shariar’s video, SRAM modules are burning hot to the touch due to a possible internal short to ground:
In my case there were some faint water damage marks on the IC so for <$20 I decided to go ahead replaced all the ICs too, just in case. After that, it would be a piece of cake repair, right? Order chips, desolder those and substitute? Wrong!
After hot air desoldering the SRAM chips, and replacing them with fresh ones, everything stays the same. My intuition tells me that this is a software issue with possible main (non-static, different chips on board) RAM (system) memory corruption. **The installed memory is 64MB and this thing runs an old version of VxWorks:
VxWorks operating system version "5.5.1" , compiled: "Sep 25 2009, 08:56:03"`So before replacing those (cheap) RAM chips too:
I take some RAM data snapshots thanks to the open WDB Agent (builtin debugger) to see if and how the actual OS code is being loaded (and uncompressed) right on RAM.
Memory dump, colorized
The Metasploit team at rapid7 poked with the WDB debugger back in 2010 and the auxiliary script that dumps the whole memory contents comes in handy in this case. That way, hopefully, we don’t need to poke around with the compressed VxWorks.bin base OS binary as other VxWorks reversing folks have done.
Pre RAM desoldering I wasn’t sure if the loader was having some issues allocating memory. The colorized representation of bytes is taken with the excellent binvis.org online tool.
Pre RAM desoldering
Post RAM soldering
Later on I found out a handy command to run a memtest with VxWorks:
-> DoRAMTest
Address Error (Load)
EXPEVT Register: 0x000000e0
Program Counter: 0x0c01d730
Status Register: 0x40000001
Access Address: 0x0000000d
c374c28 _vxTaskEntry +8 : _shell ()
c3f89ec _shell +ec : c3f8a60 ()
c3f8bb8 _shell +2b8: _execute ()
c3f8d54 _execute +94 : _yyparse ()
c40b78a _yyparse +5ca: c409a80 ()
c409b88 _yystart +848: _DoRAMTest ()
shell restarted.One of the reasons I was suspecting that the RAM was faulty was related to the active VxWorks process (task) table:
$ telnet 10.1.1.116
Trying 10.1.1.116...
Connected to 10.1.1.116.
Escape character is '^]'.
-> i
NAME ENTRY TID PRI STATUS PC SP ERRNO DELAY
---------- ------------ -------- --- ---------- -------- -------- ------- -----
tExcTask _excTask ff7a8f8 0 PEND c448de0 ff7a850 3006b 0
tLogTask _logTask ff77f58 0 PEND c448de0 ff77eac 0 0
(null) �ߐ f90df90 0 SUSPEND 4 f90df68 0 0
tShell _shell fb87684 1 READY c43e720 fb8732c 1c0001 0
tPcmciad _pcmciad ff653e0 2 PEND c448de0 ff65348 0 0
tRlogind _rlogind fdd09f8 2 PEND c373d96 fdd06b8 0 0
tWdbTask _wdbTask fb888b8 3 PEND c373d96 fb886c0 3d0002 0
tUfiClnt c081c60 fb8e574 5 PEND c373d96 fb8e508 0 0
tKbd c015fe0 fb7a5bc 25 PEND c373d96 fb7a560 0 0
tSplash _tAnimateSpl fb2b950 30 READY c43e14c fb2b8e8 0 0
tNetTask _netTask fedf468 50 PEND c373d96 fedf3fc 0 0
tPortmapd _portmapd fdd3d18 54 PEND c373d96 fdd3b50 3d0002 0
tTelnetd _telnetd fdce710 55 PEND c373d96 fdce640 0 0
tFtpdTask c00d620 fb285d4 55 PEND c373d96 fb284f8 0 0
tTelnetOut__telnetOutTa f91015c 55 READY c373baa f90fec0 0 0
tTelnetIn_f_telnetInTas f8f58f8 55 PEND c373d96 f8f55ac 0 0
tDhcpcStatec3a9700 fdd6a68 56 PEND c373d96 fdd69ec 3d0004 0
tDhcpcReadT_dhcpcRead fdd54bc 56 PEND c373d96 fdd52f8 3d0002 0
tUglInput _uglInputTas fb76cbc 60 PEND+T c373d96 fb769b0 3d0002 3
t1 c413020 fbcbf20 100 PEND c448de0 fbcbe48 0 0
tUsbIsp1161_usbIsp1161H fbc1b48 100 READY c448de0 fbc1aa0 3d0004 0
usbOhciIsr c071560 fbbf924 100 PEND c373d96 fbbf8dc 0 0
BusM A c42a6c0 fbbd6d0 100 READY c43e14c fbbd678 0 0
usbMouseLibc413020 fbbaf00 100 PEND c448de0 fbbad28 0 0
usbMouseLibc413020 fbb67c0 100 PEND c448de0 fbb6738 0 0
usbKeyboardc413020 fbaddcc 100 PEND c448de0 fbadbf4 0 0
usbKeyboardc413020 fba968c 100 PEND c448de0 fba9604 0 0
BULK_CLASS c413020 fba4f1c 100 PEND c448de0 fba4d44 0 0
BULK_CLASS_c413020 fba07dc 100 PEND c448de0 fba0754 0 0
tBulkClnt c080a00 fb9c528 100 PEND c373d96 fb9c4c0 0 0
CBI_UFI_CLAc413020 fb96f98 100 PEND c448de0 fb96dc0 0 0
CBI_UFI_CLAc413020 fb92858 100 PEND c448de0 fb927d0 0 0
usbAcmLib c413020 fb837f8 100 PEND c448de0 fb83620 0 0
usbAcmLib_Ic413020 fb7f0b8 100 PEND c448de0 fb7f030 0 0
tBigBrother_tBigBrother f91c884 100 SUSPEND 4 f91c81c 0 0
tCheckUsb _tCheckUsb f912a20 100 READY c43e14c f912608 0 0
tRootTask _usrRoot ff7fe54 103 READY c43e14c ff7fb90 1c0001 0
tDcacheUpd _dcacheUpd ff4260c 250 READY c43e14c ff425a0 0 0
tUsbKbd c413020 fbb250c 255 READY c43e14c fbb249c 0 0
value = 0 = 0x0But I digress a bit here… since the instrument seems to still work, let’s not muck around with hardware and switch to software analysis instead?
Telnet
A passwordless prompt is all it needs to get inside the instrument:
$ telnet 10.1.1.116
Trying 10.1.1.116...
Connected to 10.1.1.116.
Escape character is '^]'.
->Getting in the telnet console early enough I can see the following boot message:
(...)
got OS version V3.21
OS match failed: V2.06 != V3.21
Application running!
*** Base Platform Compiled: Sep 23 2016, 10:27:14Dumping symbols via telnet
There is a “lookup address” VxWorks command we can leverage instead to enumerate some symbols:
-> lkAddr 0
0x0c002000 _usrEntry text
0x0c002040 _sysInit text
0x0c002064 _intPrioTable text
0x0c00216c _intPrioTableSize text
0x0c002180 _ataDrv text
0x0c002e40 _ataDevCreate text
0x0c0030c0 _ataRawio text
0x0c0050e0 _testAtaRw text
0x0c005140 _mysysInWordString text
0x0c0051c0 _mysysInLongString text
0x0c005240 _batteryInit text
0x0c0052a0 _giBatteryCmd textSo performing a lkAddr 0x0c0052a0, for the last _giBatteryCmd row, gives us the next consecutive batch of symbols… writing up a dirty telnet python script to do the rest for us is fairly straightforward:
After the resulting list of address/symbol pairs .csv has been generated, we can have some fun with some of the so-called VxWorks tasks, interacting with the instrument:
-> _testKbd
KBD: Setting interrupt level to 14
0x23
0x2b
0x11
0x19
0x10
0x18
Row tally == 0
0x15
0x1d
0x13
0x1b
Collumn tally == 0
(...)The voltage rails seem as “broken” as in Shariar’s video (via the recovery GUI he finds out):
-> sysSelfTest
SELFTEST STEP 4
Failed Temperature selftest (HOT)
SELFTEST: +5.8 voltage test failed (0 mv)
SELFTEST: +24.0 voltage test failed (0 mv)
SELFTEST: -5.8 voltage test failed (-21 mv)
SELFTEST: one or more supply voltages incorrect
Self Test returning 0x00000810
value = 2064 = 0x810But those failed tests are most probably a software-level artifact since the onboard Linear Technologies DC-DC converters have simply not activated those voltage rails at this stage in the boot process. This process of enabling power rails in sequence is conveniently named power sequencing, usually controlled by PMICs.
Binwalk… meh
As warned by experts in the field, binwalk and other semi-automatic tools still don’t cut it with VxWorks blobs (and probably never will) although new tools like binaryanalysis-ng might hold some promise?:
$ docker run -v /tmp:/samples/ -v /tmp/extracted:/samples/extracted cincan/binwalk -C /samples/extracted -e /samples/anritsu.mem
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
4698492 0x47B17C Boot section Start 0x42424242 End 0x7E4242
7954393 0x795FD9 Boot section Start 0x31000000 End 0x10081400
7954853 0x7961A5 Boot section Start 0x45000000 End 0x10081400
7955545 0x796459 Boot section Start 0x14006300 End 0xE1008
(...)
8032708 0x7A91C4 Boot section Start 0x1400C422 End 0xE1008
8036439 0x7AA057 Boot section Start 0x752340 End 0xE100814
8071937 0x7B2B01 Boot section Start 0x42424242 End 0x3C3C
8702277 0x84C945 Boot section Start 0x-4D6AF9C0 End 0x10102400
9251448 0x8D2A78 Copyright string: "Copyright (C) 1998, Thomas G. Lane"
9291980 0x8DC8CC YAFFS filesystem, little endian
9611985 0x92AAD1 Microsoft executable, MS-DOS
10281976 0x9CE3F8 VxWorks operating system version "5.5.1" , compiled: "Sep 25 2009, 08:56:03"
10326056 0x9D9028 Copyright string: "Copyright 1984-2002 Wind River Systems, Inc."
10339244 0x9DC3AC Copyright string: "Copyright Wind River Systems, Inc., 1984-2003"
10410684 0x9EDABC VxWorks WIND kernel version "2.6"
10413595 0x9EE61B Copyright string: "Copyright 1999-2001 Wind River Systems."
10504249 0xA04839 Copyright string: "copyright_wind_river"
11511236 0xAFA5C4 VxWorks symbol table, big endian, first entry: [type: function, code address: 0x202F0C, symbol address: 0x402A40C]
60465640 0x39AA1E8 Base64 standard index table
66396536 0x3F52178 XML document, version: ""1."
66405752 0x3F54578 JPEG image data, JFIF standard 1.01
66405782 0x3F54596 TIFF image data, little-endian offset of first image directory: 8Good news is that there does not seem to be that much entropy for this in-memory dump, meaning no signs of encrypted sections for this ~10 year old discontinued instrument:
$ docker run -v /tmp:/samples/ -v /tmp/extracted:/samples/extracted cincan/binwalk -C /samples/extracted -N -E /samples/anritsu.mem
DECIMAL HEXADECIMAL ENTROPY
--------------------------------------------------------------------------------
0 0x0 Falling entropy edge (0.687660)
3670016 0x380000 Falling entropy edge (0.838699)
3735552 0x390000 Falling entropy edge (0.848914)
4030464 0x3D8000 Falling entropy edge (0.810111)
9437184 0x900000 Falling entropy edge (0.436021)
9535488 0x918000 Falling entropy edge (0.722818)
60129280 0x3958000 Falling entropy edge (0.823834)Next up: Gearing up with Radare2 and SuperH4 decompiler
Radare2 and its great community comes in very handy in this work since other free software alternatives like GHIDRA lacks Super-H4 support at the time of writing this. Even if it did, I would have to pre-segment the firmare sections beforehand since 66MB of memory dump might not sit well with its project loader.
On the other hand, it just took just a few hours from the original r2-decompiler author to add r2 super-h decompiler support to the already existing radare2 SuperH support and radare2 can selectively analyze slices of a binary fast.
After this initial exploration blogpost I’ll be definitely analyzing deeper on the the VxWorks software stack side with radare2 on followup posts, stay tuned!