Fixing T470 Thunderbolt firmware issue

I am the happy owner of a (refurbished) Thinkpad T470 since 2017. It is a laptop I really like for many reasons : its form factor, the hot-swapable battery, the keyboard, the trackpoint. But most importantly, there is a large amount of resources you can find on the net, as Thinkpad are historically quite common in hackers communities.

It comes with a -single- USB Type C connector which is a Thunderbolt 3 port. It may hence be used for charging (with USB PD power supply, but there is also a proprietary Lenovo power port), for connecting several monitors (DisplayPort) and standard USB devices. Pretty cool ! Except when it stops working…

The Thunderbolt controller bug

Around 3 years ago, while I was simply using the Thunderbolt port for charging, it failed. Initially I suspected the power supply. But it worked properly on other laptops and in addition other Thunderbolt devices were not detected anymore. Then I believed the port was physically broken somehow. And finally, I discovered the root of the issue: a recently discovered bug on the Thunderbolt controller firmware which caused a premature aging of the SPI Flash chip.

As the upgrade of the Thunderbolt firmware with LVFS was a real pain at the time, I ignored the 2019 fix and ended up with this arguably more annoying issue. In my case, the symptoms were quite weird. The controller was still appearing somehow (boltctl was able to power it on). Some kernel events shown the USB was partially working. But no device was working properly, neither the power supply was detected. While I can live without Thunderbolt and type-C, I was relying on it for my dual-monitor setup: only a single HDMI port is usable otherwise.

Flashing the firmware the hard way

Many people have been affected by the issue. Some related fixing it simply by upgrading the firmware “normally”, often on Windows. I spent some time trying to upgrade it using LVFS, as the controller was still showing up in my case. Unfortunately all my attempt failed, both with the official Lenovo approved firmware (still in testing repository) and with some wizardry to craft a custom upgrade archive.

So I was let with two options. Either I can still flash the Flash EEPROM manually with an updated firmware, or it is too damaged and I have to unsolder it and put a new chip. A few people reported fixing it by soldering a new chip. My SMD soldering skill is fairly limited so I hoped the first method would success…

A good motivation was this blog post of someone successfully fixing the problem on a Thinkpad P1. I bought a good quality 3M SOIC-8 clip and some MX25L80 chips in case I have to solder a new one. I connected the clip to the GPIO ports of a RaspberryPi which was gathering dust in some drawer (certainly like 50% of all existing RaspberryPi). And this lead to a huge disappointment. Or more accurately, two huge disappointments.

First, the very expensive SOIC clip (30€!) from 3M is certainly defective. Some of the pins are too short to connect properly to a SOIC chip. Second, the motherboard on the T470 is quite different from the one of the P1¹. For instance, the Thunderbolt firmware chip is not the same reference. And in fact, not even the same width…

Fighting back with the right tools

After this initial defeat, and an additional year without thunderbolt port, I decided to try it again. I bought a cheap generic SOIC clip + USB EEPROM programmer on amazon for something like 15€. The actual SPI Flash chip on the T470, a Winbond 25Q80JVSIG is quite difficult to find. I bought some 25Q16JVSI on a local hardware shop.

The T470 motherboard, with the Thunderbolt firmware SPI Flash chip highlighted

I simply connected the clip on the chip and carefully set the cable on the USB programmer (no direction or ‘first pin’ indication on this cheap stuff, had to do some manual continuity checks). As the USB programmer is based on a quite standard CH341A chip, it is well detected and supported by tools such as flashrom. After connecting everything, I was finally able to read the current content with flashrom -p ch341a_spi -r thunderbolt_firmware_original.bin !

The setup is less messy than my first attempt with loosely connected cables between the clip and the GPIO ports !

Retrieving and preparing the new firmware

After a quick inspection with hexdump of the dumped content confirmed it looks like the Thunderbolt firmware². To retrieve the new firmware, I downloaded the latest version of the Thunderbolt controller update program directly from the Lenovo’s website. It comes as a Windows executable. Running it with Wine allows for unzip its content (“Extract Only”) in some directory. The firmware is stored as a ASSISTTBT.BIN file in destination directory. A quick look with hexdump and strings show its content to be similar to the dumped firmware and its version string near the end of the file being N1QTF18W, the latest firmware version available.

However the structure is a bit different on the dumped file:

• its size is 1 MiB compared to the ~252 KiB of the new file
• the area between 0x01000 and 0x04000 is filled with various data, compared to 0xFF in the new file
• there is a second copy on the content at the second half of the Flash, not aligned identically
  • seems to duplicate the content starting after 0x04000, starting at 0x82000

The second copy is maybe not strictly required, looking like some protection against firmware corruption. I decided to mimic the structure as best as I understand, using the following commands, with firmware_n1qth09w.bin being a copy of the ASSISTTBT.BIN retrieved previously.

dd if=/dev/zero of=firmware_padded.bin bs=1024k count=1
dd conv=notrunc if=firmware_n1qth09w.bin of=firmware_padded.bin
# second copy, starting at 0x82000 and ignoring the first 0x4000 bytes of the new firmware file
dd conv=notrunc if=firmware_n1qth09w.bin of=firmware_padded.bin bs=1k skip=16 seek=520

Being satisfied enough, I attempted to flash it.

flashrom -p ch341a_spi -w firmware_padded.bin

Disappointment and happy ending

After rebooting the laptop, I started first being happy (it booted, at least I destroyed nothing by accident!) then angry. Still no sign of improvement of the Thunderbolt port: power supply is still not detected, USB device are still unusable…

In despair, I attempted to enable/disable all possible BIOS options related even loosely to Thunderbolt. Here lies the final ingredient ! By disabling completely the Thunderbolt port (in Security > I/O Port Access), rebooting and re-enabling it… The port finally works again perfectly !

My best guess is that it writes something in the data area of the Thunderbolt firmware Flash, which was set to a default of “disabled” when I erased everything. But I will not investigate anymore for now, sometimes we may accept a seemingly magic outcome, isn’t it?

1. Some readers could ask “Wasn’t a good idea to check the motherboard before to buy the replacing chip?”. And it would be a very relevant question. ↩

2. It would have been slightly annoying to erase the BIOS chip or another firmware by accident. ↩