home

Sorry Guys, I Have to Troubleshoot My USB Drivers Before I Can Play With You

June 23, 2024 ❖ Tags: writeup, reverse-engineering, linux, operating-systems, hardware

This blog post is about a GNU/Linux rabbit hole I fell down in the belief I was chasing a mighty adventure. It was not nearly as adventurous as I had hoped, but I am nonetheless posting about it in case this information is helpful to someone else.1 My story begins with a purchase of four wireless gamepads from 8BitDo. I had done little research outside of scrolling past a few positive comments about their products on the Fediverse and viewing enough of their marketing materials to see that the controller I was interested in was supported by SteamOS2. That was enough to encourage me to put in an order, so I did, and patiently awaited their arrival. When they were finally in my hands, I plugged two of them into my media center, hoping to play some Mario Kart with Oli. They were clearly working in some capacity because RetroArch pops up a toast when it detects that a controller has been plugged in, but something was wrong. I twiddled the analog sticks and I mashed the buttons. Nothing seemed to happen.

This was potentially a hardware issue, so a natural first step is to check the kernel logs. We're lucky to find a wealth of information to help us troubleshoot. Unfortunately, some of these clues hint that the controllers may not be as well-supported under GNU/Linux as I had previously imagined. The read-out below begins at the point I plug the controller (or 2.4 GHz dongle) into my computer. 

...
[ 1184.030714] usb 3-1.2: new full-speed USB device number 3 using ehci-pci
[ 1184.115318] usb 3-1.2: New USB device found, idVendor=2dc8, idProduct=3106, bcdDevice= 1.14
[ 1184.115327] usb 3-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1184.115331] usb 3-1.2: Product: 8BitDo Ultimate Controller
[ 1184.115333] usb 3-1.2: Manufacturer: 8BitDo
[ 1184.115336] usb 3-1.2: SerialNumber: 36d4f8d817e4
[ 1184.127567] input: 8BitDo Pro 2 Wired Controller as /devices/pci0000:00/0000:00:1d.0/usb3/3-1/3-1.2/3-1.2:1.0/input/input28
[ 1184.127627] usbcore: registered new interface driver xpad
[ 1184.139303] input input28: unable to receive magic message: -121
[ 1185.336638] usb 3-1.2: USB disconnect, device number 3
[ 1185.517720] usb 3-1.2: new full-speed USB device number 4 using ehci-pci
[ 1185.599178] usb 3-1.2: New USB device found, idVendor=057e, idProduct=2009, bcdDevice= 2.00
[ 1185.599187] usb 3-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1185.599190] usb 3-1.2: Product: Pro Controller
[ 1185.599193] usb 3-1.2: Manufacturer: Nintendo.Co.Ltd.
[ 1185.599195] usb 3-1.2: SerialNumber: 000000000001
[ 1185.601479] input: Nintendo.Co.Ltd. Pro Controller as /devices/pci0000:00/0000:00:1d.0/usb3/3-1/3-1.2/3-1.2:1.0/0003:057E:2009.0005/input/input29
[ 1185.601541] hid-generic 0003:057E:2009.0005: input,hidraw4: USB HID v1.11 Joystick [Nintendo.Co.Ltd. Pro Controller] on usb-0000:00:1d.0-1.2/input0
[ 1188.920613] usb 3-1.2: USB disconnect, device number 4
[ 1189.109706] usb 3-1.2: new full-speed USB device number 5 using ehci-pci
[ 1189.193909] usb 3-1.2: New USB device found, idVendor=2dc8, idProduct=3106, bcdDevice= 1.14
[ 1189.193918] usb 3-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1189.193921] usb 3-1.2: Product: 8BitDo Ultimate Controller
[ 1189.193924] usb 3-1.2: Manufacturer: 8BitDo
[ 1189.193926] usb 3-1.2: SerialNumber: 36d4f8d817e4
[ 1189.194505] input: 8BitDo Pro 2 Wired Controller as /devices/pci0000:00/0000:00:1d.0/usb3/3-1/3-1.2/3-1.2:1.0/input/input30
[ 1189.206821] input input30: unable to receive magic message: -121
[ 1190.456603] usb 3-1.2: USB disconnect, device number 5
[ 1190.637736] usb 3-1.2: new full-speed USB device number 6 using ehci-pci
[ 1190.719644] usb 3-1.2: New USB device found, idVendor=057e, idProduct=2009, bcdDevice= 2.00
[ 1190.719653] usb 3-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1190.719657] usb 3-1.2: Product: Pro Controller
[ 1190.719659] usb 3-1.2: Manufacturer: Nintendo.Co.Ltd.
[ 1190.719662] usb 3-1.2: SerialNumber: 000000000001
[ 1190.721853] input: Nintendo.Co.Ltd. Pro Controller as /devices/pci0000:00/0000:00:1d.0/usb3/3-1/3-1.2/3-1.2:1.0/0003:057E:2009.0006/input/input31
[ 1190.721910] hid-generic 0003:057E:2009.0006: input,hidraw4: USB HID v1.11 Joystick [Nintendo.Co.Ltd. Pro Controller] on usb-0000:00:1d.0-1.2/input0
[ 1191.104940] usb 5-2: device descriptor read/64, error -110
[ 1191.320864] usb 5-2: reset high-speed USB device number 2 using xhci_hcd
[ 1194.040579] usb 3-1.2: USB disconnect, device number 6
[ 1194.220713] usb 3-1.2: new full-speed USB device number 7 using ehci-pci
[ 1194.305496] usb 3-1.2: New USB device found, idVendor=2dc8, idProduct=3106, bcdDevice= 1.14
[ 1194.305505] usb 3-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1194.305509] usb 3-1.2: Product: 8BitDo Ultimate Controller
[ 1194.305511] usb 3-1.2: Manufacturer: 8BitDo
[ 1194.305514] usb 3-1.2: SerialNumber: 36d4f8d817e4
[ 1194.306180] input: 8BitDo Pro 2 Wired Controller as /devices/pci0000:00/0000:00:1d.0/usb3/3-1/3-1.2/3-1.2:1.0/input/input32
[ 1194.318768] input input32: unable to receive magic message: -121
[ 1195.576570] usb 3-1.2: USB disconnect, device number 7
[ 1195.756718] usb 3-1.2: new full-speed USB device number 8 using ehci-pci
[ 1195.838587] usb 3-1.2: New USB device found, idVendor=057e, idProduct=2009, bcdDevice= 2.00
[ 1195.838595] usb 3-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1195.838599] usb 3-1.2: Product: Pro Controller
[ 1195.838601] usb 3-1.2: Manufacturer: Nintendo.Co.Ltd.
[ 1195.838604] usb 3-1.2: SerialNumber: 000000000001
[ 1195.840810] input: Nintendo.Co.Ltd. Pro Controller as /devices/pci0000:00/0000:00:1d.0/usb3/3-1/3-1.2/3-1.2:1.0/0003:057E:2009.0007/input/input33
[ 1195.840984] hid-generic 0003:057E:2009.0007: input,hidraw4: USB HID v1.11 Joystick [Nintendo.Co.Ltd. Pro Controller] on usb-0000:00:1d.0-1.2/input0
[ 1199.160545] usb 3-1.2: USB disconnect, device number 8
[ 1199.341723] usb 3-1.2: new full-speed USB device number 9 using ehci-pci
[ 1199.425711] usb 3-1.2: New USB device found, idVendor=2dc8, idProduct=3106, bcdDevice= 1.14
[ 1199.425720] usb 3-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1199.425724] usb 3-1.2: Product: 8BitDo Ultimate Controller
[ 1199.425726] usb 3-1.2: Manufacturer: 8BitDo
[ 1199.425729] usb 3-1.2: SerialNumber: 36d4f8d817e4
[ 1199.426262] input: 8BitDo Pro 2 Wired Controller as /devices/pci0000:00/0000:00:1d.0/usb3/3-1/3-1.2/3-1.2:1.0/input/input34
[ 1199.438796] input input34: unable to receive magic message: -121
...

Let's break down what we're being told here.

  1. When I plug the controller into my computer, it advertises itself as "8BitDo Pro 2 Wired Controller" (2dc8:3106), which the kernel recognizes and loads the xpad module to handle.
    1. xpad is for devices that emulate various Xbox controllers.
  2. evdev attempts to receive a "magic message" but fails with an error code of -121.
  3. A disconnect event occurs (prompted by the controller – I didn't unplug anything), and the controller re-appears as "Nintendo.Co.Ltd. Pro Controller" (057e:2009).
  4. Another disconnect, and the controller re-appears as it did initially.
    1. Same "magic message" error.
  5. Another disconnect, and we're back to pretending to be a Switch controller.
    1. This time, there's a device descriptor read error.
  6. Repeat this again.
  7. Give up after advertising one last time as 2dc8:3106.

Peering into the xpad source code, it's clear that this device is supported by my kernel, but for some reason, the controller isn't sending anything. If I cat the device and wiggle the analog sticks, nothing shows up.

Is it a problem with the controller?

At this point, I wanted to rule out that the problem wasn't the hardware itself, so I fired up my Windows 10 VM. QEMU supports USB pass-through, whereby the guest operating system effectively has direct access to the device. If the controller would work on a computer that's actually running Windows, then it should hypothetically work in a virtual machine with the controller passed through.

Typically, you specify either a vendor and product ID to pass through (which uniquely identifies a particular brand and model of USB device), or a host bus and host address (which is an identifier that the kernel allocates to a device when it's plugged in). But because the vendor/product ID pair is oscillating, and because Linux is going to allocate a new host address for the controller each time it re-connects (because it is seen as a new device each time), I instead pass through an entire USB port3.

I'm met with one disconnect, initially, but I then have a "Pro Controller" device that works. If I disconnect the controller from Windows and open SDLJoytest-GL on my host operating system.. I get input! From this, we know that the issue is with my kernel.

I took a bit of a break at this point, but came across this GitHub Gist about getting the controller to work in Linux. The udev rule didn't do me any good because I'm on a recent enough kernel that upstream xpad already has the vendor/product ID for this controller, but the comment a few pages down from Adrian Wilkins suggests that the disconnect/reconnect behavior is because the controller is trying to figure out if it's connected to a Nintendo Switch.

Does it work on the Steam Deck?

The 8BitDo website claims that the controller works on the Steam Deck, so I pulled mine out to verify and.. indeed, it does, without any of the disconnect/reconnect dance. 

[ 2940.018861] usb 1-1.1: new full-speed USB device number 7 using xhci_hcd
[ 2940.197350] usb 1-1.1: New USB device found, idVendor=2dc8, idProduct=3106, bcdDevice= 1.14
[ 2940.197360] usb 1-1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 2940.197363] usb 1-1.1: Product: 8BitDo Ultimate Controller
[ 2940.197365] usb 1-1.1: Manufacturer: 8BitDo
[ 2940.197367] usb 1-1.1: SerialNumber: 36d4f8d817e4
[ 2940.305610] input: 8BitDo Pro 2 Wired Controller as /devices/pci0000:00/0000:00:08.1/0000:04:00.3/usb1/1-1/1-1.1/1-1.1:1.0/input/input53
[ 2940.305743] usbcore: registered new interface driver xpad
[ 2940.317356] input input53: unable to receive magic message: -121
[ 2940.370371] input input53: unable to receive magic message: -121
[ 2940.437335] input input53: unable to receive magic message: -121
[ 2941.108365] input input53: unable to receive magic message: -121
[ 2941.375444] input: Microsoft X-Box 360 pad 1 as /devices/virtual/input/input54

So, what gives? I'd initially thought that 8BitDo might have submitted a patch to Valve to add support for their controllers in the SteamOS kernel fork, but I cloned the sources for the SteamOS version I was running and found that xpad.c hadn't been altered at all from upstream. I didn't see anything strange in lsmod, and lsusb told me that it really is xpad that's managing the controller. At this point I thought that the host detection might be implemented by something in userland, so I broke out Wireshark to investigate traffic on the USB hub.

sorry-guys-i-have-to-troubleshoot-my-usb-drivers-before-i-can-play-wireshark.png
Figure 1: Two Wireshark (QT) traces: one from my workstation, and one from the Steam Deck. The delta is highlighted and explained in the following paragraphs.

It takes a close eye, but we can see that the Steam Deck's kernel sends a USB interrupt before polling the controller for interrupts.

If you aren't familiar with USB, think of this as a socket. The Steam Deck's kernel is doing a write before doing a read, whereas my kernel is doing a read first, and this seems to be confusing the controller. (In the "bad" trace, the response to the URB_INTERRUPT in is -ENOENT.)

The interrupt that the Steam Deck sends to the controller is just three bytes: 01 03 02. Simple enough. Let's see if sending those bytes over fixes it. 

#!/usr/bin/env python

import usb.core
import usb.util

dev = usb.core.find(find_all=True)

for d in dev:
    if d.idVendor == 0x2dc8 and d.idProduct == 0x3106:
        d.detach_kernel_driver(0)
        d.write(0x02, '\x01\x03\x02')
        print(d.read(0x02, 0x64))
        usb.util.dispose_resources(d)
        d.attach_kernel_driver(0)

And, indeed, it does. On both my workstation and my media center, if I run this script after the controller gives up on its detection routine, it miraculously begins to work.

Now, running a Python script manually (or even via udev rule) is an unappealing solution to me, so I reached for the hammer to nail down a (marginally) solution. 

@@ -1936,7 +1936,7 @@ static void xpad_led_disconnect(struct usb_xpad *xpad) { }

 static int xpad_start_input(struct usb_xpad *xpad)
 {
-       int error;
+       int error, status;

        if (xpad->xtype == XTYPE_XBOX360) {
                error = xpad_start_xbox_360(xpad);
@@ -1947,6 +1947,13 @@ static int xpad_start_input(struct usb_xpad *xpad)
        if (usb_submit_urb(xpad->irq_in, GFP_KERNEL))
                return -EIO;

+       status = usb_interrupt_msg(xpad->udev, usb_sndintpipe(xpad->udev, 2), "\x01\x03\x02", cpu_to_le16(3), 0, 10000);
+
+#ifdef DEBUG
+       dev_dbg(&xpad->intf->dev,
+                   "JLK: %s - interrupt message 1 returned %d\n", __func__, status);
+#endif
+
        if (xpad->xtype == XTYPE_XBOXONE) {
                error = xpad_start_xbox_one(xpad);
                if (error) {

Despite seeing the "magic packet" in Wireshark traces now, I was still seeing the same disconnect/connect behavior. And after panicking my running kernel a couple of times seeing where else I could try to put my hack, I decided to take a more methodical approach. Surely, if the stack for this driver on SteamOS is the same as it is in upstream Linux, then there's some code in my worktree that sends the 01 03 02 sequence. Let's try to find that, and determine why it isn't firing.

We don't have to search far. I found what I was looking for in about 15 minutes. 

...
/*
 * set the LEDs on Xbox360 / Wireless Controllers
 * @param command
 *  0: off
 *  1: all blink, then previous setting
 *  2: 1/top-left blink, then on
 *  3: 2/top-right blink, then on
 *  4: 3/bottom-left blink, then on
 *  5: 4/bottom-right blink, then on
 *  6: 1/top-left on
 *  7: 2/top-right on
 *  8: 3/bottom-left on
 *  9: 4/bottom-right on
 * 10: rotate
 * 11: blink, based on previous setting
 * 12: slow blink, based on previous setting
 * 13: rotate with two lights
 * 14: persistent slow all blink
 * 15: blink once, then previous setting
 */
static void xpad_send_led_command(struct usb_xpad *xpad, int command)
{
        struct xpad_output_packet *packet =
                        &xpad->out_packets[XPAD_OUT_LED_IDX];
        unsigned long flags;

        command %= 16;

        spin_lock_irqsave(&xpad->odata_lock, flags);

        switch (xpad->xtype) {
        case XTYPE_XBOX360:
                packet->data[0] = 0x01;
                packet->data[1] = 0x03;
                packet->data[2] = command;
                packet->len = 3;
                packet->pending = true;
                break;
...

Hm.. that looks interesting. And it wouldn't be firing because… 

jakob@endseal /usr/src/linux $ grep XPAD_LED .config
# CONFIG_JOYSTICK_XPAD_LEDS is not set

Ah.

Sure enough, recompiling xpad with CONFIG_JOYSTICK_XPAD_LEDS resolves the issue.

What did we learn, then? That the 8BitDo Ultimate Controller firmware detects that it's plugged into a PC by… seeing if the USB host tries to turn its home button LED on.

Neat. I spent my Sunday morning elucidating the world with this useless fact.

Now, if you'll excuse me, I'll be using one of these four controllers to play GoldenEye 64. By myself, mind you, because I live alone and have no friends.4

—

Footnotes:

1

I also want to believe that putting something out there will help to ameliorate my case of chronic Writer's block – even if it isn't as grandiose as the articles I typically publish here.

2

This is a more consumer-friendly way of communicating that the hardware works under GNU/Linux.

3

That StackExchange page explains how to do this with command-line arguments, but I prefer to hot-plug USB devices with QEMU so I can connect and disconnect devices without having to restart the virtual machine. I specified -monitor stdio -usb on the command line so that I would have the QEMU monitor accessible, and a usb-host device allocated to the guest, and I then passed through my USB port with the monitor command device_add usb-host,hostbus=3,hostport=1.1,id…=

4

This is tongue-in-cheek. No need to worry about whether I'm doing okay. This, as well as the title, are jocular nods at the stereotype that GNU/Linux users are socially awkward.

Comments for this page

    Click here to write a comment on this post.