I was working with the SieraWireless LTE device drivers recently and I got into a strange issue with one of the drivers called GobiSerial. I have compiled this driver code in the same way as described here. But when I try to insert this in the Pi kernel I get the below error:

pi@raspberrypi:~ $ sudo insmod GobiSerial.ko
insmod: ERROR: could not insert module GobiSerial.ko: Unknown symbol in module
pi@raspberrypi:~ $ dmesg
[ 65.759262] GobiSerial: Unknown symbol usb_serial_generic_open (err -2)
[ 65.759290] GobiSerial: Unknown symbol usb_serial_suspend (err -2)
[ 65.759328] GobiSerial: Unknown symbol usb_serial_generic_resume (err -2)
[ 65.759348] GobiSerial: Unknown symbol usb_serial_deregister_drivers (err -2)
[ 65.759407] GobiSerial: Unknown symbol usb_serial_generic_write (err -2)
[ 65.759427] GobiSerial: Unknown symbol usb_serial_register_drivers (err -2)


To debug this issue first I would like to know if the symbols like usb_serial_generic_open exists in the Pi kernel or not. Apparently not, but just to verify that I would use the /proc/kallsyms tool provided by Linux Kernel. This utility prints all the symbols (exported) available in the kernel space. So I would try to grep the missing symbols from all the avialable symbols in the kernel like this:

pi@raspberrypi:~ $ cat /proc/kallsyms | grep -i "usb_serial_generic_open"
pi@raspberrypi:~ $

In the above command nothing is returned indicating my hunch the symbols are not present (exported/available) in the Pi Kernel.
Now, by the look of it the symbols should be coming from a Kernel module/driver usb_serial, this is my hunch by the look of the symbols which are missing. My next hunch is some Kernel module are not inserted unless there is a reason for it. Like if you haven’t inserted anything on the USB serial the usb_serial driver won’t be inserted into the kernel. Let’s try this. I will attach the SieraWireless LTE device into the USB port of the Pi. Before I do this I usually clean my Kernel messages buffer as below:

sudo dmesg -c

I do this so that when I insert the device I can only focus on the kernel messages coming out for this device. Now insert the device, wait for a second and then try the dmesg again, and you should be able to see something like this on the console:

[ 252.661876] usb 1-1.3: reset low-speed USB device number 4 using dwc_otg
[ 253.511884] usb 1-1.5: reset low-speed USB device number 5 using dwc_otg
[ 261.551829] usb 1-1.2: new high-speed USB device number 8 using dwc_otg
[ 261.682434] usb 1-1.2: config 1 has an invalid interface number: 8 but max is 3
[ 261.682451] usb 1-1.2: config 1 has no interface number 1
[ 261.683111] usb 1-1.2: New USB device found, idVendor=1199, idProduct=9091, bcdDevice= 0.06
[ 261.683123] usb 1-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 261.683133] usb 1-1.2: Product: Sierra Wireless EM7565 Qualcomm® Snapdragon™ X16 LTE-A
[ 261.683143] usb 1-1.2: Manufacturer: Sierra Wireless, Incorporated
[ 261.683153] usb 1-1.2: SerialNumber: UF92377624041506
[ 261.686019] QMAP Disabled
[ 261.688132] GobiNet 1-1.2:1.8 eth1: register 'GobiNet' at usb-3f980000.usb-1.2, GobiNet Ethernet Device, d2:bd:e1:49:f1:08
[ 261.689996] USB Speed : USB 2.0
[ 261.805791] usbcore: registered new interface driver usbserial_generic
[ 261.805832] usbserial: USB Serial support registered for generic
[ 261.806590] usbcore: registered new interface driver cdc_wdm
[ 261.814689] usbcore: registered new interface driver qcserial
[ 261.814756] usbserial: USB Serial support registered for Qualcomm USB modem
[ 261.815707] qcserial 1-1.2:1.0: Qualcomm USB modem converter detected
[ 261.816223] usb 1-1.2: Qualcomm USB modem converter now attached to ttyUSB0
[ 261.819226] qcserial 1-1.2:1.2: Qualcomm USB modem converter detected
[ 261.819635] usb 1-1.2: Qualcomm USB modem converter now attached to ttyUSB1
[ 261.819970] qcserial 1-1.2:1.3: Qualcomm USB modem converter detected
[ 261.820236] usb 1-1.2: Qualcomm USB modem converter now attached to ttyUSB2
[ 261.820411] usbcore: registered new interface driver qmi_wwan
[ 261.920268] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready

Looks like the default linux kernel serial drivers been just inserted. Curious huh? Now, check if our missing symbol is back in the Kernel or not as below:

pi@raspberrypi:~ $ cat /proc/kallsyms | grep -i "usb_serial_generic_open"
00000000 r __ksymtab_usb_serial_generic_open [usbserial]
00000000 r __kstrtab_usb_serial_generic_open [usbserial]
00000000 t usb_serial_generic_open [usbserial]

Hurray!! It is back. Now try inserting the GobiSerial driver module again:

pi@raspberrypi:~ $ sudo insmod GobiSerial.ko
pi@raspberrypi:~ $

Seems like the previous error didn’t appear. Lets check the dmesg again, and here we go:

[ 326.647915] usbserial: USB Serial support registered for GobiSerial
[ 326.647943] GobiSerial: 2019-11-22/SWI_2.39:GobiSerial


Issue resolved.
But soon I realized that attaching the device to the RPI hw is actually insmod-ing the qcserial and the qmi_wwan kernel modules which are Qualcomm drivers. But if you are dealing with the device EM7565, in the release notes it prohibits to have qcserial and qmi_wwan drivers in the kernel and prescribes to blacklist these in the below file:
/etc/modprobe.d/blacklist-modem.conf.


$ sudo vim /etc/modprobe.d/blacklist-modem.conf
# Uncomment these entries in order to blacklist unwanted modem drivers
# blacklist snd-atiixp-modem
# blacklist snd-intel8x0m
# blacklist snd-via82xx-modem
blacklist qcserial
blacklist qmi_wwan

If you do that these two drivers won’t be loaded when you attach your EM7565 device to the USB port. So, once I do that and restart the system my above resolution to the problem stops working again :(.

Actually qcserial driver is loading the usbserial and once you take the qcserial out of the equation, it all stops working again.

So finally what I did was to:

sudo insmod /lib/modules/4.19.75-v7+/kernel/drivers/usb/serial/usbserial.ko
sudo insmod GobiSerial.ko

and it works fine.

This works but not yet the best way to sort this problem. To sort this problem in the cleanest possible way would be to install the GobiNet and GobiSerial and then do a demod to resolve the dendencies. Follow the below steps for this:

$ export OUTPUTDIR=/lib/modules/`uname -r`/kernel/drivers/net/usb/
$ mkdir -p $OUTPUTDIR
$ cp -f GobiNet.ko $(OUTPUTDIR)
$ cp -f GobiSerial.ko $(OUTPUTDIR)
$ depmod
$ modprobe GobiNet
$ modprobe GobiSerial

Now, restart the system, and you should have everything in place.
To be sure do the below:

pi@raspberrypi:~ $ lsmod | grep -i "Gobi*"
GobiSerial 20480 1
usbserial 40960 3 GobiSerial
GobiNet 438272 0

When we do depmod, it resolves the dependancies of the various modules in the Kernel /lib/modules/ folder. Then at the time of modprobe the dependecy drivers/modules are loaded first and then the actual module of interest.