Unknown symbol usb_serial* in GobiSerial LTE Linux Kernel driver

I was working with the SieraWireless LTE device drivers recently and I got into a strange issue with one of the drivers called GobiSerial. I have compiled this driver code in the same way as described here. But when I try to insert this in the Pi kernel I get the below error:

pi@raspberrypi:~ $ sudo insmod GobiSerial.ko
insmod: ERROR: could not insert module GobiSerial.ko: Unknown symbol in module
pi@raspberrypi:~ $ dmesg
[ 65.759262] GobiSerial: Unknown symbol usb_serial_generic_open (err -2)
[ 65.759290] GobiSerial: Unknown symbol usb_serial_suspend (err -2)
[ 65.759328] GobiSerial: Unknown symbol usb_serial_generic_resume (err -2)
[ 65.759348] GobiSerial: Unknown symbol usb_serial_deregister_drivers (err -2)
[ 65.759407] GobiSerial: Unknown symbol usb_serial_generic_write (err -2)
[ 65.759427] GobiSerial: Unknown symbol usb_serial_register_drivers (err -2)

To debug this issue first I would like to know if the symbols like usb_serial_generic_open exists in the Pi kernel or not. Apparently not, but just to verify that I would use the /proc/kallsyms tool provided by Linux Kernel. This utility prints all the symbols (exported) available in the kernel space. So I would try to grep the missing symbols from all the avialable symbols in the kernel like this:

pi@raspberrypi:~ $ cat /proc/kallsyms | grep -i "usb_serial_generic_open"
pi@raspberrypi:~ $

In the above command nothing is returned indicating my hunch the symbols are not present (exported/available) in the Pi Kernel.
Now, by the look of it the symbols should be coming from a Kernel module/driver usb_serial, this is my hunch by the look of the symbols which are missing. My next hunch is some Kernel module are not inserted unless there is a reason for it. Like if you haven’t inserted anything on the USB serial the usb_serial driver won’t be inserted into the kernel. Let’s try this. I will attach the SieraWireless LTE device into the USB port of the Pi. Before I do this I usually clean my Kernel messages buffer as below:

sudo dmesg -c

I do this so that when I insert the device I can only focus on the kernel messages coming out for this device. Now insert the device, wait for a second and then try the dmesg again, and you should be able to see something like this on the console:

[ 252.661876] usb 1-1.3: reset low-speed USB device number 4 using dwc_otg
[ 253.511884] usb 1-1.5: reset low-speed USB device number 5 using dwc_otg
[ 261.551829] usb 1-1.2: new high-speed USB device number 8 using dwc_otg
[ 261.682434] usb 1-1.2: config 1 has an invalid interface number: 8 but max is 3
[ 261.682451] usb 1-1.2: config 1 has no interface number 1
[ 261.683111] usb 1-1.2: New USB device found, idVendor=1199, idProduct=9091, bcdDevice= 0.06
[ 261.683123] usb 1-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 261.683133] usb 1-1.2: Product: Sierra Wireless EM7565 Qualcomm® Snapdragon™ X16 LTE-A
[ 261.683143] usb 1-1.2: Manufacturer: Sierra Wireless, Incorporated
[ 261.683153] usb 1-1.2: SerialNumber: UF92377624041506
[ 261.686019] QMAP Disabled
[ 261.688132] GobiNet 1-1.2:1.8 eth1: register 'GobiNet' at usb-3f980000.usb-1.2, GobiNet Ethernet Device, d2:bd:e1:49:f1:08
[ 261.689996] USB Speed : USB 2.0
[ 261.805791] usbcore: registered new interface driver usbserial_generic
[ 261.805832] usbserial: USB Serial support registered for generic
[ 261.806590] usbcore: registered new interface driver cdc_wdm
[ 261.814689] usbcore: registered new interface driver qcserial
[ 261.814756] usbserial: USB Serial support registered for Qualcomm USB modem
[ 261.815707] qcserial 1-1.2:1.0: Qualcomm USB modem converter detected
[ 261.816223] usb 1-1.2: Qualcomm USB modem converter now attached to ttyUSB0
[ 261.819226] qcserial 1-1.2:1.2: Qualcomm USB modem converter detected
[ 261.819635] usb 1-1.2: Qualcomm USB modem converter now attached to ttyUSB1
[ 261.819970] qcserial 1-1.2:1.3: Qualcomm USB modem converter detected
[ 261.820236] usb 1-1.2: Qualcomm USB modem converter now attached to ttyUSB2
[ 261.820411] usbcore: registered new interface driver qmi_wwan
[ 261.920268] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready

Looks like the default linux kernel serial drivers been just inserted. Curious huh? Now, check if our missing symbol is back in the Kernel or not as below:

pi@raspberrypi:~ $ cat /proc/kallsyms | grep -i "usb_serial_generic_open"
00000000 r __ksymtab_usb_serial_generic_open [usbserial]
00000000 r __kstrtab_usb_serial_generic_open [usbserial]
00000000 t usb_serial_generic_open [usbserial]

Hurray!! It is back. Now try inserting the GobiSerial driver module again:

pi@raspberrypi:~ $ sudo insmod GobiSerial.ko
pi@raspberrypi:~ $

Seems like the previous error didn’t appear. Lets check the dmesg again, and here we go:

[ 326.647915] usbserial: USB Serial support registered for GobiSerial
[ 326.647943] GobiSerial: 2019-11-22/SWI_2.39:GobiSerial

Issue resolved.

Cross compiling custom Linux Kenel driver for Rasberry Pi Platform

Getting Rasberry Pi up and running

I am using a NOOB version for getting up and running with the Rasberry Pi. The version of my Raspberry Pi is 3 Model B (see below):


Format a micro SD card

For formatting the micro SD card I usually use the gparted utility in Ubuntu. Format the micro SD with FAT 32 filesystem format. You need to have a SD card reader for attaching the SD card to the PC. I use the below SD card and the reader for my purposes:

Once you have attached your SD card with the linux PC you are ready to format it. I am using a Ubuntu PC for my purposes. Go to the console and run the below command:

~$ sudo gparted

Screenshot from 2020-02-11 16-58-29

By default the gparted selects your hard drive (/dev/sda is actually your hard drive, did you know!). Be very cautious, not to play around with your hard drive partitions, you might soon get into a funny situation. So go to the top right corner of the gparted screen and change the drive from /dev/sda to /dev/sdb (if there are no other media attached to the PC). Look at the various field carefully and try to understand what they are. I cannot go into the details of all the fields here. What we are up to is delete everything in the SD card and get it ready for the NOOBS. I can see the below in my SD card (/dev/sdb) at the moment:

Screenshot from 2020-02-11 17-02-31

You can also see the same partitions with lsblk command on the console, I can see the below:

sda 8:0 0 465.8G 0 disk
├─sda1 8:1 0 512M 0 part /boot/efi
└─sda2 8:2 0 465.3G 0 part /
sdb 8:16 1 29.7G 0 disk
├─sdb1 8:17 1 2.4G 0 part
├─sdb2 8:18 1 1K 0 part
├─sdb5 8:21 1 32M 0 part
├─sdb6 8:22 1 256M 0 part
└─sdb7 8:23 1 27.1G 0 part

As you can see above the SD card has in total 5 partitions /dev/sdb1, /dev/sdb2, /dev/sdb5, /dev/sdb6, /dev/sdb7. I want to get rid of all of these. So what I do in gparted screen is to select/highlight each partition, right click, and then Unmount. Once the selected/highlighted partition has been Unmounted, you can right click and Delete it. Unmount and Delete each of these partitions in the SD card. Now Click on the green Tick at the top of the gparted screen and Apply. At this point you would have unallocated partition like this:

Screenshot from 2020-02-11 17-09-51

Right click on the Unallocated partition and click on New. Leave everything as default except Filesystems as FAT32, the default is ext4.

Screenshot from 2020-02-11 17-11-49

After changing the file systems type it should look like the below:

Screenshot from 2020-02-11 17-13-02

Click Add. Then again click on the green Tick button at the top, then Apply. Once done, you should see something like the below:

Screenshot from 2020-02-11 17-14-07

Now, close gparted and go back to your favourite file browser and locate the SD card, in my case it shows something like the below, an empty drive:

Screenshot from 2020-02-11 17-15-44

Get hold of the NOOBS

Go to the below link:



Click on the Download ZIP and you will get the latest vesion of the NOOBS. Or if you need a different NOOBS version other than the latest, you can get a copy of that in the below link:


For my purposes I have download the NOOBS_v3_2_1 version (NOOBS_v3_2_1.zip) from the above link. Choose the one you need and move to the next step.

Once you have downloaded the NOOB version you want, extract the NOOBS and you should have a folder called something like NOOBS_*_*. In my case it is NOOBS_v3_2_1. Go inside the folder using your file browser, select and copy everything (Ctrl-A, Ctr-C). Go back to the SD card and do a Ctrl-V. This will copy and paste everything that the NOOBS_v3_2_1 had into the SD card.

Be patient, it takes a while to copy the stuffs across into the SD card. I often make mistake of prematurely eject the drive and nothing works! So be patient! There is a circular progress bar is whirling around at the top on the file browser window, if you have noticed. Once everything copied into the SD card, safely eject the drive and insert it into the SD card slot in Pi.


Now, connect the below to the Pi:

  1. A keyboard.
  2. A monitor/TV
  3. A mouse (very useful)
  4. Connect the Pi using micro USB cable to the PC to power it up.

Once you have powered the Pi, the NOOBS will take care of booting the device and soon you should see something like this on the Monitor:


Select the Raspberry Pi Full option from the menu shown above. You will require at least ~8GB I guess. Other option is to select the LibreELEC_RPi2 which I won’t recommend at this point. After you select one option from the menu, the “install” button will get activated. Click on Install. See the below pictures for reference:

Once you have clicked on the Install the installation starts:


Wait for the installation to finish. After the installation the setup wizard comes up, follow the screens and setup the Country, Language and Timezone etc. Keep the default user account ‘pi’ and password ‘raspberry’. It may ask for restart, do restart.

Find the kernel version of NOOBS installation

Go to Pi and open a terminal. Then type the below:

pi@raspberrypi:~ $ uname -a
Linux raspberrypi 4.19.75-v7+ #1270 SMP Tue Sep 24 18:45:11 BST 2019 armv7l GNU/Linux

In the above it shows the kernel version this NOOBS installation has is 4.19.75-v7+. Now, for cross compilation we need the exact version of the Kernel source code, otherwise we will get the below error:

pi@raspberrypi:~ $ sudo insmod GobiNet.ko
insmod: ERROR: could not insert module GobiNet.ko: Invalid module format
Linux kernels for Raspberry Pi is located here:


Screenshot from 2020-02-12 12-33-57

By default the one coming right now is version 4.19.y. This doesn’t exactly match with the one we need i.e. 4.19.75-v7+. Next you can look into the Branches here:


Screenshot from 2020-02-12 12-36-18

However, I cannot see the exact match. So now I look into the Releases tab. This looks like has the one we want:


Screenshot from 2020-02-12 12-37-58

In this page I can see the Kernel release which was on the 24th Sep, 2019. This exactly matches with the one in NOOBS:

pi@raspberrypi:~ $ uname -a
Linux raspberrypi 4.19.75-v7+ #1270 SMP Tue Sep 24 18:45:11 BST 2019 armv7l GNU/Linux

Download this Kernel source code in your local machine (Ubuntu in my case). The tarball is called linux-raspberrypi-kernel_1.20190925-1.tar.gz. Extract the tarball with the below command:

tar -xvf linux-raspberrypi-kernel_1.20190925-1.tar.gz
In my setup after extraction the linux source is located in the below path:

Now, go inside the linux kernel source folder and do th below:

cd /home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1
make mrproper

This will ensure we have started with a clean kernel source.
Now, come to your Pi again. What we need now is the kernel configuration which was used for the NOOBS kernel compilation. The Kernel configuration now a days kept in the kernel itself as a module. So to grab hold of the kernel .config file do the below in Pi:

pi@raspberrypi:~ $ sudo modprobe configs
pi@raspberrypi:~ $ zcat /proc/config.gz > /tmp/.config

Copy this .config file in a sd card and bring it back to the Ubuntu PC where your downloaded Kernel source tree lives. Then copy the .config file into the linux kernel source tree as below:

vbhadra@vbhadra-8200-elit-convertable-microtower:~/Downloads/linux-raspberrypi-kernel_1.20190925-1$ sudo mount /dev/sdb1 ~/usb/
vbhadra@vbhadra-8200-elit-convertable-microtower:~/Downloads/linux-raspberrypi-kernel_1.20190925-1$ ls -la ~/usb/
total 748
-rw-r--r-- 1 vbhadra vbhadra 163003 Sep 26 02:23 .config
vbhadra@vbhadra-8200-elit-convertable-microtower:~/Downloads/linux-raspberrypi-kernel_1.20190925-1$ cp ~/usb/.config .

Download the Pi toolchain

You will need the toolchain to cross compile the Kernel and your module for Pi environment. Download the toolchain from the git hub with the below command:

git clone https://github.com/raspberrypi/tools ~/tools

Locate where your tools folder is. I usually create a rpi folder in my home folder and do everything there. So in my setup it is located here:
Now, you need to point where your tools (cross compiler) is for compiling the kernel. To do that do the below:

export CCPREFIX=/home/vbhadra/rpi/tools/arm-bcm2708/arm-bcm2708-linux-gnueabi/bin/arm-bcm2708-linux-gnueabi-

Compile the kernel to create the Module.symvers file. To compile the kernel do the below in your Ubuntu setup:


Let the above compile the Kernel which will take a while, wait patiently!

Now check if you have got a file created called in your linux folder:

vbhadra@vbhadra-8200-elit-convertable-microtower:~/Downloads/linux-raspberrypi-kernel_1.20190925-1$ ls -la Module.symvers
-rw-r--r-- 1 vbhadra vbhadra 841463 Feb 12 13:31 Module.symvers

Compile your module

I am trying to cross compile GobiNet and GobiSerial drivers fro SieraWireless LTE device EM7565.

You can grab hold of the driver rom SieraWireless website. This is not about any specific driver and their intricacies. We will only focus on the important bits to cross compile any custom driver for Raspberry Pi (hw version 3 B).

So One you have any driver code which has been compiled for any other platform (possibly) and you want to cross compile it, first thing you should do is to look at the driver Makefile. In this case, I have the below driver Makefile for GobiNet driver:

obj-m := GobiNet.o
GobiNet-objs := GobiUSBNet.o QMIDevice.o QMI.o
KDIR := /lib/modules/$(shell uname -r)/build
PWD := $(shell pwd)
OUTPUTDIR=/lib/modules/`uname -r`/kernel/drivers/net/usb/
ifeq ($(TX_URB_MONITOR), 1)
ifeq ($(TX_URB_MONITOR), 1)
GobiNet-objs += usbnet_3_0_6.o \
usbnet_2_6_35.o usbnet_3_10_21.o usbnet_3_12_xx.o usbnet_4_4_xx.o

RAWIP := 0
ifeq ($(RAWIP), 1)

#PI_KDIR := ~/k/linux-rpi-3.6.y
#PI_KDIR := /home/vbhadra/Downloads/linux-rpi-4.19.y-rt
PI_KDIR := /home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1

OW_KDIR := ~/openwrt/trunk/build_dir/target-mips_r2_uClibc-

MARVELL_KDIR := ~/toolchain/qmi_mxwell/kernel-2.6.31
MARVELL_CCPREFIX := ~/toolchain/qmi_mxwell/toolchain/bin/arm-none-linux-gnueabi-

M64_KDIR := ~/toolchain/arm64bit/linux/linux-4.4.1-devel-16.04.1
M64_CCPREFIX := ~/toolchain/arm64bit/be/aarch64ebv8-marvell-linux-gnu-5.2.1_x86_64_20151117/bin/aarch64_be-marvell-linux-gnu-

PPC_KDIR := ~/toolchain/ppc/kernel_ppc
PPC_CCPREFIX :=~/toolchain/ppc/fsl/1.1/sysroots/i686-fslsdk-linux/usr/bin/ppc64e5500-fsl-linux/powerpc64-fsl-linux-

HIKEY9_KDIR := /farm/android/Hikey_9.0_Trunk/kernel/hikey-linaro
HIKEY9_CCPREFIX := /farm/android/Hikey_9.0_Trunk/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin/aarch64-linux-android-

$(MAKE) -C $(KDIR) M=$(PWD) modules

$(MAKE) ARCH=arm64 CROSS_COMPILE=${M64_CCPREFIX} -C $(M64_KDIR) M=$(PWD) modules






install: all
mkdir -p $(OUTPUTDIR)
cp -f GobiNet.ko $(OUTPUTDIR)

#rm -rf *.o *~ core .depend .*.cmd *.ko *.mod.c .tmp_versions Module.* modules.order .cache.mk *.o.ur-safe
make -C $(PI_KDIR) M=$(PWD) clean

The only thing we are interested here is the target, in this file there is a target type called “pi”, that’s the one we want.
Look deeper into this target and associated macros:


Notice the CROSS_COMPILE is set to PI_CCPREFIX which is what you need to fix. In this example we have discussed about the tools we will use for asp Pi Kernel compilation. The same tool will come handy here as well. Point the PI_CCPREFIX to your tool/cross compiler path as below:


Other important macro is this PI_KDIR. This points to the Kernel folder against which we are going to cross compile. Point this to your previosuly downloaded kernel source folder as below:

PI_KDIR := /home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1

I also modified the clean target as below:

make -C $(PI_KDIR) M=$(PWD) clean

But this is not necessary.
Save these changes and go back to the driver folder and issue the make clean command to start with a clean slate and then issue make pi command as below:

vbhadra@vbhadra-8200-elit-convertable-microtower:~/rpi/GobiNet$ make clean
#rm -rf *.o *~ core .depend .*.cmd *.ko *.mod.c .tmp_versions Module.* modules.order .cache.mk *.o.ur-safe
make -C /home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1 M=/home/vbhadra/rpi/GobiNet clean
make[1]: Entering directory '/home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1'
make[1]: Leaving directory '/home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1'
vbhadra@vbhadra-8200-elit-convertable-microtower:~/rpi/GobiNet$ make pi
make ARCH=arm CROSS_COMPILE=/home/vbhadra/rpi/tools/arm-bcm2708/arm-bcm2708-linux-gnueabi/bin/arm-bcm2708-linux-gnueabi- -C /home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1 M=/home/vbhadra/rpi/GobiNet modules
make[1]: Entering directory '/home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1'
CC [M] /home/vbhadra/rpi/GobiNet/GobiUSBNet.o
In file included from ./include/net/sock.h:58:0,
from ./include/net/inet_sock.h:26,
from ./include/net/ip.h:31,
from /home/vbhadra/rpi/GobiNet/GobiUSBNet.c:94:
./include/linux/memcontrol.h: In function 'task_in_memcg_oom':
./include/linux/memcontrol.h:546:2: warning: return makes integer from pointer without a cast [enabled by default]
/home/vbhadra/rpi/GobiNet/GobiUSBNet.c: At top level:
/home/vbhadra/rpi/GobiNet/GobiUSBNet.c:112:2: warning: #warning "Remove memcontrol.h task_in_memcg_oom warning : replace 'return p->memcg_in_oom;' to 'return p->memcg_in_oom==NULL ? 0 : 1;' in function task_in_memcg_oom" [-Wcpp]
/home/vbhadra/rpi/GobiNet/GobiUSBNet.c:113:2: warning: #warning "Commnet '#define MEMCG_NOT_FIX' above after fix applied." [-Wcpp]
CC [M] /home/vbhadra/rpi/GobiNet/QMIDevice.o
CC [M] /home/vbhadra/rpi/GobiNet/QMI.o
LD [M] /home/vbhadra/rpi/GobiNet/GobiNet.o
Building modules, stage 2.
MODPOST 1 modules
CC /home/vbhadra/rpi/GobiNet/GobiNet.mod.o
LD [M] /home/vbhadra/rpi/GobiNet/GobiNet.ko
make[1]: Leaving directory '/home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1'

Check if there is a new kernel module been compiled called GobiNet.ko (in your case your define obj-m name). In this case the driver defined module name was GobiNet in the Makefile as below:

obj-m := GobiNet.o

So I will check if there is a GobiNet.ko been generated:

vbhadra@vbhadra-8200-elit-convertable-microtower:~/rpi/GobiNet$ ls -la GobiNet.ko
-rw-r--r-- 1 vbhadra vbhadra 217987 Feb 21 11:17 GobiNet.ko

Copy this .ko file in a sdcard and insert into a Pi. Copy the .ko file from the sdcard to home folder in Pi. Then do the below insert it into the Pi Kernel:

pi@raspberrypi:~ $ sudo insmod GobiNet.ko
pi@raspberrypi:~ $ dmesg
[ 2526.062621] GobiNet: 2019-11-22/SWI_2.60
[ 2526.064306] usbcore: registered new interface driver GobiNet

As you can see now the cross compiled GobiNet driver is up and running in the Pi Kernel.

The other kernel driver for SieraWireless LTE EM7565 device is called GobiSerial. I have compiled this driver code in the same way. But when I try to insert this in the Pi kernel I get the below error:

pi@raspberrypi:~ $ sudo insmod GobiSerial.ko
insmod: ERROR: could not insert module GobiSerial.ko: Unknown symbol in module
pi@raspberrypi:~ $ dmesg
[ 65.759262] GobiSerial: Unknown symbol usb_serial_generic_open (err -2)
[ 65.759290] GobiSerial: Unknown symbol usb_serial_suspend (err -2)
[ 65.759328] GobiSerial: Unknown symbol usb_serial_generic_resume (err -2)
[ 65.759348] GobiSerial: Unknown symbol usb_serial_deregister_drivers (err -2)
[ 65.759407] GobiSerial: Unknown symbol usb_serial_generic_write (err -2)
[ 65.759427] GobiSerial: Unknown symbol usb_serial_register_drivers (err -2)

I resolved the problem in the following post here.


How to modify kernel command line

[/contact-form] You can modify the kernel command line from the below dts file: ~/repos/solaris_src/linux/arch/mips/boot/dts/xyx/xyz.dts chosen { – bootargs = “console=ttyS0,115200n8 loglevel=8 earlycon=uart8250,mmio32,0x1CC40000,115200 rootwait ro”; + bootargs = “mem=32M console=ttyS0,115200n8 loglevel=8 earlycon=uart8250,mmio32,0x1CC40000,115200 rootwait ro”; };  

Browse Kernel code with Eclipse

This blog will try to demonstrate the steps required to use eclipse for browsing Linux kernel level source code, e.g. Linux kernel tree or device driver codes. If you have already worked with Linux kernel level source code you will know that the Linux kernel is written in C and Linux kernel space device drivers are also written in C. For c/c++ source code the eclipse version I use is called Eclipse-CDT (C/C++ Development Tooling). My development environment is Ubuntu Linux. So most of my references will be based on Ubuntu packages but these steps are pretty much similar in other Linux distros.

Read More »


Problem Statement

The main mlme processing kernel thread comes out of the wait_for_completion() immediately without waiting for the complete() to be called on the completion variable. As a result the mlme response was erroneous and the command to check the wifi link status “sudo iw wlan0 link” was returning error.

Read More »