How to replace the Kernel image in NOOBS with your custom Kernel

In one of the previous posts I have described Bringing up Rasberry Pi 3 with NOOBS. But after I brought up the Raspberry Pi (ver 3 B) with NOOBS I started having some difficultly with the NOOBS kernel version. Some of the Linux drivers I was working with works only with particular version of Linux Kernel and hence it became necessary for me to figure out how to replace the NOOBS kernel image with my own built shiny cross compiled image, so that I have the flexibility of using any version of Kernel I want. I think it could of use for you as well and hence writing the steps up here so that I can refer it later and also maybe someone out there finds it helpful. The original instructions are based out here. The rest of the post is based on Raspberry Pi version 3 B which is what I am working on right now. So the instructions are valid and tested for version 3. It may vary in other Pi hardware version, but the basic steps would remain by and large same.

Bring up Pi with NOOBS

Follow this link to first bring up your Raspberry Pi with NOOBS as described here.

Cross Compile the Kernel

Required utilities

Install the below software in your linux distribution. I am using Ubuntu hence the command to install the dependency software are as below.

$ sudo apt install git bc bison flex libssl-dev make
$ sudo apt install git bc bison flex libssl-dev make libc6-dev libncurses5-dev

Get Linux Source

git clone --depth=1 https://github.com/raspberrypi/linux

Get the toolchain

git clone https://github.com/raspberrypi/tools ~/tools

Set the tool PATH

$ echo PATH=\$PATH:~/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian/bin >> ~/.bashrc
$ source ~/.bashrc

Cross compile the kernel

cd linux/
KERNEL=kernel7

Generate the .config file

$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2709_defconfig

Compile

make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs -j 12

Prepare the SD Card from Pi

Now, you will need the SD card where you have installed NOOBS. One of the mistakes I was doing is thinking installing NOOBS means copying the extracted NOOBS files into the SD card. It is stupid I know, but instructions may not be always very clear (or your head may not be working at all on a day). So make sure you actually have followed all the steps described here. What it will ensure is that the NOOBS actually creates all the partitions required in the SD card and then copy the right files into the right partitions. Once that is done the installation is complete. This is done by NOOBS right after you do the following steps as described in the above link:

  1. Copy the extracted NOOBS files into the sd card
  2. Inserted the SD card into the Rasberry Pi (I am using version 3 B).
  3. Powered up the Pi and be patient when NOOBS does the neccessary things for your to install the Rasbian.

Generally if you have followed all the steps described in the post Bringing up Rasberry Pi 3 with NOOBS then all you need to do is to eject the SD card from the Pi and insert it into you local linux machine (where you are doing the cross compilation).

Locate the SD card in Host

Once you have attached the SD card to you local PC (picture below in case of any confusion), go to the console (I am using a Ubuntu PC).

I use the above card reader, choose you own, it doesn’t matter but essentially, you would probably need a card reader. Now, in you console issue the below command to find out where your SD card has gone:

lsblk
sda 8:0 0 465.8G 0 disk
├─sda1 8:1 0 512M 0 part /boot/efi
└─sda2 8:2 0 465.3G 0 part /
sdc 8:32 1 29.7G 0 disk
├─sdc1 8:33 1 2.4G 0 part
├─sdc2 8:34 1 1K 0 part
├─sdc5 8:37 1 32M 0 part /media/vbhadra/SETTINGS
├─sdc6 8:38 1 256M 0 part /media/vbhadra/boot1
└─sdc7 8:39 1 27G 0 part /media/vbhadra/root
sr0 11:0 1 1024M 0 rom

It looks like as above (just the interesting bits copy pasted) after issuing lsblk command on my Ubuntu console.

Now, sda is your hard drive. So leave that alone. The rest is sdc which is the SD card media in this case. Under sdc you will see the 5 partitions that NOOBS has created for us. The most important partitions for us are sdc6 and sdc7. SDC6 is FAT partition called boot and SDC7 is an ext4 type partitions called root.

Mount SD card partitions on Host Linux

We need to install the newly built kernel modules to the SD card partitions. Also, we need to copy paste other stuffs to the SD card partitions. So for that purpose we are up to mounting these partitions sdc6 and sdc7 into suitable folders on the host PC (Ubuntu in my case). To do that follow the below steps on the host PC console:

~/temp/linux$ mkdir mnt
~/temp/linux$ mkdir mnt/fat32
~/temp/linux$ mkdir mnt/ext4
~/temp/linux$ sudo mount /dev/sdb6 mnt/fat32
~/temp/linux$ sudo mount /dev/sdc6 mnt/fat32
~/temp/linux$ sudo mount /dev/sdc7 mnt/ext4 

~/temp/linux$ ls mnt/
ext4/  fat32/ 

~/temp/linux$ ls mnt/fat32/
bcm2708-rpi-b.dtb       bcm2710-rpi-2-b.dtb       bcm2835-rpi-a-plus.dtb   bcm2835-rpi-zero-w.dtb    config.txt     fixup_cd.dat        kernel7.img       overlays      start_db.elf
bcm2708-rpi-b-plus.dtb  bcm2710-rpi-3-b.dtb       bcm2835-rpi-b.dtb        bcm2836-rpi-2-b.dtb       COPYING.linux  fixup.dat           kernel7l.img      start4cd.elf  start.elf
bcm2708-rpi-cm.dtb      bcm2710-rpi-3-b-plus.dtb  bcm2835-rpi-b-plus.dtb   bcm2837-rpi-3-b.dtb       fixup4cd.dat   fixup_db.dat        kernel8.img       start4db.elf  start_x.elf
bcm2708-rpi-zero.dtb    bcm2710-rpi-cm3.dtb       bcm2835-rpi-b-rev2.dtb   bcm2837-rpi-3-b-plus.dtb  fixup4.dat     fixup_x.dat         kernel.img        start4.elf
bcm2708-rpi-zero-w.dtb  bcm2711-rpi-4-b.dtb       bcm2835-rpi-cm1-io1.dtb  bootcode.bin              fixup4db.dat   issue.txt           LICENCE.broadcom  start4x.elf
bcm2709-rpi-2-b.dtb     bcm2835-rpi-a.dtb         bcm2835-rpi-zero.dtb     cmdline.txt               fixup4x.dat    kernel7-backup.img  os_config.json    start_cd.elf

~/temp/linux$ ls mnt/ext4/
bin  boot  dev  etc  home  lib  lost+found  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
~/temp/linux$ mount 
/dev/sdc6 on /media/vbhadra/boot1 type vfat (rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remount-ro,uhelper=udisks2)
/dev/sdc7 on /media/vbhadra/root type ext4 (rw,nosuid,nodev,relatime,uhelper=udisks2)

Install the newly build module into SD card

~/temp/linux$ sudo env PATH=$PATH make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- INSTALL_MOD_PATH=mnt/ext4 modules_install

Copy the rest of the files into SD card

Copy the Kernel Image

~/temp/linux$ sudo cp mnt/fat32/$KERNEL.img mnt/fat32/$KERNEL-backup.img
~/temp/linux$ sudo cp arch/arm/boot/zImage mnt/fat32/$KERNEL.img

Copy the dtb file

~/temp/linux$ sudo cp arch/arm/boot/dts/*.dtb mnt/fat32/
~/temp/linux$ sudo cp arch/arm/boot/dts/overlays/*.dtb* mnt/fat32/overlays/
~/temp/linux$ sudo cp arch/arm/boot/dts/overlays/README mnt/fat32/overlays/

Umount the sd card partitions

~/temp/linux$ sudo umount mnt/fat32
~/temp/linux$ sudo umount mnt/ext4

Boot the Pi with the SD card

Take the SD card and insert it into the Pi back again. Then power up the Pi. The rest should be taken care by NOOBS. So wait and watch. You shouldn’t see any errors at all. In case of any error, please post it in the below comments section so that I can have a look. Once the Pi comes up open a console and check if your Pi is actually running your newly built kernel image. See the below section for that.

Verify the kernel version on Pi

pi@raspberrypi:~ $ uname -a 
Linux raspberrypi 4.19.106-v7+ #1 SMP Tue Mar 3 12:57:47 GMT 2020 armv7l GNU/Linux

As you might have already noticed the new kernel version is 4.19.106-v7+. Th earlier on my Pi was 4.19.75-v7+. Also, notice the time stamp. As it shows the time stamp in my Pi is showing today’s date and time, Tue Mar 3 12:57:47 GMT 2020, another proof that it is my cross built kernel which is running in Pi not the original NOOBS kernel which came with the NOOBS distribution.

All done, have fun!

Bringing up Rasberry Pi 3 with NOOBS

Getting Rasberry Pi up and running

I am using a NOOB version for getting up and running with the Rasberry Pi. The version of my Raspberry Pi is 3 Model B (see below):

20200211_163448

Format a micro SD card

You will possibly need to format a SD Card (if you are not lucky and have one already formatted properly for NOOBS). To format an SD Card I usually prefer gparted. I will format my SD card in FAT32 format for NOOBS. I prefer gparted for formatting media cards. You may follow the instructions in the below tutorial to use gparted if you like:

Get hold of NOOBS

To download an official distribution of NOOBS, go to the below link:

noobs

Click on the Download ZIP and you will get the latest vesion of the NOOBS. Or if you need a different NOOBS version other than the latest, you can get a copy of that in the below link:

http://downloads.raspberrypi.org/NOOBS/images/

For my purposes I have download the NOOBS_v3_2_1 version (hence, I go a NOOBS_v3_2_1.zip file in my ~/Download/ folder) from the above link. Choose the one you want and locate the zip file in your local machine.

Extract the zip file

Once you have downloaded the NOOB version you want, extract the NOOBS and you should have a folder called something like NOOBS_*_*. In my case it is NOOBS_v3_2_1. Go inside the folder using your file browser, select and copy everything (Ctrl-A, Ctr-C). Go back to the SD card and do a Ctrl-V. This will copy and paste everything that the NOOBS_v3_2_1 had into the SD card.

Be patient, it takes a while to copy the stuffs across into the SD card. I often make mistake of prematurely eject the drive and nothing works. So please be patient! There is a circular progress bar at the top on the file browser window which shows the progress of the copy (see the picture below), if you have not already noticed.

Once everything copied into the SD card, safely eject the drive and insert it into the SD card slot in Pi.

20200211_172622

Now, connect the below to the Pi:

  1. A keyboard.
  2. A monitor/TV
  3. A mouse (very useful)
  4. Connect the Pi using micro USB cable to the PC to power it up.

Once you have powered the Pi, the NOOBS will take care of booting the device and soon you should see something like this on the Monitor:

20200211_173226

Select the Raspberry Pi Full option from the menu shown above. You will require at least ~8GB I guess. Other option is to select the LibreELEC_RPi2 which I won’t recommend at this point. After you select one option from the menu, the “install” button will get activated. Click on Install. See the below pictures for reference:

Once you have clicked on the Install the installation starts:

20200211_173842

Wait for the installation to finish. After the installation the setup wizard comes up, follow the screens and setup the Country, Language and Timezone etc. Keep the default user account ‘pi‘ and password ‘raspberry‘. It may ask for restart, do restart.

Unknown symbol usb_serial* in GobiSerial LTE Linux Kernel driver

I was working with the SieraWireless LTE device drivers recently and I got into a strange issue with one of the drivers called GobiSerial. I have compiled this driver code in the same way as described here. But when I try to insert this in the Pi kernel I get the below error:

pi@raspberrypi:~ $ sudo insmod GobiSerial.ko
insmod: ERROR: could not insert module GobiSerial.ko: Unknown symbol in module
pi@raspberrypi:~ $ dmesg
[ 65.759262] GobiSerial: Unknown symbol usb_serial_generic_open (err -2)
[ 65.759290] GobiSerial: Unknown symbol usb_serial_suspend (err -2)
[ 65.759328] GobiSerial: Unknown symbol usb_serial_generic_resume (err -2)
[ 65.759348] GobiSerial: Unknown symbol usb_serial_deregister_drivers (err -2)
[ 65.759407] GobiSerial: Unknown symbol usb_serial_generic_write (err -2)
[ 65.759427] GobiSerial: Unknown symbol usb_serial_register_drivers (err -2)

To debug this issue first I would like to know if the symbols like usb_serial_generic_open exists in the Pi kernel or not. Apparently not, but just to verify that I would use the /proc/kallsyms tool provided by Linux Kernel. This utility prints all the symbols (exported) available in the kernel space. So I would try to grep the missing symbols from all the avialable symbols in the kernel like this:

pi@raspberrypi:~ $ cat /proc/kallsyms | grep -i "usb_serial_generic_open"
pi@raspberrypi:~ $

In the above command nothing is returned indicating my hunch the symbols are not present (exported/available) in the Pi Kernel.
Now, by the look of it the symbols should be coming from a Kernel module/driver usb_serial, this is my hunch by the look of the symbols which are missing. My next hunch is some Kernel module are not inserted unless there is a reason for it. Like if you haven’t inserted anything on the USB serial the usb_serial driver won’t be inserted into the kernel. Let’s try this. I will attach the SieraWireless LTE device into the USB port of the Pi. Before I do this I usually clean my Kernel messages buffer as below:

sudo dmesg -c

I do this so that when I insert the device I can only focus on the kernel messages coming out for this device. Now insert the device, wait for a second and then try the dmesg again, and you should be able to see something like this on the console:

[ 252.661876] usb 1-1.3: reset low-speed USB device number 4 using dwc_otg
[ 253.511884] usb 1-1.5: reset low-speed USB device number 5 using dwc_otg
[ 261.551829] usb 1-1.2: new high-speed USB device number 8 using dwc_otg
[ 261.682434] usb 1-1.2: config 1 has an invalid interface number: 8 but max is 3
[ 261.682451] usb 1-1.2: config 1 has no interface number 1
[ 261.683111] usb 1-1.2: New USB device found, idVendor=1199, idProduct=9091, bcdDevice= 0.06
[ 261.683123] usb 1-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 261.683133] usb 1-1.2: Product: Sierra Wireless EM7565 Qualcomm® Snapdragon™ X16 LTE-A
[ 261.683143] usb 1-1.2: Manufacturer: Sierra Wireless, Incorporated
[ 261.683153] usb 1-1.2: SerialNumber: UF92377624041506
[ 261.686019] QMAP Disabled
[ 261.688132] GobiNet 1-1.2:1.8 eth1: register 'GobiNet' at usb-3f980000.usb-1.2, GobiNet Ethernet Device, d2:bd:e1:49:f1:08
[ 261.689996] USB Speed : USB 2.0
[ 261.805791] usbcore: registered new interface driver usbserial_generic
[ 261.805832] usbserial: USB Serial support registered for generic
[ 261.806590] usbcore: registered new interface driver cdc_wdm
[ 261.814689] usbcore: registered new interface driver qcserial
[ 261.814756] usbserial: USB Serial support registered for Qualcomm USB modem
[ 261.815707] qcserial 1-1.2:1.0: Qualcomm USB modem converter detected
[ 261.816223] usb 1-1.2: Qualcomm USB modem converter now attached to ttyUSB0
[ 261.819226] qcserial 1-1.2:1.2: Qualcomm USB modem converter detected
[ 261.819635] usb 1-1.2: Qualcomm USB modem converter now attached to ttyUSB1
[ 261.819970] qcserial 1-1.2:1.3: Qualcomm USB modem converter detected
[ 261.820236] usb 1-1.2: Qualcomm USB modem converter now attached to ttyUSB2
[ 261.820411] usbcore: registered new interface driver qmi_wwan
[ 261.920268] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready

Looks like the default linux kernel serial drivers been just inserted. Curious huh? Now, check if our missing symbol is back in the Kernel or not as below:

pi@raspberrypi:~ $ cat /proc/kallsyms | grep -i "usb_serial_generic_open"
00000000 r __ksymtab_usb_serial_generic_open [usbserial]
00000000 r __kstrtab_usb_serial_generic_open [usbserial]
00000000 t usb_serial_generic_open [usbserial]

Hurray!! It is back. Now try inserting the GobiSerial driver module again:

pi@raspberrypi:~ $ sudo insmod GobiSerial.ko
pi@raspberrypi:~ $

Seems like the previous error didn’t appear. Lets check the dmesg again, and here we go:

[ 326.647915] usbserial: USB Serial support registered for GobiSerial
[ 326.647943] GobiSerial: 2019-11-22/SWI_2.39:GobiSerial

Issue resolved.
But soon I realized that attaching the device to the RPI hw is actually insmod-ing the qcserial and the qmi_wwan kernel modules which are Qualcomm drivers. But if you are dealing with the device EM7565, in the release notes it prohibits to have qcserial and qmi_wwan drivers in the kernel and prescribes to blacklist these in the below file:
/etc/modprobe.d/blacklist-modem.conf.


$ sudo vim /etc/modprobe.d/blacklist-modem.conf
# Uncomment these entries in order to blacklist unwanted modem drivers
# blacklist snd-atiixp-modem
# blacklist snd-intel8x0m
# blacklist snd-via82xx-modem
blacklist qcserial
blacklist qmi_wwan

If you do that these two drivers won’t be loaded when you attach your EM7565 device to the USB port. So, once I do that and restart the system my above resolution to the problem stops working again :(.

Actually qcserial driver is loading the usbserial and once you take the qcserial out of the equation, it all stops working again.

So finally what I did was to:

sudo insmod /lib/modules/4.19.75-v7+/kernel/drivers/usb/serial/usbserial.ko
sudo insmod GobiSerial.ko

and it works fine.

This works but not yet the best way to sort this problem. To sort this problem in the cleanest possible way would be to install the GobiNet and GobiSerial and then do a demod to resolve the dendencies. Follow the below steps for this:

$ export OUTPUTDIR=/lib/modules/`uname -r`/kernel/drivers/net/usb/
$ mkdir -p $OUTPUTDIR
$ cp -f GobiNet.ko $(OUTPUTDIR)
$ cp -f GobiSerial.ko $(OUTPUTDIR)
$ depmod
$ modprobe GobiNet
$ modprobe GobiSerial

Now, restart the system, and you should have everything in place.
To be sure do the below:

pi@raspberrypi:~ $ lsmod | grep -i "Gobi*"
GobiSerial 20480 1
usbserial 40960 3 GobiSerial
GobiNet 438272 0

When we do depmod, it resolves the dependancies of the various modules in the Kernel /lib/modules/ folder. Then at the time of modprobe the dependecy drivers/modules are loaded first and then the actual module of interest.

Cross compiling custom Linux Kenel driver for Rasberry Pi Platform

Getting Rasberry Pi up and running

To bring up your Raspberry Pi (ver 3 B) with NOOBS follow the instructions here.

Find the kernel version of NOOBS installation

Go to Pi and open a terminal. Then type the below:

pi@raspberrypi:~ $ uname -a
Linux raspberrypi 4.19.75-v7+ #1270 SMP Tue Sep 24 18:45:11 BST 2019 armv7l GNU/Linux

In the above it shows the kernel version this NOOBS installation has is 4.19.75-v7+. Now, for cross compilation we need the exact version of the Kernel source code, otherwise we will get the below error:

pi@raspberrypi:~ $ sudo insmod GobiNet.ko
insmod: ERROR: could not insert module GobiNet.ko: Invalid module format

Linux kernels for Raspberry Pi is located here:

https://github.com/raspberrypi/linux

Screenshot from 2020-02-12 12-33-57

By default the one coming right now is version 4.19.y. This doesn’t exactly match with the one we need i.e. 4.19.75-v7+. Next you can look into the Branches here:

https://github.com/raspberrypi/linux/branches

Screenshot from 2020-02-12 12-36-18

However, I cannot see the exact match. So now I look into the Releases tab. This looks like has the one we want:

https://github.com/raspberrypi/linux/releases

Screenshot from 2020-02-12 12-37-58

In this page I can see the Kernel release which was on the 24th Sep, 2019. This exactly matches with the one in NOOBS:

pi@raspberrypi:~ $ uname -a
Linux raspberrypi 4.19.75-v7+ #1270 SMP Tue Sep 24 18:45:11 BST 2019 armv7l GNU/Linux

Download this Kernel source code in your local machine (Ubuntu in my case). The tarball is called linux-raspberrypi-kernel_1.20190925-1.tar.gz. Extract the tarball with the below command:

tar -xvf linux-raspberrypi-kernel_1.20190925-1.tar.gz

In my setup after extraction the linux source is located in the below path:

/home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1

Now, go inside the linux kernel source folder and do th below:

cd /home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1
make mrproper


This will ensure we have started with a clean kernel source.
Now, come to your Pi again. What we need now is the kernel configuration which was used for the NOOBS kernel compilation. The Kernel configuration now a days kept in the kernel itself as a module. So to grab hold of the kernel .config file do the below in Pi:

pi@raspberrypi:~ $ sudo modprobe configs
pi@raspberrypi:~ $ zcat /proc/config.gz > /tmp/.config

Copy this .config file in a sd card and bring it back to the Ubuntu PC where your downloaded Kernel source tree lives. Then copy the .config file into the linux kernel source tree as below:

vbhadra@vbhadra-8200-elit-convertable-microtower:~/Downloads/linux-raspberrypi-kernel_1.20190925-1$ sudo mount /dev/sdb1 ~/usb/
vbhadra@vbhadra-8200-elit-convertable-microtower:~/Downloads/linux-raspberrypi-kernel_1.20190925-1$ ls -la ~/usb/
total 748
-rw-r–r– 1 vbhadra vbhadra 163003 Sep 26 02:23 .config
vbhadra@vbhadra-8200-elit-convertable-microtower:~/Downloads/linux-raspberrypi-kernel_1.20190925-1$ cp ~/usb/.config .

Download the Pi toolchain

You will need the toolchain to cross compile the Kernel and your module for Pi environment. Download the toolchain from the git hub with the below command:

git clone https://github.com/raspberrypi/tools ~/tools

Locate where your tools folder is. I usually create a rpi folder in my home folder and do everything there. So in my setup it is located here:
/home/vbhadra/rpi/tools
Now, you need to point where your tools (cross compiler) is for compiling the kernel. To do that do the below:

export CCPREFIX=/home/vbhadra/rpi/tools/arm-bcm2708/arm-bcm2708-linux-gnueabi/bin/arm-bcm2708-linux-gnueabi-

Compile the kernel to create the Module.symvers file. To compile the kernel do the below in your Ubuntu setup:

make ARCH=arm CROSS\_COMPILE=${CCPREFIX} -j 12

Let the above compile the Kernel which will take a while, wait patiently!

Now check if you have got a file created called in your linux folder:

vbhadra@vbhadra-8200-elit-convertable-microtower:~/Downloads/linux-raspberrypi-kernel_1.20190925-1$ ls -la Module.symvers
-rw-r–r– 1 vbhadra vbhadra 841463 Feb 12 13:31 Module.symvers

Compile your module

I am trying to cross compile GobiNet and GobiSerial drivers fro SieraWireless LTE device EM7565.

You can grab hold of the driver rom SieraWireless website. This is not about any specific driver and their intricacies. We will only focus on the important bits to cross compile any custom driver for Raspberry Pi (hw version 3 B).

So One you have any driver code which has been compiled for any other platform (possibly) and you want to cross compile it, first thing you should do is to look at the driver Makefile. In this case, I have the below driver Makefile for GobiNet driver:

obj-m := GobiNet.o
GobiNet-objs := GobiUSBNet.o QMIDevice.o QMI.o
KDIR := /lib/modules/$(shell uname -r)/build
PWD := $(shell pwd)
OUTPUTDIR=/lib/modules/`uname -r`/kernel/drivers/net/usb/
#KBUILD_CFLAGS += -DQOS_SIMULATE
#KBUILD_CFLAGS += -DTX_XMIT_SIERRA -DTX_URB_MONITOR
ifdef TX_URB_MONITOR
ifeq ($(TX_URB_MONITOR), 1)
KBUILD_CFLAGS += -DTX_XMIT_SIERRA -DTX_URB_MONITOR
else
TX_URB_MONITOR:=0
endif
else
TX_URB_MONITOR:=0
endif
ifeq ($(TX_URB_MONITOR), 1)
	ccflags-y:=-DTX_URB_MONITOR
	GobiNet-objs += usbnet_3_0_6.o \
	            usbnet_2_6_35.o usbnet_3_10_21.o usbnet_3_12_xx.o usbnet_4_4_xx.o
endif

RAWIP := 0
ifeq ($(RAWIP), 1)
	ccflags-y:=-DDATA_MODE_RP
endif

PI_KDIR := ~/k/linux-rpi-3.6.y
PI_CCPREFIX=~/toolchain/rpi/tools-master/arm-bcm2708/arm-bcm2708-linux-gnueabi/bin/arm-bcm2708-linux-gnueabi- 

OW_KDIR := ~/openwrt/trunk/build_dir/target-mips_r2_uClibc-0.9.33.2/linux-ar71xx_generic/linux-3.8.11
OW_CCPREFIX=~/openwrt/trunk/staging_dir/toolchain-mips_r2_gcc-4.6-linaro_uClibc-0.9.33.2/bin/mips-openwrt-linux-

MARVELL_KDIR := ~/toolchain/qmi_mxwell/kernel-2.6.31
MARVELL_CCPREFIX := ~/toolchain/qmi_mxwell/toolchain/bin/arm-none-linux-gnueabi-

M64_KDIR := ~/toolchain/arm64bit/linux/linux-4.4.1-devel-16.04.1
M64_CCPREFIX := ~/toolchain/arm64bit/be/aarch64ebv8-marvell-linux-gnu-5.2.1_x86_64_20151117/bin/aarch64_be-marvell-linux-gnu-

PPC_KDIR := ~/toolchain/ppc/kernel_ppc
PPC_CCPREFIX :=~/toolchain/ppc/fsl/1.1/sysroots/i686-fslsdk-linux/usr/bin/ppc64e5500-fsl-linux/powerpc64-fsl-linux-

HIKEY9_KDIR := /farm/android/Hikey_9.0_Trunk/kernel/hikey-linaro
HIKEY9_CCPREFIX := /farm/android/Hikey_9.0_Trunk/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin/aarch64-linux-android-

all:
	$(MAKE) -C $(KDIR) M=$(PWD) modules

m64:
	$(MAKE) ARCH=arm64 CROSS_COMPILE=${M64_CCPREFIX} -C $(M64_KDIR) M=$(PWD) modules

ppc:
	$(MAKE) ARCH=powerpc CROSS_COMPILE=${PPC_CCPREFIX} -C $(PPC_KDIR) M=$(PWD) modules

marvell:
	$(MAKE) ARCH=arm CROSS_COMPILE=${MARVELL_CCPREFIX} -C $(MARVELL_KDIR) M=$(PWD) modules

pi: 
	$(MAKE) ARCH=arm CROSS_COMPILE=${PI_CCPREFIX} -C $(PI_KDIR) M=$(PWD) modules

ow: 
	$(MAKE) ARCH=mips CROSS_COMPILE=${OW_CCPREFIX} -C $(OW_KDIR) M=$(PWD) modules

hikey9:
	$(MAKE) ARCH=arm64 CROSS_COMPILE=${HIKEY9_CCPREFIX} -C $(HIKEY9_KDIR) M=$(PWD) modules
install: all
	mkdir -p $(OUTPUTDIR)
	cp -f GobiNet.ko $(OUTPUTDIR)
	depmod
clean:
	rm -rf *.o *~ core .depend .*.cmd *.ko *.mod.c .tmp_versions Module.* modules.order .cache.mk *.o.ur-safe

The only thing we are interested here is the target, in this file there is a target type called “pi”, that’s the one we want.
Look deeper into this target and associated macros:

pi:
$(MAKE) ARCH=arm CROSS_COMPILE=${PI_CCPREFIX} -C $(PI_KDIR) M=$(PWD) modules

Notice the CROSS_COMPILE is set to PI_CCPREFIX which is what you need to fix. In this example we have discussed about the tools we will use for asp Pi Kernel compilation. The same tool will come handy here as well. Point the PI_CCPREFIX to your tool/cross compiler path as below:

PI_CCPREFIX=/home/vbhadra/rpi/tools/arm-bcm2708/arm-bcm2708-linux-gnueabi/bin/arm-bcm2708-linux-gnueabi-

Other important macro is this PI_KDIR. This points to the Kernel folder against which we are going to cross compile. Point this to your previosuly downloaded kernel source folder as below:

PI_KDIR := /home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1

I also modified the clean target as below:

clean:
make -C $(PI_KDIR) M=$(PWD) clean

But this is not necessary.
Save these changes and go back to the driver folder and issue the make clean command to start with a clean slate and then issue make pi command as below:

~/rpi/GobiNet$ make clean
 #rm -rf *.o *~ core .depend .*.cmd *.ko *.mod.c .tmp_versions Module.* modules.order .cache.mk *.o.ur-safe
 make -C /home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1 M=/home/vbhadra/rpi/GobiNet clean
 make[1]: Entering directory '/home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1'
 make[1]: Leaving directory '/home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1'
 vbhadra@vbhadra-8200-elit-convertable-microtower:~/rpi/GobiNet$ make pi
 make ARCH=arm CROSS_COMPILE=/home/vbhadra/rpi/tools/arm-bcm2708/arm-bcm2708-linux-gnueabi/bin/arm-bcm2708-linux-gnueabi-  -C /home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1 M=/home/vbhadra/rpi/GobiNet modules
 make[1]: Entering directory '/home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1'
 CC [M]  /home/vbhadra/rpi/GobiNet/GobiUSBNet.o
 In file included from ./include/net/sock.h:58:0,
 from ./include/net/inet_sock.h:26,
 from ./include/net/ip.h:31,
 from /home/vbhadra/rpi/GobiNet/GobiUSBNet.c:94:
 ./include/linux/memcontrol.h: In function 'task_in_memcg_oom':
 ./include/linux/memcontrol.h:546:2: warning: return makes integer from pointer without a cast [enabled by default]
 /home/vbhadra/rpi/GobiNet/GobiUSBNet.c: At top level:
 /home/vbhadra/rpi/GobiNet/GobiUSBNet.c:112:2: warning: #warning "Remove memcontrol.h task_in_memcg_oom warning : replace 'return p->memcg_in_oom;' to 'return p->memcg_in_oom==NULL ? 0 : 1;' in function task_in_memcg_oom" [-Wcpp]
 /home/vbhadra/rpi/GobiNet/GobiUSBNet.c:113:2: warning: #warning "Commnet '#define MEMCG_NOT_FIX' above after fix applied." [-Wcpp]
 CC [M]  /home/vbhadra/rpi/GobiNet/QMIDevice.o
 CC [M]  /home/vbhadra/rpi/GobiNet/QMI.o
 LD [M]  /home/vbhadra/rpi/GobiNet/GobiNet.o
 Building modules, stage 2.
 MODPOST 1 modules
 CC      /home/vbhadra/rpi/GobiNet/GobiNet.mod.o
 LD [M]  /home/vbhadra/rpi/GobiNet/GobiNet.ko
 make[1]: Leaving directory '/home/vbhadra/Downloads/linux-raspberrypi-kernel_1.20190925-1'


Check if there is a new kernel module been compiled called GobiNet.ko (in your case your define obj-m name). In this case the driver defined module name was GobiNet in the Makefile as below:

obj-m := GobiNet.o

So I will check if there is a GobiNet.ko been generated:

vbhadra@vbhadra-8200-elit-convertable-microtower:~/rpi/GobiNet$ ls -la GobiNet.ko
-rw-r–r– 1 vbhadra vbhadra 217987 Feb 21 11:17 GobiNet.ko


Copy this .ko file in a sdcard and insert into a Pi. Copy the .ko file from the sdcard to home folder in Pi. Then do the below insert it into the Pi Kernel:


pi@raspberrypi:~ $ sudo insmod GobiNet.ko
pi@raspberrypi:~ $ dmesg
[ 2526.062621] GobiNet: 2019-11-22/SWI_2.60
[ 2526.064306] usbcore: registered new interface driver GobiNet

As you can see now the cross compiled GobiNet driver is up and running in the Pi Kernel.

The other kernel driver for SieraWireless LTE EM7565 device is called GobiSerial. I have compiled this driver code in the same way. But when I try to insert this in the Pi kernel I get the below error:

pi@raspberrypi:~ $ sudo insmod GobiSerial.ko
insmod: ERROR: could not insert module GobiSerial.ko: Unknown symbol in module
pi@raspberrypi:~ $ dmesg
[ 65.759262] GobiSerial: Unknown symbol usb_serial_generic_open (err -2)
[ 65.759290] GobiSerial: Unknown symbol usb_serial_suspend (err -2)
[ 65.759328] GobiSerial: Unknown symbol usb_serial_generic_resume (err -2)
[ 65.759348] GobiSerial: Unknown symbol usb_serial_deregister_drivers (err -2)
[ 65.759407] GobiSerial: Unknown symbol usb_serial_generic_write (err -2)
[ 65.759427] GobiSerial: Unknown symbol usb_serial_register_drivers (err -2)


I resolved the problem in the following post here.

How to integrate a custom device driver into kernel tree

Find out where you want to place the device driver source code. In my case I had to integrate a device driver for a device called ring oscillator. The category of the device was not clear to me and I couldn’t find a suitable place under “linux/drivers” folder except “linux/drivers/misc”. So I decided to place my driver source code under linux/drivers/misc. You may decide to place it where you think  it is appropriate. It is up to to decide.

Read More »

How to modify kernel command line

[/contact-form] You can modify the kernel command line from the below dts file: ~/repos/solaris_src/linux/arch/mips/boot/dts/xyx/xyz.dts chosen { – bootargs = “console=ttyS0,115200n8 loglevel=8 earlycon=uart8250,mmio32,0x1CC40000,115200 rootwait ro”; + bootargs = “mem=32M console=ttyS0,115200n8 loglevel=8 earlycon=uart8250,mmio32,0x1CC40000,115200 rootwait ro”; };