A split-screen image depicting a coding environment on an x86-64 host and a terminal output in QEMU. The host displays C++ code to print 'Hello, ARM64!', and the QEMU section shows compilation details for ARM64 architecture. The background is dark with a modern aesthetic.

The idea of cross-compilation and emulation

Often in our development, we need to write code for hardware that is different from the hardware where we are doing the development. Now, writing code is not enough to be sure that we are heading in the right direction. We need to compile it and test it. In some cases, we would possibly have the option of transferring our developed code to the hardware where it is supposed to run, installing the compiler on that machine, and then compiling and running it. For example, if I am writing code and I want it to run on a Raspberry Pi 4, we can transfer the source code to the Pi, install the necessary compiler on the Pi 4, and compile and test it. However, that can be a tedious process, and there are also dependencies like the availability of hardware and the test environment. To help with this, we have something called cross-compilation and emulation.

In cross-compilation, we develop the source code on our favorite host machine, such as an x86-64 computer, install a cross-compiler, and then cross-compile it for the target hardware, like, let’s say, a Raspberry Pi 4. In this post, I will explain how this works and then explain how emulation can help us test our cross-compiled code in an ARM64 Linux virtual machine without having the physical hardware.

Cross Compilation

We will exercise cross-compilation on a host x86-64 machine that runs Ubuntu 22.04 Linux. To achieve cross-compilation on the host, the first thing we need to do is install a cross-compiler that is meant for the targeted hardware. We are going to cross-compile for a Raspberry Pi 4, and hence we need to find and install the appropriate cross-compiler. Let’s do that step by step.

1. Check the host operating system and architecture

Run these commands in the host Ubuntu terminal:

cat /etc/os-release
uname -m

The relevant output was:

PRETTY_NAME="Ubuntu 22.04.5 LTS"
VERSION_ID="22.04"
VERSION_CODENAME=jammy
x86_64

uname -m identifies the architecture reported by the running kernel. Here, the host is x86-64, and the executable will target AArch64.

2. Install the cross-compiler and user-mode emulator

Now, Raspberry Pi 4 has the ARM64 architecture. So, we need a cross-compiler that can run on an x86-64 machine but can build for ARM64. Later on, we are going to use QEMU to emulate an ARM64 Linux machine. Hence, we will install the ARM64 Linux cross-compiler, QEMU user-mode emulation, and the executable inspection tools on the host to build, run, and inspect our ARM64 executable.

sudo apt update
sudo apt install g++-aarch64-linux-gnu qemu-user \
binutils-aarch64-linux-gnu file

The cross-compiler package brings in the target development libraries needed by this example. binutils-aarch64-linux-gnu provides tools such as aarch64-linux-gnu-readelf for inspecting the generated executable. Check the installed versions:

aarch64-linux-gnu-g++ --version
qemu-aarch64 --version

The version lines were:

aarch64-linux-gnu-g++ (Ubuntu 11.4.0-1ubuntu1~22.04.3) 11.4.0
qemu-aarch64 version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.31)

The name aarch64-linux-gnu-g++ identifies the target architecture, Linux platform and GNU environment. The compiler itself runs on the host; its generated machine code and linked libraries are for the target.

3. Create the C++ source file

For testing our cross-compilation setup, let’s create a source file in the host.

Example 1: arm64_cross_compile_test.cpp

#include <iostream>
int main()
{
#if defined(__aarch64__)
std::cout << "Compiled for AArch64\n";
#else
std::cout << "Compiled for another architecture\n";
#endif
std::cout << "Pointer size: " << sizeof(void*) << " bytes\n";
}

The preprocessor selects the message during compilation. GCC defines __aarch64__ for its AArch64 target, so the first message identifies the compilation target. sizeof(void*) reports the size of a pointer in the compiled program’s data model. It is eight bytes in this ARM64 Linux build.

4. Cross-compile the program

Build the executable on the host:

aarch64-linux-gnu-g++ \
-std=c++17 -O2 -Wall -Wextra -Wpedantic \
arm64_cross_compile_test.cpp \
-o arm64_cross_compile_test

The toolchain selects the AArch64 target even though it is running on an x86-64 host.

5. Inspect the generated executable

Use file command on the host to identify the executable:

$ file arm64_cross_compile_test
arm64_cross_compile_test: ELF 64-bit LSB pie executable,
ARM aarch64, version 1 (SYSV), dynamically linked,
interpreter /lib/ld-linux-aarch64.so.1,
BuildID[sha1]=8970793dcb93c7145182434ae2ffe9d9d27f6c85,
for GNU/Linux 3.7.0, not stripped
  • ELF = executable format.
  • 64-bit LSB identifies a 64-bit, little-endian machine.
  • ARM aarch64 identifies the target architecture.
  • The executable is dynamically linked and requests the ARM64 loader /lib/ld-linux-aarch64.so.1.
  • The pie executable description means it is a position-independent executable.

For more detail, you may inspect the ELF header, program headers, and dynamic section:

aarch64-linux-gnu-readelf -h arm64_cross_compile_test
aarch64-linux-gnu-readelf -l arm64_cross_compile_test
aarch64-linux-gnu-readelf -d arm64_cross_compile_test

The relevant header fields were:

Class: ELF64
Data: 2's complement, little endian
Type: DYN (Position-Independent
Executable file)
Machine: AArch64
Entry point address: 0x940

The ELF type DYN is also used for shared objects. In this file, the interpreter and PIE information identify it as a position-independent executable. The entry point is the startup address in the executable, which is distinct from the C++ main function.

The dynamic section contained these direct dependencies and flags:

(NEEDED) Shared library: [libstdc++.so.6]
(NEEDED) Shared library: [libc.so.6]
(FLAGS) BIND_NOW
(FLAGS_1) Flags: NOW PIE

NEEDED lists the shared libraries required to run the program: libstdc++.so.6 is the C++ standard library, and libc.so.6 is the C standard library. The libraries can require further libraries themselves. The NOW flags request symbol binding during loading.

Emulation Using QUEMU

Our cross-compiled binary is ready to be tested, but we don’t have hardware. How do we test it then? For testing the cross-compiled binary, we will use an emulation tool called QEMU. Follow the steps below to set up QEMU and run the binary on it.

6. Run the executable with QEMU user-mode emulation

Before setting up the full ARM64 Linux virtual machine, we will use QEMU’s user-mode emulation to test our cross-compiled program. This is a comparatively lightweight test where QEMU’s user-mode emulator executes the ARM64 instructions and translates Linux system calls for the host’s kernel, so we can quickly run and test the cross-compiled program without first booting a guest operating system.

qemu-aarch64 -L /usr/aarch64-linux-gnu ./arm64_cross_compile_test
Compiled for AArch64
Pointer size: 8 bytes

QEMU user-mode emulation executes the target program and translates supported Linux system calls for the host. It uses the host kernel. The -L argument provides the target path prefix used to find the ARM64 loader and libraries installed with the cross-toolchain.

7. Compare dynamic and static linking

Build a second executable with -static on the host:

aarch64-linux-gnu-g++ \
-std=c++17 -O2 -Wall -Wextra -Wpedantic -static \
arm64_cross_compile_test.cpp \
-o arm64_cross_compile_test_static
file arm64_cross_compile_test_static
qemu-aarch64 ./arm64_cross_compile_test_static
arm64_cross_compile_test_static: ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=326e50c10208239bb4611123da9eea16749d7c88, for GNU/Linux 3.7.0, not stripped
Compiled for AArch64
Pointer size: 8 bytes

For this program, static linking includes the required library code in the executable. QEMU therefore runs it without the separate ARM64 loader and shared-library prefix. The executable still contains ARM64 instructions and uses Linux services. Compare file sizes and inspect the static executable’s dynamic section:

ls -lh arm64_cross_compile_test arm64_cross_compile_test_static
aarch64-linux-gnu-readelf -d arm64_cross_compile_test_static
-rwxrwxr-x 1 vbhadra vbhadra 9.2K Oct 3 09:00 arm64_cross_compile_test
-rwxrwxr-x 1 vbhadra vbhadra 2.1M Oct 3 09:05 arm64_cross_compile_test_static
There is no dynamic section in this file.

8. Install the full-system QEMU tools

The next stage creates a separate ARM64 Linux machine. Install these packages on the host:

sudo apt install qemu-system-arm qemu-system-gui \
qemu-efi-aarch64 qemu-utils cloud-image-utils wget openssh-client

On Ubuntu 22.04, qemu-system-arm supplies the ARM system emulators, including qemu-system-aarch64. The GUI package supplies the window interface. The EFI package supplies ARM64 firmware, qemu-utils supplies disk tools, and cloud-image-utils supplies cloud-localds for preparing first-boot configuration.

9. Download the ARM64 Ubuntu image

Create a dedicated directory on the host:

mkdir -p ~/qemu_arm64_lab
cd ~/qemu_arm64_lab

Download the ARM64 Ubuntu Server cloud image:

wget https://cloud-images.ubuntu.com/releases/jammy/release/ubuntu-22.04-server-cloudimg-arm64.img

This image contains an installed operating system and boots through UEFI.

10. Create the guest’s working disk

Create a QCOW2 working disk backed by the downloaded image, then expand its virtual capacity. Run these commands once during initial setup, with the guest stopped:

qemu-img create -f qcow2 \
-F qcow2 \
-b "$PWD/ubuntu-22.04-server-cloudimg-arm64.img" \
ubuntu_arm64_lab.qcow2
qemu-img resize ubuntu_arm64_lab.qcow2 12G

11. Prepare the guest account and first-boot configuration

In ~/qemu_arm64_lab on the host, create the cloud-init user configuration:

cat > user-data <<'EOF_USER_DATA'
#cloud-config
hostname: arm64-lab
users:
- name: lab
groups: [sudo]
shell: /bin/bash
sudo: "ALL=(ALL) ALL"
lock_passwd: false
plain_text_passwd: 'Arm64Lab2026'
ssh_pwauth: true
chpasswd:
expire: false
EOF_USER_DATA

This creates the lab account with Bash, gives it access to sudo, and enables password authentication for SSH. expire: false keeps the configured password usable without a forced change at first login.

The password shown here is the temporary password used in the exercise. For a new guest, choose a password of your own before creating the seed disk. The user-data file and generated seed disk contain the configured password in plain text. Create the instance metadata:

cat > meta-data <<'EOF_META_DATA'
instance-id: arm64-lab-001
local-hostname: arm64-lab
EOF_META_DATA

Create the configuration disk:

cloud-localds seed.img user-data meta-data

cloud-localds packages the files into a disk that cloud-init can read through its NoCloud data source. QEMU will attach this disk alongside the operating-system disk.

12. Start the ARM64 guest in a QEMU window

From ~/qemu_arm64_lab on the host, launch the virtual machine:

qemu-system-aarch64 \
-name arm64-lab \
-machine virt \
-accel tcg \
-cpu cortex-a72 \
-smp 2 \
-m 2048 \
-bios /usr/share/qemu-efi-aarch64/QEMU_EFI.fd \
-drive if=none,file=ubuntu_arm64_lab.qcow2,format=qcow2,id=osdisk \
-device virtio-blk-pci,drive=osdisk \
-drive if=none,file=seed.img,format=raw,id=seed \
-device virtio-blk-pci,drive=seed \
-netdev user,id=net0,hostfwd=tcp:127.0.0.1:2222-:22 \
-device virtio-net-pci,netdev=net0 \
-display gtk \
-serial vc \
-monitor none

The command stays running while the guest is active. Open a second host terminal for SSH and file transfers.

The options configure the following parts of the machine:

OptionPurpose
-machine virtSelects the generic ARM platform
-accel tcgUses software CPU emulation for the ARM64 guest on x86-64
-cpu cortex-a72Selects the emulated CPU model
-smp 2Presents two virtual CPUs
-m 2048Allocates 2,048 MiB of guest RAM
-bios ...QEMU_EFI.fdLoads the installed ARM64 UEFI firmware
-drive ...id=osdisk and its -deviceAttach the working disk through a VirtIO block device
-drive ...id=seed and its -deviceAttach the first-boot configuration disk
-netdev user,...Provides user-mode networking and SSH port forwarding
-device virtio-net-pci,...Presents a VirtIO network interface to the guest
-display gtkOpens a QEMU window
-serial vcConnects the guest serial port to a console in that window
-monitor noneDisables the separate QEMU monitor console

The window displays the Ubuntu Server text console. Select the serial console entry, usually serial0, from the View menu if it is not already selected. Allow first-boot initialization to finish, then press Enter if needed to expose the login prompt.

Log in with username lab and the password configured in user-data. Password input normally appears without characters being echoed.

13. Connect to the guest through SSH

The guest can also be reached from a second host terminal. First, the running QEMU process was checked with:

pgrep -af qemu-system-aarch64

The process had PID 15392 and used the launch options above. Process IDs change between runs. Connect from the host:

ssh -p 2222 lab@127.0.0.1

The forwarding rule sends connections to the host’s loopback port 2222 to the guest port 22. The address 127.0.0.1 in this command refers to the host; QEMU forwards the connection into the guest. After login, the prompt becomes:

lab@arm64-lab:~$

To inspect the guest directly, run these commands at the guest prompt:

uname -m
cat /etc/os-release

The architecture should be aarch64. The SSH welcome message already identified the guest kernel as AArch64. An SSH session is also convenient for pasting commands and copying terminal output. The QEMU serial console remains available for guest-side execution in the window.

14. Transfer the cross-compiled executable

Use a host terminal for the transfer. If the current terminal is connected to the guest, either open another host terminal or run exit to leave that SSH session. Copy the dynamic executable from the host’s home directory into the guest:

scp -P 2222 ~/arm64_cross_compile_test lab@127.0.0.1:/home/lab/

The executable is now stored in the guest’s /home/lab directory.

15. Run the program inside the guest

At the guest prompt, run:

cd /home/lab
chmod +x arm64_cross_compile_test
./arm64_cross_compile_test

16. Shut down and restart the guest

Shut down from either the guest’s SSH session or its QEMU console:

sudo poweroff

Enter the guest password if prompted. Let Ubuntu finish shutting down before working on its disk files.

To restart, return to ~/qemu_arm64_lab on the host and repeat the QEMU launch command from step 12. The working disk retains the transferred executable and guest account. Disk creation, resizing and initial seed creation are performed during setup.

There you go, we cross-compiled a C++ program, brought up our QEMU emulation environment with the target guest OS on it, transferred the cross-compiled binary, and ran it on the QEMU platform, confirming it runs in the emulated ARM64 Linux environment without having to have the hardware itself. Happy cross-compiling!


Discover more from Tech For Talk

Subscribe to get the latest posts sent to your email.

Leave a Reply