Using QEMU in cross compiling for embedded systems

1. QEMU
in
Cross Building
Tetsuyuki Kobayashi
1

2. Who am I?
 20+ years involved in embedded systems
 10 years in real time OS, such as iTRON
 10 years in embedded Java Virtual Machine
 Now Gcc, Linux, QEMU, Android, …
 Blogs
 http://d.hatena.ne.jp/embedded/ (Personal)
 http://blog.kmckk.com/ (Corporate)
 Twitter
 @tetsu_koba
2

3. Today's topic
 About QEMU
 Native build vs Cross build
 Easy but too slow: building on
system emulator
 Fast but too complex:
Scratchbox
 Just simple : building on user
mode QEMU
3

4. Today's topic
 About QEMU
 Native build vs Cross build
 Easy but too slow: building on
system emulator
 Fast but too complex:
Scratchbox
 Just simple : building on user
mode QEMU
4

5. About QEMU
 http://wiki.qemu.org/
 QEMU is
a generic and open source
machine emulator
and virtualizer.
5

6. QEMU is used …
 As virtualizer,
 KVM
 Xen
 VirtualBox
 Host CPU == Target CPU
 Today I don't focus on this.
6

7. QEMU is used …
 As a system emulator,
 Host CPU != Target CPU
 Many open source projects such as:
 Android
 Maemo
 Symbian
 ARM port of Open Solaris
 ...
7

8. QEMU has …
 Virtual disk.
 Many of virtual devices.
 Tiny Code Generator (TCG) to execute fast.
 Port redirector, Virtual network card.
 Bootloader to boot Linux kernel.
 gdbserver to connect gdb.
 …
8

9. User mode QEMU
 Emulate only user mode of linux.
 Not all system calls implemented,
but enough for building.
9

10. Today's topic
 About QEMU
 Native build vs Cross build
 Easy but too slow: building on
system emulator
 Fast but too complex:
Scratchbox
 Just simple : building on user
mode QEMU
10

11. Native build vs Cross build
 Native build – compile and run on
the same machine.
 Cross build – compile and run on
the different arch machine.
 Embedded system does not have enough
CPU and memory for native building.
11

12. Configure problem
 In many open source project
$ configure
$ make
$ make install
This suppose to be native build.
Configure generate many small program
and compile and run to check environment.
This does not work on cross build.

13. Today's topic
 About QEMU
 Native build vs Cross build
 Easy but too slow: building on
system emulator
 Fast but too complex:
Scratchbox
 Just simple : building on user
mode QEMU
13

14. Building on system QEMU
 Apt-get works. Very easy to install
compilers and libraries
 Too slow to compile
 Emulating the whole system including kernel.
 Cannot take advantage of SMP becase
QEMU runs on single thread.

15. Today's topic
 About QEMU
 Native build vs Cross build
 Easy but too slow: building on
system emulator
 Fast but too complex:
Scratchbox
 Just simple : building on user
mode QEMU
15

16. ScratchBox
 Magical environment for cross
building
 running ARM executable using user
mode QEMU
 invoking the cross compiler on native
16

17. ScratchBox 1
 http://www.scratchbox.org/
 Maemo project on Nokia
 mechanism
 chroot
 binfmt_misc
 scratch-box-aware tool chains
17

18. ScratchBox 2
 http://www.freedesktop.org/wiki/Software/sbox2
 Another implementation of scratch box
 Tool chain agnostic
 Hosted by Freedesktop.org
 mechanism
 Wrap libc by PRELOAD magic of ld.so
 Hook system calls
 Insert QEMU when exec target binary
 Path name translation by LUA script 18

19. ScratchBox* is
 Pretty fast to compile
 Very hard to customize and
troubleshooting
19

20. Today's topic
 About QEMU
 Native build vs Cross build
 Easy but too slow: building on
system emulator
 Fast but too complex:
Scratchbox
 Just simple : building on user
mode QEMU
20

21. Let's try simpler way
 Just use user mode QEMU
 Use chroot and binfmt_misc
 Compiler running on QEMU, too
21

22. binfmt_misc
 Mechanism to execute foreign executable.
 ex) Used in Wine for dos executable.

$ cat /proc/sys/fs/binfmt_misc/qemu-arm
enabled
interpreter /usr/bin/qemu-arm-static
flags: OC
offset 0
magic
7f454c4601010100000000000000000002002800
mask
ffffffffffffff00fffffffffffffffffeffffff
$
22

23. Chroot
 Make dedicated directory tree
 Used for public ftp server to make
sand box
23

24. Static linked QEMU
 Nice idea to avoid conflict in chrooted target
root file system
 Dynamically linked QEMU needs so many library
files, which are likey conflict file names to target
ones.
 Static linked QEMU is stand alone. Just copy one
file to target root file system.
 I found this idea in rootstock script in Ubuntu
24

25. Just Try
 Make arm root file system by rootstock on
Ubuntu 10.04
$ sudo apt-get install rootstock
$ mkdir arm-ubuntu
$ cd arm-ubuntu/
$ sudo rootstock --fqdn arm-lucid --login user
--password user
It takes more than 10 minutes ...
$ mkdir root
$ cd root
$ sudo tar xvf ../armel-rootfs-201005031740.tgz
25

26. Just Try
$ ls
bin dev home lost+found mnt proc sbin srv tmp var
boot etc lib media opt root selinux sys usr
$ uname -m
x86_64
$ sudo chroot . /bin/bash
#
# uname -m
armv7l
26

27. Build time comparison
 80min (on system QEMU)
 8min (user mode QEMU, make -j4)
 http://blog.kmckk.com/archives/2342452.html
Not yet try on scratchbox.
It would be faster if I could.
27

28. /proc
 ps command fails

# ps
Cannot find /proc/version - is /proc mounted?

 Don't worry. Mount /proc
# mount proc /proc -t proc
# ps
PID TTY TIME CMD
15434 ? 00:00:00 bash
15440 ? 00:00:00 ps
#
28

29. Networking
 Failed to resolve name
 OK, just add ip address to /etc/hosts
# cat /etc/hosts
127.0.0.1 localhost ubuntu1004
127.0.1.1 arm-lucid
91.189.88.36 ports.ubuntu.com
Then apt-get works. So far, it's enough.
29

30. Distcc
 http://code.google.com/p/distcc/
 Preprocessing and linking locally.
 Compiling and assembling remotely.
 Invoke cross compiler remotely.
 Distcc will enhance build time.
30

31. Q&A
 & Demo
Thank you for listening!
Any comments to blogs are welcome.
31