QEMU take 2: A New Compiler for IZ2S

I'm not entirely clear how I got derailed from my efforts with the Arch zipit and my script cleanup projects, but whatever, I'll roll with it.  At some point in mid April I found myself reexamining the qemu setup that I'd abandoned over a year ago.  My goal back then was to be able to compile and test IZ2S zipit goodies without the clunky scratchbox VM I've been using for several years now.  

I didn't exactly get there, but I did reach a point where I could run SDL programs like gmenu2x in a 320x240 pixel zipit framebuffer, simulated in a QEMU window.  That could be nice for testing purposes, but not so much for compiling due to the limited memory in the zipit simulation.  It's almost like compiling on the zipit itself.  However, an IZ2S chroot running in qemu user-mode *could* be made to run a compiler with all the memory of the host machine.  Most of the work on that was already done by Tycho at brainfisheatfrishbrain.com based on the qemu setup outlined here.  Unfortunately Tycho vanished from the zipit IRC channel, apparently without quite finishing.  And I ran out of Holiday time, also without finishing.  Naturally some other zipit project eventually distracted me...

Anyhow here I was, back on track with the qemu setup, and things seemed to be falling into place this time.  There were small gaps in Tycho's documentation that stumped me before.  Where it says, "Create a chroot", what it really means is to make a chroot for the host system.  These were subtle bits that I missed the first time around.  Of course it could also be that my host laptop is more up to date this time due to a recent disk crash and reinstall with the current version of LMDE.  I took some notes and I think this fake shell script captures what I've done so far, using the tarball from 29 november 2013 (SD iz2s FAT) as a starting point.

I'm pretty sure the old gcc-4.1.1 compiler in that tarball still works, just like it does on the SD card.  So if you just want a compiler, and don't care that it's showing it's age, you can use that one after some minimal setup in the chroot.  The setup is mostly softlinks to the busybox apps and other things that aren't immediately available in the chroot, like they are when you boot from the SD card.  Pretty simple stuff.

I took it a little further and eventually got to the point where I could compile things in the IZ2S chroot with the gcc-4.2.4 that Tycho built.  But then I realized it was linking things against the glibc dynamic linker instead of uclibc.  Ouch.  I ran into that problem long ago when I built my older ARM native IZ2S gcc-4.1.1.  Not only was this a nuisance on the SD card, but it prevented me from using x86 tools from the host OS like make, bash, and autoconf to improve compile times in the IZ2S chroot.  Because of this I can't really recommend using the 4.2.4 compiler.  However I could still use it to build a newer compiler, which I did.  I'm pretty sure the old arm oabi has been obsoleted in the most current gcc releases, so I selected 4.4.7 from March 2012 as a reasonable candidate.  It's the most recent of the 4.4.x series, started in 2009, and it's in the sweet spot of the compilers currently used for openwrt on the zipit.   I've seen hints that maybe a 4.7.x compiler can be made to produce oabi executables, so I may eventually try that someday.

Once I got the compiler running, I recompiled Tycho's binutils to link against ld-uClibc.so.0 and then used the new compliler to rebuild itself, completely purging ld-linux.so.2 from the system. 

Now I had to test it, so I tried it on the OpenMSX that came in the brainfish IZ2S tarball.  Turns out, OpenMSX has a really crazy build system.  Not only does it require python, but it also insists on rebuilding everything from scratch, every time.  I originally passed it over for IZ2S due to the build system, but Tycho put python, and tcl, and all the required pieces in place so I went for it.  It took forever due to the full rebuilds, and because the chroot gcc seems slower than the old scratchbox VM gcc.  I guess that makes sense because even though scratchbox is running in a virtual machine, it uses a cross compiler whereas the IZ2S chroot wraps an arm native gcc in the the qemu-user emulation.  A VM for the same architecture beats emulation.

When I finally got an openMSX executable, it still took me a few days to figure out how to run it on the zipit.  MSX never really went anywhere in the US so I didn't know all that much about it.  Eventually I figured out where to put all the BIOS rom images and such, and got it into the CBIOS screen.

Not very exciting.  Now I had to figure out where to get something interesting to run in it.  Since I don't own any MSX hardware or software I tracked down a few homebrew roms.  Bombjack is sorta cute, but it seems like maybe the sound and gameplay is a bit off.  I suspect at 312MHz the zipit is just ever so slightly underpowered.  For comparison I also ran the openmsx executable that Tycho made with the older gcc-4.1.1 and the arm assembly code disabled.  I couldn't tell the difference.  Oddly, the sound worked about the same on both builds, despite the comment about sound not working.  I checked and noticed the brainfish IZ2S tarball was still using the old libSDL with broken sound support.  Tycho probably never knew his openmsx executable was actually fully functional.

I may need to recompile it one more time if I can't figure out how to modify the keyboard config file to give me access to the Dingux on screen display.  (Without that I have to run it from an ssh session if I want to quit.)  Then hopefully I can put together some IZ2S packages for gcc-4.4.7, openMSX, python, tcl, and whatever other useful libs Tycho built before I get distracted by another project.

I also need to sort out and publish some readable follow up instructions for installing gcc-4.4.7 into the IZ2S chroot, and using it to build things.

I've already got a tracking number for mozzwald's z2 breakout board, which means it's been shipped, so the distraction free window is rapidly closing...


Update:

I sure called that one.  Six months and counting and I still haven't followed up on this.  So I hastily zipped up the gcc-4.4.7 executables.  I hope I got them all.  There actually seems to be multiple copies of the binutils executables.  I suspect only the ones in the gcc/bin directory are actually required.

gcc-4.4.7-iz2s.zip

Unzip it to your SD card and source the compiler script to set up all the paths and such.

  ".   /mnt/sd0/bin/compiler"
 

Here's the instructions I posted in the comments section on how to put together the qemu chroot.  You should be able to unzip the compiler into that instead of compiling it yourself.

Qemu setup instructions

Or, use this tarball of the entire qemu iz2s setup.  You'll still have to follow some of the instructions to start it up in quemu user mode.  It's got the compiler, python, openmsx, etc already built and ready to go.  Thanks mozzwald for hosting it.

iz2s_qemu.tar.bz2

For historical purposes, here's a link to the scratchbox VM with the older gcc-4.1.1 compiler.

Zipit Z2 Development V2.zip

Update 2 (11/2018):

After a few years of neglect I decided to put the qemu compiler to work once again, and make a new IZ2S build of dropbear.  The old cyphers are deprecated and the workaround command line args to ssh into the zipit are painful.  I could handle something like -1, but -oKexAlgorithms=+diffie-hellman-group1-sha1 is about 40 letters too many to commit to memory.  So I resurrected the qemu chroot and went to work compiling the latest dropbear.  It took a few tries to remember some of the tricky techniques required to produce "optimal" executables for both the internal flash jffs and for the SD card.  For the jffs, small size is critical, so I built the dropbearmulti target with minimal features and compressed it with the lzma version of upx.  An IZ2S SD card uses the FAT filesystem, so individual apps work better than dropbearmulti, because softlinks are not available on FAT.

I've uploaded a zip file to the usual place, with dropbearmulti for the jffs and the individual apps for the SD.

Meanwhile, I discovered some gaps in the tool set on the IZ2S qemu image.  The upx and sstrip programs were absent, and libcrypt.a was unavailable.  I believe libcrypt is actually in the qemu image with the older v4.1 compiler, and simply needs to be copied from the older gcc libs in /usr/local/share/gcc/lib to the usr/lib directory.  But it's not in the v4.4.7 compiler zip or the zips for for the IZ2S libs and includes.  So I think I need to put together a small "toolchain_extras" zip file with the missing bits.

Also, since I can't seem to remember, I'll point out here that CFLAGS="-march=armv5te -mtune=xscale" is nice for the zipit processor.  And do NOT use soft-float on IZ2S.  The stock kernel is OABI and the libs are compiled with default float settings.  Back in olden times that meant hard float instructions with kernel emulation of the FPA via exceptions, which is why floating point code is so dreadfully slow on IZ2S.  If you run readelf -h libgcc.a you'll see lots of zeros in the flags and ABI bits, indicating the defaults were used.  So if you get hard/soft float mismatches from the linker (like I did when I accidentally grabbed the soft-float libcrypt.a from openwrt) then you should compare readelf header output of the mismatched files to see what's up.

Here's a readelf -h header sample from the IZ2S libgcc.a for reference:

ELF Header:
  Magic:   7f 45 4c 46 01 01 01 61 00 00 00 00 00 00 00 00
  Class:                             ELF32
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            ARM
  ABI Version:                       0
  Type:                              REL (Relocatable file)
  Machine:                           ARM
  Version:                           0x1
  Entry point address:               0x0
  Start of program headers:          0 (bytes into file)
  Start of section headers:          700 (bytes into file)
  Flags:                             0x0
  Size of this header:               52 (bytes)
  Size of program headers:           0 (bytes)
  Number of program headers:         0
  Size of section headers:           40 (bytes)
  Number of section headers:         9
  Section header string table index: 6




Another tip to remember... If you're making an SDL program for IZ2S, don't forget to link with -pthread before -lSDL and -pthread again after.  This prevents a nasty segfault in the startup code -- before it even gets into main().