Try, Try, Again

Previously I mentioned the premature demise of my openwrt and IZ2S development laptops.   I have backups for both, but they're on far from optimal machines.  So I've been itching to get something going with native compilers on the zipit.  Lately I've been having some luck with the aboriginal linux native compiler on my openwrt zipit, so I thought I'd give the other aboriginal native compiler some attention on the IZ2S zipit.

Running gcc -v tells me the native compiler I've been trying to use in IZ2S was built with --disable-multilib and --with-float=soft, which is incompatible with the hard float settings in all the IZ2S shared libs built with the scratchbox VM.  That's too bad because the xscale processor in the zipit has no FP hardware, so the soft-float setting makes more sense than forcing the kernel to trap whenever a function call attempts to pass an arg via nonexistant FP registers.  But even more disappointing, it means the aboriginal gcc is pretty much limited to producing static executables for IZ2S.  Someday I should try to use it to build a static version of nightsky and see if soft-float really is 10 times faster than trapping all the FP calls.

The scratchbox instructions on the wiki mention arm-gcc4.1-uclibc20061004 so I fetched gcc 4.1.1 and attempted to build it with the scratchbox cross-compiler.  I think the last time I built a compiler was way back before C++ even existed, so I was more than a bit intimidated by the complexity of a more modern compiler.  My first attempt to build gcc in VM didn't go so well, so i thought maybe I could build it right on the zipit with the aboriginal compiler.  I very quickly thought better of that plan.  Unzipping gcc on the zipit used way too much disk space.  In fact, I panicked watching it unzip all that stuff and hit the hard reset button, which naturally hosed the FAT filesystem on the SD card.  Poking around in the wreakage reminded me that the FAT filesystem doesn't support softlinks, probably a deathblow to building gcc.  Not to mention the zipit is too slow, has no memory, and the SD card would take a serious beating.

I came to my senses and decided to try again in scratchbox.  This time I did some homework.  Running gcc -v reveals the config settings used to build the scratchbox compiler.  Running ld -V tells what config settings were used to build the scratchbox binutils.  I took some notes and read (ok skimmed) the gcc build instructions.  So this time I built binutils first, configured to install in the /usr/local/share/gcc directory so I could keep it segregated on the zipit SD card.  That went fairly well, so I duplicated most of the config settings from the cross-compiler and attempted to build a native gcc.  It went somewhat better this time.  Near the end of the build it tried to use xgcc (a limited build of gcc) to build something - maybe libgcc and/or the final gcc executable.  But xgcc was an arm executable built in an earlier stage, so it segfaulted when run in the scratchbox environment.  I patched the gcc/Makefile to replace ./xgcc with gcc so it would use the scratchbox cross-compiler to finish the job.  This actually seemed to work and make install put what looked like a native compiler in /usr/local/share/gcc.

I gave it a whirl on the zipit and discovered a few problems compiling a basic hello.c program.  First of all, it couldn't find stdio.h.  I'm not sure how to get make install to put everything in the /usr/local/share/gcc directory.  So meanwhile I had to fetch the /usr/include and /usr/lib directories from scratchbox.  I used zip to pack these up for transfer to the zipit so it would make file copies instead of softlinks for the FAT filesystem used by IZ2S.  I also had to track down libc.so.0 because apparently libc.so is just a script that points to the real libc.so.0 somewhere in bowels of scratchbox.  I edited the libc.so script to remove the scratchbox paths so libc.so.0 could reside in the same directory as libc.so on the zipit.  I also made a compiler.sh script add the /usr/local/share/gcc/bin to the PATH and softlink the various lib and include directories into the proper places in the /usr and /usr/local sections of the filesystem.

This let me build the executable, but it wouldn't run.  It was looking for the /lib/ld-linux.so.2 dynamic linker instead of the /lib/ld-uClibc.so.0 I had on the zipit.  I fixed this temporarily with a softlink, which got the hello program running, but was quite unsatisfying.  After a bit of internet research I went back to the gcc build and edited gcc-4.4.1/gcc/config/arm/linux-elf.h to make LINUX_TARGET_INTERPRETER use the uClibc dynamic linker.  Reconfigure, rebuild, again.

Now hello.c compiles and runs.  Yeah!  So I moved on and I tried to build an IZ2S version of imgv.  It compiles and links against the IZ2S SDL libs.  Yes!  Couldn't do that with the aboriginal compiler.  But it segfaults when I run it.  Drat!  Actually, some of it runs because when I forgot to set the SDL env vars it rotated the screen before segfaulting.  I think that means it was actually doing some of the SDL stuff.  I ran imgv through strace, and it appears to blame gettimeofday?  Weird, but getting closer.  I ccould almost feel it.  I've had mysterious segfaults due to bugs in the ancient IZ2S uclibc so I also compiled imgv with the scratchbox compiler to compare.  It also segfaults on getimeofday so the compiler is off the hook on that one.

I went to my backup SDL test program - fgui.  This also segfaulted, but with a little debugging I tracked it down to the SDL_init call.  It was set to initialize SDL_EVERYTHING including the SDL_CDROM code.  I probably built SDL without cdrom support, so I reduced the init to SDL_VIDEO only and voila, it runs on the zipit.Looking back at imgv I see it inits SDL_VIDEO and the SDL_TIMER.  I bet the timer code is triggering the segfault when it trys to use the buggy uclibc gettimeofday function.  I might just have to rebuild SDL with some new fixes, or see if I can link in the gettimeofday function from libiberty.a instead.  And I should probably review the SDL_Init calls in some of the SDL programs I'd previously abandoned due to mysterious segfaults (like bunjallo and zgv).

So the good news is the native IZ2S compiler appears to work.  Yes!  Here's a screenshot of the freshly compiled fgui SDL test app in action on the zipit.

Now I just need to figure out how best to trim it down and package it up.  Can I get rid of the enormous pre-compiled c++ headers?  Apparently --enable-libstdcxx-pch is on by default.  This is actually a bit of a tough call.  The precompiled headers use 36MB of the precious SD space, but I suspect if I ever do any C++ they'll save gobs of time on the super slow zipit, by appending --include bits/stdc++.h to CXXFLAGS?  Maybe I should run some time tests.

Here's the native compiler (23MB).  iz2s-gcc.zip
The C++ precompiled headers (7MB).  iz2s-gcc-pch.zip
And a snapshot of my usr/local/lib and include dirs (40MB).  iz2s-gcc-usrlocal.zip

The native compiler contains a script (/bin/compiler) that you should source in order to setup some softlinks and environment variables.  You may want to edit he bin/compiler script if you want to enable a swapfile for compiling bigger things.

Update:

I moved the SDL sources to the zipit and did some debugging on the segfault.  Turns out it was pthreads that was causing the segfault in the sdl_timer code, not gettimeofday.  So I did some searching on the internet and discovered some references to problems running pthreads from shared libs (like say SDL).  The fix is in the linking.  I linked with -pthread before -lSDL and -pthread again (for good luck) after the -lSDL and the other SDL libs and now imgv runs just fine.  So does zgv.  Maybe uclibc has some fake pthread functions that get in the way if you don't explicitly link in the real thing?  I don't know, and I also have no idea right now what's the difference between -pthread and -lpthread, so I have some more reading to do...

Here's an imgv package for IZ2S: imgv-iz2s.zip
And here's a zgv executable for IZ2S: zgv-iz2s.zip

And finally, this is what bunjalloo looks like on the zipit.

Sorry for the poor lighting.  It's sorta ironic considering the headline article in the magazine I used for background.