QEMU take 2: A New Compiler for IZ2S

I'm not entirely clear how I got derailed from my efforts with the Arch zipit and my script cleanup projects, but whatever, I'll roll with it.  At some point in mid April I found myself reexamining the qemu setup that I'd abandoned over a year ago.  My goal back then was to be able to compile and test IZ2S zipit goodies without the clunky scratchbox VM I've been using for several years now.  

I didn't exactly get there, but I did reach a point where I could run SDL programs like gmenu2x in a 320x240 pixel zipit framebuffer, simulated in a QEMU window.  That could be nice for testing purposes, but not so much for compiling due to the limited memory in the zipit simulation.  It's almost like compiling on the zipit itself.  However, an IZ2S chroot running in qemu user-mode *could* be made to run a compiler with all the memory of the host machine.  Most of the work on that was already done by Tycho at brainfisheatfrishbrain.com based on the qemu setup outlined here.  Unfortunately Tycho vanished from the zipit IRC channel, apparently without quite finishing.  And I ran out of Holiday time, also without finishing.  Naturally some other zipit project eventually distracted me...

Anyhow here I was, back on track with the qemu setup, and things seemed to be falling into place this time.  There were small gaps in Tycho's documentation that stumped me before.  Where it says, "Create a chroot", what it really means is to make a chroot for the host system.  These were subtle bits that I missed the first time around.  Of course it could also be that my host laptop is more up to date this time due to a recent disk crash and reinstall with the current version of LMDE.  I took some notes and I think this fake shell script captures what I've done so far, using the tarball from 29 november 2013 (SD iz2s FAT) as a starting point.

I'm pretty sure the old gcc-4.1.1 compiler in that tarball still works, just like it does on the SD card.  So if you just want a compiler, and don't care that it's showing it's age, you can use that one after some minimal setup in the chroot.  The setup is mostly softlinks to the busybox apps and other things that aren't immediately available in the chroot, like they are when you boot from the SD card.  Pretty simple stuff.

I took it a little further and eventually got to the point where I could compile things in the IZ2S chroot with the gcc-4.2.4 that Tycho built.  But then I realized it was linking things against the glibc dynamic linker instead of uclibc.  Ouch.  I ran into that problem long ago when I built my older ARM native IZ2S gcc-4.1.1.  Not only was this a nuisance on the SD card, but it prevented me from using x86 tools from the host OS like make, bash, and autoconf to improve compile times in the IZ2S chroot.  Because of this I can't really recommend using the 4.2.4 compiler.  However I could still use it to build a newer compiler, which I did.  I'm pretty sure the old arm oabi has been obsoleted in the most current gcc releases, so I selected 4.4.7 from March 2012 as a reasonable candidate.  It's the most recent of the 4.4.x series, started in 2009, and it's in the sweet spot of the compilers currently used for openwrt on the zipit.   I've seen hints that maybe a 4.7.x compiler can be made to produce oabi executables, so I may eventually try that someday.

Once I got the compiler running, I recompiled Tycho's binutils to link against ld-uClibc.so.0 and then used the new compliler to rebuild itself, completely purging ld-linux.so.2 from the system. 

Now I had to test it, so I tried it on the OpenMSX that came in the brainfish IZ2S tarball.  Turns out, OpenMSX has a really crazy build system.  Not only does it require python, but it also insists on rebuilding everything from scratch, every time.  I originally passed it over for IZ2S due to the build system, but Tycho put python, and tcl, and all the required pieces in place so I went for it.  It took forever due to the full rebuilds, and because the chroot gcc seems slower than the old scratchbox VM gcc.  I guess that makes sense because even though scratchbox is running in a virtual machine, it uses a cross compiler whereas the IZ2S chroot wraps an arm native gcc in the the qemu-user emulation.  A VM for the same architecture beats emulation.

When I finally got an openMSX executable, it still took me a few days to figure out how to run it on the zipit.  MSX never really went anywhere in the US so I didn't know all that much about it.  Eventually I figured out where to put all the BIOS rom images and such, and got it into the CBIOS screen.

Not very exciting.  Now I had to figure out where to get something interesting to run in it.  Since I don't own any MSX hardware or software I tracked down a few homebrew roms.  Bombjack is sorta cute, but it seems like maybe the sound and gameplay is a bit off.  I suspect at 312MHz the zipit is just ever so slightly underpowered.  For comparison I also ran the openmsx executable that Tycho made with the older gcc-4.1.1 and the arm assembly code disabled.  I couldn't tell the difference.  Oddly, the sound worked about the same on both builds, despite the comment about sound not working.  I checked and noticed the brainfish IZ2S tarball was still using the old libSDL with broken sound support.  Tycho probably never knew his openmsx executable was actually fully functional.

I may need to recompile it one more time if I can't figure out how to modify the keyboard config file to give me access to the Dingux on screen display.  (Without that I have to run it from an ssh session if I want to quit.)  Then hopefully I can put together some IZ2S packages for gcc-4.4.7, openMSX, python, tcl, and whatever other useful libs Tycho built before I get distracted by another project.

I also need to sort out and publish some readable follow up instructions for installing gcc-4.4.7 into the IZ2S chroot, and using it to build things.

I've already got a tracking number for mozzwald's z2 breakout board, which means it's been shipped, so the distraction free window is rapidly closing...


Update:

I sure called that one.  Six months and counting and I still haven't followed up on this.  So I hastily zipped up the gcc-4.4.7 executables.  I hope I got them all.  There actually seems to be multiple copies of the binutils executables.  I suspect only the ones in the gcc/bin directory are actually required.

gcc-4.4.7-iz2s.zip

Unzip it to your SD card and source the compiler script to set up all the paths and such.

  ".   /mnt/sd0/bin/compiler"
 

Here's the instructions I posted in the comments section on how to put together the qemu chroot.  You should be able to unzip the compiler into that instead of compiling it yourself.

Qemu setup instructions

Or, use this tarball of the entire qemu iz2s setup.  You'll still have to follow some of the instructions to start it up in quemu user mode.  It's got the compiler, python, openmsx, etc already built and ready to go.  Thanks mozzwald for hosting it.

iz2s_qemu.tar.bz2

For historical purposes, here's a link to the scratchbox VM with the older gcc-4.1.1 compiler.

Zipit Z2 Development V2.zip

Update 2 (11/2018):

After a few years of neglect I decided to put the qemu compiler to work once again, and make a new IZ2S build of dropbear.  The old cyphers are deprecated and the workaround command line args to ssh into the zipit are painful.  I could handle something like -1, but -oKexAlgorithms=+diffie-hellman-group1-sha1 is about 40 letters too many to commit to memory.  So I resurrected the qemu chroot and went to work compiling the latest dropbear.  It took a few tries to remember some of the tricky techniques required to produce "optimal" executables for both the internal flash jffs and for the SD card.  For the jffs, small size is critical, so I built the dropbearmulti target with minimal features and compressed it with the lzma version of upx.  An IZ2S SD card uses the FAT filesystem, so individual apps work better than dropbearmulti, because softlinks are not available on FAT.

I've uploaded a zip file to the usual place, with dropbearmulti for the jffs and the individual apps for the SD.

Meanwhile, I discovered some gaps in the tool set on the IZ2S qemu image.  The upx and sstrip programs were absent, and libcrypt.a was unavailable.  I believe libcrypt is actually in the qemu image with the older v4.1 compiler, and simply needs to be copied from the older gcc libs in /usr/local/share/gcc/lib to the usr/lib directory.  But it's not in the v4.4.7 compiler zip or the zips for for the IZ2S libs and includes.  So I think I need to put together a small "toolchain_extras" zip file with the missing bits.

Also, since I can't seem to remember, I'll point out here that CFLAGS="-march=armv5te -mtune=xscale" is nice for the zipit processor.  And do NOT use soft-float on IZ2S.  The stock kernel is OABI and the libs are compiled with default float settings.  Back in olden times that meant hard float instructions with kernel emulation of the FPA via exceptions, which is why floating point code is so dreadfully slow on IZ2S.  If you run readelf -h libgcc.a you'll see lots of zeros in the flags and ABI bits, indicating the defaults were used.  So if you get hard/soft float mismatches from the linker (like I did when I accidentally grabbed the soft-float libcrypt.a from openwrt) then you should compare readelf header output of the mismatched files to see what's up.

Here's a readelf -h header sample from the IZ2S libgcc.a for reference:

ELF Header:
  Magic:   7f 45 4c 46 01 01 01 61 00 00 00 00 00 00 00 00
  Class:                             ELF32
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            ARM
  ABI Version:                       0
  Type:                              REL (Relocatable file)
  Machine:                           ARM
  Version:                           0x1
  Entry point address:               0x0
  Start of program headers:          0 (bytes into file)
  Start of section headers:          700 (bytes into file)
  Flags:                             0x0
  Size of this header:               52 (bytes)
  Size of program headers:           0 (bytes)
  Number of program headers:         0
  Size of section headers:           40 (bytes)
  Number of section headers:         9
  Section header string table index: 6




Another tip to remember... If you're making an SDL program for IZ2S, don't forget to link with -pthread before -lSDL and -pthread again after.  This prevents a nasty segfault in the startup code -- before it even gets into main().

Scripts and Scraps

While working on qemacs, gmu, and other assorted goodies for the zipit Arch image, I built up a small collection of scripts to make life easier for myself.  Alsa is not set out of the box with reasonable defaults for the zipit, so a small script takes care of that.  In order to compile I need to run make, which only works properly if the zipit is keeping track of time.  So I made a script to connect the wifi, and dug up another script to set the clock via http.  Arch runs heavy compared to openwrt or iz2s (systemd and bash vs busybox) so I needed a script to enable swap in order for gcc to have enough memory to actually do anything.  You get the idea.  I really must start a github repository for Arch zipit tools before I lose track of this stuff.

Anyhow, after success with gmu I attempted to build gmenu2x for the zipit Arch image.  I was compiling on the zipit itself, so it took forever.  To speed it up I tinkered with zram at high priority with failover to the lower priority swapfile, but that tended to fail rather than failover.  Locking up the zipit did not improve compile time.  Oh well, so for coding and pacman updates I disable zram and switch to a 64 megabyte swap file.  Otherwise I just stay with the zram.  I built my arch gmenu2x with the IZ2S gmenu2x  code fork because I'm more familiar with it than the openwrt branch. Unfortunately I had to modify the code to use the Alsa API for volume control because mozzwald's kernel wasn't configured with OSS support.  I suppose that means I'll have to start another github fork of gmenu2x for Arch.  I'm not too excited with that idea as the Arch alsalib (like all things Arch) is full featured and heavy and tends to push gmenu2x into swap.  So I convinced mozzwald to add OSS to his kernel build.  However I still need to go back and rebuild gmenu2x to test it. 

AFAIK nobody's built any zipit specific power management utilities for Arch yet, so I added keyboard and LCD backlight controls to gmenu2x, but it needs more work.  For example, there's no screen blanker for graphics mode yet, so you close the lid on gmenu2x and the backlights stay on, unless you dial them down manually first.  Hmmm, that got me thinking the IZ2S utilities and some unimplemented features I'd put on the backburner for all that stuff.  So I gathered up the old scripts, and various scraps of code from pastebin, and assembled an IZ2S utilities repository on github.  Then I got to work on some of those backburner utility projects.  First I folded the power management and LED control multi-call-binaries into one super utility mcb.  This allowed me to add the missing keyboard backlight restore function to lid switch monitor, without calling a separate executable.  I built it with tiny executable settings, so someday I'll make yet another jffs image and this'll save a bunch of space on the flash, and possibly some runtime RAM as well.

I also cleaned up the fake mouse demon and added code to intercept key press events on the zipit's side mounted volume rocker button.  If the control or alt modifier keys are active when the rocker is pressed then the demon steals the key events and adjusts the backlights instead.  It's almost just like I promised way back when I published my preferred "universal" keyboard layout.  While I was in the code I also (finally) added functions to get some use out of the play and stop buttons, when they're being interpreted as the left and right fake mouse buttons (probably most of the time given the general lack of mouse aware programs on the zipit).  Previously I'd added play and stop icons on the gmenu2x settings screen, which is nice, but the actual physical play and stop buttons are available all the time.  Much better.  I've currently got them sending fake key events to tty4 if anything like mpg123 is running there, but you can edit the OnPlay or OnStop scripts if you want them to do something else. 

The play and stop button work got me thinking about the youtube and movgrab.  I long ago lost track of the SD card where I originally set all that up for IZ2S, and couldn't find a pastebin with my IZ2S youtube scripts, so I started over and came up with something that mostly works.  Live streaming only works well with the postage stamp size video.  Perhaps the stdin and stdout buffers are limited (like named pipes) to 64K in the aging stock zipit kernel?   So to work around, I let movgrab spool things to a file on the SD card and pointed mplayer at that instead of a pipe.  It works well for short full screen videos, but eventually breaks up on the longer ones.  That's ok, I suppose.  The movgrab process eventually gets it all and you can play it back later with slightly better settings.

Slightly better is still sorta iffy.  I swear it looks better in real life.

I suspect maybe youtube is throttling the stream, so I tried to get mplayer to display the progress bar and let me monitor how far along it was getting.  But apparently the old IZ2S mplayer was compiled without progress bar support.  It's a pretty old version.  Maybe it's time for an upgrade?

 The tiny 3gp video streams ok, but there's not much to see.

I started on a new mplayer build for IZ2S, disabling this and that -- things that will never work on the zipit -- and patching the ffmpeg code for uclibc.  But what I've come up with so far is still too fat and bloated.  If I can't build a reasonably lean and mean mplayer with progress bar support, then perhaps I can convince the existing mplayer to leave a few rows of pixels on the bottom of the screen untouched.  Maybe I can put up my own progress bar with a small framebuffer program running in the background, monitoring the movgrab download progress while simultaneously tracking the estimated play time?  I did some experiments with libFB and libpsf, but nothing conclusive yet. Also, while I can have the background process send pause and play commands to mplayer if the download buffer gets too low, what happens if the user (me) manually tells mplayer to pause, or even quit.  How's the background process gonna know about it?  Gotta do some more thinking on this...

I'll try and package up the goodies and drop them here when I get a chance.

Ok, here's the Arch gmenu2x, gmu, and qemacs in /usr/local/*.
zipit-arch-local.tgz

Here's the assorted helpful scripts that I've got in the Arch home directory. 
archome.tgz

Update - December 2015:

Mozzwald made an updated arch image with the oss enabled kernel, and I recently acquired some cheap 32GB micro sd cards, so I decided to try it out.  Dropbear is a real package now in the arm arch distribution, so that's pre-installed on the new image.  Nice.  This also has the backlight blanking demon so the screen will blank even when closing the lid on an SDL program.  Good stuff!

Since this was a new install, it gave me the chance to document the steps required to get the gmu and gmenu2x in the zipit-arch-local tarball above running.

Right after installing mozzwald's image on the sd card, I installed the two tarballs above.  Then I put it in a zipit and booted.  When it was up and running, the first thing I had to do was look under the battery for the mac address so this zipit wouldn't conflict with my other arch zipit on the wifi.  Install the correct mac and connect to the wifi.

   ifconfig wlan0 hw ether xx:xx:xx:xx:xx:xx
   wifi-menu

Next you want to install some packages.  You'll need to turn off zram and use just the swapfile, otherwise pacman will run out of RAM.  Maybe zram and a swapfile work ok together for small things, but I never got them working well together for pacman.

   swapoff /dev/zram0
   swapon /swapfile2

Then update the packages, and install some good stuff.

   pacman -Syu

For SDL you have to use --nodeps twice or pacman will pull in all sorts of bloated X11 junk.  The dependencies for the other sdl libraries are ok though.  Same for alsa-utils.

   pacman -S sdl --nodeps --nodeps
   pacman -S sdl_ttf
   pacman -S sdl_image
   pacman -S sdl_gfx
   pacman -S alsa-utils

 I use mpg132 for the internet radio app in gmenu2x.  Openssh is handy for scp.  And fbgrab is handy for screenshots to prove you did it.

   pacman -S mpg123
   pacman -S openssh
   pacman -S fbgrab

Once you have all this stuff you need to setup your environment for SDL so it knows to use the framebuffer and alsa drivers.  You'll use some of the scripts from the archome tarball above.

   .    /root/sdlenv
   .    /root/alsaon

I probably want to set the init system to do this and the mac address on every boot up, but I'm not there yet.  This should be enough to run gmu, but I still need to create a wrapper script for gmenu2x in /usr/local/bin, and maybe plug it into the init system.

Don't forget to fix the top display settings.  Scroll to the bottom for instructions.

Out of Focus

Adding the SDL-rtf driving directions viewer to PSPMaps got me thinking once again about word procssing on the zipit.  Every so often I find myself wishing for something like Wordpad, to go just a wee bit outside the domain of the traditional text editor and wander into the territory the word processor.  This was one of those times.  The example viewer program included with the SDL_rtf sources was compact and simple, and solved what appeared to be half of the problem, drawing the formatted text onto the screen.  All that remained to be done was the editing functions, and some sort of user interface.

Once upon a time I got fairly close to the goal, with flwriter and nanox, but was derailed by a particulary nasty early winter snowstorm and never got back on track.  I dug up the old executable and took some screen shots of the user interface, which unfortunately requires a mouse, or the mouse emulator.  However, it really looks almost useable.  I really must finish that up someday.

Meanwhile, could I do something similar on SDL to add a simple editing interface to the SDL_rtf example?  The one requirement would be a small widget set in C using SDL for the backend.  It all seemed so simple.  But in fact it was not.

While C++ GUI widget toolkits are truely a dime a dozen, finding anything for plain vanilla C can be quite a chore.  Finding such a beast that runs on SDL is even more difficult.  I did find AGAR, but something about it just didn't fit my plans.  Perhaps it used cmake or one of those crazy build systems based on python.  Or maybe it pulled in some other huge library as a dependency.  I can't remember, so I'll have to get back to it someday and at the very least jot down what it was that turned me away.

With nowhere to turn I looked backwards, at an old blog post.  I once used a throwaway simple GUI sample program called fgui to prove out the native iz2s compiler on the zipit.  I pulled up the code from my fgui github fork, only to realize it was not actually the same fgui.  I eventually tracked down a copy of that one on an old SD card, although I can't find it anywhere on the internet.  But in the meantime I was looking at an entirely different fgui.  One with perhaps a bit more promise.

The github fgui had just 3 (or 4 if you count a label) basic widgets.  And it wasn't particularly pretty.  Plain old boxy widgets with a seriously ugly font.  But it was written well enough that I could understand it quickly, and certain features made it a good fit for the zipit.  For example, while it could potentially support a mouse, the widget focus traversal and activation is currently all keyboard driven.  I like that.  It also has no dynamic memory allocation calls and it's example test code made a very small executable, under 20K.  Excellent for the zipit.  Plus it doesn't take over the entire SDL program.  In fact it's only connection to SDL is a single drawpixel function and an fgui event handler hook somewhere in the main loop of the SDL program.  Nice modular design.  I like it.

Naturally I had to ruin it.  So I swapped the ugly bitmap font with an SDL_ttf font, breaking the pristine modularity somewhat, but not actually too much.  I may someday restore the ugly built-in font as a make/compile time option, or possibly a runtime fallback option, or both.  But first I'll have to dig up a better looking bitmap font...

fgui demo with ttf font and "fancy" buttons

The next thing I did was add a bit of texture to the flat boxy buttons.  Nothing too modern, but I can now imagine developing a GUI with it that I'm not embarrassed to show.  I also roughed in a checkbox widget, a list box widget, and pretended to make a menubar several times out of the few existing widgets.  I came to the conclusion that I need to add some widget traversal setup functions so I can complement the TAB key with the arrow keys (when appropriate) to change widget focus and get around on the screen.  I also need to modify the widget focus mechanism to allow for menu hotkeys, and to develop a way for parents to pass along focus to their child widgets.  Then maybe I can make a real menubar.

After which the RTF editor code should be a piece of cake.

Oh, wait.  Did I forget to mention my lack of focus?  At some point during the fgui development I got distracted with an sdl asteroids game that looked too good not to play with.  So I compiled a version for the zipit.  I'll try to post it here, when I get a chance.

I also went a bit off the rails and added a theme color option to the fgui widget code.  Here's an example running a green theme with a totally fake mockup menubar widget.

Ha,  fooled ya!  The menubar is truly fake, and I really did add theme colorizing functions in a moment of weakness (It was surprisingly easy due to the well thought out  fgui design).  But in reality, that's actually a screenshot of fgui running in Arch linux on the zipit.  Apparently the Arch fbgrab program only knows about green, or possibly I used it wrong.  But who cares.  The cool thing is I finally loaded the zipit Arch image onto an SD card, installed the Arch packages for gcc, make, sdl, and sdl_ttf  and was able to compile and run fgui right there on the zipit itself.  To keep things on the small side I installed the Arch SDL package without dependencies, and then used the SDL environment variables to select the built in fbcon driver and set the correct rotation for the zipit screen.  Arch runs a bit heavy for the zipit's paltry 32 megabytes of RAM, so every little savings helps.

So now what should I do?  Finish fgui and work on the SDL_rtf editor program, or play some more with Arch linux on the zipit?  I can't decide.

Ok, I guess I decided Arch Linux is more fun right now.  I compiled qemacs on the Arch zipit so I could be more productive, and got busy.  I'm considering installing X11 to see how well (poorly?) that behaves, so I built the zipit mouse emulator and tried it out in the old fgui SDL demo.  I also compiled gmu with mp3 support so I could play some internet radio.  Here it is in the simulated retro green theme imposed by fbgrab.  In real life it's much prettier.

I think next up I'm gonna have to build proper working version of fbgrab for Arch.

Back on the Map

Last time around I did some updates to libmikmod and the sdl-mixer library.  Considering my previous lack of testing on them, I decided it'd probably be best to be a wee bit more thorough this time.  So I tracked down some oldschool galaga style games and gave them a zipit makeover.  I'd really prefer to get a nice version of centipede, joust, or one of the many zelda clones on the zipit, but didn't see anything that struck me as easy, so space shooters was it this time.

I started with dodgin' diamond and all went well.  Even the two player mode seen above appears to work, although it's difficult to get 2 different hands on the zipit keyboard at the same time.   Next I decided to give the fancier looking powermanga game a try.  I thought I maybe powermanga would be a more challenging port, given the code size and some floating point bits, but it was pretty smooth right out of the box on the medium sound quality setting.  So I'll take that as a win and move on.

I still wanted to leverage my success with the fixed point supertux patch into something more substantial, so I decided to see if I could do anything with protracker.  I'd like a few more tracker options on the zipit, and my little piggy tracker port was stuck on a blank screen for some reason.  Meanwhile protracker looked almost feasible.  I did a quick and dirty screen hack to remove the pixel doubling and chopped last 15 rows of the display to fit it onto the zipit screen.  It actually looks fairly decent, but appears to require the fake mouse controls on the dpad buttons, and all sorts of internal mixer and filter code needs a conversion to fixed point if I ever want to hear more than an occasional squeek out of it.

Unfortunately, about halfway through the float to fixed point conversion, we hit the Christmas holiday season and the zipit irc channel came alive with chatter.  Someone mentioned the pspmaps location lookup appeared to be broken.  My attention deficit disorder took control and off I went into fixit mode.  It didn't take long to discover that google had shut off the V2 googlemaps API a while back.  The V3 options were JSON or XML.  No sign of the CSV and KML options from V2.  Oh well.  I'd been meaning to tinker with JSON again someday.  And now that day had arrived.  I checked out a few small JSON libraries for C, but I'd seen cJSON before in the opentyrian code, and it looked simple enough, so I went with that.  After a modicum of testing, I had a working fix for both the location and driving direction lookups, so I bumped the version number.

Someday I'd like to add an option to use the OpenStreetMap lookup as well.  That way there's a backup solution, just in case google decides to obsolete their API again.  Also, I suspect openstreetmap might make it easier to add support for more permanent offline map sets.

I noticed the original pspmaps site was gone, but there was a github project with a few odd fixes from just before the google API change.  So I forked it, got back up to speed with git (after maybe two years of rust), and rebased the openwrt-zipit pspmaps project off my github fork instead of the dead link to the past.  I still haven't restored my openwrt-zipit build system after the recent hard disk crash, so hopefully somebody will notice and rebuild the openwrt pspmaps package.

Here's the goodies:

dd2-iz2s.zip

powermanga-iz2s.zip

I still need to package up pspmaps.  For now here's a new executable and urls.txt file for pspmaps on openwrt.  Just untar it in the pspmaps directory.

pspmaps-update_2_4_wrt.tgz

Do the same for the iz2s executable.

pspmaps-update-2_4-iz2s.zip

I'll have to do another blog post if ever get one of those trackers working.

In the meantime I should probably say something about the Navit GPS/map program since it came up in the same IRC conversation as pspmaps.

By following the TomTom build recipe I got it to compile for iz2s, as a possible offline map alternative to pspmaps.  Apparently it wants to be in Germany?  However it crashed when I attempted to bring up the internal gui, so it clearly still needs some work...

Update:

Ok, I fixed the crash and found the maemo build recipe, which had some more clues on how to configure it better for the zipit, like converting the svg icons to png, and discouraging floating point math.

I still have a few more icons to track down and convert, but most of the menus look all right now.  Perhaps it's time to fetch some more map data, other than that sample patch in Germany.

So I visited the Navit Planet Extractor website and fetched me a rectangle of OpenStreetMap data.  The smallest chunk it wanted to give was around 80 megabytes, so I went with that.  It sorta worked.  I could search for towns and streets contained within the map data, but the map itself was a mess.  It was very slow to update and missing (or simply not displaying) features that I know should be there.  Perhaps the 32 megabytes of RAM on the zipit is not enough?  I've read some notes that suggest Navit is not the most efficient app in the world.  I'll have to play with it some more and see it there's any way to make it feel like a useful tool on the zipit.  Or maybe I'll just grab the 700MB prepackaged blob of north east corridor USA map data from the Planet Extractor and see if I can make the zipit pop...

Or maybe I should try to use something like gmapcatcher to fetch map tiles for pspmaps.  I might have to reorganize the pspmaps cache to be compatible with the directory structure though.  It looks like it might be close.

Update 2:

It actually was pretty close.  Well, not really.  The pspmaps cache is a simple list of N numbered png files.  The meaning of those files is stored in a separate disk.dat file as a list of x,y,z coordinates (and a number for the specific map service), one entry for each numbered file in the cache.   It's these entries that match the directory and file structure of the gmapcatcher data, exactly.  They also match up with the OpenStreetMap Slippy map tilenames convention.  That meant it required just a few new lines of code in tile.c to convince pspmaps to check the offline gmapcatcher data before resorting to a network lookup.

Here you can see where pspmaps used some offline tiles saved by gmapcatcher.  I used two different map servers so I could be sure.  On the bottom left is a tile fetched from google by pspmaps.  The rest of the image is made up of tiles pulled from the offline files saved by gmapcatcher.  I forget which map service it was using, but you can easily see the difference.

Anyhow, I'll eventually put some sort of Navit package for iz2s here, and another revision of pspmaps too.

Update 3:

I've run out of vacation time so I zipped up a Navit package for iz2s.

navit-iz2s.zip

I haven't yet tried any of the large pre-made data sets, so I don't really know if it's actually useful.  I'll get back to it eventually.

Meanwhile, I think pspmaps has a lot of untapped potential.  I put the code in my github repository and added my offline map code to work with the OSM_tile data from gmapcatcher.  I also fixed it to remember a preferred default map service, and tweaked the code to allow you to skip map services that don't interest you.  

I've got a few other small changes that I'd like to make.  The simplistic linear searching disk cache implementation could use some work to allow it to scale up.  Maybe hash table lookups, or at the very least least make it do a binary tree search.  I'd also like to add some simple versions of the gmapcatcher functions.  A small radius local area capture with a few autozoom levels would be nice.  And an OSM_tile format offline map export function would make it possible to use the zipit instead of gmapcatcher in a pinch.  And of course I still want OpenStreetMap location and driving directions lookup as an alternative to google.

Some wilder ideas include better driving directions, with access to the text version of the route, and maybe markers for the turns.  Point of Interest data might also be fun to play with.  Maybe OpenPOIs or the OpenStreetMap Overpass_API would work?  I could probably use some of the 16x16 png icons from Navit as the markers...

Hopefully I'll have something worthy of a release soon...

Ok, this has the offline stuff and allows you to comment out map services that you don't want in urls.txt.  See the readme.txt file for details.  Once again it's just an updated executable with urls.txt and a readme file, not a full package.

pspmaps-update-2_4-1-iz2s.zip

Same for the wrt update...

pspmaps-update-2_4_1-wrt.tgz

Update 4:

As life would have it, I actually got a chance to try out pspmaps in a real world situation. I had to visit a company near Boston to get me some nifty software training for work.  Christmas flew by and I still don't have a smart phone with a data plan, so I loaded up the offline cache in pspmaps on the zipit.  I also generated some driving directions.  It worked, but just barely.  The screen is just a wee bit small to see enough of the map while driving, the thin blue directions line was barely visible at night on the road, and I really could have used the text directions.

I can't do much about the screen size, so I widened the blue line in the code and got to work on parsing the text out of the JSON driving directions.

 Wider is better

I'm using the SDL_rtf sample app as a mockup for the display.  It works reasonably well for my purposes (and I still dream of a minimal word processor app on the zipit someday) so I think I might stick with it when I fold the code back into pspmaps.

While I'm at it, maybe I should try to hook up the flite speech synthesis from the bard package to read the text out loud.  At the very least I should try and feed this sample text to flite and see if the pronunciation is intelligible.  That should be an interesting exercise.

Update 5:

Ok, I haven't tried to hook up flite, but I did fold the SDL_rtf driving directions code back into pspmaps.  I really like the way it autowraps the text for me because I wasn't looking forward to writing that part myself.  I had a bit of confusion with latin1 text until I realized the rtf unicode escape sequence \uN is apparently not yet supported in sdl_rtf.  However testing reveals I can simply convert the directions to extended ASCII from the 8 bit latin1 codepage and use that, as seen here.  

 Better watch out for the speed cameras!

The last release of SDL_rtf is more than 10 years old, but it's had a few unreleased updates on the SDL mercurial repository.  I fetched the most recent update for SDL-1.2, from 2012, but didn't see anything that looked like unicode escape sequence support.  So for now I'm sticking with the SDL_rtf v0.1 code.  I added SDL_rtf to the list of openwrt-zipit packages but I don't have a compiler set up to build an actual package for it.  Because of that, I decided to static link SDL_rtf into some pspmaps executables.  

Here's another round of executable updates (and the urls.txt config files).

pspmaps-2_4_2-iz2s.zip

pspmaps-2_4_2-wrt.tgz

The Backspace key gets you to the text directions screen on the zipit.

Faking it, Part 2

Last week I had some pretty good luck with a quick and easy build of the wetspot2 game that I discovered on the gcw zero recent news page.  So I thought maybe I'd go for a twofer and cherry pick another easy target off that list.  I measured things up and decided the last-mission game also looked ripe for the picking.  Naturally, things weren't nearly so simple.

Remember last month when I confessed to pushing a fake build of libmikmod in order to fascilitate a quickie build of SDL_mixer, which I needed to get sdlbasic running?  Well, it turns out libmikmod wasn't the only joker in the pack.  Apparently I also skimped on the SDL_mixer build itself.  I vaguely remember that was somewhat of a challenge to compile because I wanted it to use libmpg123 instead of libsmpeg.  But that involved a bunch of finicky hand patching.  When I finally got that going I failed to check out the other build options.  Turns out the default ogg support is the floating point version, not the fixed point libtremor code that works on the zipit.  Oops.

Which brings us back around to the last-mission story.  I was getting nowhere.  It would load a blank black screen, and then appeared to hang.  So I added some debug printfs and discovered it was actually running behind the black screen, loading a bunch of ogg sound files at something shy of a snails pace.  If I waited long enough, just short of an eternity, it would actually bring up the splash screen and allow me to attempt a game.  Sorta, kinda.  Press a button, then wait a minute for the the floating point ogg code to process the associated squeek or beep sound.   Repeat.  Not much fun at all, really.

I checked the SDL_mixer readme file and sure enough there was an option for libtremor that I hadn't selected.  That might work better than the fake floating point support on the zipit.  So I did a rebuild on the library and now the game loads and plays at reasonable speed on the zipit.  As an added bonus, the new SDL_Mixer lib also fixed up playmus to work on ogg files.  I wonder what else I might have missed.  Say, doesn't supertux use ogg?  Perhaps I should take another look at that.

Meanwhile I've got a new SDL_mixer and last-mission package to put up here...

libSDL_mixer-iz2s-tremor.zip
last-mission-iz2s.zip

Update:

So, I fetched supertux and applied the 4 patches from the openwrt-zipit project before compiling.  The resulting executable took a while to load, but it ran and the music played ok.  However the controls were sluggish and the frame refresh rate was pretty darn awful.  Digging into the code revealed floating point math all over the place.  Oh well.  But wait, I don't give up that easy.  I ran some internet searches and found a fixed-point patch for a slightly older version of supertux that was done for the zaurus.  I also found some tantalizing hints that someone might have updated them to the newer code, but could't locate the goods.  The outdated patch was pretty large, but I gave it a shot and then attempted to manually merge the rejected parts.

As far as I can tell so far it seems to work.  The controls are smooth and the game is much more playable.

So now I just gotta pack it up and make an updated version of the patch...

supertux-iz2s.zip
supertux-0.1.3-fp.patch

Two Steps Back...

Seems like only yesterday I installed a brand new 500GB harddisk in my tired old laptop so I'd have plenty of room to work on the zipit stuff.  But apparently it was in there for almost exactly two years last week when it suddenly decided that was enough.  Looking back I suppose it wasn't all that sudden.  It started humming a few hours before it froze, but at the time I thought it was the CPU fan.  Oh well, live and learn.  Shoulda done more frequent backups.

The most recent backup of my iz2s development VM was over 6 months old, and the openwrt backup was even older.  Ouch.  So I dug up a much smaller, slower drive from my junk pile and put the latest LMDE linux on it to get back up and running before the long holiday weekend.  Turns out the 500GB drive wasn't completely dead yet.  I could run it for a few minutes as an external usb drive before it would seize up, just long enough to pull off the iz2s dev stuff.  Phew!  Openwrt was another story though.  I tried a few times, but never got more than half of it.  I gotta think about it before I try again so I don't brick the drive first.  Maybe I can use rsync to resume where the last attempt gave up?  Then I need to come up with a better backup strategy for the long term that plays well with my inherent laziness.  

Meanwhile I had to test the restored iz2s VM and make sure all was well.  I was already running out of holiday so I chose an easy target.  I saw the wetspot2 game mentioned in the recent releases section of the gcw zero forums and thought it might be fun on the zipit.  Since it was developed in C and SDL it looked like a good fit.  The game compiled ok right out of the box, but I eventually tweaked the makefile a teensy bit for the zipit.  Then I changed the 2 player keyboard settings to get a little more separation on the cramped iz2s keyboard, but that was it.

Single player mode uses the dpad arrows to move the crab and the control key (...) to push the blocks.  For two players the tab (Zipit) key is used to move the blocks for player 1.  Player  2 gets the S, Control, Z, and X keys to move the crab, and the space bar to push the blocks.  I actually tried this with one of the kids and it's not too bad.  The player with the arrow keys has a slight advantage though.  I know this because I had the other keys and lost.  It had to be the keys.

wetspot2-iz2s.zip

Oh yeah, I almost forgot.  While the laptop was out of commission I found some time to dig into the iz2s wifi config scripts.  Every so often they'd connect, but fail to get a dhcp lease.  Subsequent retries never seemed to work, and so I was always forced to reboot whenever it happened.  But this time it occurred during my laptop downtime, so I had no excuse not to investigate.  Nothing better to do...

It didn't actually take very long to see what was going on.  The dhcp request failed but the wpa_supplicant demon was left running, and that's what interfered with my retries.  So I patched the script to kill any running wpa_supplicant before starting a new one.  Now a retry usually succeeds.

With that fix in the bag I decided to tackle another nagging wifi issue.  I finally ported the iz2s Easy Wifi Configurator script to work on the jffs.  Sometimes I prefer the dialog driven EWoC script to connect because there's less typing, making it easier to use in a dimly lit location.   The main issue was working out a suitable substitute for awk to fetch the selected line out of the list of SSIDs in one of the dialogs.  A one, two punch of head followed by tail did the trick.  Shoulda thought of that a long time ago...

ewoc-iz2jffs.sh

Can't Fake the Funk

I'm a bit of a sucker for puzzle games like sokoban, especially when they play the oldschool chiptunes for background music.  So a long time ago when my kids went through school and they had some fancy TI graphing calculators, I thought maybe I'd eventually inherit one and hack it to play some silly puzzle games like blockdude.  But, kids being kids, they either lost or broke the darn things and I forgot about it.

Time passed, and then a year or so ago I'd just finished patching up a zipitized build of the Funktracker Gold music tracker (and mod file player) when I spotted a gp2x version of blockdude.  I tweaked the keyboard settings and compiled it for the zipit but couldn't quite get the bleeping sound to work.  My blog post at the time had the title "Game Sounds" so providing blockdude without sound was never gonna fly.  Worse yet, I could not even solve the first level.  Embarrassing!

So I moved on and built some silent console games instead.  I don't think I've passed the first or second level on any of those either (I sense a trend here...) but at least everything seems to work.

Recently I was thinking about fixed point math again, which led me to discover the ancient gp2x version of SDL includes a really nice fixed point math module, with arm assembler code.  Nice.  That in turn led me to some opengl implementations which also incorporate the fixed point module.  I absolutely must explore that some day, but meanwhile it got me thinking again about chiptunes, mod trackers, and eventually my problems with blockdude.

This time I realized the first sound blockdude plays is from a mod file with the title soundtrack.  I remembered that I'd done a quick and dirty build of libmikmod, way, way back when I was trying to compile sdlbasic.  At the time, I only needed libmikmod because it was required to build the SDL_mixer library.  But I didn't have any mod files handy to test it, so I simply used the default config to fake it, and then promptly forgot about it.

So now I did some testing with blockdude and the SDL_mixer playmus utility and discovered that not only did my build of libmikmod refuse to make a peep, it also locked up the sound driver so that nothing else could use it either.  Not good.  So I fetched the latest libmikmod-3.1.18 bugfix release code and applied the config settings from the zipit openwrt makefile.  With the new libmikmod installed, playmus and blockdude could crank out the cheezy chiptunes in all their funky low-fi glory on my zipit.  I'm still stuck in level 1 though...

Here's the goodies.

libmikmod-3.1.18-iz2s.zip  (also includes the playmus utility from SDL_mixer)

blockdude-iz2s.zip

Update:

Made it to level 2.  I feel much smarter now.