A Little Less Failure This Time?

This week started out much like the last.  More adventures in failure.  The guys on Zipit IRC seemed to be struggling with an improperly rotated links -g running on DirectFB in openwrt.  I pointed out Ray Haque's secret sauce for that problem, but failed to get it done.  So I got the bright idea to try and build it myself, with SDL and all my favorite patches from the IZ2S world.  Sound familiar?  Yeah, and it started out exactly the same way.  One failure after another. 

To start with, I had an old failed rockbox for openwrt that I never quite got working right.  It runs, but horribly, and eventually I lost interest and moved on.  Shoulda took that as a warning.  Stay away.  But no, I fired up the old P3 with my openwrt setup on it and updated all the git archives following Anarsoul's instructions.  At least I thought that's what I was doing.  Apparently the browser still had a cached version of the instructions from last February when I was working on the rockbox build.  And the cached instructions contained a typo.  A dash character in one of the feed names prevented me from building the qi repositories with the links package that I wanted.  For several hours!  The guys on IRC were quite helpful, and we eventually worked out the problem only to discover the instructions on the internet were already fixed, and probably had been for months.

See.  There's the typo, right there.  Can you spot the difference?

Oh well, moving on.  So I started the P3 building the goodies and watched for a few minutes before... guess what?  Hard disk failure!  Aargh!  Stupid old 80GB external drive developed a bad superblock.  I sorta remembered this drive gave me trouble before so I decided to abandon it and get a newer one rather than try to fix it.  Someday maybe I'll try to recover the bad rockbox build with one of the backup superblocks.

The next day I nabbed a 500GB drive from one of the kids and started over.  I refetched all the openwrt archives, followed the correct instructions to setup the feeds and started the build again.  Then I went to sleep, exhausted but hopeful.  The build failed a few times on supertux, the kbd package, and libmpg123 so I had a few restarts to do the next day.  But eventually it built everything else I'd selected.  Gotta see what's up with kbd and libmpg123 since I may want them someday.  But for now I'm gonna build with IGNORE_ERRORS=m to keep it going after failures.

Anyhow, the links package compiled so I just needed to apply my patch from IZ2S and rebuild.  Or so I thought.  I tested the patch in the directory with the links sources and it applied cleanly.  Cool.  However when openwrt compiles, it wants to wipe out the directory with the package sources, start fresh, and apply it's own patches.  But it took a while to figure out how to remake my patch the way openwrt wants it.  You need to make it run with patch -p0 so the destination directory name in the patch must match exactly with the directory name in the package tarball used by openwrt.  At least I think that's the case.  It's hard to see exactly what the byzantine openwrt build system is doing, even with the V=99 command line option.

I got some more hints from the IRC guys on how to reproduce the openwrt build environment so I could manually step through the patching and configuring and try and fix the broken parts.  I don't think I ever completely reproduced the environment but got close enough to figure out some of the trouble spots.  Apparently for openwrt you need to make a makefile for each package that runs configure to create a makefile for the package.  Hmmm.  I examined the existing meta-makefile and attempted to convert the tricks used to configure directfb into tricks that would configure links for SDL.  It didn't exactly work.  The openwrt build process was successfully applying my patch, but failing during the configure stage because it couldn't find SDL.  I tried looking at some other packages like nupdf and sdlwidgets for SDL config hints, but to no avail.  As many things do, the answer crystalized in my sleep, and when I woke up I found the sdl-config program I needed in the openwrt-zipit/staging_dir/target-arm_v5te_uClibc-0.9.33_eabi/usr/bin directory.  I fixed up the meta-makefile with that path.  Now the configure stage managed to find SDL and moved on... only to fail writing the new makefile.  Syntax error in the patched configure script!  Did I already say arrgh!?

Now, the original SDL patch from the internet required me to rerun autoconf to create a shiny new configure script (and Makefile.in) with SDL options.  I did that for my IZ2S build, but I have no idea how to do that in the openwrt environment.  Also, different versions of autoconf tend to generate a completely different output files.  And configure is over 600MB!  So I'd attempted to make a small patch for the configure script that would paste in only the changes involving SDL and skip all the gratuitous changes caused by the different version of autoconf. That got me all the way to the syntax error.  I took another look at my patch, and the modified configure script, but that's not work meant for humans.  Hand editing 600MB of butt ugly machine generated script file is not my cup of tea.  Even the mighty emacs with it's color syntax highlighting and electric parenthesis matching wasn't helping me deal with the horrendous tab formatting.  So instead I gave up and changed the openwrt meta-makefile to replace the whole 600MB configure script with the one I'd regenerated earlier with a different autoconf on a different PC.  Oh well.  It's kludgy, but it works.   

Not Fail!  Yeah!

The new openwrt Makefile, patches, some zipit friendly config files, and key docs are on github.

I put up the links-sdl_2.3pre1_pxa.ipk package for openwrt zipits, with config files and a kbd cheatsheet.

Here's my wrt-env.sh script (modified from slugs).

Update:   Anarsoul added PKG_FIXUP:=autoreconf to the Makefile to auto-magically regenerate the correct configure script as part of the openwrt build process.  And projectgus pulled it all into the master github project, so you should be able to get the links package from the usual place.  Cool!

Meanwhile, here's a qemacs executable for openwrt.  I still need to learn how make it into a real openwrt-zipit package...

I Don't Know What I Was Thinking.

Boy, that'll teach me to make promises on Friday the 13th, just before taxes are due.  The taxes got done, but not much else.  And now I'm really having trouble remembering where I stashed all the zipit goodies I promised to make available.  I swear I'll get them posted, real soon now...

In the meantime, whilst avoiding packaging up the old goodies, I somehow found myself on an old Nintendo DS homebrew site, which reminded me of the spiffy Bunjalloo web browser.  It runs quite well (with graphics even) in the limited 4MB and small screen real estate of the NDS.  Plus there's some notes about building it for linux with SDL, and it uses the tiny matrixssl library for shttp connections.  I've been looking for an excuse to use that one, but I'm far too lazy to graft it onto one of the other zipit browsers.  So off I went, merrily avoiding what I should be doing, and attempting to port bunjalloo instead.  Turns out it uses some (python based?) WAF build system that I've never heard of, and a bunch of stuff from the NDS devkitarm toolchain.  Well, python and WAF were quite unhappy with my scratchbox build environment, so I quickly ditched them and moved to script based compiling.  I had to resort to Windows to use the NDS oriented grit tool to convert some png files to .c and .h source code because I'm too lazy to pull down all it's requirements and build a linux version.

Then, just as I was reaching 100% of the sources compiled I ran up against a wall.  Oops, the linux build requires opengl.  Apparently some 2D opengl calls match up well against the native NDS hardware, so an opengl layer makes sense for portability.  But not really for my purposes.  Opengl is not so great for the zipit, with it's conspicuous lack of floating point hardware.  However on closer inspection, it looked like most of the opengl calls were using the integer versions of the functions.  Hmm, that got me thinking.  Apparently I still have a few brain cells left over from olden times, when DSLinux wasn't quite dead yet.  Just before the DSI came out and flatlined the last remaining DSLinux effort, I managed to sneak in a really lame port of the opengl version of the classic Elite space game.  I dug through the archives and sure enough, there was the integer only tiny-SDL-OpenGL lib I used for the port.  I think it needs a few tweaks to make it suitable for bunjalloo, and then there's the whole business of reconfiguring the screen and keyboard code, but the wall has been breached. 

Hopefully bunjalloo doesn't animate or use 100% of the CPU in a busy loop, because the opengl example programs are not exactly top performers.  Here's the well known gears program grinding out a whopping 2 frames per second on the zipit.

The shiny spinning triangle isn't a whole lot snappier.

Hopefully I can continue to restrain myself and not try to port any of my old opengl based CAD programs to the zipit.

 Although this does look almost useable in 320x240 pixels...

Must, resist, temptation...

For your  amusement, here's a zip with the tinygl sources.  The gears and triangle executables are also included in the examples directory. 
tinygl-iz2s.zip

And here's what bunjalloo currently looks like running in the debugger on ubuntu.

As you can see, I have plenty of work to do on the screen layout before I can even attempt to run it on the zipit.

Update:  I dug through the code, looking for all numbers near 256 and 192 (the NDS screen dimensions), and replaced all the 192s with 120 to use a half zipit screen instead of one of the NDS screens.  It sorta works but the scrollbar looks weird constrained to the bottom half of the display.  So I think I really want to make it use only one 240 pixel high screen instead.

I was able to confirm that I can use bunjalloo to access my gmail account (cool!), however hotmail seems to really want javascript.

The Long Way Around the Top

I'm not exactly satisfied with the ebook contents page produced by zpub, so I thought maybe I could make zpub build the html using the same algorithm as the original script.  But that uses the bc utility to assemble it's javascript and xml contents, and bc isn't included with IZ2S, so I figured I'd build it and see if it helped. 

At around the same time, I somehow got to wandering around on the nanonote wiki where I happened upon a shiny picture of htop running on the nanonote.  Ooh, shiny... Had to have it!  So I compiled it and tweaked the screen and keys a bit to better fit the Z2.  I love the way it lets you scroll to the right to see the entire command line.  Way better than the usual run-of-the-mill top utility!

Also around the same time I got to thinking about how I still really wanted a C compiler for IZ2S.  What's wrong with me?  That's just stupid.  But sometimes the only way to get these crazy ideas out of my head is to just go ahead and do it.  So I did.  Well... Ok, it's not really a compiler per se, but the PicoC interpreter is probably a better fit for IZ2S anyhow.  Since I had some previous trouble with floating point math support on the zipit I built two versions: one without floating point support, and one with it.

Unfortunately, before I got to test it all that much, I stumbled upon an SDL patch for the links browser.  What?  How did I miss that before?  With this patch I should be able to make a version of links for IZ2S that works properly, with or without the mouse, and without requiring the messed up DirectFB libs.  If it works, I can finally toss out the broken IZ2S DirectFB libs and remake them correctly from scratch whenever I get back to working on fltk, xdialog, vnc, or any of my backburner projects that need a normal working version of DirectFB.

Anyhow, I tweaked the links-sdl patch a bit for the IZ2S screen size, and for the current version of links.  I also added link traversal via TAB and BackTAB, and horizontal scrolling with the <> keys to make it more workable with zipit keyboard and the fake "mouse".  Initial testing feels good.  Gotta get it finished soon so I can patch bc back into the zpub script and see if it helps before something else pops into my head...

See?  The "mouse" actually works.

Look in the Zipit Z2 Goodie Bag for links to all the stuff.

Basically Nothing to See Here

I finally made some progress on the software backlog.  I built the r3.9 rockbox for z2lite, with a complete set of plugins that should hopefully also work on IZ2S.  But I have yet to test it and package it all up.  Oh well.  Maybe this weekend.

Meanwhile, this tiny basic interpreter caught my eye.  Yeah I know, there are plenty of basic interpreters out there, some with the same name even.  SmallBASIC even looks useful with it's FLTK widgets and all, but I have a weakness for shiny things and a colorful screenshot pulled me in.  So I patched it for the zipit keyboard and applied this patch (except for the font doubler).  It seems to work ok with the example files.

SDL_basic-1.0.2-iz2s.zip (executable and source code)

VNCx3

So it turns out there are at least three framebuffer vnc clients out there that might be useful in the IZ2S userland on the zipit.   There's the old ipaq/zaurus fbvnc that looks like a good fit for the tiny zipit screen.

http://pocketworkstation.org/fbvnc.html

There's a nice simple fbvnc here along with some other minimal fb programs.

http://litcave.berlios.de/

And finally there is directvnc which runs on top of DirectFB instead of directly on /dev/fb0.

http://drinkmilk.github.com/directvnc/

Naturally none of them really fit my needs.  The ipaq/zaurus fbvnc looks the most promising because it's got scaling, panning, and rotating already built in for use with small screens.  But the code is old and needs updating to make the keyboard and mouse work.  Also the nifty screen rotation goes the wrong way, so I can have the screen sideways, or upside down relative to the keyboard.  I borrowed the mouse code from the litcave fbvnc so at least that works now.  Panning also works.  I hooked in the battery monitoring code from rockbox, connected the backlight dimmer to some of the many variations of kbd led controls in IZ2S, and attempted to hook the volume control to alsamixer.  So if you don't mind working sideways with an onscreen keyboard instead of the real one, then you might be ok with this fbvnc as is. 

I suspect the keyboard will only take a small fix like the mouse, but the rotation is gonna be a challenge.  The rotation code is a bit of a mess with multiple unnecessary conversions back and forth between the local and remote coordinate systems, and massive hand unrolled loops to optimize the code at the point where it actually needs to draw the pixels.

The litcave fbvnc is no frills, and either uclibc doesn't like some socket call it makes, or else it's lack of password support is blocking me.  Either way, it's probably only useful to me as a source for the simpler mouse/keyboard code.

Directvnc connects and puts up a screen, rotated correctly by leveraging my previous work with DirectFB and SDL, but the mouse fails to move.  And the only keys I've verified are the Ctrl-Q key combo to quit.  I can probably borrow some working mouse code but once again the lack of panning or scaling makes this fairly useless on the zipit.

What I really need is the keyboard/mouse code from the litcave fbnc, the correctly rotated DirectFB screen from directvnc, and the rest of the goodies from the ipaq/zaurus fbvnc.  I may just end up with that eventually.

Layers upon Layers.

One of my goals for the zipit is to get gtk running on Directfb so I can have all sorts of goodies built outta gtk; like fbreader, some OpenStreetmap stuff, and even xdialog.  The Directfb folks have a recipe with about 100 steps to do this, but I'm not so good at following directions, so I haven't got it done yet.  I did manage to compile all the various bits and pieces, supposedly in the right order too.  But all the gtk programs I've tried (even the tiniest of demos) crash when I try to run 'em.  Prolly something's not configured right in one of the many layers leading up to gtk.  So I decided to start testing things from the bottom layer up.  

The first new layer in the gtk food-chain is Directfb.  Now iz2s comes with a Directfb of sorts.  It provides the g for the graphical links browser.  But it has a few warts, sorta like the iz2s alsa libs.  Once again the libs are compiled to look for things in a weird, nonexistent path.  The iz2s glinks works around this by explicitly linking with the fbdev and keyboard modules in their unusual location instead of allowing the main Directfb lib to load them on demand from the normal places.  That's fine for glinks, but it interferes with my plans to build more stuff on directfb.  Now apparently each consecutive version of iz2s has a different build of glinks.  And fortunately it turns out one of 'em was compiled with the normal lib setup.  But it had issues with the keyboard.  Oh well.  Also, I discovered the hack used to rotate the glinks display is really just a window rotate, and not a screen rotate, so the mouse does not reorient itself as well to match the display.  Once again, fine for glinks running mouseless, but not so good for me and my nefarious plans.

Then it hit me.  I've already spent a considerable amount of  time patching up SDL to solve various screen, keyboard, and sound issues.  And Directfb can be built to run layered on top of SDL in addition to running directly on fbdev.  Maybe I can fix both layers with one set of patches.  So I rebuilt Directfb with both backends enabled and tried it.  The SDL backend fixed the keyboard issue, the mouse rotation problem, and even the blinking cursor bug.  Whoo Hoo!  It doesn't support double buffer mode, but neither does the the fbdev backend, unless you run it without the 90 degree CCW rotation required to match the screen orientation with the keyboard.

I'll probably have to patch SDL yet again to add the rotated double buffering.  The SDL screen rotation code uses what they call a "shadow" buffer to hold the pre-rotated pixels.  So it's already double buffered.  The only difference is the timing of the updates.  The shadow implementation sneaks in the updates when you're not looking, whereas regular double buffering does it on page flips.  I honestly don't know how the SDL folks missed that.

Meanwhile I think I might just try to build fbvnc so I got something useful to show for all this work, not just a bunch of demos and an irrelevant mouse pointer for glinks.  And maybe I'll even do a little sidetracking and build FLTK_Directfb and then murga-lua, just in case I never make all the way up past gtk to the xdialog layer.  Otherwise, next up is the glib layer.  I gotta find me an interesting program that'll put glib through its paces without requiring a full up gtk install...

Blinker-be-Gone

Ok, so that pesky blinking text cursor interfering with all the SDL programs on IZ2S was starting to drive me nuts.  I know, there's a workaround, but who wants to make yet another wrapper script for every little SDL program I might happen to churn out.  But you know what?  I already made a new SDL lib for the ALSA sound support, so why not also patch it to get rid of that pesky blinking cursor.  Turns out the SDL_fbevents.c file already was already opening /dev/tty0 to access the keyboard, so I could also use that file descriptor to write the "\x1b[?25l" hide cursor escape sequence from the wrapper scripts whenever SDL switches into graphics mode.  One simple line of code and the nasty blinker is banished forever!  About time.

So I updated the SDL lib in here:  sdl-salsa-iz2s.zip


Still gotta replace it in all the other zip packages though...

Meanwhile I have a patch for directfp that might work, but only glinks uses directfb so far, and it still needs a wrapper script to load a different keymap.  So I'm gonna hold off on that patch.

I'm also sticking with a script for mplayer since it's a big static executable and apparently it works directly in the current tty rather than opening a free tty for the graphics.  So it needs a slightly different wrapper.

Here's what it looks like all cleaned up with the no-blink wrapper script.  You'll have to fetch the actual mplayer-rc2 executable from IZ2S 2.04.  This is just the wrapper and a config file that tells it to print as little as possible.

mplayer-iz2s.zip

A new day brings some new stuff for an old IZ2S.

Remember that bit about the double trouble?  Well, forget it.  Sometimes I have no idea what I'm talking about. 

I still don't really know what prevents me from puting a working libasound together.  It could simply be one of the zillions of ALSA config files is busted for all I know.  However I was successful with the salsa embedded ALSA lib.  I managed to build it and actually got some test programs from a beat up old Linux Journal to work.  I played with these for a bit, did some reading about "safe ALSA", and compared the way SDL wanted to use ALSA to the working mpg123 and aplay executables.  It turns out the simplistic buffering strategy used by SDL is probably ok for games, but not so good for gapless music playback on a stressed out low MHz linux box (like the zipit running IZ2S).  So I mercilessly hacked the SDL_alsa_audio.c file to make it behave more like the other programs.  Then I built a new libSDL with my changes, and with the tiny salsa library built right in. 

If you like, you can replace the old libSDL in IZ2S with sdl-salsa-iz2s.zip.  Then set SDL_AUDIODRIVER=alsa instead of dsp and go.  I recycled some rather under-utilized SDL_ALSA environment variables, just in case you want to play around and adjust the buffering scheme.  These are (lightly) documented in a text file included with the lib.

I also put together a real gmu-0.7.2-iz2s.zip package with the new libSDL since it no longer skips on IZ2S.  I must say GMU is really handy for debugging the audio code.  It's well written and easy to follow, unlike the gigantipus that is rockbox. 

I happen to like rockbox though.  So I rebuilt the IZ2S version with a more recent branch of code.  It has better (but not quit perfectly smooth) battery monitoring.  It can now change codecs on the fly and it no longer needs to move config files in and out of the /tmp directory.  It also seems to quit after 10 minutes on pause.  Perhaps that might help with the IZ2S power management?  I also let it use the standard linux file structure which prefers to install in the /usr/local directories instead of the embedded setup using the /.rockbox directory.  I'm not sure how much I like that, but for now it lets me keep the old and new versions installed at the same time for testing.  Here it is.  rockbox-iz2s-t11.zip

Finally, here's an ebindkeys executable for IZ2S with a setup to make the home key launch rockbox.  That's a single easy to find button so I can start it up in the car at night without turning on all the lights.  And here's another tip for the car.  Add a line to the startup script to soft link your music directory to /audio so it's right at the top of the rockbox file browser for quick and easy access.


Update:  It's a bit trickier to install rockbox themes on the new release.  You'll need to CD to the share directory on your SD card and temporarily rename the rockbox directory in there to .rockbox.  Then you can unzip an Ipod Video theme into the .rockbox directory.  Rename it to back to rockbox and you're done.

Also, unlike rockbox GMU only allows one key binding per function.  With the wide variation in default key mappings among IZ2S versions, this means that sometimes the volume is controlled via the +/- keys on the side of the zipit, and sometimes you'll have to use the prev/next keys.   So if you're using IZ2S 2.04 with it's default keymap and you want the volume controls on the side then you should edit the gmuinput.z2.conf file and near the bottom replace this:

Button73=280,Volume+
Button-74=281,Volume-

with this:

Button73=270,Volume+                                                                         Button74=269,Volume-

                                                                                                     
I suppose I could try to patch the code to work around this limitation.  Or maybe distribute several different GMU config files with the package.  I'll make a decision someday...