A Little Less Failure This Time?

This week started out much like the last.  More adventures in failure.  The guys on Zipit IRC seemed to be struggling with an improperly rotated links -g running on DirectFB in openwrt.  I pointed out Ray Haque's secret sauce for that problem, but failed to get it done.  So I got the bright idea to try and build it myself, with SDL and all my favorite patches from the IZ2S world.  Sound familiar?  Yeah, and it started out exactly the same way.  One failure after another. 

To start with, I had an old failed rockbox for openwrt that I never quite got working right.  It runs, but horribly, and eventually I lost interest and moved on.  Shoulda took that as a warning.  Stay away.  But no, I fired up the old P3 with my openwrt setup on it and updated all the git archives following Anarsoul's instructions.  At least I thought that's what I was doing.  Apparently the browser still had a cached version of the instructions from last February when I was working on the rockbox build.  And the cached instructions contained a typo.  A dash character in one of the feed names prevented me from building the qi repositories with the links package that I wanted.  For several hours!  The guys on IRC were quite helpful, and we eventually worked out the problem only to discover the instructions on the internet were already fixed, and probably had been for months.

See.  There's the typo, right there.  Can you spot the difference?

Oh well, moving on.  So I started the P3 building the goodies and watched for a few minutes before... guess what?  Hard disk failure!  Aargh!  Stupid old 80GB external drive developed a bad superblock.  I sorta remembered this drive gave me trouble before so I decided to abandon it and get a newer one rather than try to fix it.  Someday maybe I'll try to recover the bad rockbox build with one of the backup superblocks.

The next day I nabbed a 500GB drive from one of the kids and started over.  I refetched all the openwrt archives, followed the correct instructions to setup the feeds and started the build again.  Then I went to sleep, exhausted but hopeful.  The build failed a few times on supertux, the kbd package, and libmpg123 so I had a few restarts to do the next day.  But eventually it built everything else I'd selected.  Gotta see what's up with kbd and libmpg123 since I may want them someday.  But for now I'm gonna build with IGNORE_ERRORS=m to keep it going after failures.

Anyhow, the links package compiled so I just needed to apply my patch from IZ2S and rebuild.  Or so I thought.  I tested the patch in the directory with the links sources and it applied cleanly.  Cool.  However when openwrt compiles, it wants to wipe out the directory with the package sources, start fresh, and apply it's own patches.  But it took a while to figure out how to remake my patch the way openwrt wants it.  You need to make it run with patch -p0 so the destination directory name in the patch must match exactly with the directory name in the package tarball used by openwrt.  At least I think that's the case.  It's hard to see exactly what the byzantine openwrt build system is doing, even with the V=99 command line option.

I got some more hints from the IRC guys on how to reproduce the openwrt build environment so I could manually step through the patching and configuring and try and fix the broken parts.  I don't think I ever completely reproduced the environment but got close enough to figure out some of the trouble spots.  Apparently for openwrt you need to make a makefile for each package that runs configure to create a makefile for the package.  Hmmm.  I examined the existing meta-makefile and attempted to convert the tricks used to configure directfb into tricks that would configure links for SDL.  It didn't exactly work.  The openwrt build process was successfully applying my patch, but failing during the configure stage because it couldn't find SDL.  I tried looking at some other packages like nupdf and sdlwidgets for SDL config hints, but to no avail.  As many things do, the answer crystalized in my sleep, and when I woke up I found the sdl-config program I needed in the openwrt-zipit/staging_dir/target-arm_v5te_uClibc-0.9.33_eabi/usr/bin directory.  I fixed up the meta-makefile with that path.  Now the configure stage managed to find SDL and moved on... only to fail writing the new makefile.  Syntax error in the patched configure script!  Did I already say arrgh!?

Now, the original SDL patch from the internet required me to rerun autoconf to create a shiny new configure script (and Makefile.in) with SDL options.  I did that for my IZ2S build, but I have no idea how to do that in the openwrt environment.  Also, different versions of autoconf tend to generate a completely different output files.  And configure is over 600MB!  So I'd attempted to make a small patch for the configure script that would paste in only the changes involving SDL and skip all the gratuitous changes caused by the different version of autoconf. That got me all the way to the syntax error.  I took another look at my patch, and the modified configure script, but that's not work meant for humans.  Hand editing 600MB of butt ugly machine generated script file is not my cup of tea.  Even the mighty emacs with it's color syntax highlighting and electric parenthesis matching wasn't helping me deal with the horrendous tab formatting.  So instead I gave up and changed the openwrt meta-makefile to replace the whole 600MB configure script with the one I'd regenerated earlier with a different autoconf on a different PC.  Oh well.  It's kludgy, but it works.   

Not Fail!  Yeah!

The new openwrt Makefile, patches, some zipit friendly config files, and key docs are on github.

I put up the links-sdl_2.3pre1_pxa.ipk package for openwrt zipits, with config files and a kbd cheatsheet.

Here's my wrt-env.sh script (modified from slugs).

Update:   Anarsoul added PKG_FIXUP:=autoreconf to the Makefile to auto-magically regenerate the correct configure script as part of the openwrt build process.  And projectgus pulled it all into the master github project, so you should be able to get the links package from the usual place.  Cool!

Meanwhile, here's a qemacs executable for openwrt.  I still need to learn how make it into a real openwrt-zipit package...

Memorial Day

I really like the English version of the latest jffs dronz put together!  It's got just about everything I want on the zipit.  But I did make a few changes over the Memorial Day holiday.

First I fixed a tinyirc bug so it no longer wraps the status bar when using the large 8x14 font.  I tend to use a smaller font so I didn't notice the bug right away.  Sorry.

Then I went to work, making it just a wee bit more user friendly, at least for me.  I'm starting to appreciate gmenu2x as a program launcher, so I wanted that available right away on boot up, with no extra work.  At first I tried to get it going from the Zcovery startup script, using the toybox nohup to work around a problem with the hangup signal.  (Just like we did in olden days with the dialup modems).  But nohup wants to redirect stdout, which is troublesome for text terminal programs launched from gmenu2x.  So instead I simply fixed the bin/gmenu script to trap sighup just like on slug's jffs.  No more hangups means I can launch from the Zcovery startup script.  Yay!

With that problem solved, I modified Zcovery to first start a shell prompt in the background on tty1, then switch to tty2 to run gmenu2x.  Works like a charm.  But I'm pretty old school, so I want to get at that command prompt on tty1 at least half of the time.  To make it easy to switch between gmenu2x and tty1 I modified the three keymap files so ctrl-alt-home always goes to the command line shell on tty1.  I'm not actually sure I got this right in the calculator keymap, but I may try to eliminate the calculator keymap someday by remapping the keys inside the script like I did with zcalc.  Then I'd only need to maintain two keymaps.  Anyhow, ctrl-alt-home should work from gmenu2x or any SDL or text program launched by gmenu2x.  Then alt-rightarrow will get you to the gmenu2x text screen on tty2 and another alt-rightarrow will get you to it's graphics screen on tty3.  Sometimes the gmenu2x graphics remain on the screen after ctrl-alt-home, but you can clear that by typing cls or by pressing alt-rightarrow then alt-leftarrow to go to the gmenu text screen on tty2 and back to the command shell on tty1.  In this setup, SDL programs tend to run the graphics on tty3, leaving the text screen on tty2 in limbo.  With gmu and links you can type ctrl-z and bg on tty2 to free up the shell, however this is not advisable when running things from gmenu2x as it wants to manage both tty2 and tty3 for it's program launching scheme.

Once things were working satisfactorily, I decided to Americanize a few settings.  I changed the timezone file and switched the time format in the clock and alarm scripts to 12 hour time with no seconds because that's what I use here.  Since 99.9% of the time I only read things in English on the internet, I decided to compile an ASCII only version of links with SSL to save 200k on the jffs.  I'm seriously considering using the much smaller SSL-free ASCII only version to save even more.  I can always keep a full version of links on the SD card.  I also removed the green gmenu2x background.  I'd prefer to simply replace the wallpaper file if I ever get tired of the blue one.  That saves a few more K.

I used all that saved space to add the rockbox executable and a single codec to the jffs.  The rest of the codecs are softlinks to the jffs addon pack (or the normal IZ2S rockbox package) on the SD card.  Yeah, I know... rockbox doesn't really make any sense on the jffs because you still need an SD card for the music files, but I'm smitten by the coolness factor of it all.  Actually, now that I'm hooked on multiple ttys I may need to build a new rockbox executable because the current one doesn't realize it's supposed to give up drawing things when I switch to tty1.

 

Stock zipit boots into gmenu2x and plays the zipit sound in rockbox with no SD card.

There's still more Americanizing to do.  The gmenu2x glinks app file sends the browser to the french portal by default.  I'm OK with that for now, because it points to lots of handy zipit friendly internet sites.  I also need to switch a few other things like the gmenu2x settings tab to english.  But I think I'd like to get a build out there now because I like the new boot sequence so much.

In the future I still want to try swapping MatrixSSL into links instead of OpenSSL.  And I'd like to switch to slug's gmenu2x sources, then hookup the wifi icon and paste in the IZ2S battery monitoring code that I used for rockbox.  Both of these efforts should save considerable jffs space.  I also want to add the new set timezone script to the initial boot sequence.  Then maybe I'll get back to bunjallo for a browser choice somewhere between links and retawq in size and functionality.  Or maybe I'll see if it's possible to dim the status LEDs (like the backlights) or just turn them completely off when the alarm clock is running.  They're way too bright for night time.

I also need to figure out a better way to organize the goodie bag.  Dronz has been busy adding everything, which is great, but now there's so much in there I can't remember what it all is.  I'm not sure what would help.  Short descriptions?  Dates?  Pointers back to the original blog posts?  Gotta think about it...


Here's the Americanized jffs.  iz2jffs-v4-en.tgz
The old install instructions should suffice.

Here's the new keymap files with src, plus the new gmenu, tinyirc, and Zcovery.  That should make it easy to update the French version.  gmenu-jffs-fixes.zip

I forgot to include the tinyirc source code with the fixes the so I'll pastebin tinyirc.c

Latin Lessons (and Other Things)

Wow, it's been over a month since I did any sort of real blog post.  This must be what writers block feels like.  I've been off tinkering with so many different things that it's difficult to put enough coherent thoughts together for a decent post.  After I put up my previous "Where's the Beef?" post, I went back and edited the dead links in all my old posts to redirect them to the "Where's the Beef?" message.  It's quite a relief to be done with that.  But that's just clean up.  Nothing new.

So why am I so scatterbrained?  Well, I'm currently lugging 5 zipits around, all with different stuff on the internal flash.  In addition, I've got a pocket full of micro/mini SD cards with various versions of IZ2S, Flashstock, openwrt, etc.  And the pocket is really rough on the labels I've made for them.  They keep fading or falling off, so I can no longer remember what's on what card.  At least some of the zipit's are still labelled, and I'm testing some of the official zipit skin stickers to see if they're more resiliant, or easier to remember.

Zipit 1.  I finally loaded Mozzwalds openwrt rescue onto a virgin zipit, as a prelude to loading slug's shiny looking gmenu/gmu rescue image.  But now this particular zipit no longer likes any of my SD cards, so I'm back to being disappointed with uboot again.  I could probably leave it at home in a drawer somewhere, but I keep hoping I'll come up with a brilliant plan to restore the blob bootloader and the stock kernel via the wifi.

Zipit 2.  After I gave up on zipit 1, I decided to update a zipit from Mozzwalds initramfs uboot rescue image to slugs gmenu/gmu image.  The initramfs was functional, but limited due to the high memory use of the filesystem in RAM.  This zipit seems to have less trouble with the SD cards, even though it has the exact same uboot image.  Weird.  Anyhow, aside from my general misgivings about uboot, this one looks quite promising.  However testing revealed a problem where some component of gmu (sdl, alsa, pthreads, nanosleep, or whatever) is causing it to use 70% of the CPU instead of the expected 17% or so.  I did a test build of rockbox for openwrt a while back which also performed poorly.  I wonder if it had the same problem...

Zipit 3.  I've been updating my gmenu/gmu image for the blob bootloader and the stock kernel.  It's not as pretty as slugs, but it's still fun to work on.  I'm currently fiddling with the wifi setup scripts in an effort to add options for multiple profiles.  I've also started poking through slugs gmenu2x updates to see how hard it would be to backport the wifi setup dialogs and other changes.  I've also done some work on running a 2nd tty so I can have gmu in the background and a clock on the screen, but running in another tty.  Both ttyclock and dgclock seem to function with gmu in the background.  However this required me to map the home key as F1 so I could enter the magic ctrl-alt-F1 sequence recognized by SDL programs to escape to tty1.  I created a t2 script to launch a new tty and a cls script to rotate the tty, turn on the cursor, and clear the screen, just in case it gets hosed somehow.  I also added a deallocvt utility to the lcdval mcb so I can clean up abandoned ttys.

Zipit 4.  I've been tinkering with the rockbox/glinks image that dronz whipped up.  I built a smaller partial static glinks for sdl and added latin1 keyboard support so now there's a little bit of room to grow.  Here it is with a totally fictional foreign URL. 

I also built a slightly smaller mpg123 in case I want to grow this jffs into something with internet radio support.  And I've been tinkering with the keymap because I really want to standardize on one sticky latin15 keymap and one non-stick keymap for all my IZ2S derivatives.  I still need to do something about the other Fn keys though.

Zipit 5.  This has my original jffs base image and plenty of free space for testing things while I attempt to shrink them down.  For example, I used this one to test a latin1 build of pspmaps.  I recycled the latin1 kbd routines from glinks and made pspmaps accept latin1 locations.  Then I discovered RFC 3986 which says you now need to convert URLs to utf-8 before percent encoding and assembling your http GET request.  So I finally got my foot in the door a utf-8 app (well half utf-8 anyhow).  My son just went overseas with his classmates so I offered the zipit, just in case he needed to use it for directions.  He took his ipod instead.  C'est la vie...

With the new pspmaps you can travel from Paris to Mâcon, without crossing the Atlantic to wind up in Macon, GA.

I also used htop on zipit 5 to do some baseline performance testing of gmu and mpg123 on IZ2S based systems.  Mpg123 seems to use about 13% to 18% of the CPU to play and gmu about 2% more.  While testing I discovered that I need a .bashrc file in the home directory on the jffs to set the terminfo path if I want to run curses programs like htop in an ssh session.  Gotta add that to the next revision of all the IZ2S based jffs systems.

Finally, I'm currently investigating toybox and it's oneit utility to see if it's more efficient than own-tty or this busybox sh trickery.  Toybox is a tiny busybox alternative with some interesting features for my jffs images.  It's fairly compact and comes with a nice set of utilities that complement the stock busybox without all the overhead of the full busybox replacement from IZ2S.  I could use it to replace nc, killall, tee, and a few other standalone programs that I've built and get quite a bit more for just a few K.  Since toybox comes with a mini bzcat, I figured I might as well look for mini versions of zip and gzip to round out the compression options.  I found some, but more testing is needed.

There will be goodies when I track down where I posted them on the #zipit IRC channel...

links-latin-sdl-iz2s.zip
Here's the source, finally:  lnks-sdl-iz2s-latin-src.tar.bz2
Here's a comprehensive patch for links-2.3 on the zipit:  links-2.3-zipit.patch

pspmaps-utf8latin-iz2s.zip
pspmaps-utf8latin-iz2s-src.zip

mpg123-iz2jffs.zip

Here's toybox, deallocvt, and related scripts for using multiple vts (as shown below).
toybox-iz2jffs.zip

Source for the new lcdval mcb with deallocvt:  ez2sutils-src.zip

For some reason I felt compelled to compile the lynx browser.  lynx-iz2s.zip