Putting Links on a Diet

I got a bit ahead of myself (skipping a few stops on the roadmap) and compiled a bunjalloo executable for IZ2S.  It doesn't do anything but segfault right now, probably hitting one of those pesky uclibc bugs.  I'll have to shake that out one of these days, but I should probably stick to the original plan before attempting it.

However, I did discover something very interesting.  The size of the executable.  It came out to right around 250K upxed.  That's really, really small compared to the 2+MB upxed links executables that I've been building.  This got me thinking again about how to put links on a diet.  I know OpenSSL is significantly heavier than MatrixSSL.  But the built in font contributes most of the links bloat.  Bunjalloo gets by with only one size (small) font containing only ASCII characters as far as I know.  On the tiny zipit screen that's probably good enough.  If you think about it, the text mode of links does exactly that.

Now, I already knew the enormous font_include.c file was just a bunch of png files shoveled into C code.  But after some digging around I finally realized the graphics subdirectory contains the actual png glyph files and source to the generate_font program which created font_include.c.  Jackpot!  I pruned the glyph files down to just the basic Latin15 codes and some extra punctuation glyphs and then ran generate-font.  Next I rebuilt links with the new slimmer font_include.c and voila!  The upxed executable is almost megabyte lighter.

I tested it on some ebooks and a few websites.  So far It seems to have all the glyphs I typically run into, except for some unusual arrow characters on slashdot that weren't in the original font anyhow.  It's definitely a step up from retawq on the jffs.

I'm now positive I can get a upxed build of links (with graphics) slimmed down to less than a megabyte.  Some of the options I might try are:

•     Only one font (no monospace text)
•     No bold characters
•     ASCII only (no Latin15)
•     MatrixSSL instead of OpenSSL
•     No SSL.

Here's my first attempt, weighing in at about 1.6MB:  links-sdl-latinonly.zip

That should free up a whole bunch of space on the rockbox/links iz2jffs distro variant...

For the gmenu2x/gmu iz2jffs I save 200K right off the bat by using the existing jpeg, png, and libz shared libs required by gmenu2x instead of the static libs.  I can get another 150K by retiring retawq.  And I think removing the bold characters buys 200-300K.  If I rebuild gmenu2x without libfreetype (using the code from slug's github?) I think it all might just fit.  For now I just went only ASCII only, no bold, and no SSL for a 750K quick and dirty retawq replacement on the gmenu2x/gmu iz2jffs system.

links-iz2jffs-gmenu2x.tar.bz2

I'll try to put SSL and such back in after I see how the new gmenu2x build goes...

The Return of Zpub

While comparing the screenshots of glinks and bunjalloo, I was reminded of my old zpub post.  At the time I hadn't really dug into the links code very much, so I just sorta settled for a so-so reading experience.  I'd thought the epub text in glinks was somewhat small and ugly compared to netsurf.  But since netsurf requires the "mouse" I was a bit aprehensive about using it for an ereader.  I also remembered that dronz had tweaked his glinks config for ereading, so I decided to do a little experimentation with zpub and glinks.

The first change I made was to bump up the text size to something closer to the fonts used in a typical paperback book.  I'm pretty happy with an html-user-font-size setting of 18, but 16 is also not too shabby.  With the larger font, it makes sense to reduce the html-margin setting to 1 character to waste a minimal amount of space on the sides.  To bad you can't set it to 1 or 2 pixels.

While I was working on the margins I decided to see what I could do about that big fat ugly scroll bar.  The default G_SCROLL_BAR_WIDTH setting in setup.h is 12 pixels, which is extremely wasteful on the zipit.  Simply reducing it by 4 pixels gives me another full line of text with a font size of 18.

The other thing that really bugs me about glinks is the ugly default battleship gray background color.  You can change the colors of the menus, status bars, and even the scroll bar, but the html page background color must be set with in the document with a <body bgcolor="#FFFFDD"> tag or else you get the hard coded default_bg_g color from the default.c file.  I may try to code up an html-default-bgcolor command line and config file option to set this at runtime, but for now I just compiled in a yellow tinged off white color #FFFFDD that reminds me an old paperback book.

Another thing I noticed was the occasional odd characters in the text. Turns out the books I fetched from Project Gutenberg are utf8 formatted. I wouldn't be surprised if all epub documents are utf8.  Gotta review the specs someday.  Anyhow, I simply added the html-assume-codepage utf8 setting to the zpub script and now all is right with the text.  Check it out.

Remember how it used to look?  The new look is sooo much more warm and inviting, and actually displays even better on the zipit itself because the hardware rotated screen orientation washes out some of the yellow and possibly better matches my glinks subpixel anti-aliasing orientation setting.  Compare to this.

Gotta see if I can add that links command line option, then get the goodies posted and resume work on the bunjalloo code.

Update:  That turned out to be pretty easy, just a few ifdefs in the default.c file.  But now I've got a few more ideas.  I read about a third of the "Little Fuzzy" novel last night, and when I put down the zipit I realized the links bookmark system is probably inadequate for ereading.  I'd really like to save the scroll bar position so I can resume where I left off.  I'm gonna see if I can use the M and R keys to Memorize and Recall a scroll bar bookmark.  I might also look into defining the L and D keys in links to Lighten and Darken the LCD backlight for an adjustable day or night reading experience.  Either that or I'll have to setup the keymap so I can switch to another virtual terminal to adjust the lights and maybe check the battery level.

Update 2:  I finally finished reading my first full e-novel on the zipit and I can honestly say the experience was actually better than a paperback book in one way because I was able to pocket the zipit and take it with me.  I did find and fix a bug in zpub during my test read that briefly prevented me from viewing the 2nd half of the novel.  The M and R key scrollmark patch was fairly simple, and really made it easy to put down the zipit and pick it up again later to continue reading.  I decided against patching dimmer keys into links because that should be done at a system level so all apps can benefit.  I just need to work out which key combos make sense for all apps.

Here's the updated IZ2S zpub and links executables:  zpub2-iz2s.zip

Here's the modified links sources:  links-sdl-iz2s-zpub.tar.bz2
Here's a comprehensive patch for links-2.3 on the zipit:  links-2.3-zipit.patch

And here's a patch with just the ereading modifications.  http://pastebin.com/weJbzHWT

A jffs version will come later.  I need to make a zpub-lite that works without bash, and bc.  And I'll either have to provide a mini unzip executable, or build zpub-lite as an executable so I can use zlib. Maybe another mcb...

Moving Forwards

I finally posted the source code for the links browser with all my changes in the Friday the 13th blog entry.  Phew!  What a relief it is to get that done.

In the meantime, I've made some real progress on bunjalloo.  I read through all of the display code and consulted with some online docs about the NDS "main" and "sub" screens, their features, and operating modes.  Once I had it all marked up with comments, and thought I had a fair understanding of what was going on, I started making some changes to the code.

First I replaced all the occurrences of hard coded NDS screen sizes with SCREEN_WIDTH and SCREEN_HEIGHT.  Then I redefined those to match the Zipit screen size of 320x240, and tweaked the size of an opengl texture to accommodate the larger width.  This got me a double sized zipit screen display like this.

But I can only display the bottom "main" screen on the zipit so next I tweaked the SDLHandler.cpp code to use a single screen sized SetVideoMode() call, and tweaked the mouse button code to match the new screen size.  That gave me something like this, which should work on the zipit.

I really like the way bunjalloo can rotate the bottom toolbar around the screen to the right side as seen above.  It can also hide the toolbar so it doesn't obscure anything at all.  The scroll bar looks nice while using a minimal amount of screen real estate.  For comparison here's glinks displaying the same web site.  It's clearly more sophisticated than bunjalloo, but did they really have to make all the widgets so hideous?  Just look at that scroll bar.  It's the orthopedic shoe of scroll bar widgets.

I also double checked http:\\m.gmail.com to make sure I could still retrieve my gmail on the reduced zipit sized screen.  It's a bit slower than I'd like, but seems to work.

Unfortunately, in it's current form the bunjalloo code is still wasting considerable time and effort updating the video buffers for the "sub" screen even though it's no longer displayed.  This still needs work.  However, the (shared lib) bunjalloo executable is currently about 700K on x86 ubuntu (250K upxed).  That's significantly smaller than glinks, so I might just find a use for this when I'm all done.  Here's a roadmap of what I'm planning to do with it if all goes well.

• Compile without the WAF (python) build system.  Done
• Remove hardcoded NDS screen size numbers.  Done
• Change the SDL screen size to fit zipit screen.  Done
• Replace all opengl code (and unwanted libs) with one SDL Blit.  Done
• Remove unused SDL_mixer audio code from SDLHandler.cpp  Done
• Eliminate drawing to the emulated NDS upper "sub" screen.
• Remove (ifdef?) "sub" screen datastructs, leaving only "main" screen code.
• Fix SDL kbd code to use unicode field, and make it more responsive.
• Add support for ENTER, BS, ESC keys from real kbd on kbd gui.
• Add link navigation via tab, shift-tab, and enter keys instead of z2mouse.
• Consider changing arrow keys (and others) to match glinks keybindings.
• Create real makefiles in all source code directories.
• Build on Zipit.
• Optionally link bunjallo cache dir to /tmp/bunjalloo/cache for jffs.
• Try to build it without libstdc++ (to reduce exe size).
• Try other build options to reduce exe size for a better fit on the zipit jffs.

I've put the code onto my github site so you can see what's changed from the original sources.

Had another thought.  If I repackage my gmenu2x/gmu jffs distro to supplement retawq with bunjalloo I might get a better fit with a shared libstdc++ since I'll have 2 apps using it.  Something to think about...

I Don't Know What I Was Thinking.

Boy, that'll teach me to make promises on Friday the 13th, just before taxes are due.  The taxes got done, but not much else.  And now I'm really having trouble remembering where I stashed all the zipit goodies I promised to make available.  I swear I'll get them posted, real soon now...

In the meantime, whilst avoiding packaging up the old goodies, I somehow found myself on an old Nintendo DS homebrew site, which reminded me of the spiffy Bunjalloo web browser.  It runs quite well (with graphics even) in the limited 4MB and small screen real estate of the NDS.  Plus there's some notes about building it for linux with SDL, and it uses the tiny matrixssl library for shttp connections.  I've been looking for an excuse to use that one, but I'm far too lazy to graft it onto one of the other zipit browsers.  So off I went, merrily avoiding what I should be doing, and attempting to port bunjalloo instead.  Turns out it uses some (python based?) WAF build system that I've never heard of, and a bunch of stuff from the NDS devkitarm toolchain.  Well, python and WAF were quite unhappy with my scratchbox build environment, so I quickly ditched them and moved to script based compiling.  I had to resort to Windows to use the NDS oriented grit tool to convert some png files to .c and .h source code because I'm too lazy to pull down all it's requirements and build a linux version.

Then, just as I was reaching 100% of the sources compiled I ran up against a wall.  Oops, the linux build requires opengl.  Apparently some 2D opengl calls match up well against the native NDS hardware, so an opengl layer makes sense for portability.  But not really for my purposes.  Opengl is not so great for the zipit, with it's conspicuous lack of floating point hardware.  However on closer inspection, it looked like most of the opengl calls were using the integer versions of the functions.  Hmm, that got me thinking.  Apparently I still have a few brain cells left over from olden times, when DSLinux wasn't quite dead yet.  Just before the DSI came out and flatlined the last remaining DSLinux effort, I managed to sneak in a really lame port of the opengl version of the classic Elite space game.  I dug through the archives and sure enough, there was the integer only tiny-SDL-OpenGL lib I used for the port.  I think it needs a few tweaks to make it suitable for bunjalloo, and then there's the whole business of reconfiguring the screen and keyboard code, but the wall has been breached. 

Hopefully bunjalloo doesn't animate or use 100% of the CPU in a busy loop, because the opengl example programs are not exactly top performers.  Here's the well known gears program grinding out a whopping 2 frames per second on the zipit.

The shiny spinning triangle isn't a whole lot snappier.

Hopefully I can continue to restrain myself and not try to port any of my old opengl based CAD programs to the zipit.

 Although this does look almost useable in 320x240 pixels...

Must, resist, temptation...

For your  amusement, here's a zip with the tinygl sources.  The gears and triangle executables are also included in the examples directory. 
tinygl-iz2s.zip

And here's what bunjalloo currently looks like running in the debugger on ubuntu.

As you can see, I have plenty of work to do on the screen layout before I can even attempt to run it on the zipit.

Update:  I dug through the code, looking for all numbers near 256 and 192 (the NDS screen dimensions), and replaced all the 192s with 120 to use a half zipit screen instead of one of the NDS screens.  It sorta works but the scrollbar looks weird constrained to the bottom half of the display.  So I think I really want to make it use only one 240 pixel high screen instead.

I was able to confirm that I can use bunjalloo to access my gmail account (cool!), however hotmail seems to really want javascript.