Monday, May 14, 2012

Putting Links on a Diet

I got a bit ahead of myself (skipping a few stops on the roadmap) and compiled a bunjalloo executable for IZ2S.  It doesn't do anything but segfault right now, probably hitting one of those pesky uclibc bugs.  I'll have to shake that out one of these days, but I should probably stick to the original plan before attempting it.

However, I did discover something very interesting.  The size of the executable.  It came out to right around 250K upxed.  That's really, really small compared to the 2+MB upxed links executables that I've been building.  This got me thinking again about how to put links on a diet.  I know OpenSSL is significantly heavier than MatrixSSL.  But the built in font contributes most of the links bloat.  Bunjalloo gets by with only one size (small) font containing only ASCII characters as far as I know.  On the tiny zipit screen that's probably good enough.  If you think about it, the text mode of links does exactly that.

Now, I already knew the enormous font_include.c file was just a bunch of png files shoveled into C code.  But after some digging around I finally realized the graphics subdirectory contains the actual png glyph files and source to the generate_font program which created font_include.c.  Jackpot!  I pruned the glyph files down to just the basic Latin15 codes and some extra punctuation glyphs and then ran generate-font.  Next I rebuilt links with the new slimmer font_include.c and voila!  The upxed executable is almost megabyte lighter.

I tested it on some ebooks and a few websites.  So far It seems to have all the glyphs I typically run into, except for some unusual arrow characters on slashdot that weren't in the original font anyhow.  It's definitely a step up from retawq on the jffs.

I'm now positive I can get a upxed build of links (with graphics) slimmed down to less than a megabyte.  Some of the options I might try are:
  •     Only one font (no monospace text)
  •     No bold characters
  •     ASCII only (no Latin15)
  •     MatrixSSL instead of OpenSSL
  •     No SSL.
Here's my first attempt, weighing in at about 1.6MB:  links-sdl-latinonly.zip

That should free up a whole bunch of space on the rockbox/links iz2jffs distro variant...

For the gmenu2x/gmu iz2jffs I save 200K right off the bat by using the existing jpeg, png, and libz shared libs required by gmenu2x instead of the static libs.  I can get another 150K by retiring retawq.  And I think removing the bold characters buys 200-300K.  If I rebuild gmenu2x without libfreetype (using the code from slug's github?) I think it all might just fit.  For now I just went only ASCII only, no bold, and no SSL for a 750K quick and dirty retawq replacement on the gmenu2x/gmu iz2jffs system.

links-iz2jffs-gmenu2x.tar.bz2

I'll try to put SSL and such back in after I see how the new gmenu2x build goes...

7 comments:

  1. Awesome ! It works perfectly !

    ReplyDelete
  2. Bonjour,
    Here is my french gmu-gmenu2x-links-rockbox-jffs:
    http://prototypeur.net/dataz/my-fr2-jffs.tgz
    And here is the optionnal part to copy to a SD card:
    http://prototypeur.net/dataz/sd0-fr2.zip
    Please be indulgent! And tell me if it's worth
    to translate and share it after your correction.
    Thanks a lot.

    ReplyDelete
  3. ...
    -I must comment the commands I don't use in Zcovery,
    in the place of removing them.
    -I must add an explicit readme file, with the names of
    the contributors, the licences, and a help to install.
    -I must correct a bunch of errors with nupdf:
    -I forgot to copy the config files /mnt/ffs/share/nupdf
    They are here: http://prototypeur.free.fr/nupdf-jffs.zip
    -/mnt/ffs/gmenu2x/sections/applications/nupdf has a typo,
    sdO in place of sd0
    -/mnt/sd0/lib/bin/nupdf has a typo, "$file" in place of $*
    Then, the gmenu2x file selector works nicely with nupdf.

    ReplyDelete
  4. I'll try it. It looks very close to my personal goal system. But I'm hoping to have the optional executables and libs reside in the exact same place on the SD card as they would on an IZ2S SD card. I haven't decided how to do that yet though. I'd also like to keep rockbox.bin on the jffs, which I should be able to do by switching from OpenSSL to matrixSSL, but that'll probably take a while since I'm unfamiliar with both libs.

    ReplyDelete
  5. You know, I'm not a programmer. I learn and sometimes I find some tricks. You're right, keeping lib and bin names for the optionnal SD card directory can drive to confusion. I must change this. Rockbox alone, without SD card, has no use. It's why I decided to move it. If all is in one dir, it's easy for a beginner to copy it on each of his SD cards.
    And about the big place I found in ffs, gmenu calls libjpeg, but do not use it! I have replaced this lib (800K) by a symlink to libpng, and I'm lucky, it works !
    I'm very happy with this version, and because it was my first symlinking :) I don't forget that without your work,
    all this could not exist, so thanks again. I don't forget too that I owe you a serial modded Z2 with the cable, I hope I will be able to send it to you soon. Cheers!

    ReplyDelete
  6. I've been examining your jffs tarball and it's quite nice. I spotted the libjpeg trick. Clever! Symlinking is really handy. I wonder if you can use a symlink to put the rockbox codec directory back on the SD card and save some more space on the jffs.

    Meanwhile, I'll have to see if I can build gmenu2x without libjpeg so we can put the real libjpeg on the SD card someday. I can also build a slightly smaller glinks for your jffs since you have libpng and libz as shared libs.

    ReplyDelete
  7. Ok, I cut some fat off the libs and was able to restore libjpeg, which is now used by links, as well as gmu, and gmenu2x (sorta). I also switched links to use the shared libpng and libz for a small savings. I added the .bashrc needed for ssh and updated lcdval to the latest version with deallocvt and clear applets. Maybe a few other small changes.
    So far I've only briefly tested wifi, links, and gmu. I have a script that boots right into gmenu unless you press a key, but I have to track it down so it's not in here:

    http://www.datafilehost.com/download-5f8f2d24.html

    It has a little over 500K available so I can still add the rockbox executable later if I decide to. But I'd like to do more testing as is first.

    ReplyDelete