The goodies you seek are here. and/or here. Read on if you want to know why.
But first, a note about the goodies. Much of it is intended for either iz2s or iz2jffs. They're basically the same thing except iz2jffs is much smaller and resides on the internal flash, whereas iz2s is installed on an SD card. Both are hacked up systems that make use of the stock kernel that comes pre-installed on the zipit. It's an ancient kernel, but I like it for my purposes. Now since the entire thing is hacked together there's no real package management system for beginners. Sorry about that. You might just have to learn your way around the linux file system if you want to get a lot out of this.
Anyhow, if you want to use lots of this stuff, then your best bet is to boot from an iz2s SD card. Otherwise, if you know what you're doing, you can use all of the goodies with an iz2jffs setup, but it takes a little extra work to get it to recognize the libraries and paths used by the stuff. The addz goodie pack does this for you for some of the older goodies that can't possibly fit on the jffs. You'd have to do something similar to the addz pack to adapt the newer goodies to work with the jffs. I may do it someday, but right now I'm booting from the SD card most of the time.
And now for the story of the goodie bag...
It's been said that the internet giveth and the internet taketh away. And sometimes that's true. It's also been said that the internet never forgets, but that's another story. This one is about give and take. Previously my cable provider of 10 years decided to eliminate their crappy (but free!) web hosting service because absolutely nobody was using it, except me. So the Zipit Goodie Bag had to pack up and move. Rkdavis graciously provided a new home for it, but it's fallen into a bit of disrepair lately. I suspect he's taken in too much raspberry pi vapor, and will be back when the buzz wears off.
In the meantime I've been keeping things patched together by hosting new goodies at datafilehost.com. But already some of the links have gone dead due to lack of interest. Imagine that! My 40MB IZ2S emacs package has gone missing due to lack of downloads. Don't you people realize the awsome power of a fully operational emacs? Even my own son has turned against me, recently declaring allegiance to vi instead of emacs. I've seen this movie before and let me just say, it doesn't end well for the zipit.
But now Dronz has provided the Zipit Goodie Bag with a new summer home in France. So it's time for some spring cleaning. I'm planning to prune all the dead links in the blog and redirect them to this post, where I still have the editing power to keep things up to date. From now on, this post will tell you to where all the goodies actually reside.
Here's the Zipit Goodie Bag Summer Home, including the new jffs goodies.
And here's the old tried and true (but in need of an update) Zipit Goodie Bag.
Fun with software, gizmos, virtual bricks, and other geeky toys.
Friday, March 30, 2012
Monday, March 12, 2012
In the Zone
I decided to use my renewed focus on the jffs to try and see if I could squeeze gmenu2x onto the stock zipit jffs along with gmu. (I've already fielded a few not so subtle hints from my co-worker that the linux command line is perhaps not the easiest way to use the zipit.) This wasn't gonna be easy though because gmenu2x is fairly bloated with C++, freetype, and lots of png files.
The first thing I did was to cut it back to a minimal installation with just the default skin, only one translation (french), and fewer icons and the smallest background image file I could find. See above. I also ran all the images through pngcrush for good measure. This resulted in a gmenu2x installation that was almost small enough for the jffs. Unfortunately it didn't quite work when run from the stock busybox shell. I had to build a tiny killall executable to enable my previous uclibc workaround from the iz2s build. Then I had to make sure to "exec gmenu2x" from my gmenu wrapper script or it would complain about various tty ailments whenever it finished running an app. Weirdness.
Next I attempted to link gmenu2x without libstdc++ according to these instructions. But gmenu2x uses STL strings, dynamic casts, and more from libstdc++ so the best I could do was skip exceptions and static link with libstdc++.a. The shared lib is about 800k stripped, and would would use about 300K compressed on the jffs, so I managed to squeeze a little bit out of it. I recently discovered the jffs seems to garbage collect on boot up, which means I may have a reliable way to compare the space usage. So I'll probably try to install the full lib someday to see exactly how much I saved. Actually, I suspect I'll hold off on that until I find another C++ program that I just gotta have...
I also tried static linking libfreetype, but that didn't save anything. A stripped libfreetype is 400K, or about 200K compressed on the jffs, so I bit the bullet and stuck it on there. This meant I could also toss in the dgclock program for only 20K if could trick it to use the same font as gmenu2x. So I moved the /mnt/ffs/gmenu2x/skins/Default/font.ttf file to /mnt/share/fonts/font.ttf and replaced it in it's original location with a soft link. Then I soft linked the mplus-2p-medium.ttf file used by dgclock to font.ttf and voila! The dgclock works. The gmenu2x font file is fairly compact but I can barely tell the difference in the dgclock program. It might have slightly narrower glyphs. Maybe it'll be good enough for a few more small SDL-ttf programs.
This fit onto the zipit, but just barely, so I swapped out the wpa-supplicant and wpa-passphrase for my cut down versions. The ones from the V1 firmware (and IZ2S) are fairly large, and the ancient wpa-supplicant was having troubles connecting reliably to WPA2 routers anyhow. The new smaller version actually seems to work better. This got me to about 700K free on the jffs. That's enough room for some more small goodies.
While working with dgclock, I noticed the zipit appeared to be stuck in a different timezone (somewhere in europe?) instead of EST5EDT. I'm not sure what caused it, but eventually I deleted the /mnt/ffs/TZ file and rebooted. The timezone setting script popped up. I entered EST5EDT, and it recovered. Then I realized I was off by an hour because the 2007 daylight savings rules weren't incorporated into the iz2s uclibc. Fortunately Dom pointed out the uclib TimeZone fix for the US 2007 daylight savings change in one of the comments here. Down at the bottom it mentions the magic EST5EDT,M3.2.0,M11.1.0 setting for the /mnt/ffs/TZ file. EST5EDT,M3.2.0,M11.1.0 is US Eastern Time with Daylight Savings Time starting on the 3rd month (March), 2nd week, day 0 (Sunday) and continuing until the 11th month (November), 1st week, day 0 (Sunday).
Here's a combo jffs image containing the base, gmu and gmenu2x packages to make it an easier install for newbs. Note: I said "easier", not "easy".
gmu-gmenu2x-jffs.tgz Here's the install instructions.
And apparently the Goodie Bag was offline for a while. Sorry. I think I'll try and mirror it...
The first thing I did was to cut it back to a minimal installation with just the default skin, only one translation (french), and fewer icons and the smallest background image file I could find. See above. I also ran all the images through pngcrush for good measure. This resulted in a gmenu2x installation that was almost small enough for the jffs. Unfortunately it didn't quite work when run from the stock busybox shell. I had to build a tiny killall executable to enable my previous uclibc workaround from the iz2s build. Then I had to make sure to "exec gmenu2x" from my gmenu wrapper script or it would complain about various tty ailments whenever it finished running an app. Weirdness.
Next I attempted to link gmenu2x without libstdc++ according to these instructions. But gmenu2x uses STL strings, dynamic casts, and more from libstdc++ so the best I could do was skip exceptions and static link with libstdc++.a. The shared lib is about 800k stripped, and would would use about 300K compressed on the jffs, so I managed to squeeze a little bit out of it. I recently discovered the jffs seems to garbage collect on boot up, which means I may have a reliable way to compare the space usage. So I'll probably try to install the full lib someday to see exactly how much I saved. Actually, I suspect I'll hold off on that until I find another C++ program that I just gotta have...
I also tried static linking libfreetype, but that didn't save anything. A stripped libfreetype is 400K, or about 200K compressed on the jffs, so I bit the bullet and stuck it on there. This meant I could also toss in the dgclock program for only 20K if could trick it to use the same font as gmenu2x. So I moved the /mnt/ffs/gmenu2x/skins/Default/font.ttf file to /mnt/share/fonts/font.ttf and replaced it in it's original location with a soft link. Then I soft linked the mplus-2p-medium.ttf file used by dgclock to font.ttf and voila! The dgclock works. The gmenu2x font file is fairly compact but I can barely tell the difference in the dgclock program. It might have slightly narrower glyphs. Maybe it'll be good enough for a few more small SDL-ttf programs.
While working with dgclock, I noticed the zipit appeared to be stuck in a different timezone (somewhere in europe?) instead of EST5EDT. I'm not sure what caused it, but eventually I deleted the /mnt/ffs/TZ file and rebooted. The timezone setting script popped up. I entered EST5EDT, and it recovered. Then I realized I was off by an hour because the 2007 daylight savings rules weren't incorporated into the iz2s uclibc. Fortunately Dom pointed out the uclib TimeZone fix for the US 2007 daylight savings change in one of the comments here. Down at the bottom it mentions the magic EST5EDT,M3.2.0,M11.1.0 setting for the /mnt/ffs/TZ file. EST5EDT,M3.2.0,M11.1.0 is US Eastern Time with Daylight Savings Time starting on the 3rd month (March), 2nd week, day 0 (Sunday) and continuing until the 11th month (November), 1st week, day 0 (Sunday).
Here's a combo jffs image containing the base, gmu and gmenu2x packages to make it an easier install for newbs. Note: I said "easier", not "easy".
gmu-gmenu2x-jffs.tgz Here's the install instructions.
And apparently the Goodie Bag was offline for a while. Sorry. I think I'll try and mirror it...
Wednesday, March 7, 2012
Uncontrollable Urges
In the midst of all my utf8 sleuthing I got the urge to revisit my problem using bash (instead of the busybox shell) to type extended characters with the European keymap. Running loadkeys eurokeymap.map says, "assuming iso-8859-15 euro". Hmm, 8859... I went back to Prototypeur, followed the french wikipedia link and finally realized my error... Ah ha! That's not utf8. It's just a full 8bit codepage. The french wikipedia page is actually clearer than the english version because the compact table layout makes it obvious at a glance that this is single byte encoding. The english table is bloated and confusing with octal numbers and redundant hex values. Who really cares about the octal these days? Octal's about as useful as Roman numerals. The future belongs to hex.
To make it work I want bash to run 8bit clean instead of 7bit only. Maybe that'll help for for utf8 too. Here's how.
Make a .inputrc file in the home directory and put this in it. Or try a global /etc/inputrc file.
As if that weren't enough, while all of this was going on I finally got around to packaging up the gmu music player for the jffs. This came about because a co-worker suddenly decided he was ready for his zipit that I'd promised to deliver (as soon as it was done). You know how that goes with software. It's never really done... Anyhow, I came up with 3 potential configurations for the jffs on this zipit: rockbox for mp3s and games, gmu for mp3s and internet radio, or mplayer and some simple bash wrapper scripts like these for mp3s, internet radio, and possibly some video. The gmu choice was selected so I finally put that together with all the required libs. There's still about 1.2MB on the jffs for extra goodies, so I really need to identify some programs that make use of all those shared libs to get the most bang for the buck...
Meanwhile, in testing the gmu package, I discovered that my latest base jffs was horribly busted. The timezone script was missing all the /dev/tty0 redirection that's required for scripts run before owntty in the boot sequence. Oops, that little mistake left you hanging on a blank screen, waiting for you to select a timezone from an invisible list. Not good. Here's a replacement timezone script for the 2 people who downloaded the broken jffs base.
tz-iz2jffs.zip
Here's a gmu package suitable for the stock jffs. I preloaded the playlist with a few stations.
gmu-0.8-iz2jffs.tar.bz2
To make it work I want bash to run 8bit clean instead of 7bit only. Maybe that'll help for for utf8 too. Here's how.
Make a .inputrc file in the home directory and put this in it. Or try a global /etc/inputrc file.
# .inputrc controls how programs using the readline library behave.
# This includes bash.
### Make Bash 8bit clean ###
# Enable 8bit input
set meta-flag on
set input-meta on
# Turns off 8th bit stripping
set convert-meta off
# Keep the 8th bit for display
set output-meta on
As if that weren't enough, while all of this was going on I finally got around to packaging up the gmu music player for the jffs. This came about because a co-worker suddenly decided he was ready for his zipit that I'd promised to deliver (as soon as it was done). You know how that goes with software. It's never really done... Anyhow, I came up with 3 potential configurations for the jffs on this zipit: rockbox for mp3s and games, gmu for mp3s and internet radio, or mplayer and some simple bash wrapper scripts like these for mp3s, internet radio, and possibly some video. The gmu choice was selected so I finally put that together with all the required libs. There's still about 1.2MB on the jffs for extra goodies, so I really need to identify some programs that make use of all those shared libs to get the most bang for the buck...
Meanwhile, in testing the gmu package, I discovered that my latest base jffs was horribly busted. The timezone script was missing all the /dev/tty0 redirection that's required for scripts run before owntty in the boot sequence. Oops, that little mistake left you hanging on a blank screen, waiting for you to select a timezone from an invisible list. Not good. Here's a replacement timezone script for the 2 people who downloaded the broken jffs base.
tz-iz2jffs.zip
Here's a gmu package suitable for the stock jffs. I preloaded the playlist with a few stations.
gmu-0.8-iz2jffs.tar.bz2
Tuesday, March 6, 2012
Terminally Insane
This one is a long rambling mess. Sorry in advance...
I found a simple program at the Linux Productivity Magazine to display the curses ACS special characters. When I tried it I discovered that some looked wrong, and some were missing on the zipit. All of the box drawing characters there but the arrows were garbled, the "lantern" or vertical tab char was missing, and all of the 1 3 5 7 9 single scan line glyphs were missing except for the ones that do double duty as the dash and the underline.
I took a peek at the unicode table from my console font and discovered the unicode mappings for the scan line glyphs were way up in the Chinese section of the giant unicode table. So I moved them back down into the 0x2xxx range, rebuilt the font file and voila, the scan lines were there. Turns out ACS_LANTERN was just missing from my mcurses code. The arrows were a different story. They're not included in the 32 characters of the DEC special graphics set. I need to print a unicode character in order to display them, so for now I went with the default standby ASCII substitutes of <>^v for the arrows and # for ACS_BOARD and ACS_BLOCK.
It drives me nuts that I've got all these nice glyphs in the font and yet have no way to display them. So I went looking for some simple C code to print a unicode character. This turns out to be all tangled up with a monster sized locale and nls system on my desktop linux box.
There are about a dozen environment variables that control all sorts of settings. So I tried setting these up for US english with UTF-8 according to the notes on the internet. Not only did this have no effect, but when I try to set LC_CTYPE something (bash, I think) spits back an error message. WTF? It's just an environment variable! Shut up and set it. You might think its some sort of conspiracy to force a huge bloated locale database onto the zipit in /usr/lib/locale. Ok, it's only 2.5M but that's too much for the jffs and I think the FAT filesystem of IZ2S will bloat that way up because it can't use soft or hard links to avoid duplication. Plus I really don't need the zipit to know what order I want my socks sorted, or what time in the afternoon to serve me tea. I just want to be able to print all of the characters I loaded into my font. Is that too much to ask?
Well I did some soul searching, and some google searching and eventually unearthed the magic escape sequences to put the terminal into UTF-8 mode and back (ESC%G and ESC%@). Armed with this knowledge I wrote a short C program to print the ACS_BLOCK character. And it worked!
Well, it worked in my gnome term on the desktop linux, and it worked on the zipit linux console in both bash and the busybox ash shell. But it didn't work in gnu screen or dvtm. Fooey!. Actually it turns out I can make it work in screen if I issue the ESC%G from a shell script first and then run "screen -U". Ok. I can live with that, but dvtm just eats the ESC% and spits out the G. (I can't remember, but I think this script may have finally let me type UTF-8 at the bash command prompt, instead of just in the busybox ash shell.)
Fortunately the dvtm code is pretty straightforward (unlike screen, which may have a secret built-in nethack mode?). I found the ESC handler code and added a little patch to eat the entire ESC%G sequence and toggle the internal is_utf8 flag. But then dvtm just prints three nonsense characters for my 0xe2,0x86,0x88 sequence instead of converting to the proper unicode BLOCK glyph. The man pages tell me mbrtowc and the reverse wcrtomb functions are under the influence of LC_CTYPE. It's that darn conspiracy again!
Back to the internet for a workaround. This sorta spells out the problem and it gives a hint about the leader of the conspiracy: glibc. I'd bet uclibc doesn't even know about those locale files. Anyhow, I found some magic c code to convert utf8 to utf16 with no mention of the evil LC_CTYPE conspiracy. It worked somewhat. Debugging showed everything converted ok inside dvtm, but ncurses still eats the final result, no matter what form I try to push through. Probably in league with the conspiracy. So now what to do?
Maybe I should try to build a small ncurses example and get it working before moving back to dvtm.
Or maybe I should do something about the nl_langinfo() function. This function is used by both the dvtm vt.c init code and the ncurses ncurses/tinfo/lib_setup.c file to determine if the tty is in UTF-8 mode. The nl_langinfo function is in the uClibc libc.so.0 library and I suspect it's just a stub, hardcoded to POSIX and C locales. According to the uclibc FAQ you can compile in UTF-8 locales but that makes it significantly bigger, depending on the number of locales. (http://uclibc.org/FAQ.html) The IZ2S libc is pretty small at just under 300K. Plus you supposedly had to fetch uClibc-locale-030818.tgz manually to build the locale support, so I doubt it's in there. Grep found the word POSIX, but no mention of UTF, LC, or LANG. At least uClibc seems to be on my side, resisting the bloat, just not in a useful way that lets me use my font.
It looks like I could undefine HAVE_LANGINFO_CODESET and recompile ncurses so it uses an internal nc_get_locale() function instead. But then I might have to undefine HAVE_LOCALE_H so nc_get_locale uses getenv instead of setlocal(LC_CTYPE, 0) which always returns "C" instead of "en_US.UTF-8" (or anything.UTF-8). And what about the NCURSES_NO_UTF8_ACS environment variable hack? I may need to set that once I'm in utf8 mode in ncurses. There appear to be some special case hacks in the code to do this for gnu screen and when TERM=linux.
I ended up with a special version of nl_langinfo() that scans the environment vars for UTF-8. I also bypassed setlocale() in the ncurses lib_setup.c file and now I see a UTF-8 locale. But the meta and control conversions are taking precedence over the characters. e2 94 a4 is printed as M-b ~T M-$. And the block glyph e2 96 88 is printed as M-b ~V ~H. Darn it! Ncurses is a tough nut to crack!
Anyhow, dvtm is off the hook. I can't even get a simple standalone ncursesw program to print utf8. Worse, I can't even get the test programs that come with ncurses to print utf8! I'm either building it wrong, or something in uclibc is broken. Right now, I can't even tell if ncursesw is even using the wide char routines, though it is 20K larger than the non-wide ncurses. Stupid speed optimization macros make it really hard to decypher the code. Besides, it's more likely that some other fn like isprint or isascii from uclibc is steering ncurses the wrong way. Apparently ncurses uses just about every single function that deals with text somewhere. No wonder it's so fat. And yes, it uses isprint() in multiple places, and the man page implies isprint() needs a working setlocale() to handle utf8. However the uclibc implementation of isprint is a friggin ugly macro in ctype.h, so I can't easily replace it. The macro doesn't look like it even considers utf-8. That means multiple hacks will be needed in the obtuse ncurses code instead. Ouch.
Meanwhile the ncurses source directory has a wcwidth.h file that looks suspiciously like the substitute wcwidth fn I dug up. And there are tantalizing hints in the test directory that just maybe there's a configure option to use libutf8 instead of the libc defaults. Is that just for the test programs? Maybe I should try it. Then if it works try it with my test program, and then dvtm if that works. Unfortunately I think it's a hidden configure option. configure --help=short gives no clue how to change it.
My list of hacks was getting sorta big, so I decided to try libutf8 to replace the whole pile of crap. It's kinda big though at 160K, and the easier to use shared lib LD_PRELOAD plug version wouldn't link in my scratchbox toolchain. So in the end, all I can say is that I'm still working on on it...
Someday when I get things together I'll post the updated fonts, and maybe an new dvtm-wide.
I found a simple program at the Linux Productivity Magazine to display the curses ACS special characters. When I tried it I discovered that some looked wrong, and some were missing on the zipit. All of the box drawing characters there but the arrows were garbled, the "lantern" or vertical tab char was missing, and all of the 1 3 5 7 9 single scan line glyphs were missing except for the ones that do double duty as the dash and the underline.
I took a peek at the unicode table from my console font and discovered the unicode mappings for the scan line glyphs were way up in the Chinese section of the giant unicode table. So I moved them back down into the 0x2xxx range, rebuilt the font file and voila, the scan lines were there. Turns out ACS_LANTERN was just missing from my mcurses code. The arrows were a different story. They're not included in the 32 characters of the DEC special graphics set. I need to print a unicode character in order to display them, so for now I went with the default standby ASCII substitutes of <>^v for the arrows and # for ACS_BOARD and ACS_BLOCK.
It drives me nuts that I've got all these nice glyphs in the font and yet have no way to display them. So I went looking for some simple C code to print a unicode character. This turns out to be all tangled up with a monster sized locale and nls system on my desktop linux box.
There are about a dozen environment variables that control all sorts of settings. So I tried setting these up for US english with UTF-8 according to the notes on the internet. Not only did this have no effect, but when I try to set LC_CTYPE something (bash, I think) spits back an error message. WTF? It's just an environment variable! Shut up and set it. You might think its some sort of conspiracy to force a huge bloated locale database onto the zipit in /usr/lib/locale. Ok, it's only 2.5M but that's too much for the jffs and I think the FAT filesystem of IZ2S will bloat that way up because it can't use soft or hard links to avoid duplication. Plus I really don't need the zipit to know what order I want my socks sorted, or what time in the afternoon to serve me tea. I just want to be able to print all of the characters I loaded into my font. Is that too much to ask?
Well I did some soul searching, and some google searching and eventually unearthed the magic escape sequences to put the terminal into UTF-8 mode and back (ESC%G and ESC%@). Armed with this knowledge I wrote a short C program to print the ACS_BLOCK character. And it worked!
Well, it worked in my gnome term on the desktop linux, and it worked on the zipit linux console in both bash and the busybox ash shell. But it didn't work in gnu screen or dvtm. Fooey!. Actually it turns out I can make it work in screen if I issue the ESC%G from a shell script first and then run "screen -U". Ok. I can live with that, but dvtm just eats the ESC% and spits out the G. (I can't remember, but I think this script may have finally let me type UTF-8 at the bash command prompt, instead of just in the busybox ash shell.)
Fortunately the dvtm code is pretty straightforward (unlike screen, which may have a secret built-in nethack mode?). I found the ESC handler code and added a little patch to eat the entire ESC%G sequence and toggle the internal is_utf8 flag. But then dvtm just prints three nonsense characters for my 0xe2,0x86,0x88 sequence instead of converting to the proper unicode BLOCK glyph. The man pages tell me mbrtowc and the reverse wcrtomb functions are under the influence of LC_CTYPE. It's that darn conspiracy again!
Back to the internet for a workaround. This sorta spells out the problem and it gives a hint about the leader of the conspiracy: glibc. I'd bet uclibc doesn't even know about those locale files. Anyhow, I found some magic c code to convert utf8 to utf16 with no mention of the evil LC_CTYPE conspiracy. It worked somewhat. Debugging showed everything converted ok inside dvtm, but ncurses still eats the final result, no matter what form I try to push through. Probably in league with the conspiracy. So now what to do?
Maybe I should try to build a small ncurses example and get it working before moving back to dvtm.
Or maybe I should do something about the nl_langinfo() function. This function is used by both the dvtm vt.c init code and the ncurses ncurses/tinfo/lib_setup.c file to determine if the tty is in UTF-8 mode. The nl_langinfo function is in the uClibc libc.so.0 library and I suspect it's just a stub, hardcoded to POSIX and C locales. According to the uclibc FAQ you can compile in UTF-8 locales but that makes it significantly bigger, depending on the number of locales. (http://uclibc.org/FAQ.html) The IZ2S libc is pretty small at just under 300K. Plus you supposedly had to fetch uClibc-locale-030818.tgz manually to build the locale support, so I doubt it's in there. Grep found the word POSIX, but no mention of UTF, LC, or LANG. At least uClibc seems to be on my side, resisting the bloat, just not in a useful way that lets me use my font.
It looks like I could undefine HAVE_LANGINFO_CODESET and recompile ncurses so it uses an internal nc_get_locale() function instead. But then I might have to undefine HAVE_LOCALE_H so nc_get_locale uses getenv instead of setlocal(LC_CTYPE, 0) which always returns "C" instead of "en_US.UTF-8" (or anything.UTF-8). And what about the NCURSES_NO_UTF8_ACS environment variable hack? I may need to set that once I'm in utf8 mode in ncurses. There appear to be some special case hacks in the code to do this for gnu screen and when TERM=linux.
I ended up with a special version of nl_langinfo() that scans the environment vars for UTF-8. I also bypassed setlocale() in the ncurses lib_setup.c file and now I see a UTF-8 locale. But the meta and control conversions are taking precedence over the characters. e2 94 a4 is printed as M-b ~T M-$. And the block glyph e2 96 88 is printed as M-b ~V ~H. Darn it! Ncurses is a tough nut to crack!
Anyhow, dvtm is off the hook. I can't even get a simple standalone ncursesw program to print utf8. Worse, I can't even get the test programs that come with ncurses to print utf8! I'm either building it wrong, or something in uclibc is broken. Right now, I can't even tell if ncursesw is even using the wide char routines, though it is 20K larger than the non-wide ncurses. Stupid speed optimization macros make it really hard to decypher the code. Besides, it's more likely that some other fn like isprint or isascii from uclibc is steering ncurses the wrong way. Apparently ncurses uses just about every single function that deals with text somewhere. No wonder it's so fat. And yes, it uses isprint() in multiple places, and the man page implies isprint() needs a working setlocale() to handle utf8. However the uclibc implementation of isprint is a friggin ugly macro in ctype.h, so I can't easily replace it. The macro doesn't look like it even considers utf-8. That means multiple hacks will be needed in the obtuse ncurses code instead. Ouch.
Meanwhile the ncurses source directory has a wcwidth.h file that looks suspiciously like the substitute wcwidth fn I dug up. And there are tantalizing hints in the test directory that just maybe there's a configure option to use libutf8 instead of the libc defaults. Is that just for the test programs? Maybe I should try it. Then if it works try it with my test program, and then dvtm if that works. Unfortunately I think it's a hidden configure option. configure --help=short gives no clue how to change it.
My list of hacks was getting sorta big, so I decided to try libutf8 to replace the whole pile of crap. It's kinda big though at 160K, and the easier to use shared lib LD_PRELOAD plug version wouldn't link in my scratchbox toolchain. So in the end, all I can say is that I'm still working on on it...
Someday when I get things together I'll post the updated fonts, and maybe an new dvtm-wide.
Wednesday, February 22, 2012
Remix
With the tiny mcurses build of ttyclock under my belt, I set my sights on some larger targets to mcursify. First up was the alsamixer. Even with the salsa rebuild it's still quite large due to the big fat lump of static ncurses embedded in the executable. I added a few more stub functions to mcurses and made a new 60K (before upx) alsamixer for a savings of 100K. Not bad, if only it worked. Segfault! Rather than wade through the voluminous verbiage of alsa goodness, searching for the offending mcurses function, I decided to look elsewhere for perhaps a smaller starter mixer. Turns out there's not much out there and most of it predates ALSA, using the earlier OSS mixer controls.
I like the simple compact look of rexima though. So I built a 19K mcurses version, hoping it might just work. And it did. Sorta. It came up with only three controls displayed: PCM, PCM2, and IGAIN. WTF? I twiddled them anyhow. PCM is what it says, the mostly useless master volume control. PCM2 controled the headphone volume, and I think IGAIN handled the volume for weird zipit microphone input. With no way to control the tiny zipit speaker, this wasn't gonna work. Fortunately I went to the net and discovered these are just the default mappings from the emulated OSS mixer to ALSA mixer controls, and you can override them by writing the correct settings to /proc/asound/card0/oss_mixer in your boot up scripts. On IZ2S and it's derivatives, that place would be at the end of the setup-alsa.sh script. So I whipped up some changes and settled on the handful of controls I'm likely to use. I also squeezed the rexima display down to zipit size, and added a splash of mcurses color.
Once I had the emulated OSS mixer under control I thought it might be fun to dig through the old curses music players. Who knows what sort of nuggets might be in that pile. I decided to try building the orpheus music player because it claimed make use of mpg123, which is included in IZ2S, and it mentioned internet radio support. It's internationalized C++ so it's probably too bloated for the jffs. Because of that I went with a regular ncurses build, after tweaking the screen layout and key bindings with the usual zipit fixes. It looks ok, and it seems to work, but I don't think it's really what I want. It did however prove that OSS mixer based programs can be viable on IZ2S.
That's orpheus up top working its way through a funky Eddie Hazel playlist, with rexima mixin it up down below in the bottom dxvt window.
Here's a 21K rexima executable with source (including mcurses) and a modified IZ2S setup-alsa.sh. You can borrow the new oss settings from the bottom of the alsa script if you want to use this on the jffs.
I like the simple compact look of rexima though. So I built a 19K mcurses version, hoping it might just work. And it did. Sorta. It came up with only three controls displayed: PCM, PCM2, and IGAIN. WTF? I twiddled them anyhow. PCM is what it says, the mostly useless master volume control. PCM2 controled the headphone volume, and I think IGAIN handled the volume for weird zipit microphone input. With no way to control the tiny zipit speaker, this wasn't gonna work. Fortunately I went to the net and discovered these are just the default mappings from the emulated OSS mixer to ALSA mixer controls, and you can override them by writing the correct settings to /proc/asound/card0/oss_mixer in your boot up scripts. On IZ2S and it's derivatives, that place would be at the end of the setup-alsa.sh script. So I whipped up some changes and settled on the handful of controls I'm likely to use. I also squeezed the rexima display down to zipit size, and added a splash of mcurses color.
Once I had the emulated OSS mixer under control I thought it might be fun to dig through the old curses music players. Who knows what sort of nuggets might be in that pile. I decided to try building the orpheus music player because it claimed make use of mpg123, which is included in IZ2S, and it mentioned internet radio support. It's internationalized C++ so it's probably too bloated for the jffs. Because of that I went with a regular ncurses build, after tweaking the screen layout and key bindings with the usual zipit fixes. It looks ok, and it seems to work, but I don't think it's really what I want. It did however prove that OSS mixer based programs can be viable on IZ2S.
Here's a 21K rexima executable with source (including mcurses) and a modified IZ2S setup-alsa.sh. You can borrow the new oss settings from the bottom of the alsa script if you want to use this on the jffs.
Monday, February 20, 2012
Curses!
My plans for a cheesy SDL_ttf console font hack are now officially on the backburner. Too many distractions. I'll get back to it real soon though, I swear!
The #zipit IRC channel is abuzz with all sort of openwrt chatter. It's too hard to ignore so I finally got off my butt, took inventory of my messy stack of Micro SD cards, and found one that worked well enough with uboot to load openwrt. Actually the uboot that came with the flashstock rescue system seems much more robust with the SD cards than I remember. Maybe it's fixed now. I'll have to triple check that inventory and see how many, if any, SD cards still fail to boot.
Anyhow, the recent openwrt system from projectgus had a busted wifi setup, so mozzwald posted his test tarball with working wifi and a few extra goodies like gmenu2x and gmu. It's quite shiny and seems to work pretty well, so I'm gonna have to play with it. I'm just not sure what to do next. My git foo is kinda week, so when I got confused trying to clone the code from the projectgus github I switched over to anarsouls instructions and finished the job based on that. I hope the resulting odd mix of git clone archives is ok. It's building right now on a slow USB drive hanging off my old PIII laptop, so it should be done cooking by sometime next week. Then I'll try to build an openwrt version of rockbox and see how that goes.
So what did I accomplish last week? Err, not much.
I was a bit bothered that the smallest a static ncurses program will get down to is about 120K. But the shared lib is over 400K, so I need to find at least 4 ncurses programs that I've just gotta have to make installing the shared lib worth it on the jffs. A 120k static executable for ttyclock seems excessive. So I went looking for a mini ncurses alternative to make a smaller clock. There are actually quite a few out there. I settled on mcurses because it came with a nice demo program and didn't need too much work to build the ttyclock. I added a fake "immediate" (as in unbuffered) mode version of the windowed curses functions and it. I shuffled the ttyclock code a teensy bit to make it work. Then I added a bunch of patches to add some features and fix the date format bug, plus make it default to ISO 8601 dates instead of whatever weird euro-format it was stuck on. The resulting executable is around 16K, which puts the static mcurses overhead at about 1/10th of the size you get with ncurses. I'd call that a win.
Here it is in borderless ISO-date mode, preparing to replace the old standby clock radio.
ttyclock-mcurses.zip
I also finished off the 5x8 font and repackaged dvtm along with the rxvt terminfo files it seems to want.
The rest of the goodies will be posted real soon...
The #zipit IRC channel is abuzz with all sort of openwrt chatter. It's too hard to ignore so I finally got off my butt, took inventory of my messy stack of Micro SD cards, and found one that worked well enough with uboot to load openwrt. Actually the uboot that came with the flashstock rescue system seems much more robust with the SD cards than I remember. Maybe it's fixed now. I'll have to triple check that inventory and see how many, if any, SD cards still fail to boot.
Anyhow, the recent openwrt system from projectgus had a busted wifi setup, so mozzwald posted his test tarball with working wifi and a few extra goodies like gmenu2x and gmu. It's quite shiny and seems to work pretty well, so I'm gonna have to play with it. I'm just not sure what to do next. My git foo is kinda week, so when I got confused trying to clone the code from the projectgus github I switched over to anarsouls instructions and finished the job based on that. I hope the resulting odd mix of git clone archives is ok. It's building right now on a slow USB drive hanging off my old PIII laptop, so it should be done cooking by sometime next week. Then I'll try to build an openwrt version of rockbox and see how that goes.
So what did I accomplish last week? Err, not much.
I was a bit bothered that the smallest a static ncurses program will get down to is about 120K. But the shared lib is over 400K, so I need to find at least 4 ncurses programs that I've just gotta have to make installing the shared lib worth it on the jffs. A 120k static executable for ttyclock seems excessive. So I went looking for a mini ncurses alternative to make a smaller clock. There are actually quite a few out there. I settled on mcurses because it came with a nice demo program and didn't need too much work to build the ttyclock. I added a fake "immediate" (as in unbuffered) mode version of the windowed curses functions and it. I shuffled the ttyclock code a teensy bit to make it work. Then I added a bunch of patches to add some features and fix the date format bug, plus make it default to ISO 8601 dates instead of whatever weird euro-format it was stuck on. The resulting executable is around 16K, which puts the static mcurses overhead at about 1/10th of the size you get with ncurses. I'd call that a win.
Here it is in borderless ISO-date mode, preparing to replace the old standby clock radio.
ttyclock-mcurses.zip
I also finished off the 5x8 font and repackaged dvtm along with the rxvt terminfo files it seems to want.
The rest of the goodies will be posted real soon...
Friday, February 10, 2012
Off Track
I was planning to hack up a cheesy console font version of SDL_ttf this week so I could build some bloat free versions of dgClock, the jlime fileselector, and possibly gmenu2x suitable for squeezing onto the stock zipit jffs. And then I was gonna attempt to build a skinny version of the retro the maemo flipclock. But that woulda involved lots of planning and testing, and whatnot. Instead I got distracted by some shiny new things.
I don't remember how, but I was reading something that led me to the dtach utility which led me to the dvtm virtual terminal manager. Or maybe it was the other way around. Perhaps I was reading about twin and that led me there. I don't know. Anyhow, if you can't give up the jffs space to run gnu screen, then maybe these two lighter utilites are a better fit.
To get a better idea of what dvtm was capable of I added the line characters to a few fonts that are smaller than the one true 6x10 font I've been using. Here's a shot of dvtm running with a 5x8 font from the ben nanonote mailing list archive. I also patched the lines into the atari4x8 font so at least I did something related to fonts...
As if this wasn't enough distraction from my planned activites... These days everyone and their dog seems to have some version of openwrt running on their zipit. I got hooked into their discussion on IRC because there's now a version of gmu for openwrt that fits on the openwrt rescue jffs. It's not as nice as mine because you have to crank it up to 412MHz to beat the skippies. But still, I should'a been done with my jffs version by now. Maybe I can still be first with a cool alarmclock radio gmu package.
Meanwhile, due to all the chatter on the zipit IRC channel about openwrt and it's mini versions of everything, I got motivated to try and build a WPA-only version of wpa_supplicant for the jffs. I didn't quite get there because that's rumored to come out around 50K, and I only got mine down to just over 100K. It prints a few ioctl warning messages but seems to work so far with the WPA settings on my cheepo home wifi router. I also got onto an open access point without a hitch, but that's probably cheating.
If you're daring and need the room on the jffs you can realize quite a savings over the 640k stock wpa_supplicant. I also built a smaller version of wpa_passphrase but I still don't know exactly what that does, so mine might just do nothing. In a pinch you can always boot off the SD card and use the full wpa_* utilities from EZ2S to connect to one of those "enterprise" wifi routers that you might see at work.
dvtm-dtach-iz2s.zip
wpa-jffs.zip
I don't remember how, but I was reading something that led me to the dtach utility which led me to the dvtm virtual terminal manager. Or maybe it was the other way around. Perhaps I was reading about twin and that led me there. I don't know. Anyhow, if you can't give up the jffs space to run gnu screen, then maybe these two lighter utilites are a better fit.
To get a better idea of what dvtm was capable of I added the line characters to a few fonts that are smaller than the one true 6x10 font I've been using. Here's a shot of dvtm running with a 5x8 font from the ben nanonote mailing list archive. I also patched the lines into the atari4x8 font so at least I did something related to fonts...
Meanwhile, due to all the chatter on the zipit IRC channel about openwrt and it's mini versions of everything, I got motivated to try and build a WPA-only version of wpa_supplicant for the jffs. I didn't quite get there because that's rumored to come out around 50K, and I only got mine down to just over 100K. It prints a few ioctl warning messages but seems to work so far with the WPA settings on my cheepo home wifi router. I also got onto an open access point without a hitch, but that's probably cheating.
If you're daring and need the room on the jffs you can realize quite a savings over the 640k stock wpa_supplicant. I also built a smaller version of wpa_passphrase but I still don't know exactly what that does, so mine might just do nothing. In a pinch you can always boot off the SD card and use the full wpa_* utilities from EZ2S to connect to one of those "enterprise" wifi routers that you might see at work.
dvtm-dtach-iz2s.zip
wpa-jffs.zip
Subscribe to:
Posts (Atom)