Tuesday, January 16, 2018

Long Time No Talk

I haven't done much worth writing about lately.  Or maybe I've just simply been too busy doing other things.  I don't know...  So I'm just gonna jot down a few brief notes for now to keep this zombie blog twitching.

On the zipit IRC we recently started tinkering with Devuan.  So far, it feels pretty similar to the old Arch distro from two years ago, except you have to mount the SD card on your PC in a qemu chroot to fetch packages.  That's not as difficult as it sounds, so it's not huge burden.  I noticed that several of the executables I built for Arch, like qemacs and gmu, actually work on Devuan as is.  Probably because they're both based on glibc.  Anyhow, that's handy.  But I'm still getting up to speed compiling things in the Devuan chroot.  So far I've built a working version of the ziptuner, but not much else.

Meanwhile there's a few new tidbits for the bleeding edge openwrt.  Here's a picture of the new wavemon package running on the zipit with the 5x8 font. 


The Fn keys shown on the botton don't exist on the zipit, but you can still get to the other screens by pressing the first letter of the screen name.  So the keyboard support seems good enough for me on the zipit.  However, a few strings in the messages could possibly be shortened to make it all fit.  Maybe I'll whip up a patch for that.  I'd also like to make a patch to build it with libnl-tiny to make it smaller, but that's not really a priority

What else is there?  About the only bits I remember are a small update to the ebindkeys package to make the stop button work with the default gmu shift-esc key combo.  Maybe I did some more work on the unfinished stppc puzzles package?


Part 2.  The Long Lost Holiday Special.

As you can see above, somehow despite a personal record, due to procrastination, of 2.5 weeks off for Christmas and another full week for Thanksgiving, I managed to accomplish next to nothing during the holiday season of 2017.  Well, maybe I got caught up on some classic science fiction reading from the 50's, 60's, and 70's.  But nothing worth mentioning here.  So instead I decided to relive some former glory and attempt to recollect the events that transpired near the end of 2016.  Around that time I had a really nice burst of energy and enthusiasm that has gone undocumented here for far too long.  I need to sort it out and finally start tidying up some of the loose ends before I forget it all completely.

 At the very least I'd like to rediscover my recipe for building an iwmmxt enhanced mplayer executable for openwrt.  I'm almost 99% certain that a zipit running bleeding edge openwrt was publicly demonstrated to play video streams (with proper lipsync) that were located with the links browser through mozzwalds zipit friendly youtube search portal, then fetched with movegrab and piped into an iwmmxt enhanced mplayer for a seamless streaming experience.  For now I might be happy if I could simply recover enough notes to make an iwmmxt howtobuild.txt file and check it into the files directory of the openwrt mplayer github project.  Unfortunately it turns out the whole sequence that culminated in that public demo involved more cheating and hackery than I care to remember.  The bleeding edge movgrab package was even more challenging to put together than the iwmmxt enhanced mplayer, but we'll get to that later.

Anyhow, I'd also like figure out what sort of loose ends I left when I created the openwrt rockbox package around the same time, so can pick up the slack and clean it up.  Right now I'm thinking the battery monitor code was unfinished and maybe I need to borrow some eq files from the iz2s build?  This could turn out to be really important to me.  Here's why.  After 10 years in the same old car I finally gave up and got a new one.  Apparently the new car can play music off a usb stick, or an iphone, or whatever else you might have via the aux input jack.  But the software is a total mess.  The USB interface only recognizes mp3 files and presents them on screen in a confusing mishmash of unrecognizable numbered folders.  I mean who in the world thought that listing random numbers for folder names instead of the *actual* folder names from the USB stick was a good idea?  At least the auto maker Honda was nice enough to include an aux port, so the zipit is still golden if I can keep it running for another 10 years or so.  By then I expect to have a robot valet to drive my flying car and manage my music collection.

So, here's the story.  It started in the run up to Christmas 2016, when I managed to put out a few updates to various zipit goodies, mostly for the bleeding edge openwrt.  For example, at some point I made a bleeding edge tinyirc executable.  It's handy for a zipit internal flash installation, if for some reason you can't get an SD card going.  There's better IRC clients in the bleeding edge package repo if you're running from an SD card. Than in May 2016 I finally fixed a rather annoying "feature" of qemacs that had been driving me nuts for some time.  Normally, in full up gnu emacs, whenever you type a close bracket (be it round, squiggly, or square) the electric parenthesis matching function flashes the cursor briefly over the matching opening bracket and then returns to your current position.  I love that feature when developing code.  But the qemacs version of this was failing miserably on the return.  You can probably imagine the confusion that causes when you're feverishly coding, only to realize the text you're typing is inserting itself into the wrong place, and has been for quite a while -- ever since you closed your last parenthesis.  It was especially horrifying pasting a large block of code only to see it crumple in upon itself in slow motion, creating a total jumbled mess of nonsense to clean up.  I finally had enough and patched the code and uploaded a bleeding edge executable.  Eventually I remembered to update the github repositories too, so the current bleeding edge openwrt package should also contain the fix.  I probably ought to upload a new iz2s executable, or maybe even make an update the old uz2s image from 2015, when I find some time.

In October I picked up speed heading into the holiday season and compiled a zgps executable for bleeding edge, and also updated the pspmaps and navit executables for iz2s. Toward the end of October we got a request for opus codec support, so I built libopus and gmu10 to make use of it for the bleeding edge  openwrt.  These might be real packages at this point, but I can't remember.

As the holidays approached I determined to get the video player integrated into bleeding edge openwrt, just like it was in IZ2S and the older openwrt system based on uClibc.  I wanted to browse for music videos on mozzwald's youtube search portal and have them play in realtime, right from the links browser.  We had a working mplayer package, right?  So I figured we just needed to fix up some musl compile issues in the movgrab package and add some configuration glue to the links package.  How naive...

After fixing some minor compile issues with musl header files, I discovered movgrab had a problem with the musl implementation of select or poll.  Been there, done that with gpsd and such.  So I rewrote a FDselect wrapper function in the movgrab libUseful code to add a timeout. otherwise it would block for something like forever when printing out the available video formats, and when fetching the actual video stream url.  Or so I thought... Eventually I got to thinking it was a socket reuse problem in musl, and not actually another problem with select.

Unfortunately I never solved it.  Instead I compiled a cheezy static version movgrab on a zipit with the old openwrt.  That got it built with uclibc instead of musl.  But I didn't have the ssl or crypto libs and include files on the zipit where I did the compiling, so instead I built a really bare bones movgrab.  What a mess, but it seemed to work, even on the bleeding edge zipit. If you look real close at the bleeding edge movegrab package files on github you might notice the static movgrab executable in the files directory.  When you build the package, it goes to all the trouble to compile a musl executable and then slips in the old static executable instead.  :)  Wow.  I am a total cheater.

Sadly it longer seems to work with youtube, probably due to the missing ssl or whatever restrictions fell out of the recent Amazon Google spat over access rights.  But maybe I can finally spend some quality time now and fix up the musl build the right way.  Or else try to compile an up to date static build in the devuan chroot we've been using lately.  Hmm...possibilities.

Anyhow next up was the links config glue.  However while testing various implementations it was revealed that the lipsync was off.  The sound would play at normal speed, but the video would gradually fall further and further behind.  Apparently it was like this in the iz2s and the old openwrt.  We were so close, so we twiddled with all the obscure mplayer mplayer runtime config settings, but it could NOT be convinced to coax it into sync.  So we had to go for some compile
optimization, the tricky iwmmxt code buried deep in the guts of mplayer.

At first I thought, naively again, that it was simply a matter of fixing the openwrt mplayer package makefile to add the mplayer iwmmxt config setting.  But no...  nothing is ever so simple with the zipit.  Openwrt insisted on adding -march=armv5te and -mcpu=armv5te to the compiler switches, overriding my iwmmxt setting.

So I made a script to do a swap in the makefile and then a rebuild.  That almost worked.  But -- and this is where my memory is failing me -- I could swear that I also had some compile problems somewhere in that mess that forced me to turn off iwmmxt for a couple of files.  But I don't remember if I did that by editing the source code, or by recompiling them individually with the armv5te settings.  I also vaguely remember getting it to build and then having to debug some segfaults caused by the iwmmxt code.  Maybe that's what triggered me to rebuild some of it with the armv5te settings.  I need to dig this up from the IRC logs and from my openwrt build area so I can reproduce it.  Anyhow at some point I got it to build.  The lipsync worked, so I stashed the executable and finished up the glue code.

And here's the proof.

High on success, I buried all memory of my sins and moved onto rockbox in an insane attempt to turn that into a viable bleeding edge openwrt package.  Apparently rockbox also had a problem with the musl select function.  I wish I could remember the details of that fix.  I couldn't even remember if I built the plugins.  Anyhow, I got rockbox to work, and got it to build as an openwrt package from a much more recent rockbox source snapshot.  I also managed somehow to include all the plugins in that package.  But I believe the keymaps for those plugins were just some quick and dirty hacks to make them compile, and so they probably still need some serious work.  And there were some changes to the rockbox battery handler code that prevented the zipit battery detection from working. However I abandoned rockbox at this point, and moved on.  A year later, I couldn't remember why.  But most likely the reason I stopped was that openwrt was fighting me whenever I tried to recompile it with the standard openwrt trick:

  make package/rockbox/compile V=99

Normally that would build only the rockbox package, and give me some nice verbose compiler output for it.  But instead of the detailed compiler messages I needed, it was feeding me some useless error message that told me absolutely nothing.  This got to be very frustrating, so I moved on in March (without polishing up the loose ends) when I got a request to help integrate my ancient ldglite project with into LPub3D.


Part 3: Moving Forward


I now suspect my woes with the openwrt build system might've been caused by some extra directories like junk and gitfiles that I'd created inside the rockbox package feed directory to hold some temporary backups and test files.  Weird!  In retrospect, I'm thinking maybe gitfiles was a bad choice on my part. I gotta remember from now on to hide any backup or working directories in the files directory.  That seems to be a safe place to stash temporary junk without breaking the build.

Once the build system was working, I finally felt like I could make some progress again with rockbox.  So examined the android battery support code and fixed up the zipit battery code in a similar way to make it work once again.  Only a year or so late...  Anyhow, the rockbox package has finally been updated with a working battery monitor.  There are some hints in the android code that rockbox could be made to work off the system volume control and also possibly use the system headphone detection, so I may tinker with that.

I also want take another look at movgrab to try and match up socket closes with each and every open.  See if that makes it allocate new sockets properly in musl...  I might also add more calls to fflush(stdout) so musl doesn't block it on printf calls with no linefeed, kinda like I did once upon a time with tinyfiledialogs.

Meanwhile, there's also a few other odds and ends that I'd like to remember here if I can find the time.

I made handy setclock utility for times when I want to edit files with proper timestamps, but the network is down.

I compiled drumtoy but did not make it a full package yet.

I also compiled a very simple game, ativayeben as sort of a turkey day special.