Sound advice - blog

Over the last month or so my collegues and I have been going through a small modernisation process at work. This involves picking up a few new pieces of software that themselves have no dependencies and including them into our software base (and our safety cases). Much of the work is still at a prototype stage, and the goals are varied (one is the production of a new HMI).

We don't pick up new technology very often, and whenever we do it tends to be what others might consider old technology. I spent the last two days adding a custom plugin interface to thttpd in order to provide RESTful (polling-based) XML data from our system without resorting to using CGI or third-party software containing more than a few thousand lines of C-equivalent code. If this part of the prototype pans out it should free me up in selecting solutions for the actual HMI side (it's always easier to say "use HTTP" instead of "link against this library, make sure you call the right init functions...."). As to whether it will pan out, that's a little up in the air. Even if it doesn't, it's gratifying to get SCADA and train control data into a form you can view easily in mozilla. A little XInclude and XSLT should be all that's required to put together a web page for simple management functions. Performance is reasonable, too. My initial CGI hack took .19 seconds for wget to retrieve a single datum. The plugin interface does it in approximately .01 seconds.

XML is making my life easier. We've already had an arms-length association with gnome's libxml2 and libxslt through the xsltproc program we use to generate some of our source code from XSLT templates. The relationship was close enough to require coverage in our last safety case, the success of which adds weight for its inclusion in our software proper. I've also been looking a little higher up the tool-chain and am definately considering the use of SVG to define shapes on schematic displays.

XForms is also a technology that's piqued my interest. Byron breifly compared and contrasted XForms 1.0 and WebForms 2.0 in a recent blog entry from the perspective of a web content provider. He mentions the promised inclusion of XForms in Mozilla, but also of interest to me is the recent announcement of inclusion in openoffice 2.0. These are positive steps for the standard and seem to imply that it is not too complicated to be usefully implemented.

In my business, though, I think the best I can hope for is to develop a forms capability "inspired by" XForms. There's no way I could pull in the mozilla technology stack as part of my HMI. The connection between XML Schema and XForms puts it even further from my roadmap with libxml2 only supporting XML Schema in limited ways. It seems that outside the windows world and the rarified air of w3.com many prefer Relax NG.

XForms itself obviously wouldn't solve my HMI problem. The problem is not well-defined, but does include aspects of forms entry, list handling, schematic (and iconic) data representations, and fast (non-polling subscription) updates. That combined with infinite configurability for project needs starts to look a little daunting if you stare it stright in the eye. The development of the HMI will continue to be an interesting and poorly-defined problem for the forseeable future with pressures and expectations beyond our capabilities, but I'm sure I'll get there with something useful in the end.

Benjamin

I just read this entry from one of the planet freedesktop bloggers.

It reminds me of an email sent around my place of work with a title something like "Can anyone tell me what this function does?". It came from real source code in our system at the time and contained a function at least a few hundred lines long. The very first line of the function was an unconditional "return 0;". Just as the author had, most readers spent a few minutes delving through the complex control flow of the function before looking at the code in in a linear fashion and seeing that first "the function does nothing" line.

Dead code could very well be the worst sin in software. I you want to keep past code around for future reference, use a revision control system. Don't clutter up the current codebase with historical oddities, especially when they appear at first to still be used.

After more than four months of very occasional half-hearted hacking TransactionSafe has reached version 0.4. This version is titled "Basic Register" or alternatively "I was too embarrassed to release yet another version which still didn't support basic transaction entry!".

This version includes a few interesting design points, although they may be obscured by a fairly rushed and stream-of-conciousness flow of python programming. As with previous versions I've based the user Load/Save paradigm around a commit-as-soon-as-you-can model. There is no save button because the file is always saved. Pity there is no undo. Well, that can come later ;)

Another feature is my attempt to use "processes" sort-of-modelled after the Erlang message passing system. Essentially, each object gets told when to execute by an evalaute scheduler. At that time their input is already there (as it has been passsed on by another process) and ready to be processed into output. I think I've been able to achieve a reasonable amount of decoupling between components using this model, although it is still highly-experimental (meaning non-obvious and hard to read).

Third on my list of novelties is the way the program interacts with the sqlite database. The database itself (with some transformations applied) is considered the model in my MVC, and I have walked a path of updating everything using the same inefficient simple algorithm rather than custom-coding update features. You can see this in the diference between the account tree (which is based on the micro-update model) and the register (which is the newer code based on a single update mechanism). The round-trip to the database occurs often (every time a single field is updated), and I even went so far as to use the python difflib to determine which parts of the gtk tree model that makes up my register to update. That is to say, the gtk tree model likes to be updated minimally so that the user is still in the same place they were in the tree before the update. I've taken the raw tree model from disk each time, and passed it through a diff algorithm filter to determine what that minimal update is without the need for custom code for inferring.

This version makes use of triggers for database updates, particularly to maintain certain database invariants such as "all transactions balance". I've waited this long to do it because for a long time I wasn't sure exactly how I was going to enforce that constraint. Now I maintain a Balance account that must always exist as transactionEntryId 0, and is referred to in every transaction at least once (multiple currencies each get their own balance entry). This meshes with the current user interface/database interaction paradigm nicely.

Now that the triggers are in place, it should be possible for other applications to touch the database without too much risk of trashing the underlying conceptual model. This is core to my intentions for this work and is a step up from the gnucash line of a closed file format under a open (but subject-to-change) C api deep inside the gnucash codebase.

This version is still based on a "core" financial data model. As yet it doesn't even include a default currency for accounts, let alone account types that might direct the register to call the Debit and Credit columsn something friendlier. I'm hoping that I'll get around to designing and demonstrating the use of multiple rdf schemas in the same SQL database at some point to allow expansion for various application needs (share price download could use another schema integrated into the same database, for example).

In the end this version is significant because it represents my second motivation point in the development of this software. My current list is:

1. Actually contribute some open source software, instead of just bitching (complete, v0.1)
2. Try not to be completely full of hot air by getting transaction entry working (complete, v0.4)
3. Make this program a usable alternative to Quicken for my own finances (todo)
4. Establish a small user-base (todo)
5. Establish a developer community (todo)
6. Profit! (todo. damn.)

Hopefully the energy required to overcome these peaks of resistence will not be too great :) At present I'm still just a little pissant who can't manage to play with any of the big boys and who has therefore unreasonbly set out on his own. The best I could hope for at present is to fracture an already anemic developer and user home and small business financial software community. <sigh/>

Benjamin.