Sound advice - blog

Tales from the homeworld

My current feeds

Sun, 2005-May-29

Symphony Operating System

For those who read planet.gnome.org even less frequently than I do[1], Nat Friedman has spotted a very neat little desktop mockup by the name of Symphony.

For those who are into user interface design, this looks quite neat. They've done away with pop-up menus on the desktop, replacing them with a desktop that becomes the menu. I suspect this will greatly improve the ability to navigate such menus. Users won't have to hold the mouse down while they do it. They also use the corners of the desktop as buttons to choose which menu is currently in play. Very neat. I'd like to see it running in practice.

Oooh, their development environment Orchestra sounds interesting, too. I wonder how will it will support REST principles, and how it will isolate the data of one user from that of another user...

Benjamin

[1] Actually, Nat is someone I currenly have a subscription to.

Sun, 2005-May-29

Dia

I've spent some of this afternoon playing with Dia. I have played with it before and found it wanting, but that was coming from a particular use case.

At work I've used Visio extensively, starting with the version created before Microsoft purchased the program and began integrating it with their office suite. As I've mentioned previously I use a public domain stencil set for authoring UML2 that I find useful in producing high-quality print documentation. When I used Dia coming from this perspective I found it very difficult to put together diagrams that were visually appealing in any reasonable amount of time.

Today I started from the perspective of using Dia as a software authoring tool, much like Visio's standard UML stencils are supposed to support but with my own flavour to it. Dia is able to do basic UML editing and because it saves to an XML file (compressed with gzip) it is possible to actually use the information you've created. Yay!

I created a couple of xsl stylesheets to transform a tiny restricted subset of Dia UML diagrams into a tiny restricted subset of RDF Schema. I intend to add to the supported set as I find a use for it, but for now I only support statements that indicate the existence of certain classes and of certain properties. I don't currently describe range, domain, or multipicity information in the RDFS, but this is only meant to be a rough scribble. Here's what I did:

  1. First, uncompress the dia diagram:
    $ gzip -dc foo.dia > foo.dia1
  2. Urrgh. That XML format looks terrible:

        <dia:object type="UML - Class" version="0" id="O9">
          <dia:attribute name="obj_pos">
            <dia:point val="20.6,3.55"/>
          </dia:attribute>
          <dia:attribute name="obj_bb">
            <dia:rectangle val="20.55,3.5;27,5.8"/>
          </dia:attribute>
    

    It's almost as bad as the one used by gnome's glade! I'm particularly averse to seeing "dia:attribute" entities when you could have used actual XML attributes and saved everyone a lot of typing. The other classic mistake they make is to assume that a consumer of the XML needs to be told what type to use for each attribute. The fact is that the type of a piece of data is the least of a consumer's worries. They have to decide where to put it on the screen, or which field to insert it into in their database. Seriously, if they know enough to use a particular attribute they'll know its type. Just drop it and save the bandwidth. Finally (and for no apparent reason) strings are bounded by hash (#) characters. I don't understand that at all :) Here's part of the xsl stylesheet I used to clean it up:

      <xsl:for-each select="@*"><xsl:copy/></xsl:for-each>
      <xsl:for-each select="dia:attribute[not(dia:composite)]">
        <xsl:choose>
          <xsl:when test="dia:string">
            <xsl:attribute name="{@name}">
              <xsl:value-of select="substring(*,2,string-length(*)-2)"/>
            </xsl:attribute>
          </xsl:when>
          <xsl:otherwise>
            <xsl:attribute name="{@name}">
              <xsl:value-of select="*/@val"/>
            </xsl:attribute>
          </xsl:otherwise>
        </xsl:choose>
      </xsl:for-each>
      <xsl:apply-templates select="node()">
        <xsl:with-param name="parent" select="$parent"/>
      </xsl:apply-templates>
    

    Ahh, greatly beautified:
    $ xsltproc normaliseDia.xsl foo.dia1 > foo.dia2
    <dia:object type="UML - Class" version="0" id="O9" obj_pos="20.6,3.55" obj_bb="20.55,3.5;27,5.8" elem_corner="20.6,3.55"...
    This brings the uncompressed byte count for my partular input file from in excess of 37k down to a little over 9k, although it only reduces the size of the compressed file by 30%. Most importantly, it is now much simpler to write the final stylesheet, because now I can get at all of those juicy attributes just by saying @obj_pos, and @obj_bb. If I had really been a cool kid I would probably have folded the "original" attributes of the object (type, version, id, etc) into the dia namespace while allowing other attributes to live in the null namespace.

  3. So now that is complete, the final stylesheet is nice and simple (I've only cut the actual stylesheet declaration, including namespace declaration):

    <xsl:template match="/">
    <rdf:RDF>
            <xsl:for-each select="//dia:object[@type='UML - Class']">
                    <xsl:variable name="classname" select="@name"/>
                    <rdfs:Class rdf:ID="{$classname}"/>
                    <xsl:for-each select="dia:object.attributes">
                    <rdfs:Property rdf:ID="{concat($classname,'.',@name)}"/>
                    </xsl:for-each>
            </xsl:for-each>
            <xsl:for-each select="//dia:object[@type='UML - Association']">
                    <rdfs:Property rdf:ID="{@name}"/>
            </xsl:for-each>
    </rdf:RDF>
    </xsl:template>
    

    Of course, it only does a simple job so far:
    $ xsltproc diaUMLtoRDFS.xsl foo.dia2 > foo.rdfs

    <rdf:RDF xmlns...>
      <rdfs:Class rdf:ID="Account"/>
      <rdfs:Property rdf:ID="Account.name"/>
      <rdfs:Class rdf:ID="NumericContext"/>
      <rdfs:Property rdf:ID="NumericContext.amountDenominator"/>
      <rdfs:Property rdf:ID="NumericContext.commodity"/>
    ...
    

My only problem now is that I don't really seem to be able to do anything much useful with the RDF schema, other than describe the structure of the data to humans which the original diagram does more intuitively. I do have a script which constructs an sqlite schema from rdfs, but I really don't have anything to validate the rdfs against. I don't have any program that will validate RDF data against the schema that I'm aware of. Perhaps there's something in the Java sphere I should look into.

The main point, though, is that Dia has proven a useful tool for a small class of problems. Schema information that can be most simply described in a graphical format and is compatible with Dia's way of doing things can viably be part of a software process.

I think this is important. I have already been heading down this path lately with XML files. Rather than trying to write code to describe a constrained problem space, I've been focusing on nailing down the characteristics of the space and putting them into a form that is human and machine readible (XML) but is also information-dense. The sparsity of actual information in some forms of code (particularly those dealing with processing of certain types of data) can lead to confusion as to what the actual pass/fail behaviour is. It can be hard to verify the coded form against a specification, and hard to reverse-enginer a specification from existing code. The XML approach allows a clear specification, from which I would typically generate rather than write the processing code. After that, hand-written code can pass that information on or process it in any appropriate way. That hand-written code is improved in density because the irrelevant rote parts have been removed out into the XML file.

So what this experiment with Dia means to me is that I have a second human- and machine- readible form to work with. This time it is in the form of a diagram, and part of a tool that appears to support some level of extension. I think this could improve the software process even more for these classes of problem.

Benjamin

Fri, 2005-May-20

Project Harmony

Everyone else has gotten their two cents in, so I had better too. I think Project Harmony as proposed is a good thing.

Novell won't back Java[1]. Redhat won't ship Mono. So where does this leave open source?

I think a lot of the current interest in open source didn't come from "Linux" per se, but tools such as Apache and Perl. They were free of cost. They fit with the needs of users. They filled the needs of users better than commercial equivalents, and it was clear that a commercial clone of either would fail.

Apache is still "The number one HTTP server on the Internet"[2], but I think open source is losing the language war. It's been years since exciting things were really happening that commercial software wasn't doing better.

C and C++ are dead languages[3], but they continue to be the foundation of most of the free software out there. The effort that's going into supporting that system of libraries is wasted, because the programmers of today don't really want to talk to you if you don't have garbage collection. They don't want to talk to you if you have a language as complicated as C++[4]. They don't want to talk to you if you can't make DOM and SAX XML models talk to each other. They don't want to talk to you if you can't tell them straight to their face which HTTP client code to use, or which interface specification to code to when they want to write a serverlet. We're playing silly buggers with ideas that should be dead to computer science, and the commercial players are lightyears ahead in delivering actual user functionality.

Perl's CPAN was a killer app when C++ was big. If you wanted an implementation of a particular feature you just picked it up. Now-a-days most such features are part of the standard library of your langauge. Why would you want another source? That's what we're missing.

We're still arguing about which languages we need to use to build core desktop components on! Heavens' sakes! When a programmer opens up the open source toolkit for the first time we can't give them an adequte place to start.

You could code in C, but that's like using punchcards for getting user functions in place. You could code in C++, but that's far too complicated. It won't give you access to basic functionality, nor will it provide an upgrade path for when the next big thing does arrive. Also, if you're not coding in C you have to think very hard about how your code will be reused. Chances are, if it's not in C you can forget about it... but if you do use C you'll spend the rest of your life writing and maintaining bindings for every language under the sun.

We used to be able to solve all our problems with the open source tools we had, but they were smaller problems back then and the productivity we demanded from ourselves was less. Now its hard to know where to start when implementing something as simple as an accounting system.

There are many reasons to get behind Mono. It has the makings of a great piece of technology for delivering user functions. Unfortunately, it has the legal stink surrounding it that comes from being a clone of commerical software from an company that makes money off IP lawyering. It seems that Java is currently unencumbered. Despite Sun's recent cosyness with Microsoft, they seem to be good citizens for now.

From my perspective we either have to pick one of the two horses, or invent a new language again. If we pick that third option we'd better be ready to make it fill needs that can't be filled by the current solutions, or it simply won't be swallowed. If we can't pick Mono, then Java has to be our launching point. Once we have a useful Java platform, then we can think about whether we want to stick with Java longer term or whether we want to fork in the same way Microsoft forked to create C#. Any way we go its a technical and legal minefield out there and we don't really have a 500 pound gorilla to clear the way for us.

I suppose I had better clearly state my position on python, perl and friends going forwards. To be frank I don't hold out much hope. Parrot seems to be stagnant. It has been "in the early phases of its implementation" for as long as I can recall, having had its 0.1 release in December 2001. Without a common virtual machine, I don't see Perl and Python talking to each other freely any time soon. If that virtual machine doesn't also run the code in your mythical compiler-checked uber-language replacement for C# and Java then they won't talk to each other at the library level either. As far as I am concerned it is the the lack of ability to write a library and know it will be able to be used everywhere its needed that is at the core of what I hate about working with open source tools right now. I don't care how easy it is to write in your personal favourite langauge, or how elegant it is. I just want to know whether I'll be spending the rest of my life writing and rewriting it just to end up with the same amount of funcationlity I started with. It's just as frustrating from the library user perspective, too. I'm forever looking at libraries and thinking "I need that functionaliy, I wonder if its available in a python binding?"[5]. I don't care how easy such a binding would be to create. I don't want to maintain it! The code's author probably doesn't want to maintain it either! There has to be a better way!

So... what's left?

I think we end up with three options for making open source software authoring bearable again:

  1. Pick a language, and have everyone use it
  2. Pick a calling convention (including contracts regarding garbage collection, mutability and the like), and have everyone use it
  3. Start from scratch and find another way for pieces of software to communicate with each other (such as http over the local desktop), and have everyone use it.

Given these options, I applaud project Harmony for making the attempt at positioning us with a reasonable starting point. I hope they succeed. Organisationlly, I trust Apache more than GNU with their existing classpath infrastructure to do the job and get it right. Apache have a lot of Java experience, and that includes dealing with through Java Community Process with Sun and other Java interests. On the technical level they seem to be taking a bigger-picture approach than classpath, also. I think they can get a wider base of people onto the same page than the Simultaneously, I think we should continue experimenting with alternative ways for software to communicate. Maybe we'll meet in the middle somewhere.

Benjamin

[1] Ok, that's a huge oversimplification

[2] Accoring to their website

[3] ... or dead langauges walking, on their way to the electric chair

[4] As a C++ programmer I thank the Java luminaries for removing features like operator override!

[5] I consider python the most productive language presently for writing software in the open source sphere. I don't think its a good language once you hit a few thousand lines of code... I need compile-time checks to function as a programmer.

Thu, 2005-May-19

How about local CGI?

Sharing libraries between applications so they can access common functionality is the old black. Talking to an application that emodies that functionality is the new black. It's failed before with ideas like CORBA, and looks to fail again with SOAP, but REST is so simple it almost has to work.

REST can work on the web in well-understood ways, but what about the desktop? I've pondered about this before, but I have a few more concrete points this time around.

The http URI scheme describes clearly how to interact with network services given a particular URL. The file scheme is similarly obvious, assuming you want to simply operate on the file you find. HTTP is currenly more capable, because you can extend it with CGI or servelets to serve non-file data in GET operations and to accept and respond to other HTTP commands in useful ways. File is not so extensible.

This is silly, really, because files are actually much more capable than http interfaces to other machines. You can decide locally which application to use to operate on a file. You can perform arbitrary PUT and GET operations with the file URI, but you know you'll always be dealing with static data rather than interacting with a program behind the URI.

How's this for a thought? Use the mime system to tell you which application to run for a particular URI pointing to the local file system. Drop the "net" part of a http URI for this purpose, so http:/home/benc/foo.zip becomes our URL. To look it up, we start the program we associate with HTTP on zip files and start a HTTP conversation with it. Using REST principles this URI probably tells us something about the ZIP file, perhaps its content.

Now comes the interesting part. Let's extend the URI to allow the HTTP zip program to return URI results from within the file. Let http:/home/benc/foo.zip/bar.txt refer to the bar.txt inside foo.zip's root directory. We download it via HTTP.

Now everyone who can speak this "local" dialect of HTTP can read zip files. I can read zip files from my web browser, or my email client. So long as they all know how to look up the mime type, start the responsible application, and connect to that application. A completely independent server-side application that offers the same interface for arj files can be written without having to share code with the zip implemenetation, and they can be plugged in without reference to each other to increase the functionality of programs they've never heard of.

The tricky bits are

  1. Deciding on a URI scheme.
    The use of http may cause unforseen problems
  2. Deciding how to connect to the program.
    Hopefully looking up the mime rules will be simple and standard for a given platform, but you also need to connect to the app you start. It could be done with pipes (in which case you would need a read pipe and a write pipe rather than the traditional bi-directional network socket), or via some other local IPC mechanism.
  3. Deciding how to connect to the program.
    Maybe the actual mime side of things is more contraversial than I'm making out. To be honest, my suggestion is not 100% in line with how web servers themselves would handle the situation. There are probably suggestions to be made and incorporated into the model if it is to be quite as functional as these beasties.
  4. Implementing it everywhere.
    If the functaionlity provided by this interface was so killer compared to what you could achieve across the board by reusing libraries I think it would catch on pretty quickly. Just think about the hours wasted in building language bindings for C- libraries these days. Microsoft want to get rid of the problem by having everyone agree on a common virtual machine and calling convention. Others see a problem with using Microsoft's common virtual machine ;) Perhaps HTTP is so universal that it would see adoption on the desktop, just as it has seen universal adoption on the Internet itself.

Oh, and don't forget to include HTTP subscription in the implementation ;)

Benjamin

Wed, 2005-May-18

We're hiring

The company I work for[1] is hiring

Benjamin

[1] I don't know. You tell me which page to link to...

Sun, 2005-May-15

Coding for the compiler

In my last column I wrote about deliberately architecting your code in such a way that the compiler makes your code more likely to be correct. I used an example lifted from Joel Spolksy regarding "safe" and "unsafe" strings. In indicated they should be made different types so the that the compiler is aware of and can verify that the rules associated with assining one to the other or using one in place of the other are followed. I thought I'd bring up a few more examples of coding conventions I follow to use the compiler to best effect.

Don't initialise variables unless you have a real value for them

Junior programmers are often tempted to initialise every variable that they declare, and do so immediately. They do so to avoid messy compiler warnings, or because they think they are making the code cleaner and safer by providing a value where there would otherwise be no value.

That's a bad idea. Whether it be initialising an integer to zero or a pointer to null whenever you give a value to a variable you are lying to your tools. You're giving a value to the variable that no use to your program, but at the same time you are quelling the concerns of both your compiler and any memory analyser you might be using (such as purify).

Don't declare variables until you have a real value for them

This rule answers the question about how you deal with the warnings that would come from not assigning values to your variables. The simple pseudo-code you should always follow when declaring a variable is either one of:

MyType myVariable(myValue);

or

MyType myVariable;
if (condition)
{
	myVariable = myValue1;
}
else
{
	myVariable = myValue2;
}

Any other form is pretty-well flawed. In the first case, the variable is always assigned to a useful value, and it happens immediately. In the second case, every branch must make a concious decision about what to set your variable to. Sometimes variables come in pairs, or other tuples that need to be initialised together. Sometimes your conditions and branching are much more complicated than in this case. By not initalising the varaible up front and initialising in every branch you allow the compiler to find any cases you might have missed. If you set an initial value for the variable before a conditional that may assign values to it you stop the compiler complaining about branches where the variable is not assigned to, and you allow the introduction of bugs through unconsidered branches.

Always declare your else branch

Every "if" has an implicit "else". You need to consider what happens in that case. The variable assignment structure I've laid out forces you to declare your else branch(es) with the content you might otherwise have placed in the variable's declaration. This is a good thing. Considering all branches is a critial part of writing good code, and it is better to be in the habit of doing it than in the habit of not. Even if you don't have content for one side of the branch or the other, place a comment to that effect in the branch. Explain why. This point won't help the compiler find bugs in your code, but it is good to keep in mind.

Only assign to "external" variables once per loop iteration

This is a bit hard to write down as a hard and fast rule, but it is another one of those variable initialisation issues. Sometimes you'll have a loop which changes a variable declared outside of the loop. You'll be tempted to add one or subtract one at various points in the loop, but it's the wrong approach if you want help from the compiler. Instead, declare a variable which embodies the effect on the external variable. Use the same technique as I've outlined above to allow the compiler to check you've considered all cases. At the end of the conditionals that make up the body of your loop, apply the effect by assignment, addition, or whatever operation you require.

string::size_type pos;
do
{
	string::size_type comma(myString.find(',',pos));
	string::size_type newpos;
	if (comma == string::npos)
	{
		// None found
		newpos = string::npos;
	}
	else
	{
		myString[comma] = ';';
		newpos = comma+1;
	}
	pos = newpos;
}
while (pos != string::npos);

On the subject of loops...

Use a single exit point for loops and functions

When you have a single exit point, your code is more symmetrical and that tends to support the treatement of variables and cases that I'm suggesting. What you're looking for is...

  1. To make sure you clearly enumerate and consider every case your function has to deal with, and
  2. To give the compiler every opportunity to spot effects you may not have specified for each case

It is these principles at the core of my general mistrust of exceptions as a general unusual case-handling mechanism. I prefer explicit handling of those cases. I won't go over my position on exceptions again just yet... suffice as to say they're certianly evil under C++ and I'm still forming an opinion as to whether the compiler-support exceptions recieve under Java is sufficient to make them non-evil in that context :). Give me a few more years to make up my mind.

Under C++, I consider the following loop constructs to be useful:

Pre-tested loops

while (condition) { /* body */ }
for (
	iterator it=begin();
	it!=end();
	++it
	) { /* body */ }
for (
	iterator it=begin();
	it!=end() && other-condition;
	++it
	) { /* body */ }

Mid-tested loops

for (;;) { /* body */; if (condition) break; /* body*/ }

Post-testd loops

do { /* body */ } while (condition);
for (;;) { /* body */; if (loop-scoped-condition) break; }

Don't use default branches in your switch statements

This goes back to general principle of making all your cases explicit. When you change the list of conditions (ie. the enumeration behind a switch statement) the compiler will be able to warn you of cases you haven't covered. This isn't the case when you have a default. In those cases you'll have to find a way to search for each switch statement based on the enumeration and assess whether or not to create a new branch for it.

I'm happy for multiple enumeration value to fold down to a single piece of code, for example:

switch (myVariable)
{
case A:
	DoSomething1();
	break;
case B:
	DoSomething2();
	break;
case C:
case D:
case E:
case F:
	// Do nothing!
	break;
}

When you add "G" to the condition the compiler will prompt you with a warning that you need to consider that case for this switch statement.

You have a typing and classing system, so use it!

This is the point of my previous entry. Just because two things have the same bit representation doesn't mean they're compatible. Declare different types for the two and state exactly what operations you can do to each. Don't get caught assigning a 64-bit integer into a 32-bit slot without range checking and explicit code ok-ing the conversion. Wrap each up in a class that stops you doing stupid things with them.

Use class inheritence sparingly

By class inheritence I mean inheritence of class B from class A, where A has some code. In Java this is called extension. It is usually safe to say that class B inherits interface A, but not class A. When you inherit from a class instead of an interface you are aliasing two concepts:

  1. What you can do with it
  2. How it works

Sharing what you can do with an object between multiple classes is the cornerstone of Object-Oriented design. Sharing how classes work is something I don't think we have a syntactically simple solution to, even after all these years. Class inheritence is one idea that requires little typing, but it leads to finding out late in the piece that what you want to do with two classes is the same, but their implementation needs to be more different than you expected. At that point it takes a lot more typing to fix the problem.

So declare interfaces and inherit from interfaces where possible instead of from classes. Contain objects, and delegate your functions to them explicitly ahead of inheriting from ther classes. Sometimes class inheritence will be the right thing, but in my experience that's more like five cases out of every hundred.

I've strayed off the compiler-bug-detection path a little, so I'll finish up with two points.

Use the maximum useful warning settings

They're there for a reason

Treat warnings as errors

There's nothing worse than seeing 100 warnings when you compile a single piece of code, and have to try and judge whether one of them is due to your change or not. Warnings become useless if they are not addressed immediately, and they are your compiler's best tool for telling you you've missed something. Don't think you're smarter than the machine. Don't write clever code. Dumb your code down to a point the compiler really does understand what you're writing. It will help you. It'll will help other coders. Think about it.

Benjamin

Sat, 2005-May-14

Hungarian Notation Revisited

I don't subscribe to Joel Spolksy's blog Joel On Software, but I am subscribed to the blog of Sean McGrath. He has interesting things to say that differ from my opionions often enough to be a source of interest. This week he linked to Joel's article revisiting Hungarian notation, a very interesting read in which he dispells the notion that hungarian was ever meant to convey ideas like "int" and "double". Instead, he revisits the early incarnations of this idea and uses it to convey concepts such as "safe", "unsafe", "row identifier", and "column identifier".

I like Sean's python-centric view of the world, but I disagree with it. Joel's article focuses on a few examples, one of which is ensuring that all web form input is properly escaped before presenting it back to the user as part of dynamic content. He talks about using a "us" prefix for unsafe string variables and functions that deal with strings that haven't yet been escaped. He uses a "s" prefix for safe strings that have been escaped. Sean takes this to mean that all typing is poorly construed and that python's way is right. He ponders that maybe we should be using this "classic" hungarian to specify the types of variables and functions so that programmers can verify they are correct on a single line without referring to other pieces of code.

Personally, I take the opposing view. I don't think that programmers should be the ones who have to check every line to make sure it looks right. That is the job of a compiler.

My view is that Joel is wrong to use a naming convention to separate objects of the same "type" (string) into two "kinds" (safe, and unsafe). That is exactly the reason strongly-typed object-oriented languages exist. By creating a safe-string and an unsafe-string class it is possible for the compiler to do all of the manual checking Joel implies in his article.

I see python's lack of ability to state the expectations a paritcular function has on its parameters as flawed for "real work". It may be that the classic strongly-typed OO Language constructs aren't the right way to do it, but it is clear that expectations do exist of a functions parameters. The function expects to be able to perform certain operations on them, and as yet python doesn't have a way for a compiler or interpreter to analyse those expectations and see they are met.

Joel lists four levels of maturity for programmers in a particular target language.

  1. You don't know clean from unclean.
  2. You have a superficial idea of cleanliness, mostly at the level of conformance to coding conventions.
  3. You start to smell subtle hints of uncleanliness beneath the surface and they bug you enough to reach out and fix the code.
  4. You deliberately architect your code in such a way that your nose for uncleanliness makes your code more likely to be correct.

I would add a fifth:
You deliberately architect your code in such a way that the compiler makes your code more likely to be correct.

Joel uses hungarian to train his level four code smells nose. I use types to do the same thing. I think my way is better when the capability exists in the language. In python, it seems like the use of Hungarian Notation fills a real need.

Benjamin

Sun, 2005-May-08

Gnome 2.10 Sticky Notes

According to the Gnome 2.10 "what's new":

sticky notes now stay on top other windows, so you can't lose them. To hide the notes, simply use the applet's right-click menu.

So now sticky notes have two modes:

  1. In my way, or
  2. Out of sight, out of mind

I do not consider this a feature. If you are using sticky notes with Gnome 2.8 or earlier, I do not recommend upgrading!

Update: I've placed a comment on the existing bug in Gnome bugzilla.

Benjamin

Sun, 2005-May-08

Describing REST Web Services

There has been some activity lately on trying to describe REST web services. I don't mean explain, people seem to be getting the REST message... but to describe them so that software can understand how to use them. I saw this piece during early april. The idea is to replace the SOAP-centric WSDL with a REST-centric language. Essentially it describes how to take a number of parameters and construct a URI from them. Here is their example:

<bookmark-service xmlns="http://example.com/documentation/service">
	<recent>http://example.com/{user}/bookmarks/"</recent>
	<all>http://example.com/{user}/bookmarks/all"</all>
	<by-tag>http://example.com/{user}/bookmarks/tags/{tag}/"</by-tag>
	<by-date>http://example.com/{user}/bookmarks/date/{Y}/{M}/"</by-date>
	<tags>http://example.com/{user}/config/"</tags>
</bookmark-service>

Frankly, I don't agree. I've had some time to think about this, and I am now prepared to call it "damn foolish". This is not the way URIs are supposed to be handled. Note the by-date URI, which has an infinite number of possible combinations (of which some finite number will be legal, for the pedants). Look at the implicit knowledge required of which user parameters and which tag parameters are legal to use. It's rubbish.

There are two things you need to do to make things right. First off, there is a part of the URI which is intended to be constructed by the user agent rather than the server. This part is unremarkably called the query part, and exists after any question mark and before any fragment part of a URI reference. I consider this part to be the only appropriate space for creating URIs that are subject to possibly infinite variation. The {Y} (year) and {M} (month) parameters must follow a question-mark, as must {tag}.

The second thing you need to do is use RDF to describe the remaining (finite) set of URIs. Somewhere at the top of the URI path hierarchy you create a resource that lists or allows query over the set of users. If that's not appropriate, you give your user an explicit URI to their representation in the URI space. I, for example, might reside at "http://example.com/fuzzyBSc". At that location you place a resource that explains the rest of the hierarchy, namely:

<bookmark-service
	xmlns="http://example.com/documentation/service"
	rdf:about="http://example.com/fuzzyBSc"
	>
	<recent rdf:resource="http://example.com/fuzzyBSc/bookmarks/"/>
	<all rdf:resource="http://example.com/fuzzyBSc/bookmarks/all"/>
	<by-tag rdf:resource="http://example.com/fuzzyBSc/bookmarks/tags/"/>
	<by-date rdf:resource="http://example.com/fuzzyBSc/bookmarks/date/"/>
	<tags rdf:resource="http://example.com/fuzzyBSc/config/"/>
</bookmark-service>

Separately, and tied to the rdf vocabulary at "http://example.com/documentation/service" you describe how to construct queries for the by-tag and by-date resources. I don't even mind if you use the braces notation to do that (although clearly more information than just the name this method provides will be required, for example a date type wouldn't go astray!):

<URIQueryConstruction>
	<!-- tag={tag} -->
	<Param name="tag" type="xsd:string">
</URIQueryConstruction>
<URIQueryConstruction>
	<!-- Y={Y}&M={M} -->
	<Param name="Y" type="xsd:integer">
	<Param name="M" type="xsd:integer">
</URIQueryConstruction>

Alternately, you make http://example.com/fuzzyBSc/bookmarks/tags/ contain a resource that represents all possible sub-URIs. Again, this would be a simple RDF model presumably encoded as an RDF/XML document. Another alternative again would be to incorporate the tags model into the model at "http://example.com/fuzzyBSc/".

So I know that http://example.com/fuzzyBSc/bookmarks/tags/?tag=shopping (or http://example.com/fuzzyBSc/bookmarks/tags/shopping at the implementor's discretion) will lead me to shopping-related bookmarks owned by me, and that http://example.com/fuzzyBSc/bookmarks/date/?Y=2005&M=05 will fetch me this month's bookmarks.

Personally, I think that some REST proponents are confusing (finite) regular URI and (infinite) query URI spaces too much. There are important distinctions between them, particularly that you can deal with finite spaces without needing client applications to construct URIs using implicit knowledge that might be compiled into a program based on an old service description or explicit knowledge gleaned from the site.

The RDF approach (when available) allows sites to vary the construction of these finite URIs from each other so long as they maintain the same RDF vocabularies to explain to clients which resources serve their specific purposes. The substitution approach must always follow the same construciton or a superset of a common construction. This is probably not so important for a very specific case like this bookmark service, but may be important for topics of wider interest such as "how do I find the best air fare deal?". Using the RDF approach allows you to navigate using a very general scheme with few constraints on URI construction to sites that require detailed knowledge to be shared between client and server to complete an operation.

Update: A link summarising various current attempts

Benjamin