Sound advice - blog

Tales from the homeworld

My current feeds

Sun, 2007-Feb-25

Remixing REST: Verbs and Interaction Patterns

I have been interested in the boundaries between classical object-orientation and REST for many years. This article attempts to explore the bounaries in one particular area. One of REST's core tenets is that of the uniform interface. Is the uniform interface as important as REST suggests? Could it be done any differently?

A significant proportion of the work that I do involves integrating software components or physical devices from different vendors into a single architecture. This usually involves writing a protocol converter for the purpose, often a one-off converter for a particular customer contract. Internally, we have needed to do this kind of thing less and less as we have embraced the REST style. Instead of inventing a new protocol or new IDL whenever we write a new application we have been tending for some time now to reuse an existing HTTP-derived protocol. We can then focus on document types. Do we need a new one, or will the one of the ones we have already in use do the job?

The need to limit verbs has long been a teaching of REST proponents, but the motivation isn't always abundantly clear. It seems we can look at the web and see that the nouns greatly outnumber the verbs, and see that the web seems to work well because of it. So let me have a go at coming up with a simple reasoning:

Ad hoc interoperation between two components of an architecture relies on those components being capable of participating in a particular common interaction pattern. An interaction pattern between a client and a server involves one or more request messages being sent from the client to the server, and one or more response messages being to the client. Today's Web constrains the interactions to one request and one response per interaction. The interaction is decomposed into request verb and document type, and the response verb and document type. Headers are sometimes also important parts of the interaction.

In traditional Object-Orientation we are used to writing code every time we write a new class or interface. We write new code to implement the classes, and write new code to interact with the classes. Two objects are unlikely to interoperate unless we plan for that interoperation. Interface classes and design patterns can help us decouple classes from each other, however we must still typically design and choose an interaction pattern for a specific functional purpose.

This is all well and good when we control both end-points of the conversation, or when the interface is encapsulated in an industry standard such as the servlet interface. However the Web introduces a broader problem set. We start to need an interface that decouples components from each other, even though they belong to different industries. We need standards that are more generic, lest we have to start writing new browser code every time a web site is added to the Internet.

Let's inspect the Web interaction pattern some more. We have roughly four to eight verbs to work with, with about... urgh... forty-three response verbs. That gives you around 172-344 possible request/response interactions on the web. You also need to multiply that out by the number of content types, so theoretically we have thousands or even tens of thousands of possible interations happening on the Web. That's probably too many.

In practice only a few response verbs are used, and hopefully Waka will make some sort of headway in this respect. If we reduced the response verbs to their basic types we would be left with only about twenty important interaction patterns on the Web, and only clients get the really raw deal. A server needs to understand all of the possible requests that make sense, but doesn't need to undersand any response that it doesn't plan on using. A client should understand all of the requests that are meaningful based on its own specification, but should also understand all of the responses that might be returned to it.

If client and server both implement the request and response verbs that make sense and they both know how to exchange the same document types, they should be able to be configured rather than coded to work together. This is hugely important in big architecture, where it is rarely possible to influence the other side of a conversation into following your individual, corporate, or even industry-specific specifications.

To my mind the difference between design and architecture is one of configuration control. At one extreme we have design. Design is controlled by a single agency, and deployed with a single version number. You can construct a design in a very freeform way, because you test and deploy it as a unit. It makes sense to maintain rigid control over typing. You would rather find inconsistency problems at compile time than have to pick them up during testing.

Pure architecture is the other extreme. An archicture component is deployed as a single entity, but when it is upgraded none of the other architecture components are redeployed. Consistency is no longer a concern, and checking for consistency is extremely counterproductive. It is much more important to interoperate with a range of components an component versions built and deployed by different agencies.

In the middle of these two extremes is a kind of half-design, half-architecture scenario. I'll call it system design. You might version or deploy different components of a system separately, but you own all of the components can do a big upgrade if you need to. System design has characteristics of both design and of architecture. Like architecture, you want to avoid enforcing consistency at build time between components. They might be deployed against various versions of the other components. Like design, you can add special interactions and local conventions. You control both ends of the conversation, so can be sure that your special conventions will be understood correctly.

Another way to look at system design is as a sub-architecture. Your system may participate in a wider architecture over which you have no control, in a smaller architecture over which you have some control, and yet another in which you have significant or total control. The ideal implementation of these architectures would use the interactions that are standard in the widest architecture whenever they are applicable, then scale down to specifics as special semantics are required. An example of this might be to use a HTTP GET request whenever a client wants to retrieve any kind of data from a server, but still allow special interactions such as LoadConfiguration when nothing from the HTTP sphere is a good match.

The widest possible architecture today is the Web, making HTTP and its methods hard to ignore. It seems they should be the defacto standard whenever they are appropriate. However, SOAP appears to be solving real-world problems in smaller architectures or designs today. The two are clearly not compatible on the wire, however gateways between the two protocols may be viable when a WS-*-based architecture facilitates interactions that can be cleanly mapped to HTTP. Two approaches are possible to create a mapping in a WS-* architecture. You could define a WSDL that covered HTTP-compatible interaction, or you could construct individual WSDL to deal with each interaction that HTTP supports. Given these interfaces it would be straightforward for components of the WS-* architecture to also participate in the wider architecture.

While gateways are a short-term technique that can be used to bring these architectures closer together, they don't really solve the longer-term issues. We should be prepared to identify a longer-term objective that allows the needs of both architectures to be met with a single technology set. This could be achieved by starting out every conversation as HTTP, but quickly upgrading to a more sophisticated protocol whenever it is supported. Fielding has suggested this will be a technique used by his Waka protocol, and it could likewise be adopted for a HTTP-compatible SOAP mechanism. However with both WAKA and SOAP the advantages of the new protocol would have to significantly outweigh the costs of effectively replacing the architecture of the Web. I see any such protocol as spending decades incubating in the enterprises of this world before they become remotely important components of the actual Web.