Sound Advice - patterns

Tales from the homeworld

My current feeds

Published: Mon Jul 14 21:18:08 EST 2008

Updated: Sat Nov 15 17:45:06 EST 2008

Guide

Architecture Control Patterns

Basic Transfer Patterns

Transfer directionWho knows when to transfer?Applicable Transfer Pattern
Server -> ClientClientGET
Server -> ClientServerCyclic GET, or Subscribe
Client -> ServerClientPUT

Advanced Transfer Patterns

Content Patterns

Structural Patterns

Published: Tue May 3 20:35:23 EST 2011

Updated: Wed May 4 08:01:20 EST 2011

Delta Encoding

Intent

Keep one or more clients up to date with the state of large resource.

Motivation

Use case 1: Software updates

A server has a program that it wants to distribute to its clients. After an initial installation it hopes to be able to keep its clients up to date for minimal cost. It wants to send each client only the differences that client needs in order for the client to complete its update.

Different clients may be up to different revisions of the software, meaning that different updates may be needed for different clients. The server does not wish to keep track of individual clients and where they are up to, and does want to make effective use of caching infrastructure.

Some clients will be too far out of date, and it won't be worth supplying the patch updates to them. For these clients the server wants to redistribute the whole of the new program version as an update.

Use case 2: Publishing a log

A server is keeping a log of access to a large software system. This server is one of a collection of many thousand that are all generating logs that must be analysed centrally. At various points in the architecture there are servers that aggregate logs for servers that are nearby, and transmit the aggregated logs to several secure locations where the logs can be viewed and interrogated.

Applicability

Delta Encoding is useful wherever it is likely to save time, save processing, save bandwidth, or better inform consumers about changes to a large resource to send updates to that resource instead of sending the whole representation each time it changes.

It depends on clients that know what they want to fetch, and when, and who can follow a simple algorithm to bring themselves up to date and stay up to date. It is friendly to caching, and supports the normal features available to GET requests such as content negotiation.

Structure

Delta Encoding pattern structure

Participants

Client
  • Performs an initial FetchMain
  • Records the time reference returned as part of the normal fetch response
  • Whenever it wants to be brought up to date:
    • Performs a FetchDelta, specifying its current time reference
    • Records the time reference of the response
    • Repeats this process until a NoDeltaYet response is returned
  • If the client receives a DeltaGone response to a delta fetch invokes an appropriate recovery mechanism, such as invoking FetchMain()
Server
  • Tracks changes that occur to the main resource as they occur, or is otherwise able to generate a delta for the resource from a known point in time
  • Provides a time reference in the FetchMain() response that can be used to bring clients up to date from this point in time
  • Provides a time reference in the FetchDelta() response that can be used to bring clients up to date from this point in time
  • Returns NoDeltaYet for FetchDelta() calls for the next time reference until new data is produced
  • For each FetchDelta() call old returns the difference from the specified point of time to the current point in time for the main URL
  • If the server is unable to provide deltas indefinitely far back in time, then return DeltaGone for any FetchDelta calls that nominate a time reference for which the server does not wish to produce deltas

Collaboration

Delta Encoding sequence
  • Client issues a series of requests to Server to bring itself up to date, until it sees a NoDeltaYet or a DeltaGone response.
  • Server generally brings client up to date in one request, providing a full delta from the point in time indicated in the delta URL to the current time
  • If the client fetches a delta that has no data yet, Server responds with NoDeltaYet
  • If the client fetches a delta that can no longer be generated, Server responds with DeltaGone
  • On seeing a DeltaGone response for a delta the client takes corrective action, such as re-fetching the main URL

Consequences

Although a delta encoding protocol exists for use in conjunction with HTTP it is geared towards keeping a cache up to date with a particular representation of a resource. This approach for delta encoding specified in the Implementation section below allows a more semantic differences model to be applied. Origin resources can be of any media type, and content negotiation is supported. Deltas too can be of any media type and support content negotiation. Deltas do not need to be taken by all clients as literal instructions on how to modify the current resource representation to bring it up to date. Instead, they can be taken as a list of changes to the main resource that can be processed independently of the whole main resource state for so long as fetches to delta resources do not result in a DeltaGone response.

Caching proxies may exist between client and server. This mechanism works well with caching proxies, including those that are not aware of the delta encoding pattern in use. The main resource and each of the delta resources can be queried and cached independently by multiple layers, and deltas can be distributed through a cache hierarchy for so long as clients are reasonably likely to be querying from the same point in time or a small collection of different points in time.

If the server provides similar cache control metadata for delta URLs as it does for the main URL then the client will generally be brought up to date within one request within the staleness allowed by those cache directives. However, if the server nominates a long max-age for deltas then clients will not be brought up to date just by issuing one request. For this reason the client continues fetching until it gets confirmation from the server that no more updates are available.

This pattern is stateless. It does not keep track of any individual clients, and instead tracks a series of legal time references and hands these out to clients. Each client is responsible for storing the time reference that it is up to. The amount of storage space required on the server is proportional to the size of its delta buffer, plus the number of concurrent requests being made at any given time. The total number of consumers does not directly affect the amount of storage space required on the server.

Time references generally need to be shared across all servers across a cluster in order to allow clients to access deltas on each server.

Implementation

Delta Encoding can be implemented with HTTP using the following mappings:

FetchMain
GET main URL HTTP/1.1
MainSuccess(representation, time reference)
HTTP/1.1 200 OK
Content-Type: representation media type
Link: <next delta URL, including time reference>; rel="Delta"

representation bytes
GetDelta
GET delta URL, including time reference HTTP/1.1
DeltaSuccess(representation, time reference)
HTTP/1.1 200 OK
Content-Type: representation media type
Link: <next delta URL, including time reference>; rel="Next"

representation bytes
NoDeltaYet()
HTTP/1.1 204 No Content
DeltaGone
HTTP/1.1 410 Gone

Sample Code

if (nextDelta == null)
{
fetch_main:
	Request mainRequest;
	mainRequest.url="http://example.com/log"
	Response mainResponse = fetch(mainRequest);
	nextDelta = mainResponse.links["Delta"];
	processMain(mainResponse);
}
else
{
	Request deltaRequest(nextDelta);
fetch_another:
	Response deltaResponse = fetch(request);
	if (response.code == NoContent)
	{
		// Break out of loop
	}
	else if (response.code == Gone)
	{
		jump fetch_main;
	}
	else
	{
		processDelta(deltaResponse);
		jump fetch_another;
	}
}

Related Patterns

Published: Sun Sep 21 23:21:56 EST 2008

Updated: Sun Sep 21 23:21:55 EST 2008

Enqueue

Intent

Create a series of entries in a server-maintained set or queue.

Motivation

A business to business order processing system requires a reliable means of submitting, updating, and cancelling purchase orders. The system components at either end were developed by separate vendors to fit with each company's IT infrastructure. Both customer and supplier businesses require their side of the system to interoperate with other suppliers and with other customers they may have, respectively.

Each business upgrades their capabilities in line with their own business objectives. This often means that early adopters have moved on several times before the most conservative businesses make a jump to the latest approach and standards. In the mean-time, any given component on the server or client side must face an eclectic mix of counterparts and must interoperate with them correctly.

The Enqueue pattern is focused around the creation of server-side records in this message exchange. The PUT Pattern is used for subsequent updates and deletion requests.

Applicability

Enqueue is appropriate whenever the objective is creation of one or more pieces of server-side state based on information available to the client.

Structure

Enqueue pattern structure

Participants

Client
  • Keeps a base URL that provides context for the Create operation
  • Is capable of encoding its information in any form that it can legally and reasonably be encoded to.
  • Selects the most appropriate encoding based based on an initial guess. Subsequent requests are based on the on the supplied weighted acceptable types list if a Type Not Understood response is returned.
  • Issues the Create request
  • Is responsible for overall successful execution of the operation, including modifications to the request and resubmissions of the request
  • May or may not be able to determine whether the Create was successful or not when its response is lost, depending on the implementation
  • May or may not be able to control the order in which Create operations are processed, depending on the implementation
  • Aborts the operation on a failure response, on a resubmission response that cannot or will not be satisfied, or on a lost response after too many retries.
Server
  • Checks that the type of the document is understood before performing significant processing
  • Selects the kind of object to create based on the supplied base URL
  • (optional) Is configured with mechanism to require the client to resubmit their request with or without modifications
  • Allows the Create to operate initially
  • May provide a mechanism by which the Create operation is safe to repeat
  • May provide a mechanism by which a series of Create operations are guaranteed to be processed in order
  • Is capable of parsing all forms that the data might be encoded in that are semantically rich enough to use
  • Selects the right parser implementation to use based on the returned document type
  • Returns a Success response only once the creation can be considered permanent, allowing the client to forget it. The definition of permanent will depend on the possible consequences of client forgetfulness. It would typically range from "information updated on disk" to "information replicated to all sites, and stored to backup media" for important data.

Collaboration

  • Client issues requests to Server via the Request Interface, modifying and resubmitting its request as needed until:
    1. A success response is elicited
    2. A response is lost, and the client is unable to safely repeat its request
    3. The client is unable to make changes required by a Resubmit response
    4. Client policy prevents either changes required by a Resubmit response, or further resubmissions in general

Consequences

The Enqueue pattern creates a mechanism to create a series of new resources on the server side, but the safety and efficacy of the operation is dependent on a specific implementation.

The use of an acceptable types list in a Types Not Understood response means that clients and servers built during different phases of the architecture will generally be able to communicate. Document-based communication has a degree of flexibility built in with must-ignore parameters. The acceptable types list fills a gap when incompatible changes occur to the set of document types, for example a new type deprecates an old type such as atom depreciating rss for news feed syndication.

An explicit failure response allows problems in the architecture to be reported and repaired as required. The resubmit feature allows temporary or permanent changes to the architecture to be accommodated by components without explicit reconfiguration, simplifying management. Note, however, the potential security implications of allowing one component to reconfigure others. A predefined policy for which modifications are permitted and which are to be treated as failure cases can be useful in security-sensitive environments. Incorrect reconfiguration of Create requests can lead to the client issuing incorrect requests to other Servers, so policy should generally be tighter than that enforced for GET requests.

A client that has a series of resources it wants to create should generally face no impediment to sending the requests in parallel. However, this depends on the specifics of the resources being created. This pattern can be misused to produce a general messaging passing escape valve from the normal application of REST patterns. In such cases, in-order processing of requests may be required.

A naive implementation of this pattern using POST does not produce reliable results when the response to a Create request is lost. The create may or may not have gone through, and the client is not able to distinguish between the two cases. PUT is used below as the preferred alternative.

CONNECT may be used to tunnel through intermediaries if ordered processing of requests is required, producing reliable communications within the lifetime of the tunnelled connection.

Implementation

Enqueue can be implemented with HTTP using the following mappings:

Create(base url, document, type)
POST base url+globally unique identifier HTTP/1.1
Content-Type: type
Expect: 100-continue

document

A POST request may be used for Create, however this does not result in reliable behaviour when the response is lost. Using PUT allows the request to be safely repeated.

Success(location url)
Before reading document:
HTTP/1.1 100 Continue
After reading and processing document:
HTTP/1.1 200 OK

Note that 100 Continue handling is optional

All 2xx series response codes can be treated as Success responses for PUT. The location url is the one generate as part of the PUT request.

If POST was used as the Create method, the location url is returned in the response's Location header.

Type Not Understood()
HTTP/1.1 415 Unsupported Media Type
Accept: weighted acceptable types list
Fail(reason)
HTTP/1.1 400 Bad Request

reason

Unknown 1xx series response codes can be treated as a Fail for Create. 3xx series codes that are not understood should be treated as Fail. 4xx series response codes are Fail, except for 401 Unauthorised and 407 Proxy Authentication Required. These are Resubmit responses and should only be treated as failures if they are not understood. 404 Not Found and 410 Gone are excluded from the failed codes list for DELETE requests, as they may be returned as the result of a duplicate request that has already succeeded. 5xx series responses should be treated as Fail, except for 503 Service Unavailable and 504 Gateway Timeout. These are Resubmit and Response Lost responses, respectively.

Resubmit(required changes)

Any of: 301 Moved Permanently, 302 Found, 303 See Other, 305 Use Proxy, 307 Temporary Redirect, 401 Unauthorized, or 407 Proxy Authentication Required.

Response Lost()

Any loss of communication before a response is received. This may include application or TCP/IP level timeouts, or an explicitly terminated connection. The 504 Gateway Timeout response is also equivalent to Response Lost, and indicates a loss occurred somewhere past the TCP connection made directly by the client.

Sample Code

Request request;
request.url="http://example.com/publication-date/first-edition"
information = date(2008-07-05)
request.document_and_type = information.default_encode()

try_again:
switch (request())
{
Success(location url):
	// Save away the location url for further PUT and DELETE requests.
	// A GET to this URL may reveal additional metadata and links.

Type Not Understood(weighted acceptable types list):
	// Re-encode information in an acceptable form
	request.document_and_type =
		information.encode(
			weighted acceptable types list
			)
	jump try_again;

Fail(reason):
	log(reason)

Resubmit(required_changes):
	if policy(request, required_changes)
		request.modify(required_changes)
		jump try_again;
	else
		log("Policy forbids request modification");

Response Lost():
	if request.is_idempotent()
		if policy(request, no required changes)
			jump try_again;
		else
			log("Too many retries");
	else
		log("Response lost - Human intervention required");
}

Known Uses

POST is widely used to create state on the web by submitting form data to servers. Having a human at the console mitigates some of the risks inherent in the possibility of losing responses to the Create request.

Related Patterns

Published: Wed Jul 2 20:38:52 EST 2008

Updated: Sun Sep 21 22:49:29 EST 2008

GET

Intent

Transfer a defined set of information on demand from its owner the server to an anonymous client. Do so in a form the client understands.

Motivation

A server wants to express the current state of a light bulb for access by a home automation system. Originally, the system measured whether the light was on or off. As time goes on, the server begins to measure additional attributes such as operating temperature and expected lifetime of the unit.

The bulb status server is built into the lighting assembly, and is sold to work with automation systems developed over a number of generations. Some of these systems expect a simple on/off status indicator. Others can make use of the additional information available.

The home automation system may be dealing with bulb servers from different generations, also. A newer home automation system will have to deal with both old and new bulb servers.

The GET pattern is a tool to permit clients to access the light bulb data in a form they understand. It does not require a close match of versions between the client and server, allowing the interface between the two to evolve over time without breaking compatibility.

The GET pattern also has a performance aspect. Clients seek to minimise load on the server in processing their requests, load on the network in transferring messages, or on themselves in processing responses. The automation system may be based on low power components that cannot sustain high rates of transactions. Alternatively, the system could be large scale supporting thousands of bulbs at a large site or more across multiple sites. Clients also need to be able to deal with possible error conditions, including communication failures.

The GET pattern provides a clean client/server separation that is able to survive independent upgrades of each over time, exercises control over traffic and processing waste, and deals with possible errors.

Applicability

GET is appropriate whenever a client wants to acquire the whole of the information behind a known URL, and can decide when it wants to issue the request (subject to a cache miss). Here are some common means by which a URL is discovered:

  1. Direct entry allows a user to enter the URL through an input device
  2. Configuration allows a document to be prepared ahead of time with links that have particular meaning to the client
  3. Hyperlinking is a generalisation of configuration. The document that contains meaningful links may be acquired from anywhere, including an earlier completed GET request
  4. Construction is the assembly of information available to the client into a URL format agreed with the server. This may be achieved by populating a form supplied by the server in an earlier GET request.

Methods of determining when to issue a GET request include:

  1. One-shot, the issuing of a request at a predefined time, when the URL first becomes known, or when the client needs access to the information
  2. Cyclic, the issuing of a request at a predefined rate while the client is active
  3. On cache expiry, the issuing of a request whenever a cache entry expires. Note that this requires the server to set a maximum age on cache entries, something that is not always provided.
  4. Otherwise-triggered, the issuing of a request based on some form of back-channel that indicates information at a given URL may have changed

Structure

GET pattern structure

Participants

Client
  • Keeps a URL that lets it access the Server
  • Issues a GET request that includes a condition and weighted acceptable types list
  • Is capable of parsing all forms that the data might be encoded in that are semantically rich enough to use
  • Selects the right parser implementation to use based on the returned document type
  • (optional) Retains a cache of past successful GET responses and their related cache control information
  • Is responsible for overall successful execution of the operation, including modifications to the request and resubmissions of the request
  • Treats a lost response as equivalent to a Resubmit response with no required changes
  • Aborts the operation on a failure response, on a resubmission response that cannot or will not be satisfied, or on a lost response after too many retries.
Server
  • Evaluates any condition supplied in the GET request before performing significant processing
  • Selects the information to return based on the supplied URL
  • (optional) Is configured with mechanism to require the client to resubmit their request with or without modifications
  • Guarantees that a GET request is a read-only operation that is never interpreted by the server as a request to "buy an airline ticket". The server may choose to update log files and other information, but is not free to behave as if the client has requested or authorised the change.
  • Can return the requested information in various formats. Any format which the client might reasonably request with its acceptable types list should be supported.
  • Selects the most appropriate encoding based on the supplied weighted acceptable types list and any preference it may have itself, and returns the document in that format

Collaboration

  • Client issues requests to Server via the Request Interface, modifying and resubmitting its request as needed until:
    1. A success response is elicited
    2. The request condition is not met, meaning that the cached response is still valid
    3. A failure response is elicited
    4. The client is unable to make changes required by a Resubmit response
    5. Client policy prevents either changes required by a Resubmit response, or further resubmissions in general

Consequences

The GET pattern introduces a Uniform Interface for transferring identified sets of information from server to client. Clients and servers of different ages can communicate without impediment, and communication failures can be overcome.

The use of an acceptable types list in a GET request means that clients built during different phases of the architecture will generally be able to communicate. Document-based communication has a degree of flexibility built in with must-ignore parameters. The acceptable types list fills a gap when incompatible changes occur to the set of document types, for example a new type deprecates an old type such as atom depreciating rss for news feed syndication.

An explicit failure response allows problems in the architecture to be reported and repaired as required. The resubmit feature allows temporary or permanent changes to the architecture to be accommodated by components without explicit reconfiguration, simplifying management. Note, however, the potential security implications of allowing one component to reconfigure others. A predefined policy for which modifications are permitted and which are to be treated as failure cases can be useful in security-sensitive environments.

The potential exists in common transports such as HTTP for requests sent down parallel TCP connections or pipelined requests to be processed in a different order to that in which they actually return to Client. This could cause the client to become confused by "seeing" an older state after a more recent state. A simple solution is to hold off sending a GET request to a given URL when the previous related GET has not yet returned.

A client that is holding off sending the next GET request should queue the first such request for the identified URL. After this point it should not queue another GET request to the URL until the previous has been transmitted. There is no point queuing up multiple GET requests for the same URL. If the first request has not been issued by the time motivation to issue a second request comes around, a single request will fulfil the motivation behind both.

Twin consequences of the GET pattern are that interim states at URLs may be missed, and that the architecture as a whole does not become overloaded as the architecture is put under stress. Each GET retrieves the current state of the resource, so rapid changes may see the next GET arrive several changes after an earlier GET. These states will be lost unless an additional buffering mechanism is employed. The client will read back the current state rather than the old transitional states.

The flip-side of this behaviour is that clients are never stuck reading old data. They come completely up to date quickly and process the latest information available. Many algorithms for real-time processing will behave better under this scenario than if they are fed through old changes. The GET pattern can be adapted to a buffering model for algorithms that are sensitive to losses of interim states.

Implementation

GET can be implemented with HTTP using the following mappings:

GET(url, condition, weighted acceptable types list)
GET url HTTP/1.1
Accept: weighted acceptable types list
If-condition
Success(document, type, cache)
HTTP/1.1 200 OK
Content-Type: type
Cache-Control: cache

document

All 2xx series response codes can be treated as Success responses for GET

Condition Not Met()
HTTP/1.1 304 Not Modified
Fail(reason)
HTTP/1.1 400 Bad Request

reason

Unknown 1xx series response codes can be treated as a Fail for GET. 300 Multiple Choices is a non-implementable Resubmit response for automated clients, so should also be treated as Fail alongside other 3xx series codes that are not understood. 4xx series response codes are Fail, except for 401 Unauthorised and 407 Proxy Authentication Required. These are Resubmit responses and should only be treated as failures if they are not understood. 5xx series responses should be treated as Fail, except for 503 Service Unavailable and 504 Gateway Timeout. These are Resubmit and Response Lost responses, respectively.

Resubmit(required changes)

Any of: 301 Moved Permanently, 302 Found, 303 See Other, 305 Use Proxy, 307 Temporary Redirect, 401 Unauthorized, or 407 Proxy Authentication Required.

Response Lost()

Any loss of communication before a response is received. This may include application or TCP/IP level timeouts, or an explicitly terminated connection. The 504 Gateway Timeout response is also equivalent to Response Lost, and indicates a loss occured somewhere past the TCP connection made directly by the client.

Sample Code

Request request;
request.url="http://example.com/publication-dates"
if cache_manager.fresh(request.url)
{
	// Do nothing. Our cache entry is still fresh.
}
else if (blocked())
{
	// Only queue one request for the URL
	request_pending(url) = true
}
else
{
try_again:
	request.accept=parser.accept
	request.condition=cache_manager.condition(request.url)

	switch (request())
	{
	Success(document, type, cache):
		cache_manager.update(document, type, cache)
		process(parser(document, type))

	Condition Not Met():
		// Do nothing.
		// We have already processed the
		// latest data with our last request.

	Fail(reason):
		log(reason)

	Resubmit(required_changes):
		if policy(request, required_changes)
			request.modify(required_changes)
			jump try_again
		else
			log("Policy forbids request modification")

	Response Lost():
		if policy(request, no required changes)
			jump try_again
		else
			log("Too many retries")
	}
}

Known Uses

GET is widely used on the Web, both under direct human control and under automation. Various aspects of GET are not always used well.

Common errors in applying the GET pattern include:

  • "Unsafe" GET handling by servers, where GET is treated as an update request.
  • Not including the acceptable types list, meaning that the deployed client component will not be readily handled by an upgraded server component. The wrong document type may be returned.
  • Returning the wrong type with a document, or using heuristics based on the URL to determine which parser to invoke for a returned document.
  • Using type identifiers that are too generic. Type specifications such as application/xml or application/rdf+xml could match multiple formats for the return of data, application/atom+xml and application/atom+rdf+xml allow the server to choose a document to return that is more likely to be understood when the client parses it.
  • Returning a different document from the same URL based on session state. GET requests should not create information on the server that has to be tracked and available when the client's next GET request arrives. The client should be able to issue its next request at any time without further coordination with the server. Sessions may be used to short-cut expensive processing over a series of requests. However, an expired or lost session should not cause a given request to fail or fail to be understood.

Related Patterns

Published: Wed Jul 2 20:38:52 EST 2008

Updated: Sat Jul 5 21:53:36 EST 2008

Layering

Intent

Intercede between client and server to achieve performance or functionality benefits.

Motivation

Server S wants to serve get requests for five clients, but does not have resources to serve GET requests for all directly. A layer of indirection is added between the server and its clients that shields the server from requests for resources that already have cached documents. The cache returns responses on behalf of the server, reducing the need to generate responses from scratch.

Applicability

  1. Caching Proxy: A proxy that is trusted by a set of clients to act on their behalf. A chain of trust may be established from proxy to proxy in a well-managed network. Caching proxies are generally employed in networks where there the information being transferred is non-secret. Proxies are generally bypassed for secret communications. Protocol handling libraries for REST clients often act both as a Caching Proxy for the client of its API and a gateway from an API-based REST model to that of a specific protocol.
  2. Gateway: A proxy trusted by a set of clients to speak a protocol or participate in patterns not widely understood throughout the architecture. Gateways are generally temporary measures as different architectures come into contact with each other, before they consolidate their organisational structures to effectively deal with their common requirements.
  3. Firewall: A proxy that inspects requests passed through it to make sure they fit defined policy boundaries. The firewall might limit PUT access to certain URLs, remove portion of requests that it considers dangerous, or monitor patterns of requests to actively detect attempts at intrusion.
  4. Content Distribution Network: A proxy trusted by the server to answer requests on behalf of clients. The CDN is part of a chain of trust that extends outward from the server, and can be considered to be operated by the same agency as the server's. CDNs are used to provide local service to clients that may be distant from the main data centre, and may have various forms of private access to Server-side data not generally afforded to clients.

Structure

Layering pattern structure

Participants

Intermediary
  • Accepts requests from Client and generates requests to Server. Requests are submitted in whole, so can be passed on without significant modification as-is. The Intermediary is capable of performing alternate processing on behalf of the server such as returning a cache entry or manipulating requests and responses.

Collaboration

  • Client issues requests to Intermediary instead of Server. Intermediaries may be discovered by explicit configuration of Client, implicitly as part of the network configuration, or as an explicit redirection response from Server (see HTTP's 305 Use Proxy response).
  • Intermediary processes the request. It answers it itself or optionally modifies the request and passes it on to Server.
  • Server accepts requests from Intermediary and processes them as if they had come from Client.
  • The server's response is processed by Intermediary, optionally modified, and returned to Client.

An arbitrary number of layers can be added, each treating the next as Server and the previous as Client.

Consequences

The Layering pattern increases latency for requests that are forwarded unmodified, but can provide performance and functionality improvements when caching or other modifications are applied successfully.

Introducing layers increases the complexity of security, architecture evolution, and simple component testing models. Security and trust become an issue as intermediaries move further from the ownership of either Server or Client. Intermediaries literally are the feared man in the middle of security parlance. An intermediary that chooses to function against the interests of either Client or Server will lead to compromised communication.

Evolution of the architecture can be stymied by intermediaries. Upgrade of client and server to participate in a new operation or pattern is often not sufficient. The presence of old intermediaries under control of neither the Client's agency or the Server's agency mean that work-arounds and fallbacks may be required when they otherwise would not be required.

Testing of components can also suffer. Testing against common Server implementations is not sufficient if an intermediary introduces an unknown factor. Intermediaries may re-order pipelined request messages, splitting a single TCP connection at the Client end into multiple connections by the time they reach the Server. This is a potential source of out of order processing and other mischief that testing only a Client/Server combination may fail to reveal.

Implementation

Sample Code

Known Uses

Layering is widely used on the Web. It is used to implement caching in most HTTP client libraries. Caches are still sometimes used in ISPs to reduce traffic costs on behalf of clients that issue GET requests to similar sets of URLs. Layering is very commonly used as part of the network on the server-side, with load balancing and vertical scaling techniques strongly leveraging this pattern. Content Distribution Networks are becoming more commonly leveraged to serve a global audience with low latency.

Related Patterns

Published: Sat Nov 8 13:10:10 EST 2008

Updated: Sat Nov 15 17:43:18 EST 2008

Optimistic PUT Transaction

Intent

Safely modify a defined set of server-owned information when the starting state for that information is not known. The content is transmitted in a form the server understands.

Motivation

The PUT pattern is applicable when the clients wants to see the server reach a known state. It issues its idempotent PUT request repeatedly until the pattern either succeeds or fails. However, sometimes the target state that the client desires will depend on the starting state of the resource.

For example, a logical operation may require a vehicle's speed setting to be increased by five units. Simply PUTing a 5 to a speed resource won't work. It would set the speed to 5 only. Creating a speed increment resource and issuing a PUT to that resource would not work either: Several increment operations may be issued, and the idempotency of PUT requires that the increment only take effect once.

A simple solution in many cases is to apply an optimistic PUT transaction. This pattern allows the current state of the speed resource to be sampled before setting it to a higher value, and does so safely when a limited number of concurrent clients are attempting similar operations.

Applicability

An Optimistic PUT Transaction is appropriate when a client wants to update the information behind a known and predetermined URL based partially on its current content and partially on its own processing. A small number of concurrent clients can be supported, however the expected time to completion of the pattern may increase towards infinity when the number of active clients is high.

Structure

Optimistic PUT Transaction pattern structure

Participants

Client
  • Keeps a URL that lets it access the Server
  • Has a defined set of document types that contain enough information to sample, modify, and resubmit to the server.
  • Provides this list to the server as part of its GET request.
  • Retries and resubmits its GET as per the GET pattern
  • Transforms the document
  • Issues a PUT request with the transformed document, its type, the url, and the condition returned in the GETSuccess response (or DELETE if the transformed document is the null document).
  • Selects the most appropriate encoding based based on an initial guess. Subsequent requests are based on the on the supplied weighted acceptable types list if a Type Not Understood response is returned.
  • Retries and resubmits the PUT or DELETE as per the PUT pattern
  • Retries the whole pattern application if Condition Not Met is returned from the PUT request
  • Is responsible for overall successful execution of the operation, including modifications to the requests and resubmissions of the request
  • Treats a lost response as equivalent to a Resubmit response with no required changes
  • Aborts the operation on a failure response, on a resubmission response that cannot or will not be satisfied, or after too many retries.
Server
  • Checks that the type of the document and list of acceptable types is understood before performing significant processing
  • Selects the information to return and update based on the supplied URL
  • (optional) Is configured with mechanism to require the client to resubmit their request with or without modifications
  • Allows the PUT and GET to operate initially
  • Guarantees that the PUT and GET are safe to repeat. That is: Server updates its state to match a PUT initially. However, a PUT of the same state must not be interpreted as a request to modify state. It should return a Success response without any further action.
  • Returns a PUTSuccess response only once the information update can be considered permanent, allowing the client to forget it. The definition of permanent will depend on the possible consequences of client forgetfulness. It would typically range from "information updated on disk" to "information replicated to all sites, and stored to backup media" for important data.

Collaboration

Optimistic PUT Transaction Normal Case
  • Client issues requests to Server via the Request Interface, modifying and resubmitting its request as needed until:
    1. A success response is elicited
    2. The client is unable to make changes required by a Resubmit response
    3. Client policy prevents either changes required by a Resubmit response, or further resubmissions in general
Optimistic PUT Transaction Retry Case

Consequences

The Optimistic PUT Transaction pattern introduces a Uniform Interface for modifying server-side state. Clients and servers of different ages can communicate without impediment, and communication failures can be overcome. It partially fills a gap left by the PUT pattern for operations that are not logically idempotent, but only applies when the number of concurrent clients is bounded and sufficiently small.

Multiple required applications of the pattern may be queued in the client and executed in order. This pattern does not completely replace the previous state of the server-side resource, therefore subsequent requests cannot trivially be collapsed into one another. However, a well-understood sequence of transformations may be able to be logically combined to improve efficiency and reduce the risk of unbounded memory growth. For example, a sequence of additions and subtractions to a speed setting URL could be combined on the client side before being submitted at the first possible opportunity as an aggregate number to the Server.

Twin consequences of collapsing multiple transformations together are that interim transitions at URLs may never arrive at the Server, and that the architecture as a whole does not become overloaded as it is put under stress. Clients voluntarily discard intermediate states, so a server might have to make several internal transitions in order to catch up.

The flip-side of this behaviour is that servers are never stuck processing old data. They come completely up to date quickly. Many algorithms for real-time processing will behave better under this scenario than if they are fed through old requests. The pattern can be adapted to PUT to a new URL for each request for algorithms that are sensitive to discarding of interim states.

Implementation

PUT can be implemented with HTTP using the following mappings:

GET(url, weighted acceptable types list)
GET url HTTP/1.1
Accept: weighted acceptable types list
PUT(url, condition, document, type)
PUT url HTTP/1.1
Content-Type: type
If-Match: condition (entity tag)

document

DELETE (PUT null) is a DELETE request to the URL in HTTP

GETSuccess(document, type, condition)
HTTP/1.1 200 OK
Content-Type: type
ETag: condition (entity tag)

document

All 2xx series response codes can be treated as Success responses for GET.

PUTSuccess()
HTTP/1.1 200 OK

All 2xx series response codes can be treated as Success responses for PUT. If the request was a DELETE, then 404 Not Found and 410 Gone are also treated as Success codes.

Fail(reason)
HTTP/1.1 400 Bad Request

reason

Unknown 1xx series response codes can be treated as a Fail for PUT and GET. 3xx series codes that are not understood should be treated as Fail. 4xx series response codes are Fail, except for 401 Unauthorised and 407 Proxy Authentication Required. These are Resubmit responses and should only be treated as failures if they are not understood. 404 Not Found and 410 Gone are excluded from the failed codes list for DELETE requests, as they may be returned as the result of a duplicate request that has already succeeded. 5xx series responses should be treated as Fail, except for 503 Service Unavailable and 504 Gateway Timeout. These are Resubmit and Response Lost responses, respectively.

Resubmit(required changes)

Any of: 301 Moved Permanently, 302 Found, 303 See Other, 305 Use Proxy, 307 Temporary Redirect, 401 Unauthorized, or 407 Proxy Authentication Required.

Response Lost()

Any loss of communication before a response is received. This may include application or TCP/IP level timeouts, or an explicitly terminated connection. The 504 Gateway Timeout response is also equivalent to Response Lost, and indicates a loss occurred somewhere past the TCP connection made directly by the client.

Sample Code

url="http://example.com/publication-date/first-edition"
latest_update = 5; // metres per second
transform[url] += latest_update; // Fold transformations together
if (blocked())
{
	// Record that we still need to send a request. We have already
	// folded our transformations together, above.
	request_pending(url)
}
else
{
	try 
	{
	retry_get:
		(document, type, condition) = GET(url, "text/plain")
	}
	catch (...)
	{
		// etc
	}

	resource_state = decode(document, type);

	// Transformation made
	resource_state += transform[url]

	newdocument = encode(resource_state, type);

	try
	{
		PUT(url, condition, newdocument, type)
	}
	catch (...)
	{
		// etc
	}
}

Known Uses

Related Patterns

Published: Sat Jul 5 21:43:39 EST 2008

Updated: Sat Nov 8 14:38:45 EST 2008

PUT

Intent

Request that a defined set of server-owned information be created or replaced based on client-provided content. The content is transmitted in a form the server understands.

Motivation

A business to business order processing system requires a reliable means of submitting, updating, and cancelling purchase orders. The system components at either end were developed by separate vendors to fit with each company's IT infrastructure. Both customer and supplier businesses require their side of the system to interoperate with other suppliers and with other customers they may have, respectively.

Each business upgrades their capabilities in line with their own business objectives. This often means that early adopters have moved on several times before the most conservative businesses make a jump to the latest approach and standards. In the mean-time, any given component on the server or client side must face an eclectic mix of counterparts and must interoperate with them correctly.

The PUT pattern provides a clean client/server separation that is able to survive independent upgrades of each over time, minimise traffic and processing waste, and deals with possible errors. Unlike GET, this is not a data synchronisation exercise for the client. The client is not trying to acquire a current snapshot of server state for processing. Instead, it is transferring information to the server which it may thereafter quickly forget. PUT is a pattern for information hand-over.

Applicability

PUT is appropriate whenever a client wants to create or replace the whole of the information behind a known and predetermined URL, and can decide when it wants to issue the request. Here are some common means by which a URL is discovered:

  1. Direct entry allows a user to enter the URL through an input device
  2. Configuration allows a document to be prepared ahead of time with links that have particular meaning to the client
  3. Hyperlinking is a generalisation of configuration. The document that contains meaningful links may be acquired from anywhere, including an earlier completed GET request
  4. Construction is the assembly of information available to the client into a URL format agreed with the server. This may be achieved by populating a form supplied by the server in an earlier GET request. This is a particularly useful approach when combined with a globally-unique identifier for the reliable automated submission of new data to a server.

Structure

PUT pattern structure

Participants

Client
  • Keeps a URL that lets it access the Server
  • Is capable of encoding its information in any form that it can legally and reasonably be encoded to.
  • Selects the most appropriate encoding based based on an initial guess. Subsequent requests are based on the on the supplied weighted acceptable types list if a Type Not Understood response is returned.
  • Issues the PUT or DELETE request. DELETE is a special case of PUT that is equivalent to a PUT of the null state.
  • Is responsible for overall successful execution of the operation, including modifications to the request and resubmissions of the request
  • Treats a lost response as equivalent to a Resubmit response with no required changes
  • Aborts the operation on a failure response, on a resubmission response that cannot or will not be satisfied, or on a lost response after too many retries.
Server
  • Checks that the type of the document is understood before performing significant processing
  • Selects the information to update based on the supplied URL
  • (optional) Is configured with mechanism to require the client to resubmit their request with or without modifications
  • Allows the PUT to operate initially
  • Guarantees that the PUT is safe to repeat. That is: Server updates its state to match a PUT initially. However, a PUT of the same state must not be interpreted as a request to modify state. It should return a Success response without any further action.
  • Is capable of parsing all forms that the data might be encoded in that are semantically rich enough to use
  • Selects the right parser implementation to use based on the returned document type
  • Returns a Success response only once the information update can be considered permanent, allowing the client to forget it. The definition of permanent will depend on the possible consequences of client forgetfulness. It would typically range from "information updated on disk" to "information replicated to all sites, and stored to backup media" for important data.

Collaboration

  • Client issues requests to Server via the Request Interface, modifying and resubmitting its request as needed until:
    1. A success response is elicited
    2. The client is unable to make changes required by a Resubmit response
    3. Client policy prevents either changes required by a Resubmit response, or further resubmissions in general

Consequences

The PUT pattern introduces a Uniform Interface for handing over identified sets of information from client to server. Clients and servers of different ages can communicate without impediment, and communication failures can be overcome.

The PUT pattern may require a change of thinking by developers with an imperative messaging background. PUT is not a request to make a particular change or particular kind of change to a set of data to reach an undefined end state. Instead, it is a request to make a set of data match a defined end state without specifying how the transition should be made. This is preferable to the imperative approach when it comes to network communications, because requests can be accidentally submitted multiple times without changing the meaning of the original request. Each subsequent request is interpreted as: "Please make no changes". This feature is called idempotency, and is necessary for dealing with any Response Lost message.

The use of an acceptable types list in a Types Not Understood response means that clients and servers built during different phases of the architecture will generally be able to communicate. Document-based communication has a degree of flexibility built in with must-ignore parameters. The acceptable types list fills a gap when incompatible changes occur to the set of document types, for example a new type deprecates an old type such as atom depreciating rss for news feed syndication.

An explicit failure response allows problems in the architecture to be reported and repaired as required. The resubmit feature allows temporary or permanent changes to the architecture to be accommodated by components without explicit reconfiguration, simplifying management. Note, however, the potential security implications of allowing one component to reconfigure others. A predefined policy for which modifications are permitted and which are to be treated as failure cases can be useful in security-sensitive environments. Incorrect reconfiguration of PUT requests can lead to the client issuing incorrect requests to other Servers, so policy should generally be tighter than that enforced for GET requests.

The potential exists in common transports such as HTTP for requests sent down parallel TCP connections or pipelined requests to be processed in a different order to that in which they were issued by Client. This could cause the server to use the wrong update, while the client sees only success responses. A simple solution is to hold off sending a PUT request to a given URL when the previous related PUT has not yet returned.

A client that is holding off sending the next PUT request should queue the first such request for the identified URL. After this point it should replace the queued PUT with each new request to the URL until the previous has been transmitted. There is no point queuing up multiple PUT requests for the same URL. The latest request should completely replace the effect of any previous one. This directs the server to transition its information directly from its starting state to its end state. In doing so, it can provide any internal optimisations available for efficient processing.

Twin consequences of the PUT pattern are that interim states at URLs may be lost, and that the architecture as a whole does not become overloaded as it is put under stress. Clients voluntarily discard intermediate states, so a server might have to make several internal transitions in order to catch up.

The flip-side of this behaviour is that servers are never stuck processing old data. They come completely up to date quickly. Many algorithms for real-time processing will behave better under this scenario than if they are fed through old requests. The pattern can be adapted to PUT to a new URL for each request for algorithms that are sensitive to discarding of interim states.

Implementation

PUT can be implemented with HTTP using the following mappings:

PUT(url, document, type)
PUT url HTTP/1.1
Content-Type: type
Expect: 100-continue

document

DELETE (PUT null) is a DELETE request to the URL in HTTP

Success()
Before reading document:
HTTP/1.1 100 Continue
After reading and processing document:
HTTP/1.1 200 OK

Note that 100 Continue handling is optional

All 2xx series response codes can be treated as Success responses for PUT. If the request was a DELETE, then 404 Not Found and 410 Gone are also treated as Success codes.

Type Not Understood()
HTTP/1.1 415 Unsupported Media Type
Accept: weighted acceptable types list
Fail(reason)
HTTP/1.1 400 Bad Request

reason

Unknown 1xx series response codes can be treated as a Fail for PUT. 3xx series codes that are not understood should be treated as Fail. 4xx series response codes are Fail, except for 401 Unauthorised and 407 Proxy Authentication Required. These are Resubmit responses and should only be treated as failures if they are not understood. 404 Not Found and 410 Gone are excluded from the failed codes list for DELETE requests, as they may be returned as the result of a duplicate request that has already succeeded. 5xx series responses should be treated as Fail, except for 503 Service Unavailable and 504 Gateway Timeout. These are Resubmit and Response Lost responses, respectively.

Resubmit(required changes)

Any of: 301 Moved Permanently, 302 Found, 303 See Other, 305 Use Proxy, 307 Temporary Redirect, 401 Unauthorized, or 407 Proxy Authentication Required.

Response Lost()

Any loss of communication before a response is received. This may include application or TCP/IP level timeouts, or an explicitly terminated connection. The 504 Gateway Timeout response is also equivalent to Response Lost, and indicates a loss occured somewhere past the TCP connection made directly by the client.

Sample Code

Request request;
request.url="http://example.com/publication-date/first-edition"
information = date(2008-07-05)
if (blocked())
{
	// Only queue the latest document
	// Overwrite any previous request
	request_pending(url) = information
}
else
{
	request.document_and_type = information.default_encode()

try_again:
	switch (request())
	{
	Success():
		// Do nothing. The update has completed.

	Type Not Understood(weighted acceptable types list):
		// Re-encode information in an acceptable form
		request.document_and_type =
			information.encode(
				weighted acceptable types list
				)
		jump try_again;

	Fail(reason):
		log(reason)

	Resubmit(required_changes):
		if policy(request, required_changes)
			request.modify(required_changes)
			jump try_again;
		else
			log("Policy forbids request modification");

	Response Lost():
		if policy(request, no required changes)
			jump try_again;
		else
			log("Too many retries");
	}
}

Known Uses

PUT is widely used less widely than GET on the Web, and is primarily a feature of automation.

Related Patterns

Published: Sat Nov 15 17:41:47 EST 2008

Updated: Sat Nov 15 17:41:47 EST 2008

Pessimistic PUT Transaction

Intent

Safely modify a defined set of server-owned information when the starting state for that information is not known and the likelihood of interference during a transaction is relatively high. The content is transmitted in a form the server understands.

Motivation

The PUT pattern is applicable when the clients wants to see the server reach a known state. It issues its idempotent PUT request repeatedly until the pattern either succeeds or fails. However, sometimes the target state that the client desires will depend on the starting state of the resource.

For example, a logical operation may require a vehicle's speed setting to be increased by five units. Simply PUTing a 5 to a speed resource won't work. It would set the speed to 5 only. Creating a speed increment resource and issuing a PUT to that resource would not work either: Several increment operations may be issued, and the idempotency of PUT requires that the increment only take effect once.

A simple solution in many cases is to apply an pessimistic PUT transaction. This pattern allows the current state of the speed resource to be sampled before setting it to a higher value, and locks the resource while a transformation is applied to the document and the result is resubmitted.

Applicability

An Pessimistic PUT Transaction is appropriate when a client wants to update the information behind a known and predetermined URL based partially on its current content and partially on its own processing. Concurrent clients can be supported, however a failed client may lead to inefficient processing.

Structure

Pessimistic PUT Transaction pattern structure

Participants

Client
  • Keeps a URL that lets it access the Server
  • Has a defined set of document types that contain enough information to sample, modify, and resubmit to the server.
  • Provides this list to the server as part of its lock_and_GET request.
  • Retries and resubmits its lock_and_GET as per the GET pattern
  • Transforms the document
  • Issues a PUT request with the transformed document, its type, the url, and the condition returned in the GETSuccess response (or DELETE if the transformed document is the null document).
  • Selects the most appropriate encoding based based on an initial guess. Subsequent requests are based on the on the supplied weighted acceptable types list if a Type Not Understood response is returned.
  • Retries and resubmits the PUT or DELETE as per the PUT pattern
  • Retries the whole pattern application if Condition Not Met is returned from the PUT request
  • Submits the UNLOCK request after the PUT completes
  • Is responsible for overall successful execution of the operation, including modifications to the requests and resubmissions of the request
  • Treats a lost response as equivalent to a Resubmit response with no required changes
  • Aborts the operation on a failure response, on a resubmission response that cannot or will not be satisfied, or after too many retries.
Server
  • Checks that the type of the document and list of acceptable types is understood before performing significant processing
  • Selects the information to return and update based on the supplied URL
  • (optional) Is configured with mechanism to require the client to resubmit their request with or without modifications
  • Allows the GET_and_lock, PUT, and UNLOCK to operate initially
  • Guarantees that the GET_and_unlock, PUT, and UNLOCK are safe to repeat. That is: Server updates its state to match a PUT initially. However, a PUT of the same state must not be interpreted as a request to modify state. It should return a Success response without any further action. The UNLOCK operation should return Success if the identified lock is not currently held.
  • Returns a PUTSuccess response only once the information update can be considered permanent, allowing the client to forget it. The definition of permanent will depend on the possible consequences of client forgetfulness. It would typically range from "information updated on disk" to "information replicated to all sites, and stored to backup media" for important data.
  • Times out the lock if an UNLOCK has not been received, allowing other clients or a restarted instance of the same client to continue operating.

Collaboration

Pessimistic PUT Transaction Normal Case
  • Client issues requests to Server via the Request Interface, modifying and resubmitting its request as needed until:
    1. A success response is elicited
    2. The client is unable to make changes required by a Resubmit response
    3. Client policy prevents either changes required by a Resubmit response, or further resubmissions in general
Pessimistic PUT Transaction Retry Case

Consequences

The Pessimistic PUT Transaction pattern introduces a Uniform Interface for modifying server-side state. Clients and servers of different ages can communicate without impediment, and communication failures can be overcome. It partially fills a gap left by the PUT pattern for operations that are not logically idempotent, but only applies when the number of concurrent clients is bounded and sufficiently small.

Multiple required applications of the pattern may be queued in the client and executed in order. This pattern does not completely replace the previous state of the server-side resource, therefore subsequent requests cannot trivially be collapsed into one another. However, a well-understood sequence of transformations may be able to be logically combined to improve efficiency and reduce the risk of unbounded memory growth. For example, a sequence of additions and subtractions to a speed setting URL could be combined on the client side before being submitted at the first possible opportunity as an aggregate number to the Server.

Twin consequences of collapsing multiple transformations together are that interim transitions at URLs may never arrive at the Server, and that the architecture as a whole does not become overloaded as it is put under stress. Clients voluntarily discard intermediate states, so a server might have to make several internal transitions in order to catch up.

The flip-side of this behaviour is that servers are never stuck processing old data. They come completely up to date quickly. Many algorithms for real-time processing will behave better under this scenario than if they are fed through old requests. The pattern can be adapted to PUT to a new URL for each request for algorithms that are sensitive to discarding of interim states.

Implementation

Pessimistic PUT Transaction can be implemented with HTTP and WebDAV extensions using the following mappings:

lock_and_GET(url, weighted acceptable types list)
LOCK url HTTP/1.1
Timeout: expected operation time

And as a separate request:

GET url HTTP/1.1
Accept: weighted acceptable types list
PUT(url, lock id, document, type)
PUT url HTTP/1.1
Content-Type: type
If: lock id

document

DELETE (PUT null) is a DELETE request to the URL in HTTP

UNLOCK(url, condition, document, type)
UNLOCK url HTTP/1.1
Lock-Token: lock i
GETSuccess(document, type, condition)
HTTP/1.1 200 OK
Content-Type: type
ETag: condition (entity tag)

document

All 2xx series response codes can be treated as Success responses for GET.

PUTSuccess()
HTTP/1.1 200 OK

All 2xx series response codes can be treated as Success responses for PUT. If the request was a DELETE, then 404 Not Found and 410 Gone are also treated as Success codes.

UNLOCKSuccess()
HTTP/1.1 200 OK

All 2xx series response codes can be treated as Success responses for PUT. If the request was a DELETE, then 404 Not Found and 410 Gone are also treated as Success codes.

Lock Identifier Mismatch()
HTTP/1.1 412 Precondition Failed
Fail(reason)
HTTP/1.1 400 Bad Request

reason

Unknown 1xx series response codes can be treated as a Fail for PUT and GET. 3xx series codes that are not understood should be treated as Fail. 4xx series response codes are Fail, except for 401 Unauthorised and 407 Proxy Authentication Required. These are Resubmit responses and should only be treated as failures if they are not understood. 404 Not Found and 410 Gone are excluded from the failed codes list for DELETE requests, as they may be returned as the result of a duplicate request that has already succeeded. 5xx series responses should be treated as Fail, except for 503 Service Unavailable and 504 Gateway Timeout. These are Resubmit and Response Lost responses, respectively.

Resubmit(required changes)

Any of: 301 Moved Permanently, 302 Found, 303 See Other, 305 Use Proxy, 307 Temporary Redirect, 401 Unauthorized, or 407 Proxy Authentication Required.

Response Lost()

Any loss of communication before a response is received. This may include application or TCP/IP level timeouts, or an explicitly terminated connection. The 504 Gateway Timeout response is also equivalent to Response Lost, and indicates a loss occurred somewhere past the TCP connection made directly by the client.

Sample Code

url="http://example.com/publication-date/first-edition"
latest_update = 5; // metres per second
transform[url] += latest_update; // Fold transformations together
if (blocked())
{
	// Record that we still need to send a request. We have already
	// folded our transformations together, above.
	request_pending(url)
}
else
{
	try 
	{
	retry_get:
		(document, type, condition) = GET(url, "text/plain")
	}
	catch (...)
	{
		// etc
	}

	resource_state = decode(document, type);

	// Transformation made
	resource_state += transform[url]

	newdocument = encode(resource_state, type);

	try
	{
		PUT(url, condition, newdocument, type)
	}
	catch (...)
	{
		// etc
	}
}

Known Uses

Related Patterns

Published: Wed Jul 2 20:38:52 EST 2008

Updated: Sun Jul 6 12:25:07 EST 2008

Registry

Intent

Support serendipitous reuse across an architecture

Motivation

Producer P and consumer C want to transfer a document. P has a particular abstract data schema in mind, and C has a compatible schema. Registry ensures that P encodes and transmits the information in a form that C understands.

The methods for applying patterns of information transfer and forms in which information are encoded may be defined either by local convention or by a wider convention. Local protocols may be "jargonised" versions of more widely-applicable protocols. In time, jargon may move from a satellite architecture such as that of a particular enterprise and become standardised in a wider setting such as the Web.

Applicability

Registry is applicable for REST architectures large and small. However, architectures with only a small number of components may not need a formal registry.

Structure

Registry Structure

Participants

Document Type
  • Provides a defined way to encode a given information schema
  • May have one or more parent types, whose parser is also appropriate to read this derived document type
  • Is defined in a Document Type Specification, including accommodation of document type evolution through must-ignore semantics on information that is not understood by processors.
  • May be a target encoding for a number of a number of different information schemas, including internal information schemas for various kinds of applications. For example, HTML can be used to encode information schemas as diverse as journal entry publication and warnings about severe weather events.
  • May act as a bridge between different Information Schema. For example, ODF is able to bridge between OpenOffice and Gnome Office internal document schemas.
  • Is defined to be independent of how it is transported, and orthogonal to Operations that are defined in the architecture.
  • Is developed using the least power principle: The least powerful language that is capable of expressing a set of information is used.
  • Minimises coupling by limiting reliance on local or domain-specific conventions.
Operation
  • Provides a way to transport information around the architecture
  • Corresponds to one or more Interaction Patterns that are understood by applicable information end-points
  • Is defined to be independent of the information it transports, and the specific endpoints it transfers information between. Operations are orthogonal to the set of URLs in the architecture.
Management Body and Process
  • Manages the evolution of Document Types and Operations
  • Oversees the adoption and uptake of new or updated Document Types and Operations in the architecture
  • May depend heavily on external bodies, especially those that manage the Web
  • Typically do not have the authority to upgrade any given component of the architecture. Instead, components are upgraded over time as their respective owners see fit in line with the Registry specifications.
  • Typically do not have any control over the set of URLs in the architecture

Collaboration

  • The Management Body and Processes manages the independent evolution of Document Types and Operations in the architecture. The set of URLs is not controlled by this body.

Consequences

The orthogonal nature of document types, operations, and URLs throughout the architecture allow for freedom of evolution. This separation and evolution is supported through the major patterns of the architecture as old and new components interact, including GET and PUT.

The balance of control and freedom is capable of delivering application-level forwards and backwards compatibility that lasts for a relative eternity in our industry. For example, HTML and HTTP have been able to continuously evolve over a time-frame spanning decades.

More domain-specific document types will likely have shorter lifespans as they develop from local conventions to a point of widespread global agreement. However, as critical mass is created a semantic web where machines can communicate generally based on widely-understood conventions is likely to emerge without specific coordination or intervention.

Implementation

Sample Code

Known Uses

The IETF defines a registry and change management process that includes a list of protocol interactions for document transfer (HTTP/1.1) and legal document types (the IANA).

Related Patterns