Sound advice - patterns

Tales from the homeworld

My current feeds

Sun, 2008-Sep-21

Published: Sun Sep 21 23:21:56 EST 2008

Updated: Sun Sep 21 23:21:55 EST 2008

Enqueue

Intent

Create a series of entries in a server-maintained set or queue.

Motivation

A business to business order processing system requires a reliable means of submitting, updating, and cancelling purchase orders. The system components at either end were developed by separate vendors to fit with each company's IT infrastructure. Both customer and supplier businesses require their side of the system to interoperate with other suppliers and with other customers they may have, respectively.

Each business upgrades their capabilities in line with their own business objectives. This often means that early adopters have moved on several times before the most conservative businesses make a jump to the latest approach and standards. In the mean-time, any given component on the server or client side must face an eclectic mix of counterparts and must interoperate with them correctly.

The Enqueue pattern is focused around the creation of server-side records in this message exchange. The PUT Pattern is used for subsequent updates and deletion requests.

Applicability

Enqueue is appropriate whenever the objective is creation of one or more pieces of server-side state based on information available to the client.

Structure

Enqueue pattern structure

Participants

Client
  • Keeps a base URL that provides context for the Create operation
  • Is capable of encoding its information in any form that it can legally and reasonably be encoded to.
  • Selects the most appropriate encoding based based on an initial guess. Subsequent requests are based on the on the supplied weighted acceptable types list if a Type Not Understood response is returned.
  • Issues the Create request
  • Is responsible for overall successful execution of the operation, including modifications to the request and resubmissions of the request
  • May or may not be able to determine whether the Create was successful or not when its response is lost, depending on the implementation
  • May or may not be able to control the order in which Create operations are processed, depending on the implementation
  • Aborts the operation on a failure response, on a resubmission response that cannot or will not be satisfied, or on a lost response after too many retries.
Server
  • Checks that the type of the document is understood before performing significant processing
  • Selects the kind of object to create based on the supplied base URL
  • (optional) Is configured with mechanism to require the client to resubmit their request with or without modifications
  • Allows the Create to operate initially
  • May provide a mechanism by which the Create operation is safe to repeat
  • May provide a mechanism by which a series of Create operations are guaranteed to be processed in order
  • Is capable of parsing all forms that the data might be encoded in that are semantically rich enough to use
  • Selects the right parser implementation to use based on the returned document type
  • Returns a Success response only once the creation can be considered permanent, allowing the client to forget it. The definition of permanent will depend on the possible consequences of client forgetfulness. It would typically range from "information updated on disk" to "information replicated to all sites, and stored to backup media" for important data.

Collaboration

  • Client issues requests to Server via the Request Interface, modifying and resubmitting its request as needed until:
    1. A success response is elicited
    2. A response is lost, and the client is unable to safely repeat its request
    3. The client is unable to make changes required by a Resubmit response
    4. Client policy prevents either changes required by a Resubmit response, or further resubmissions in general

Consequences

The Enqueue pattern creates a mechanism to create a series of new resources on the server side, but the safety and efficacy of the operation is dependent on a specific implementation.

The use of an acceptable types list in a Types Not Understood response means that clients and servers built during different phases of the architecture will generally be able to communicate. Document-based communication has a degree of flexibility built in with must-ignore parameters. The acceptable types list fills a gap when incompatible changes occur to the set of document types, for example a new type deprecates an old type such as atom depreciating rss for news feed syndication.

An explicit failure response allows problems in the architecture to be reported and repaired as required. The resubmit feature allows temporary or permanent changes to the architecture to be accommodated by components without explicit reconfiguration, simplifying management. Note, however, the potential security implications of allowing one component to reconfigure others. A predefined policy for which modifications are permitted and which are to be treated as failure cases can be useful in security-sensitive environments. Incorrect reconfiguration of Create requests can lead to the client issuing incorrect requests to other Servers, so policy should generally be tighter than that enforced for GET requests.

A client that has a series of resources it wants to create should generally face no impediment to sending the requests in parallel. However, this depends on the specifics of the resources being created. This pattern can be misused to produce a general messaging passing escape valve from the normal application of REST patterns. In such cases, in-order processing of requests may be required.

A naive implementation of this pattern using POST does not produce reliable results when the response to a Create request is lost. The create may or may not have gone through, and the client is not able to distinguish between the two cases. PUT is used below as the preferred alternative.

CONNECT may be used to tunnel through intermediaries if ordered processing of requests is required, producing reliable communications within the lifetime of the tunnelled connection.

Implementation

Enqueue can be implemented with HTTP using the following mappings:

Create(base url, document, type)
POST base url+globally unique identifier HTTP/1.1
Content-Type: type
Expect: 100-continue

document

A POST request may be used for Create, however this does not result in reliable behaviour when the response is lost. Using PUT allows the request to be safely repeated.

Success(location url)
Before reading document:
HTTP/1.1 100 Continue
After reading and processing document:
HTTP/1.1 200 OK

Note that 100 Continue handling is optional

All 2xx series response codes can be treated as Success responses for PUT. The location url is the one generate as part of the PUT request.

If POST was used as the Create method, the location url is returned in the response's Location header.

Type Not Understood()
HTTP/1.1 415 Unsupported Media Type
Accept: weighted acceptable types list
Fail(reason)
HTTP/1.1 400 Bad Request

reason

Unknown 1xx series response codes can be treated as a Fail for Create. 3xx series codes that are not understood should be treated as Fail. 4xx series response codes are Fail, except for 401 Unauthorised and 407 Proxy Authentication Required. These are Resubmit responses and should only be treated as failures if they are not understood. 404 Not Found and 410 Gone are excluded from the failed codes list for DELETE requests, as they may be returned as the result of a duplicate request that has already succeeded. 5xx series responses should be treated as Fail, except for 503 Service Unavailable and 504 Gateway Timeout. These are Resubmit and Response Lost responses, respectively.

Resubmit(required changes)

Any of: 301 Moved Permanently, 302 Found, 303 See Other, 305 Use Proxy, 307 Temporary Redirect, 401 Unauthorized, or 407 Proxy Authentication Required.

Response Lost()

Any loss of communication before a response is received. This may include application or TCP/IP level timeouts, or an explicitly terminated connection. The 504 Gateway Timeout response is also equivalent to Response Lost, and indicates a loss occurred somewhere past the TCP connection made directly by the client.

Sample Code

Request request;
request.url="http://example.com/publication-date/first-edition"
information = date(2008-07-05)
request.document_and_type = information.default_encode()

try_again:
switch (request())
{
Success(location url):
	// Save away the location url for further PUT and DELETE requests.
	// A GET to this URL may reveal additional metadata and links.

Type Not Understood(weighted acceptable types list):
	// Re-encode information in an acceptable form
	request.document_and_type =
		information.encode(
			weighted acceptable types list
			)
	jump try_again;

Fail(reason):
	log(reason)

Resubmit(required_changes):
	if policy(request, required_changes)
		request.modify(required_changes)
		jump try_again;
	else
		log("Policy forbids request modification");

Response Lost():
	if request.is_idempotent()
		if policy(request, no required changes)
			jump try_again;
		else
			log("Too many retries");
	else
		log("Response lost - Human intervention required");
}

Known Uses

POST is widely used to create state on the web by submitting form data to servers. Having a human at the console mitigates some of the risks inherent in the possibility of losing responses to the Create request.

Related Patterns

Sun, 2008-Sep-21

Published: Sat Jul 5 21:43:39 EST 2008

Updated: Mon Jul 14 21:01:19 EST 2008

PUT

Intent

Request that a defined set of server-owned information be created or replaced based on client-provided content. The content is transmitted in a form the server understands.

Motivation

A business to business order processing system requires a reliable means of submitting, updating, and cancelling purchase orders. The system components at either end were developed by separate vendors to fit with each company's IT infrastructure. Both customer and supplier businesses require their side of the system to interoperate with other suppliers and with other customers they may have, respectively.

Each business upgrades their capabilities in line with their own business objectives. This often means that early adopters have moved on several times before the most conservative businesses make a jump to the latest approach and standards. In the mean-time, any given component on the server or client side must face an eclectic mix of counterparts and must interoperate with them correctly.

The PUT pattern provides a clean client/server separation that is able to survive independent upgrades of each over time, minimise traffic and processing waste, and deals with possible errors. Unlike GET, this is not a data synchronisation exercise for the client. The client is not trying to acquire a current snapshot of server state for processing. Instead, it is transferring information to the server which it may thereafter quickly forget. PUT is a pattern for information hand-over.

Applicability

PUT is appropriate whenever a client wants to create or replace the whole of the information behind a known and predetermined URL, and can decide when it wants to issue the request. Here are some common means by which a URL is discovered:

  1. Direct entry allows a user to enter the URL through an input device
  2. Configuration allows a document to be prepared ahead of time with links that have particular meaning to the client
  3. Hyperlinking is a generalisation of configuration. The document that contains meaningful links may be acquired from anywhere, including an earlier completed GET request
  4. Construction is the assembly of information available to the client into a URL format agreed with the server. This may be achieved by populating a form supplied by the server in an earlier GET request. This is a particularly useful approach when combined with a globally-unique identifier for the reliable automated submission of new data to a server.

Structure

PUT pattern structure

Participants

Client
  • Keeps a URL that lets it access the Server
  • Is capable of encoding its information in any form that it can legally and reasonably be encoded to.
  • Selects the most appropriate encoding based based on an initial guess. Subsequent requests are based on the on the supplied weighted acceptable types list if a Type Not Understood response is returned.
  • Issues the PUT or DELETE request. DELETE is a special case of PUT that is equivalent to a PUT of the null state.
  • Is responsible for overall successful execution of the operation, including modifications to the request and resubmissions of the request
  • Treats a lost response as equivalent to a Resubmit response with no required changes
  • Aborts the operation on a failure response, on a resubmission response that cannot or will not be satisfied, or on a lost response after too many retries.
Server
  • Checks that the type of the document is understood before performing significant processing
  • Selects the information to update based on the supplied URL
  • (optional) Is configured with mechanism to require the client to resubmit their request with or without modifications
  • Allows the PUT to operate initially
  • Guarantees that the PUT is safe to repeat. That is: Server updates its state to match a PUT initially. However, a PUT of the same state must not be interpreted as a request to modify state. It should return a Success response without any further action.
  • Is capable of parsing all forms that the data might be encoded in that are semantically rich enough to use
  • Selects the right parser implementation to use based on the returned document type
  • Returns a Success response only once the information update can be considered permanent, allowing the client to forget it. The definition of permanent will depend on the possible consequences of client forgetfulness. It would typically range from "information updated on disk" to "information replicated to all sites, and stored to backup media" for important data.

Collaboration

  • Client issues requests to Server via the Request Interface, modifying and resubmitting its request as needed until:
    1. A success response is elicited
    2. The client is unable to make changes required by a Resubmit response
    3. Client policy prevents either changes required by a Resubmit response, or further resubmissions in general

Consequences

The PUT pattern introduces a Uniform Interface for handing over identified sets of information from client to server. Clients and servers of different ages can communicate without impediment, and communication failures can be overcome.

The PUT pattern may require a change of thinking by developers with an imperative messaging background. PUT is not a request to make a particular change or particular kind of change to a set of data to reach an undefined end state. Instead, it is a request to make a set of data match a defined end state without specifying how the transition should be made. This is preferable to the imperative approach when it comes to network communications, because requests can be accidentally submitted multiple times without changing the meaning of the original request. Each subsequent request is interpreted as: "Please make no changes". This feature is called idempotency, and is necessary for dealing with any Response Lost message.

The use of an acceptable types list in a Types Not Understood response means that clients and servers built during different phases of the architecture will generally be able to communicate. Document-based communication has a degree of flexibility built in with must-ignore parameters. The acceptable types list fills a gap when incompatible changes occur to the set of document types, for example a new type deprecates an old type such as atom depreciating rss for news feed syndication.

An explicit failure response allows problems in the architecture to be reported and repaired as required. The resubmit feature allows temporary or permanent changes to the architecture to be accommodated by components without explicit reconfiguration, simplifying management. Note, however, the potential security implications of allowing one component to reconfigure others. A predefined policy for which modifications are permitted and which are to be treated as failure cases can be useful in security-sensitive environments. Incorrect reconfiguration of PUT requests can lead to the client issuing incorrect requests to other Servers, so policy should generally be tighter than that enforced for GET requests.

The potential exists in common transports such as HTTP for requests sent down parallel TCP connections or pipelined requests to be processed in a different order to that in which they were issued by Client. This could cause the server to use the wrong update, while the client sees only success responses. A simple solution is to hold off sending a PUT request to a given URL when the previous related PUT has not yet returned.

A client that is holding off sending the next PUT request should queue the first such request for the identified URL. After this point it should replace the queued PUT with each new request to the URL until the previous has been transmitted. There is no point queuing up multiple PUT requests for the same URL. The latest request should completely replace the effect of any previous one. This directs the server to transition its information directly from its starting state to its end state. In doing so, it can provide any internal optimisations available for efficient processing.

Twin consequences of the PUT pattern are that interim states at URLs may be lost, and that the architecture as a whole does not become overloaded as it is put under stress. Clients voluntarily discard intermediate states, so a server might have to make several internal transitions in order to catch up.

The flip-side of this behaviour is that servers are never stuck processing old data. They come completely up to date quickly. Many algorithms for real-time processing will behave better under this scenario than if they are fed through old requests. The pattern can be adapted to PUT to a new URL for each request for algorithms that are sensitive to discarding of interim states.

Implementation

PUT can be implemented with HTTP using the following mappings:

PUT(url, document, type)
PUT url HTTP/1.1
Content-Type: type
Expect: 100-continue

document

DELETE (PUT null) is a DELETE request to the URL in HTTP

Success()
Before reading document:
HTTP/1.1 100 Continue
After reading and processing document:
HTTP/1.1 200 OK

Note that 100 Continue handling is optional

All 2xx series response codes can be treated as Success responses for PUT. If the request was a DELETE, then 404 Not Found and 410 Gone are also treated as Success codes.

Type Not Understood()
HTTP/1.1 415 Unsupported Media Type
Accept: weighted acceptable types list
Fail(reason)
HTTP/1.1 400 Bad Request

reason

Unknown 1xx series response codes can be treated as a Fail for PUT. 3xx series codes that are not understood should be treated as Fail. 4xx series response codes are Fail, except for 401 Unauthorised and 407 Proxy Authentication Required. These are Resubmit responses and should only be treated as failures if they are not understood. 404 Not Found and 410 Gone are excluded from the failed codes list for DELETE requests, as they may be returned as the result of a duplicate request that has already succeeded. 5xx series responses should be treated as Fail, except for 503 Service Unavailable and 504 Gateway Timeout. These are Resubmit and Response Lost responses, respectively.

Resubmit(required changes)

Any of: 301 Moved Permanently, 302 Found, 303 See Other, 305 Use Proxy, 307 Temporary Redirect, 401 Unauthorized, or 407 Proxy Authentication Required.

Response Lost()

Any loss of communication before a response is received. This may include application or TCP/IP level timeouts, or an explicitly terminated connection. The 504 Gateway Timeout response is also equivalent to Response Lost, and indicates a loss occured somewhere past the TCP connection made directly by the client.

Sample Code

Request request;
request.url="http://example.com/publication-date/first-edition"
information = date(2008-07-05)
if (blocked())
{
	// Only queue the latest document
	// Overwrite any previous request
	request_pending(url) = information
}
else
{
	request.document_and_type = information.default_encode()

try_again:
	switch (request())
	{
	Success():
		// Do nothing. The update has completed.

	Type Not Understood(weighted acceptable types list):
		// Re-encode information in an acceptable form
		request.document_and_type =
			information.encode(
				weighted acceptable types list
				)
		jump try_again;

	Fail(reason):
		log(reason)

	Resubmit(required_changes):
		if policy(request, required_changes)
			request.modify(required_changes)
			jump try_again;
		else
			log("Policy forbids request modification");

	Response Lost():
		if policy(request, no required changes)
			jump try_again;
		else
			log("Too many retries");
	}
}

Known Uses

PUT is widely used less widely than GET on the Web, and is primarily a feature of automation.

Related Patterns

Sun, 2008-Sep-21

Published: Wed Jul 2 20:38:52 EST 2008

Updated: Sun Sep 21 22:49:29 EST 2008

GET

Intent

Transfer a defined set of information on demand from its owner the server to an anonymous client. Do so in a form the client understands.

Motivation

A server wants to express the current state of a light bulb for access by a home automation system. Originally, the system measured whether the light was on or off. As time goes on, the server begins to measure additional attributes such as operating temperature and expected lifetime of the unit.

The bulb status server is built into the lighting assembly, and is sold to work with automation systems developed over a number of generations. Some of these systems expect a simple on/off status indicator. Others can make use of the additional information available.

The home automation system may be dealing with bulb servers from different generations, also. A newer home automation system will have to deal with both old and new bulb servers.

The GET pattern is a tool to permit clients to access the light bulb data in a form they understand. It does not require a close match of versions between the client and server, allowing the interface between the two to evolve over time without breaking compatibility.

The GET pattern also has a performance aspect. Clients seek to minimise load on the server in processing their requests, load on the network in transferring messages, or on themselves in processing responses. The automation system may be based on low power components that cannot sustain high rates of transactions. Alternatively, the system could be large scale supporting thousands of bulbs at a large site or more across multiple sites. Clients also need to be able to deal with possible error conditions, including communication failures.

The GET pattern provides a clean client/server separation that is able to survive independent upgrades of each over time, exercises control over traffic and processing waste, and deals with possible errors.

Applicability

GET is appropriate whenever a client wants to acquire the whole of the information behind a known URL, and can decide when it wants to issue the request (subject to a cache miss). Here are some common means by which a URL is discovered:

  1. Direct entry allows a user to enter the URL through an input device
  2. Configuration allows a document to be prepared ahead of time with links that have particular meaning to the client
  3. Hyperlinking is a generalisation of configuration. The document that contains meaningful links may be acquired from anywhere, including an earlier completed GET request
  4. Construction is the assembly of information available to the client into a URL format agreed with the server. This may be achieved by populating a form supplied by the server in an earlier GET request.

Methods of determining when to issue a GET request include:

  1. One-shot, the issuing of a request at a predefined time, when the URL first becomes known, or when the client needs access to the information
  2. Cyclic, the issuing of a request at a predefined rate while the client is active
  3. On cache expiry, the issuing of a request whenever a cache entry expires. Note that this requires the server to set a maximum age on cache entries, something that is not always provided.
  4. Otherwise-triggered, the issuing of a request based on some form of back-channel that indicates information at a given URL may have changed

Structure

GET pattern structure

Participants

Client
  • Keeps a URL that lets it access the Server
  • Issues a GET request that includes a condition and weighted acceptable types list
  • Is capable of parsing all forms that the data might be encoded in that are semantically rich enough to use
  • Selects the right parser implementation to use based on the returned document type
  • (optional) Retains a cache of past successful GET responses and their related cache control information
  • Is responsible for overall successful execution of the operation, including modifications to the request and resubmissions of the request
  • Treats a lost response as equivalent to a Resubmit response with no required changes
  • Aborts the operation on a failure response, on a resubmission response that cannot or will not be satisfied, or on a lost response after too many retries.
Server
  • Evaluates any condition supplied in the GET request before performing significant processing
  • Selects the information to return based on the supplied URL
  • (optional) Is configured with mechanism to require the client to resubmit their request with or without modifications
  • Guarantees that a GET request is a read-only operation that is never interpreted by the server as a request to "buy an airline ticket". The server may choose to update log files and other information, but is not free to behave as if the client has requested or authorised the change.
  • Can return the requested information in various formats. Any format which the client might reasonably request with its acceptable types list should be supported.
  • Selects the most appropriate encoding based on the supplied weighted acceptable types list and any preference it may have itself, and returns the document in that format

Collaboration

  • Client issues requests to Server via the Request Interface, modifying and resubmitting its request as needed until:
    1. A success response is elicited
    2. The request condition is not met, meaning that the cached response is still valid
    3. A failure response is elicited
    4. The client is unable to make changes required by a Resubmit response
    5. Client policy prevents either changes required by a Resubmit response, or further resubmissions in general

Consequences

The GET pattern introduces a Uniform Interface for transferring identified sets of information from server to client. Clients and servers of different ages can communicate without impediment, and communication failures can be overcome.

The use of an acceptable types list in a GET request means that clients built during different phases of the architecture will generally be able to communicate. Document-based communication has a degree of flexibility built in with must-ignore parameters. The acceptable types list fills a gap when incompatible changes occur to the set of document types, for example a new type deprecates an old type such as atom depreciating rss for news feed syndication.

An explicit failure response allows problems in the architecture to be reported and repaired as required. The resubmit feature allows temporary or permanent changes to the architecture to be accommodated by components without explicit reconfiguration, simplifying management. Note, however, the potential security implications of allowing one component to reconfigure others. A predefined policy for which modifications are permitted and which are to be treated as failure cases can be useful in security-sensitive environments.

The potential exists in common transports such as HTTP for requests sent down parallel TCP connections or pipelined requests to be processed in a different order to that in which they actually return to Client. This could cause the client to become confused by "seeing" an older state after a more recent state. A simple solution is to hold off sending a GET request to a given URL when the previous related GET has not yet returned.

A client that is holding off sending the next GET request should queue the first such request for the identified URL. After this point it should not queue another GET request to the URL until the previous has been transmitted. There is no point queuing up multiple GET requests for the same URL. If the first request has not been issued by the time motivation to issue a second request comes around, a single request will fulfil the motivation behind both.

Twin consequences of the GET pattern are that interim states at URLs may be missed, and that the architecture as a whole does not become overloaded as the architecture is put under stress. Each GET retrieves the current state of the resource, so rapid changes may see the next GET arrive several changes after an earlier GET. These states will be lost unless an additional buffering mechanism is employed. The client will read back the current state rather than the old transitional states.

The flip-side of this behaviour is that clients are never stuck reading old data. They come completely up to date quickly and process the latest information available. Many algorithms for real-time processing will behave better under this scenario than if they are fed through old changes. The GET pattern can be adapted to a buffering model for algorithms that are sensitive to losses of interim states.

Implementation

GET can be implemented with HTTP using the following mappings:

GET(url, condition, weighted acceptable types list)
GET url HTTP/1.1
Accept: weighted acceptable types list
If-condition
Success(document, type, cache)
HTTP/1.1 200 OK
Content-Type: type
Cache-Control: cache

document

All 2xx series response codes can be treated as Success responses for GET

Condition Not Met()
HTTP/1.1 304 Not Modified
Fail(reason)
HTTP/1.1 400 Bad Request

reason

Unknown 1xx series response codes can be treated as a Fail for GET. 300 Multiple Choices is a non-implementable Resubmit response for automated clients, so should also be treated as Fail alongside other 3xx series codes that are not understood. 4xx series response codes are Fail, except for 401 Unauthorised and 407 Proxy Authentication Required. These are Resubmit responses and should only be treated as failures if they are not understood. 5xx series responses should be treated as Fail, except for 503 Service Unavailable and 504 Gateway Timeout. These are Resubmit and Response Lost responses, respectively.

Resubmit(required changes)

Any of: 301 Moved Permanently, 302 Found, 303 See Other, 305 Use Proxy, 307 Temporary Redirect, 401 Unauthorized, or 407 Proxy Authentication Required.

Response Lost()

Any loss of communication before a response is received. This may include application or TCP/IP level timeouts, or an explicitly terminated connection. The 504 Gateway Timeout response is also equivalent to Response Lost, and indicates a loss occured somewhere past the TCP connection made directly by the client.

Sample Code

Request request;
request.url="http://example.com/publication-dates"
if cache_manager.fresh(request.url)
{
	// Do nothing. Our cache entry is still fresh.
}
else if (blocked())
{
	// Only queue one request for the URL
	request_pending(url) = true
}
else
{
try_again:
	request.accept=parser.accept
	request.condition=cache_manager.condition(request.url)

	switch (request())
	{
	Success(document, type, cache):
		cache_manager.update(document, type, cache)
		process(parser(document, type))

	Condition Not Met():
		// Do nothing.
		// We have already processed the
		// latest data with our last request.

	Fail(reason):
		log(reason)

	Resubmit(required_changes):
		if policy(request, required_changes)
			request.modify(required_changes)
			jump try_again
		else
			log("Policy forbids request modification")

	Response Lost():
		if policy(request, no required changes)
			jump try_again
		else
			log("Too many retries")
	}
}

Known Uses

GET is widely used on the Web, both under direct human control and under automation. Various aspects of GET are not always used well.

Common errors in applying the GET pattern include:

  • "Unsafe" GET handling by servers, where GET is treated as an update request.
  • Not including the acceptable types list, meaning that the deployed client component will not be readily handled by an upgraded server component. The wrong document type may be returned.
  • Returning the wrong type with a document, or using heuristics based on the URL to determine which parser to invoke for a returned document.
  • Using type identifiers that are too generic. Type specifications such as application/xml or application/rdf+xml could match multiple formats for the return of data, application/atom+xml and application/atom+rdf+xml allow the server to choose a document to return that is more likely to be understood when the client parses it.
  • Returning a different document from the same URL based on session state. GET requests should not create information on the server that has to be tracked and available when the client's next GET request arrives. The client should be able to issue its next request at any time without further coordination with the server. Sessions may be used to short-cut expensive processing over a series of requests. However, an expired or lost session should not cause a given request to fail or fail to be understood.

Related Patterns

Mon, 2008-Jul-14

Published: Mon Jul 14 21:18:08 EST 2008

Updated: Mon Jul 14 21:18:08 EST 2008

Guide

Architecture Control Patterns

Basic Transfer Patterns

Transfer directionWho knows when to transfer?Applicable Transfer Pattern
Server -> ClientClientGET
Server -> ClientServerCyclic GET, or Subscribe
Client -> ServerClientPUT

Advanced Transfer Patterns

  • Conditional PUT
  • Resume Transfer
  • Streaming
  • Delta Encoding

Content Patterns

  • Hyperlinking
  • Code on Demand

Structural Patterns

Sun, 2008-Jul-06

Published: Wed Jul 2 20:38:52 EST 2008

Updated: Sat Jul 5 21:53:36 EST 2008

Layering

Intent

Intercede between client and server to achieve performance or functionality benefits.

Motivation

Server S wants to serve get requests for five clients, but does not have resources to serve GET requests for all directly. A layer of indirection is added between the server and its clients that shields the server from requests for resources that already have cached documents. The cache returns responses on behalf of the server, reducing the need to generate responses from scratch.

Applicability

  1. Caching Proxy: A proxy that is trusted by a set of clients to act on their behalf. A chain of trust may be established from proxy to proxy in a well-managed network. Caching proxies are generally employed in networks where there the information being transferred is non-secret. Proxies are generally bypassed for secret communications. Protocol handling libraries for REST clients often act both as a Caching Proxy for the client of its API and a gateway from an API-based REST model to that of a specific protocol.
  2. Gateway: A proxy trusted by a set of clients to speak a protocol or participate in patterns not widely understood throughout the architecture. Gateways are generally temporary measures as different architectures come into contact with each other, before they consolidate their organisational structures to effectively deal with their common requirements.
  3. Firewall: A proxy that inspects requests passed through it to make sure they fit defined policy boundaries. The firewall might limit PUT access to certain URLs, remove portion of requests that it considers dangerous, or monitor patterns of requests to actively detect attempts at intrusion.
  4. Content Distribution Network: A proxy trusted by the server to answer requests on behalf of clients. The CDN is part of a chain of trust that extends outward from the server, and can be considered to be operated by the same agency as the server's. CDNs are used to provide local service to clients that may be distant from the main data centre, and may have various forms of private access to Server-side data not generally afforded to clients.

Structure

Layering pattern structure

Participants

Intermediary
  • Accepts requests from Client and generates requests to Server. Requests are submitted in whole, so can be passed on without significant modification as-is. The Intermediary is capable of performing alternate processing on behalf of the server such as returning a cache entry or manipulating requests and responses.

Collaboration

  • Client issues requests to Intermediary instead of Server. Intermediaries may be discovered by explicit configuration of Client, implicitly as part of the network configuration, or as an explicit redirection response from Server (see HTTP's 305 Use Proxy response).
  • Intermediary processes the request. It answers it itself or optionally modifies the request and passes it on to Server.
  • Server accepts requests from Intermediary and processes them as if they had come from Client.
  • The server's response is processed by Intermediary, optionally modified, and returned to Client.

An arbitrary number of layers can be added, each treating the next as Server and the previous as Client.

Consequences

The Layering pattern increases latency for requests that are forwarded unmodified, but can provide performance and functionality improvements when caching or other modifications are applied successfully.

Introducing layers increases the complexity of security, architecture evolution, and simple component testing models. Security and trust become an issue as intermediaries move further from the ownership of either Server or Client. Intermediaries literally are the feared man in the middle of security parlance. An intermediary that chooses to function against the interests of either Client or Server will lead to compromised communication.

Evolution of the architecture can be stymied by intermediaries. Upgrade of client and server to participate in a new operation or pattern is often not sufficient. The presence of old intermediaries under control of neither the Client's agency or the Server's agency mean that work-arounds and fallbacks may be required when they otherwise would not be required.

Testing of components can also suffer. Testing against common Server implementations is not sufficient if an intermediary introduces an unknown factor. Intermediaries may re-order pipelined request messages, splitting a single TCP connection at the Client end into multiple connections by the time they reach the Server. This is a potential source of out of order processing and other mischief that testing only a Client/Server combination may fail to reveal.

Implementation

Sample Code

Known Uses

Layering is widely used on the Web. It is used to implement caching in most HTTP client libraries. Caches are still sometimes used in ISPs to reduce traffic costs on behalf of clients that issue GET requests to similar sets of URLs. Layering is very commonly used as part of the network on the server-side, with load balancing and vertical scaling techniques strongly leveraging this pattern. Content Distribution Networks are becoming more commonly leveraged to serve a global audience with low latency.

Related Patterns

Sun, 2008-Jul-06

Published: Wed Jul 2 20:38:52 EST 2008

Updated: Sun Jul 6 12:25:07 EST 2008

Registry

Intent

Support serendipitous reuse across an architecture

Motivation

Producer P and consumer C want to transfer a document. P has a particular abstract data schema in mind, and C has a compatible schema. Registry ensures that P encodes and transmits the information in a form that C understands.

The methods for applying patterns of information transfer and forms in which information are encoded may be defined either by local convention or by a wider convention. Local protocols may be "jargonised" versions of more widely-applicable protocols. In time, jargon may move from a satellite architecture such as that of a particular enterprise and become standardised in a wider setting such as the Web.

Applicability

Registry is applicable for REST architectures large and small. However, architectures with only a small number of components may not need a formal registry.

Structure

Registry Structure

Participants

Document Type
  • Provides a defined way to encode a given information schema
  • May have one or more parent types, whose parser is also appropriate to read this derived document type
  • Is defined in a Document Type Specification, including accommodation of document type evolution through must-ignore semantics on information that is not understood by processors.
  • May be a target encoding for a number of a number of different information schemas, including internal information schemas for various kinds of applications. For example, HTML can be used to encode information schemas as diverse as journal entry publication and warnings about severe weather events.
  • May act as a bridge between different Information Schema. For example, ODF is able to bridge between OpenOffice and Gnome Office internal document schemas.
  • Is defined to be independent of how it is transported, and orthogonal to Operations that are defined in the architecture.
  • Is developed using the least power principle: The least powerful language that is capable of expressing a set of information is used.
  • Minimises coupling by limiting reliance on local or domain-specific conventions.
Operation
  • Provides a way to transport information around the architecture
  • Corresponds to one or more Interaction Patterns that are understood by applicable information end-points
  • Is defined to be independent of the information it transports, and the specific endpoints it transfers information between. Operations are orthogonal to the set of URLs in the architecture.
Management Body and Process
  • Manages the evolution of Document Types and Operations
  • Oversees the adoption and uptake of new or updated Document Types and Operations in the architecture
  • May depend heavily on external bodies, especially those that manage the Web
  • Typically do not have the authority to upgrade any given component of the architecture. Instead, components are upgraded over time as their respective owners see fit in line with the Registry specifications.
  • Typically do not have any control over the set of URLs in the architecture

Collaboration

  • The Management Body and Processes manages the independent evolution of Document Types and Operations in the architecture. The set of URLs is not controlled by this body.

Consequences

The orthogonal nature of document types, operations, and URLs throughout the architecture allow for freedom of evolution. This separation and evolution is supported through the major patterns of the architecture as old and new components interact, including GET and PUT.

The balance of control and freedom is capable of delivering application-level forwards and backwards compatibility that lasts for a relative eternity in our industry. For example, HTML and HTTP have been able to continuously evolve over a time-frame spanning decades.

More domain-specific document types will likely have shorter lifespans as they develop from local conventions to a point of widespread global agreement. However, as critical mass is created a semantic web where machines can communicate generally based on widely-understood conventions is likely to emerge without specific coordination or intervention.

Implementation

Sample Code

Known Uses

The IETF defines a registry and change management process that includes a list of protocol interactions for document transfer (HTTP/1.1) and legal document types (the IANA).

Related Patterns