JDrupes HTTP Codecs 3.0.0

HTTP codecs for non-blocking I/O.

The HTTP codecs are modeled after the Java CharsetDecoder and CharsetEncoder. An HTTP decoder is an engine that transforms a sequence of bytes into a sequence of HTTP requests or responses (and streams their body data). An HTTP encoder transforms an HTTP request or response (including streamed body data) into a sequence of bytes.

The main difference between the Charset codecs and the HTTP codecs API is due to the type of the decoded data. For Charset codecs this is a homogeneous stream of chars, which is easy to handle. For HTTP codecs, it’s a mixture of headers and body data which can again consist of either bytes or chars.

Decoders

Decoders realize the Decoder interface.

Decoder diagram

Binary data received from the network stream is passed to the
Decoder.decode(java.nio.ByteBuffer, java.nio.Buffer, boolean) method in a ByteBuffer. The method consumes as much data as possible from the buffer and returns the result of the decoding process.

Decoder result

The basic information provided by the decoding process (defined in Codec.Result) is known from the Charset codecs. “Underflow” indicates that more input data is needed in order to complete the decoding of the message. “Overflow” indicates that the output buffer is full. “Close connection” is mostly set by encoders and indicates that the connection should be closed. This is explained in more detail in the next section.

Besides streams with body data, decoders such as an HTTP decoder provide the headers that precede this (payload) data. The successful decoding of a header is indicated in the result by Decoder.Result.isHeaderCompleted(). The decoded header can be retrieved with Decoder.header(). Of course, if the receive buffer is rather small and the header rather big, it may take several decoder invocations before a header becomes available.

Sometimes, a protocol requires a provisional feedback to be sent after receiving the message header. Because the decoder cannot send this feedback itself, it provides the message to be sent in such cases with Decoder.Result.response().

If a received message violates the protocol or represents some kind of “ping” message, sending back the prepared response message may be all that has to be done. These cases are indicated by Decoder.Result.isResponseOnly()).

Encoders

Encoders realize the Encoder interface.

Decoder diagram

Encoding is started with a call to Encoder.encode(MessageHeader). Subsequent calls to Encoder.encode(Buffer, ByteBuffer, boolean) fill the output buffer with the encoded header and the body data. If the information in the header indicates that the message does not have a body, Encoder.encode(ByteBuffer) can be called.

The result of the encode method is a Codec.Result that indicates whether the output buffer is full and/or further body data is required. In addition, Codec.Result.closeConnection may indicate that the connection, to which the message is sent, should be closed after sending the message. This indication is needed because closing the connection is sometimes required by protocols to complete a message exchange. As an encoder cannot close the connection itself, this must be done by the invoker (the manager of the connection).

Why Generics?

While the previous sections explain the interfaces and classes with reference to HTTP, you don’t find “HTTP” in the names or methods of the types discussed. The reason is that the API presented above can be used to handle any “HTTP like” protocol (header with payload). We need such a general interface because modern HTTP provides the upgrade mechanism that allows the client or server to switch to another protocol. This is currently mostly used for the web socket protocol. More about that later.

HTTP Codecs

An HTTP decoder is a special decoder that returns HttpMessageHeaders in its HttpDecoder.header() method (type parameter T). Of course, if the result of the decode method includes a response, it’s also of type HttpMessageHeader (type parameter R).

HTTP Decoder

In addition, it is possible to specify a maximum header length to prevent a malicious request from filling all your memory. And you can query if the decoder has reached the closed state, i.e. won’t decode more messages, because the connection should be closed (if indicated by the result) or will be closed at the other end after sending a final response.

The HTTP encoder is derived in a similar way.

HTTP Decoder

See the method description for the meaning of “pending limit”.

As you can see, we still haven’t reached the goal yet to get concrete HTTP codecs. This is because there is a difference between HTTP request messages and HTTP response messages.

HTTP request and response messages

Now we have all the pieces together. In order to write an HTTP server you need an HttpDecoder parameterized with HttpRequest as type of the decoded message and HttpResponse as type of any preliminary feedback (optionally provided by the Decoder.Result). This is what makes up an HttpRequestDecoder. And you need an HttpEncoder parameterized with HttpRequest as type of the messages to be encode, in short an HttpResponseEncoder.

For implementing an HTTP client, you need an HttpRequestEncoder and an HttpResponseDecoder.

Have a look at the classes javadoc to find out what kind of support each of the codecs provides regarding header field handling.

Protocol switching

HTTP supports a client initiated upgrade from the HTTP protocol to some other protocol on the same connection. If the upgrade request is confirmed by the server, subsequent messages from the client are sent using the new protocol. This, of course, requires using different codecs.

Those codecs, or at least a subset of their functionality, is actually already required when the confirmation response is encoded. HTTP allows the confirmation response to contain information that is related to the new protocol. Obviously, this information cannot be provided by the HTTP encoder, because it knows nothing about the new protocol.

The HTTP encoder therefore takes the following approach. When the header to be encoded contains the confirmation of a protocol switch, it uses the ServiceLoader to find an appropriate protocol provider. Protocol providers must be derived from UpgradeProvider. Whether a protocol provider supports a given protocol can be checked with the method UpgradeProvider.supportsProtocol(java.lang.String). The library contains by default the WsProtocolProvider, the probably best known use case for an HTTP protocol upgrade.

If the HttpResponseEncoder cannot find a suitable protocol provider, it modifies the response to deny the protocol switch. Else, it asks the provider to apply any require changes to the confirming response.

The HttpResponseEncoder returns an extended result type that implements the Codec.ProtocolSwitchResult interface.

ProtocolSwitchResult

When the encoder finishes the encoding of an upgrade confirmation, Codec.ProtocolSwitchResult.newProtocol() returns the name of the new protocol (in all other cases it returns null). In addition, the result also provides new codecs obtained from the plugin provider. These codecs must be used for all subsequent requests and responses.

Engines

The codecs provided here are deliberatly restricted to using Buffers at their interface. They cannot acquire or send such buffers, as this would tie this library with stream mechanisms beyond the passing of Buffers. It is therefore not possible to provide autonomous engine functionality such as automatically sending a preliminary response (as described above).

Nevertheless, the package includes a ClientEngine and a ServerEngine. Both simply group together a decoder and an encoder as required for client-side or server-side operation. As support function, they adapt themselves to any protocol change, i.e. they replace the engine’s codecs if the encoder result includes new ones.

Integration

The demo server code demonstrates how the HTTP codecs can be used to implement a single threaded, blocking HTTP server. Of course, this is not what this library is intended for. It should, however, give you an idea how to integrate the HTTP codecs in your streaming environment.

An example of integrating this library in an event driven framework can be found in the JGrapes project.

Packages
Package
Description
The entry point interfaces and classes.
Provides the API for adding “upgrade” protocols
Classes required for handling the HTTP protocol.
HTTP codecs required on the client side.
HTTP codecs required on the server side.
Classes for handling the Websocket protocol.
Special types related to HTTP.