The Elephant In The Network
HTTP, RPC, Programming Languages, Parsers And Developers
This article is an analysis of network messaging. It looks at the standards and products that are relevant to the sending and receiving of messages over network transports, with the goal of improving understanding in some of the murkier corners.
The intended audience are software developers who spend time in this space and have questions about the tools available to us for the creation of networking software. If you have ever felt frustrated with a limitation of your tools, confused about the best choice of tools, or had a background sense that a piece of the puzzle is missing, this might help.
This is a large problem space that involves an evolution of technology spanning decades, and the influence that software culture and online commerce has had on that evolution. The article wanders a bit to establish a context.
Basic Network Messaging
Reliable transports are the essential magic that networks provide. They are the foundation of distributed computing. To developers who work at this level, transports mean sockets, and sockets are about the sending and receiving of blocks of bytes.
Before those blocks of bytes are useful to an application there is some work to do covering areas like encodings, marshaling and I/O management. The goal of all that work is to turn a flat, meaningless stream of bytes into a communications channel. Instead of blocks of bytes, what you want is a two-way exchange of application messages like GetSubscriber and UpdateDevice.
A popular way to manage those application messages is to bundle them together and refer to them as an API (Application Programming Interface), with the individual messages referred to as RPCs (Remote Procedure Call).
RPC is a term coined in the 1980s which takes the call-return concept from CPU instruction sets and renders it as call and return messages (or request and response) over a network. This idea was met with immediate acceptance as a good use of a communications channel.
The major and obvious reason for that acceptance was the low-friction integration with procedural programming languages. Anyone that can use a function in their favourite language — without further intellectual gymnastics — can also use an RPC. Unfortunately this strategy has glaring deficiencies that are discussed later on. For the moment it is enough to know that RPC is just one execution model for network communications, created several decades ago by a guy called Bruce.
If you go looking for more information about APIs and RPCs then you will be inundated with further products, definitions, acronyms and abbreviations such as HTTP, paths, URLs, queries, headers, protobuf, JSON, REST, OpenAPI and gRPC.
Navigating this ocean of competing and overlapping concepts and products is difficult. A few general statements can be made;
- there are substantial toolsets available for implementation of APIs,
- the major API toolsets seem to be based on RPC and HTTP,
- HTTP is typically combined with an encoding, e.g. JSON or protobuf,
- HTTP brings encryption, authentication and message routing.
HTTP (Hypertext Transfer Protocol) was designed for moving materials between a browser and a website. It also seems to support the concept of RPC directly, with formal definitions of request and response messages. It delegates the details of RPC by carrying a generic payload on behalf of the caller. It’s up to the caller, using something like JSON, to arrange for the encoding of call parameters and return values, and pass the result to HTTP as the payload (i.e. body).
HTTP also brings encryption, authentication and message routing. These are required features for a specific target environment, i.e. the Internet. This trio of capabilities exist in HTTP to ensure private, secure communications between browsers and backends, where backends are compositions of routing servers and your API servers. Underlying technologies such as TLS, PKI and HTTP headers combine to provide the security guarantees that websites demand.
It would be hard to over-emphasize the important part that RPC and HTTP have played in the evolution of websites. Encryption and authentication have enabled online business and routing servers (i.e. gateways, tunneling and load-balancing) have enabled that business to operate on a massive scale. Together, RPC and HTTP have enabled a major aspect of modern digital life.
Not The Only Game In Town
The RPC/HTTP phenomenon is focused on a specific scenario within the wider context of computer networking and it has been ably supported by tools such as OpenAPI and gRPC. However, the success of this combo does not mean it is a good response to all networking needs, and that tools like OpenAPI should be applied to every situation.
The first issue is that not all network addresses are websites. Adoption of HTTP within your communications solution tacitly applies the website abstraction to your software. It does this through the presence of request URLs and headers such as Host; a single line of code that evaluates one of these values creates a dependency to the abstraction. Adoption of OpenAPI takes this even further by imposing the REST abstraction over your software, a model based around entities and standard operations on those entities (i.e. CRUD).
Concepts like websites and REST (which assumes that there is a database behind every network address) add value by organizing the thoughts and materials associated with implementations. It follows that mis-applying such concepts must have an equally negative effect, i.e. a networked, metering gauge is not a database. While concepts can be bent to fit a purpose it creates a project dissonance.
Secondly, there are deployment environments where the features of HTTP that are so crucial to the world of websites, are manifestly redundant.
Consider an ETL host that runs a few hundred co-operating processes with the goal of maximizing the benefits of concurrency, or an automated production line with various networked sensors and actuators, all within a LAN. Communications within these scopes does not need encryption, authentication or routing. A case can be made for encryption on a LAN but alternative communications libraries (i.e. non-HTTP) also support encryption.
Persisting with HTTP in these situations is likely to produce software that is forever consuming more system resources than it otherwise might, e.g. increased size of on-the-wire messages, doubling of byte-level processing and management of large memory blocks. In extreme cases this might be the difference between meeting performance criteria or not. Designers should also be considerate of other users where a resource is shared (e.g. the LAN) .
The Elephant In The Network
The development community has long been aware of limitations of the RPC model (e.g. server-side events or push technology, refer to long polling), but a clear and effective response to this issue has not emerged.
The root cause of these difficulties is simple — RPCs come from the world of procedural programming and procedural programming has never had a clear and effective approach to managing multiple threads of execution. The problem around RPCs is the same problem that procedural languages have with multi-threading or multi-processing; it’s easy to create a new platform thread but then it’s on its own. There is no standard mechanism within procedural programming to monitor and control (i.e. interact with) the new thread.
This is the elephant in the network.
There have been major improvements to HTTP including the addition of multiplexing. This has brought substantial improvements in speed and resilience, but without asynchronicity in the application, this changes little. For server-side events a procedural application still needs a custom implementation of long-polling (or an equivalent) involving some form of multi-threading. At best this might be able to enjoy its own dedicated channel, within the single HTTP/3 session.
OpenAPI supports some asynchronicity and also integrates with AsyncAPI. At a lower level there are also implementations of QUIC which is the protocol at the core of improvements to HTTP/3. These are alternative approaches to introducing asynchronicity into your procedural applications.
For a taste of what fully asynchronous programming looks like, refer to this library. Full disclosure — the author of this article also developed the library.
How Much Does HTTP Really Cost
This section sketches out a simple implementation of an RPC mechanism. It then introduces HTTP to the mix and makes some before-and-after comparisons. The resulting numbers give some substance and scale to the benefits of dropping HTTP from communcations where it is not needed.
An encoding is needed to transfer the details of an RPC call from a client to a server, and deliver the return value back to the client;
---> <function-id>, <arg values>
<--- <return value>
---> <function-id>, <arg values>
<--- <return value>
..The RPC call instruction includes something to identify the function — like a string, and a collection of argument values. The function name will be mapped to an actual function in the receiver, and the arguments will be passed on the subsequent call.
An example exchange using JSON might look like;
---> ["xyz", {"a": 10, "b": 20}]
<--- true
---> ["abc", {}]
<--- {"name": "constantine"}There is a 2-element, JSON list sent as the call instruction and a single value — which could be a JSON list or object — sent as the return value. The second element of the call pair is always a JSON object. Newlines terminate the messages.
The calling machinery in the receiver will look something like;
name, kw = receive_request_line()
f = lookup_function(name)
value = f(**kw)
send_response_line(value)Add HTTP And Stir
An RPC request message using HTTP might look like;
POST /basic-rpc HTTP/1.1
Host: 127.0.0.1:5090
Content-Type: application/json
Content-Length: <length>
["xyz", {"a": 10, "b": 20}]This is an obvious combination of HTTP materials and one of the JSON encodings from previous paragraphs. HTTP header values (i.e. Content-Type and Content-Length) are used to quantify the payload or body — in this case the JSON encoding.
This is a representation over 4 times the size generated for the basic encoding alone. Admittedly that statistic is at its worst for smaller messages and in a common scenario — an API over a data model — there are many requests associated with data forms; the associated JSON encodings are likely to be larger. Perhaps a more useful thing to say is that a constant number of bytes has been added to every message sent. As the message rate increases the total number of HTTP-related bytes on the network, at any one time, multiplies. A system that relies on a high rate of smaller messages will be impacted the most.
A second issue with this combo of HTTP and JSON is that it introduces a double pass at byte level. The first pass detects a proper HTTP encoding and delivers the JSON text to the second pass. Byte-by-byte processing is CPU intensive and a double pass of the same bytes is unfortunate. The CPU cost of turning bytes into messages has acquired another overhead proportional to the size of the body.
If Only Network Messaging Was Like This
Technically, it is possible to implement the optimal processing of a byte stream based on an asynchronous parser. It would look something like this;
def received_block(block, parser):
for name, kw in parser.decode(block):
f = lookup_name(name)
value = f(**kw)
parser.encode(value)A block of bytes fresh off an asynchronous socket is presented to the received_block function. The parser.decode method is a coroutine that yields decoded JSON requests. Looping on that method allows the block to contain zero or more request completions — a clean reflection of the fact that encodings can arrive in different sequences;
- starting and ending within a single I/O block,
- starting in one block, spanning zero or more intermediate blocks and ending in a trailing block.
The parser object effectively carries the current state of stream processing from one method call to the next.
Another nice benefit of this approach is that there is an immediate conversion of completed lexemes (e.g. strings and integers), and structuring of those values, as appropriate (e.g. arrays and maps). Bytes can be drip fed to the parser to build arbitrarily large and complex values, without the entire JSON input ever existing as a single, contiguous block. This is easier on memory management within a process.
This is a compelling template for implementation as it cleanly resolves multiple overlapping difficulties. However, there are several other considerations and a major design problem with most parsers, that precludes this approach.
Parsers are generally written in a synchronous style (vs the async style above) and they are greedy by design;
$ python3
>>> import json
>>> json.loads('["abc", {}]')
['abc', {}]
>>> json.loads('["abc", {}] ["xyz", {"a":1}]')
..
json.decoder.JSONDecodeError: Extra data: line 1 column 13 (char 12)The standard Python JSON decoder considers the presence of the second RPC to be an error. Writing a custom parser that accepts asynchronous blocks of bytes and yields inbound RPCs is a substantial — but not unreasonable — development task. It would also create the opportunity to design the parser for a rich type system, reducing the quantity of conversions required during stream processing. In Python, time values are stored in datetime objects. In the JSON encoding there is no time type, forcing the use of strings and adding the additional layer of conversion.
Even with the synchronous, greedy parser issue resolved you would still need to factor in other considerations such as encryption, and encryption suffers from similar issues as parsing. Libraries such as Salt do not see their role to be encryption and decryption of discrete sections of an endless byte stream. The compelling template is effectively and sadly, off the table.
Inclusion of HTTP has the following consequences;
- an increase in the size of every message,
- a double-pass of byte-level processing,
- ongoing management of large blocks of memory.
Summary
The elephant in the network is the process with no asynchronous capability. It’s connected but when you poke it, there is no response. Sending it the HouseIsOnFire message does nothing to disturb its contemplations. Otherwise known as server-side events or unsolicited messages, these stimuli have no matching receptors; this process lacks the related DNA.
Any expectation of a solution arising from work around networking libraries is, at best, based on a mis-understanding. An asynchronous networking library just moves the disjunction inside the process. A side-benefit of a real solution will be the ability to interact with threads, processes and networks, in a consistent way.
HTTP and RPC are a giant success story in the evolution of the world wide web and cloud services. This has enabled secure, online commerce at huge scale and given rise to a large selection of effective and mature tools.
There are deployment environments that have no need for what HTTP brings. Alternative approaches to messaging that do not involve HTTP are entirely realistic and the benefits of such a direction are real. This will be of maximum interest in scenarios where the network activity is chatty — a high rate of small messages — and there are specific performance goals.
Adopting a non-HTTP messaging solution in those scenarios that dont need it, may produce a clearer and more maintainable codebase. This is based on the notion that HTTP and its associated toolsets may be pushing project materials in an inappropriate direction (e.g. RESTful interfaces over non-database networking software).
Taken together, these points indicate that a real need exists in those deployment environments that do not require encryption, authentication and routing. This might include;
- complex IPC within a host,
- on-premises, distributed computing within a LAN,
- backend services within a cloud (a special case of the above).
An advanced, asynchronous solution to the messaging within these scenarios will produce more efficient software and facilitate otherwise difficult distributed behaviours. The distinction between this style of distributed computing and the messaging that occurs between browsers and websites, seems strong and unlikely to change. Different tooling is appropriate.
In the case of backend services there will be hybrid processes on the boundary between the world of RPC/HTTP messaging and the asynchronous messaging within the backend. The asynchronous messaging solution must support a minimal HTTP interface. Encryption and authentication will be handled upstream of these boundary processes in the routing servers deployed to the cloud.
