RFC |
0005 |
|---|---|
Title |
JSON over HTTP for data serialization and transport |
Sponsor |
|
Status |
Draft 💬 |
Type |
Standards |
Created |
2020-01-07 |
Otto is a distributed system in which multiple services need to communicate with one another in standardized formats with a common transport layer. Practically every component in the system acts as both a producer and a consumer of data. Consolidating all service-to-service interactions onto a single format and transport layer offers numerous benefits in tooling and shared code. It is also very important to make the Otto ecosystem easily interoperable, such that users can bring their own services into the Otto ecosystem without requiring any implementation changes from the existing services.
This specification describes the standard methoa and format for all service-to-service interaction in the Otto distributed system. The interactions in the system are expected to take two forms:
-
Synchronous: a traditional request/response model with two services, or a client and service, directly interacting with one another. This flow is expected to follow the REST API model described below.
-
Asynchronous: oriented around the Otto eventbus, whereby services can send a message to a channel and expect no response other than a confirmation that the message was received. The details for event transmission and interaction with the Eventbus is described in the Events section below.
Generally speaking, REST APIs are intended for administrative for user-interface related data requirements. For the most part, services are expected to communicate with one another using Events on the Eventbus rather than sending synchronous requests back and forth.
All REST APIs must be:
-
Versioned: The URLs should be versioned using the major version number from the service’s Semantic Versioning scheme. For example,
GET /v1/channels. -
Documented with OpenAPI:
-
Accept JSON: All requests to REST APIs must be documented JSON payloads, except in cases of binary object uploads
-
Respond with JSON: All REST APIs are expected to only respond with documented JSON payloads. Any non-JSON response, except for expected binary blobs, are to be considered by the client as a server-side error.
Errors should be communicated by the server with JSON encoded error messages and status codes:
-
4xx: Malformed request or other client errors. Clients should not attempt to retry these errors, but instead log them as local errors and try to recover.
-
5xx: Server-side failures. Clients _should* attempt to retry the requests after an exponential back-off, with some permanent failure mode after a period of 30+ minutes.
If the server does not respond with a JSON payload, the client should treat the response as if it is a recoverable error and perform an exponential back-off retry.
Services are all expected to emit and receive events over a
WebSocket
connection to the Eventbus service, e.g. http://eventbus.lan/ws/. The Eventbus acts as a quasi-central broker
of all events occurring in the Otto distributed system. Its design is discussed
in another RFC< but the message payloads sent to "channels" on the eventbus
must be JSON data structures matching JSON Schemas defined for each channel.
For cases where the WebSocket connection has errored, reset, or is otherwise unavailable, services must attempt to retry into infinity. Local error logs should be generated during these failure schenarios and the service’s health check URLs should also convey that the Eventbus connection is down.
The only errors which should be expected by services writing events to the Eventbus should be schema validation errors, which are considered to be similar to REST API client-side errors Therefor local errors should be logged, but the message should not be retransmitted.
Interprocess communication occurring via JSON over HTTP helps meet a number of the key design goals for Otto:
-
Avoiding a singular central hub
-
Extensibility must not come at the expense of system integrity
-
User-level extension of the system must be viable
The key aspect of JSON over HTTP which helps address all of these goals is that JSON and HTTP are interoperable with practically everything in the modern technology environment, from backend services, to cloud storage providers, and also user’s browsers.
By providing documented synchronous and asynchronous APIs, Otto should be able to accomodate new user-provided services into its ecosystem that add additional functionality without any substantive change to the core components shipped with Otto. One example which comes to mind is an analytics system around Otto. In its current design there is not analytics system tracking tasks executed, resources utilized, etc. If a user or provider wished to offer budget forecasting, as an example, this JSON over HTTP model with REST APIs and events via WebSockets would allow for the deployment of services which could tabulate interesting events off of the Eventbus, and easily provide the end-user with a forecast. Implementing such a system, assuming the necessary events are present in the Eventbus and the necessary data is available from existing services' REST APIs, would require zero changes to the "base" components of Otto.
Since browsers can also natively "speak" JSON over HTTP, this also allows for new and interesting user interfaces atop these services and events to be developed.
The only serious alternative considered was the use of Protocol Buffers for the data format/serialization and gRPC for the transport layer between services.
This approach would be functionally very similar, and may even offer some schema and data validation benefits when compared to JSON over HTTP.
The gRPC approach does add implementation overhead however for new clients, which must have shared binary stubs with other services generated from the Protocol Buffer IDL [1]. This increases the coupling between services, which must all be booted with shared definitions of data, but also prevents easy interoperability with "outside" services and web browsers.
Ultimately, the portability story for Protocol Buffers and gRPC pales in comparison to JSON over HTTP. For Otto, the extensibility and portability of data in the system is a key design goal.
The authentication and authorization of connections over HTTP is not subject to this specification. Additional concerns around data verification through signatures or other means are also not intended to be discussed in this specification. The securing of the transport layer is a concern which will be addressed in future extensions to this design.
There are no testing issues specifically related to this proposal. However clients and services will need to express API documentation which can be automatically tested in the future. Tooling to help this already exists, for example Dredd which works with OpenAPI/Swagger specifications.
-
actix-web: a Rust framework for building high-performance web services.
-
serde_json: a Rust library for serializing and deserializing JSON objects
-
JSON Schema: tool for specifying JSON data structure and requirements.