Skip to content
/ Yllibed.StreamMultiplexer Public

Yllibed Multiplexed Encapsulated Streaming Protocol (YMESP) - a protocol and implementation in .NET for multiplexing streams.

License

MIT license
1 star 0 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

carldebilly/Yllibed.StreamMultiplexer

BranchesTags

Repository files navigation

Yllibed Multiplexing Encapsulated Streaming Protocol (YMESP)

The Multiplexing is a mecanism to establish inner streams, multiplexed into the main stream (called low-level stream).

This protocol is in no way part of any standard and has been designed by Carl de Billy in July 2017. The name Yllibed is from his last name, once reversed.

Installing

Get it on Nuget:

NuGet

Features

  • Peers are equals: There is no "server" or "client" side of the connection. We just call them Peers.
  • Bi-directional streams: Each peer can ask for a stream to the other peer. Useful for NAT Traversal or server-side push scenarii.
  • TCP Fiendly: It has been designed to be friendly with TCP/IP network streams over Ethernet connections. The packet sizes has been designed to fit the most common network topologies on the Internet.
  • Push-Back Protection: A common problem with multiplexing is data can accumulated for a stream on the other peer, waiting to be processed. A stream-level ACK (DACK packets) mecanism will prevent this problem.
  • Pipelining: Many packets can be sent before waiting for ACT from the other peer. This will maximize the bandwidth usage and minimize DATA -> DATA-ACK -> DATA -> DATA-ACK latencies.
  • Keep-alive mecanism: To ensure the connection is kept alive, a keep-alive mecanism is built-in in the protocol. Useful for NAT Traversal to force the firewall state to stay alive.
  • Name-based streamd: When opening a stream to the other peer, a full name is used and the other peer can choose to accept or reject the request.

Features of the .NET implementation:

  • Async & Multi-threaded: Based on the .NET TPL Library and the Async Framework, the operations of the YMESP .NET implementation is both parallel (multi-threaded) and asynchroneous (non-blocking operations). The hosting application can optionnaly supply its own scheduler for multithreading operations.
  • Fully compatible with existing code: The streams are System.Net.Stream implementation. Any piece of code designed to used streams in .NET should works correctly with YMESP's virtual streams.
  • Few Dependencies: The .NET implementation of YMESP is using the native framework directly with very few dependencies, to ensure maximal interoperability. The only hard dependency is the Microsoft Immutable Collections project used for threading safety.

Sample Code using the YMESP .NET Implementation

Let's start first with a very simple code in .NET:

  public static async void Main(string[] args)
  {
    // The other peer just need to connect to it using a TcpCLient
    // It's just a sample: any stream will work!
    // Feel free to encrypt it or use another transport like a serial port!
    var server = new TcpListener(1234); // Wait for connection on TCP 1234
    var tcp = await server.AcceptTcpClientAsync();
    var lowLevelStream = tcp.GetStream();

    // Create the multiplexer
    IMultiplexer multiplexer = new Multiplexer(lowLevelStream);
    multiplexer.RequestedStream += OnRequestedStream; // register for other peers asking for streams
    multiplexer.Start(); // ensure you are handling the .RequestStream before starting it!

    // As an example, request few streams to other peer
    Stream stream1 = await multiplexer.RequestStream("A");
    if(stream1 == null)
    {
      // stream "A" rejected
      return;
    }
    Stream stream2 = await multiplexer.RequestStream("B");
    if(stream2 == null)
    {
      // stream "B" rejected
    }

    // Here do what you want with your newly created streams!

    // When finished
    multiplexer.Dispose(); // will close everything (even the lowLevelStream)
  }

  private static void OnRequestedStream(object sender, StreamRequestEventArgs args)
  {
    switch(args.Name)
    {
      case "C":
        ProcessStreamC(args.GetStream());
        return;
      case "D":
        ProcessStreamD(args.GetStream());
        return;
    }
    // after that the stream request is rejected because no handler called the args.GetStream()
  }

Vocabulary

  • Low-Level Stream : This is the stream in which "everything" is passing.
  • Peer / Other Peer: Represents one end of the low-level stream.
  • Stream (or Multiplexed Stream): This is a multiplexed stream, tunnelled in the low-level stream.
  • Stream Id: The is the 32 bits identifier to refer to a stream.
  • Stream Name: This is the name of the stream, used by the requester to "ask" to for a stream on the other peer.

Not covered by this protocol

This protocol is a building block for building complex applications.

Here some parts not covered by this protocol:

  • Establishment of the low-level stream: It could be any kind of reliable connection stream. Usually a TCP/IP stream, it could be anything else. Should work with a serial port, by example. For testing purposes, we use a YMSP stream into another one.
  • Security/Encryption: This protocol is plain-text. It means if you need something secure, you should wrap it in secure channel. System.Net.Security.SslStream in .NET is a good start if you need this.
  • DOS protection: There is no mecanism to prevent DOS-type attack on this protocol. Should be managed on the low-level stream.

Important about sub-Streams

  • Sub-streams are of type MultiplexerStream, a nested type to Multiplexer.
  • Sub-streams are buffered: it means data they won't be sent until the buffer is full or until you explicitely flush it.

Low-Level Stream Protocol - UNDER THE HOOD

  1. The first bytes (ackBytes) are sent by both peers.

    • If the bytes are not recognized by both peers, the stream is closed.
    • Default bytes is the 12 bytes Yllibed.MUX. (transmitted one byte per character, i.e. UTF-8 w/o BOM) but can be overriden in constructor.

  2. The following byte is the protocol version

    • Current version is 1 (0x01, not the character).
    • Any unsupported values from other peer should close the stream as not accepted.

  3. The following byte is a bit-flags options.

    bit usage
    0 If the CRC-32 is available on the peer.
    1 Undefined - should always be "false"
    2 Undefined - should always be "false"
    3 Undefined - should always be "false"
    4 Undefined - should always be "false"
    5 Undefined - should always be "false"
    6 Undefined - should always be "false"
    7 Undefined - should always be "false"

    Bit #0 is the LSB.

  4. The followings format will be repeated until the end of the stream:

    offset length bits format name usage
    0 2 16 ushort streamId The concerned stream number. 0 is reserved.
    2 1 8 byte type The packet type (see section below)
    3 1 8 byte undefined Reserved for future use - should always be 0x00.
    4 2 16 ushort length Length of payload (can be zero)
    7 length ... raw data payload This is the payload of the packet

Packet Types

code name meaning payload
0x00 NOP No OPeration empty
0x01 REQ Stream request windowSize (ushort) + streamName (in UTF-8 w/o BOM)
0x02 ACK Stream request accepted windowSize (ushort)
0x03 NAK Stream request rejected empty
0x04 COL Stream request collision suggestedId (ushort)
0x10 DATA Payload data raw data (max 1402 bytes) + [CRC-32 or 0x00000000]
0x11 DACK Payload data acknowledge empty
0xFE ERR Stream error errorCode (ushort)
0xFF FIN Stream closed terminationType (byte)

Requesting a stream

  1. One peer pick a tentative streamId (could be incremental or random - not important). Stream id 0 (zero) is reserved.
  2. A REQ packet is sent to the other end using the tentative streamId.
    • This packet starts with the other end's window size (unsigned 16 bits)
    • The remaining of the payload is a UTF-8 name of the stream.
  3. Wait for any packet for this streamId:
    • if ACK, we can consider the stream as opened and proceed with packets using provided window size.
    • if NAK, means the other peer rejected creating this stream
    • if COL, means a collision and must restart at step 1.
  4. Proceeed to payload data for this stream.

Data Exchange & Flow Control

  1. Any peer can send any DATA packet to the other peer and wait for a DACK. If the CRC-32 fail, an ERR_CRC_FAIL error should be sent back, followed by a FIN packet. The validation of the CRC-32 is optional but recommended. A CRC-32 with value 0x0000 means the sending peer didn't calculated it.

  2. Pipelining Mode: A peer can sent many DATA packets before waiting for corresponding DACK packets. The only restriction is the other peer's window size who should never be overreached. If this occurres, an ERR_BUFFER_OVERFLOW could be generated by other peer.

  3. While waiting for DACK packets, the corresponding write operation will be blocked (for the concerned stream only). This mecanism will prevent the common "push-back" problem well known in multiplexing protocols.

  4. Zero-length payload DATA packets are useless, but they are valid in this protocol and got no special meaning.

VERY IMPORTANT: each DATA received must result in a DACK response.

Terminating a Stream

  • As soon a peer send a FIN packet, the concerned stream is closed.
  • Buffers will be deallocated.

Stream Ids

  1. The id of the stream can be any non-zero values.
  2. The stream 0 is reserved for future signaling purposes.
  3. If a peer received an id already allocated in its table, it should simply return a COL packet with a suggested Id the other peer can use (could be ignored).
  4. When a stream is closed, its id can be reused. As soon the other peer accepts it, it's valid.

Streams ids are not exposed to the application, because no application code should rely on it.

Stream Names

The name of the stream can be anything. Any UTF-8 compliant value is accepted. The meaning of the name is application-specific.

The max length of the name is limited to the packet size. The maximum length is 1404 bytes of UTF-8 characters. As a precaution, the name length is limited 1024 characters (latin-equivalent). If you need to deal with non-latin character sets, do your maths :-).

Zero length names are valid too on this protocol. It's up to the other peer to decide if the stream request is valid or not.

The way the Stream names has been designed is to use them as service names. Use them as port number in TCP/IP or WebSocket-Protocol in the WebSocket stream negociation.

Keep-Alive Mecanism

A NOP packet could be sent on any stream (usually the pseudo-stream 0) as a keep-alive mecanism.

  • Such packet should be sent when there's nothing transfered for more than 15 seconds.
  • A peer can close the low-level connection when nothing is received for more than 30 seconds.
  • The keep-alive mecanism is for the low-level stream only. There is no timeout mecanism for streams.

Error Codes

code name meaning
0x0F01 ERR_UNKNOWN_PACKET_TYPE An unknown packet type has been sent. (a FIN should follow)
0x0F02 ERR_CRC_FAIL A DATA packet failed CRC-32 check.
0x0F03 ERR_BUFFER_OVERFLOW An unknown packet type has been sent. (a FIN should follow)
0x0F04 ERR_TIME_OUT Nothing received for too. (a FIN should follow)
0x0F05 ERR_PACKET_TOO_LONG A packet is received exceeding the limit packet size.
0x0F06 ERR_PACKET_TOO_SHORT A packet length is too short to hold required information.

Termination Types

code name meaning source
0x00 NORMAL The stream terminated normally. Peer App
0x01 PROTOCOL The stream terminated by a protocol (YMESP) error. YMESP Protocol
0x02 ERROR The stream terminated by an error (emitted by the application peer) Peer App

FAQ

Why the DATA payload is limited to 1402 bytes?

Calculations:

  • Most modern networks (WIFI, LAN and WANs) are using the Ethernet for the physical connections.
  • The Ethernet II (the most common Ethernet frame type) common frame size is 1518 bytes.
  • The Ethernet II header size is 14 bytes, and the CRC-32 is 4 bytes, leaving 1500 bytes for the IP packet.
  • Some Internet providers will used encapsulating protocols like PPPoE. This protocol takes a 8 bytes overhead, leaving 1492 bytes for the IP packet.
  • The IPv4 packet header usually take 20 bytes, while the IPv6 header is typically 40 bytes, leaving 1432 bytes for the TCP transport protocol. We'lll use IPv6 header size for the calculation.
  • The TCP header usually take 20 bytes (without options), leaving 1412 bytes for the YMESP packet.
  • The YMESP header is 6 bytes, and the DATA payload requires a 4 bytes CRC value, leaving 1402 bytes for the payload.
  • Remember this is only a tweak for performance purposes. Other network configurations will work well too.
Layer / Protocol Overhead Ov. Size Remaining
Ethernet (physical) - - 1518 bytes
Ethernet II Frame MAC Addressing 18 bytes 1500 bytes
ISP Encapsulation PPPoE Header 8 bytes 1492 bytes
IPv6 IPv6 Header 40 bytes 1432 bytes
TCP TCP Header 20 bytes 1412 bytes
YMESP Packet YMESP Header 6 bytes 1406 bytes
YMESP DATA DATA CRC-32 4 bytes 1402 bytes

Stream Name: This calculation also mean the max length for a streamName is 1404 bytes, encoded as UTF-8. Depending on the choosen characters set, the string length should be limited to 1024 latin characters.

What are the CRC-32 Settings?

This protocol is the common CRC-32 Reversed Polynomial Representation of 0xEDB88320 (the same settings as used by ZMODEM, GZip, MPEG-2, PNG and Ethernet IEEE 802.3 Frames).

The CRC-32 is used only when both peers activated it in their stream establishment.

What is the endianness of the values?

All values are transfered using the Little-Endian format, as in Intel processors. This mean the bytes are transferred in this order in the stream:

value type byte transmit order sample value sample transmit order
byte b0 0x12 0x12
ushort (16 bits) b0, b1 0x1234 0x34, 0x12
uint (32 bits) b0, b1, b2, b3 0x12345678 0x78, 0x56, 0x34, 0x12

byte 0 is the LSB

Lower-level protocols (TCP / IP / Ethernet) are using Big-Endian. We're using little endian here because it's straight-forward with .NET and Intel processors. Implementation like Mono (used by Xamarin) will also use little endian on ARM processors.

Implementations

This project contains an implementation using C#/.NET.

Other implementations are welcome.

Potential Future Improvements / ambitions

  • Performance enhancements: Less CPU, less memory-copying, more throughtput.
  • Support for dynamic window instead of relying on static window size.
  • Implement a kind of priority on substreams.
  • Ability to send unsollicited OOB (Out-Of-Band) packets to other peer on a stream.
  • Ability to "ask" to other peer a list of available streams to connect to.
  • Variable packet size based on a packet size discovery mecanism.
  • Support for other languages / platforms.
  • Add compression capabilities
  • Buy the Apple's HQ "spaceship" to host the growing working team.

Other Similar Projects / Technologies / Articles on Stream Multiplexing

About

Yllibed Multiplexed Encapsulated Streaming Protocol (YMESP) - a protocol and implementation in .NET for multiplexing streams.

Resources

Readme

License

MIT license
Activity

Stars

1 star

Watchers

3 watching

Forks

0 forks
Report repository

Releases

4 tags

Packages

No packages published

Languages