Does Subscriptions Have a Backplane to Support Multiple Servers

[RehanSaeed]

I've tried subscriptions and it works in a single server scenario. How can I make it work when you have multiple load balanced servers? Signal-R uses a concept called a backplane which is basically a data store where a list of subscribers are stored.

Ideally there would be a simple interface I can implement to add, get or delete subscriptions. I could then implement the interface and store the subcriptions in Redis for example.

The closest thing I can find is the ISubscriptionManager but that seems to also mix the concepts of storing subscriptions and executing operations.

[pekkah]

Multiple servers would be tricky to implement. Subscriptions itself would work but how would you get the transport to work? Especially when 99% time client is browser connecting trough web socket. Multiple servers where each connection goes to one server should work.

[RehanSaeed]

Not sure about the connection. How does Signal-R do this? Could Subscriptions use some parts of Signal-R to get this done?

[BenjaBobs]

The way to do it is to have a backplane that notifies all the instances, and adds the message to all the subscriptions. In my team we use Azure Service Bus for this, but you can really use anything you want, that is able to receive a message, and then send the message to multiple other services.

Something akin to this:

[JustinKaffenberger]

Just wanted to chime in on this. I love GraphQL and we will be using in our software for queries and mutations, but we will be sticking with SignalR for real time events due to elasticity concerns.

[dflor003]

Apollo GraphQL handles this nicely with Redis as the backplane. Is there no equivalent to that for dotnet? Without something like that, subscriptions have no practical use in production applications.

[SebastianStehle]

I am looking into this as well at the moment. I think the first step would be to make the subscriptions async, to be non-blocking. I think IObservable is not the correct primitive. Either IAsyncObservable or IAsyncEnumerable would be better.

[Shane32]

You are likely right, @SebastianStehle, that another choice would be better. Unfortunately IAsyncObservable isn't a part of .NET Standard. But IAsyncEnumerable can convey (1) a data event, (2) an exception, and (3) a completion, just like IObservable does. I think the biggest issue is in the design (or tooling) -- it is quite easy with IObservable and System.Reactive to write a singleton service that provides notifications to as many clients as wish to connect without any additional CPU/database load. It may be little more difficult with IAsyncEnumerable. Likely IAsyncObservable is a better choice, if only it was included in .NET Standard.

Just FYI, GraphQL.NET Server v7 does not internally block on any calls when called through its IObserver interface; notifications simply post to a queue to be transmitted to the listener(s) asynchronously. The ordering of notifications is also maintained. So you can write a completely asynchronous IObservable implementation without any blocking taking place. Of course this also means that you cannot know which events have been transmitted to the client or not (probably not guaranteed in any case, however).

So if you wish to write your implementation as IAsyncEnumerable, simply call AsyncEnumerable.ToObservable (part of System.Reactive) to convert it to an IObservable, and you should suffer no ill effects under GraphQL.NET Server v7. Most likely you can do a similar conversion from IAsyncObservable to IObservable.

Link: https://fuqua.io/Rx.NET/ix-docs/html/M_System_Linq_AsyncEnumerable_ToObservable__1.htm

I use IAsyncEnumerable in my GraphQL.NET client to pull results from a subscription, and it works very well.

[Shane32]

We may want to write connectors for various existing broadcast layer tooling (e.g. Azure Service Bus as noted above) to connect with GraphQL.NET subscription support.

[SebastianStehle]

Another question is where you make the decision what to push to a subscription.

I would say, in a typical application, only a very small subset of all events are pushed to a subscription. So you probably use domain events or event sourcing and only 1% of these events are actually consumed by a subscriber. So why would you send all these events over the backbone? I think it would be more efficient if you would handle the subscription with these steps:

1. Publish new subscriptions to all nodes.
2. Keep a copy of the subscription on all nodes that emit domain events or something similar.
3. Compare the domain events with subscriptions.
4. If the current node is also a graphql server try to handle the event locally first or publish to to the backbone otherwise.

The only question is: How do you remove subscription in case of errors? (node restart or so)

[SebastianStehle]

I am working on a small system for my personal needs: https://github.com/SebastianStehle/graphql-ext/tree/main/GraphQLExample/Subscriptions

It is just the abstraction yet, but the idea is to make subscription evaluation on the node where the events are published. The subscription is just an interface, so you can implement custom rules to evaluate who should receive events.

EDIT: I have added an example to test the abstraction.