Open source collaborative text editors

[guevara]

Open source collaborative text editors



http://bit.ly/2UZRf9D

For years now I've had this itch, trying to find the "perfect" collaborative text editor. But not just any collaborative text editor, oh no, my speciality is finding (well, at least looking for) open source real-time collaborative web-based rich text editors (i.e. not just plain text, but structured content as well). The focus of this post is thus on this quite niche category of open source tools and the tech that powers them, but if your interests are wider, you can check out this constantly updated list of editors.

Story time. Now, I've gone off the deep end 🎶 and dove into the virtually endless pool of resources on real-time collaboration many times before, and the path down, that invigorating plunge towards the bottom of the pool, has become quite familiar to me. Usually it starts with a burst of initial excitement and amazement (the jump) with the newly available low-level technologies and research papers, which is followed by a sense of being overwhelmed and wanting to surface a bit (hey, this pool is pretty deep) and explore the applied research side of the equation, followed by a period of where my outlook can perhaps be best described with: "am I really too dumb for this stuff?" (quite likely) as invariably the docs, demos and code don't quite match up or make sense, or the collaboration component on the server doesn't start, or there's an edge-case with more than two users typing in the same word, or... you get where I'm going (surfaced, gasping for air). It's difficult to pick up the subject again after this, so the next phase is a sort of winter of related ideas, being dormant and letting the harsh weather pass, when I usually let the half-working code sit for months... or years. Until we meet again, conflict-free replicated data type!

Figure 1: The dive.

Case in point is that time in 2017 I was so excited by the prospects of an algorithm (a type of CRDT) described in a research paper, that I took a few days and implemented it in JavaScript, only to find out that the paper itself has an edge-case it isn't addressing (well... maybe?). I've reached out to the author at the time and got no response, and found that this particular edge-case has been addressed in a similar algorithm by a different team, but ah, they're subtly different implementations, maybe I could port the Lisp...WAIT A MINUTE! What's going on here? I certainly do not want to be implementing a novel algorithm from a research paper and fixing its issues! What am I doing, how did I get here? Let's rewind!

What I (and probably you too, dear reader) really want is a plug-and-play editor usable in a product. One that solves the problem of real-time collaboration for you. One that enables a few connected users, thousands of miles apart, to edit a structured document by pressing keys on their keyboards and see each other's changes displayed on their screens. That's it. Editors are hard, though... and I have never been able to find such a tool.

Until now.

It's 2019 (the itch is back with a vigour), and this time not only has a magical editor appeared, but there are actually two contenders to fulfil the long-sought specification. Initially, I wanted to fly with you above the landscape and list pretty much every collaborative text editor out there, but it seems like that wouldn't add much value. Instead, let us focus on the two editors that both could be "the one".

That's still two editors, a choice to make – and it would be nice to see somewhat objectively which one is a better fit. I propose a deeply scientific grading system with the following categories (using kiwis as a standard unit of goodness):

1. Open license: adds 1 kiwi 🥝 (this is actually an exclusion criterion as well)
2. Supports images: adds 1 kiwi 🥝
3. Supports tables: adds 1 kiwi 🥝
4. Supports lists: adds 1 kiwi 🥝
5. Supports math: adds 1 kiwi 🥝
6. Supports real-time collaboration: adds 1 kiwi 🥝
7. Available required server bits: 1 kiwi 🥝
8. Supports remote cursors and selections: 1 kiwi 🥝 (so you can see other users' cursors)
9. Offline support: 1 kiwi 🥝 (this is quite tricky to determine)
10. Used in production: 1 kiwi 🥝
11. Mobile support: 1 kiwi 🥝

So, with the scoring system above, an editor can be given a maximum of 11 kiwi units of goodness. Let's get to it!

CKEditor 5

Figure 2: CKEditor 5 interface (one possible variant). From: https://ckeditor.com/docs/ckeditor5/latest/builds/guides/overview.html#available-builds

Let's start with CKEditor 5, a newly released editor from a long line of quality editors. CKEditor 5 is beautiful. Now I know, I know, it's not one of the criteria, neither does it really matter (and we might even not agree), but I just had to say it. This editor promises to take a lot of the kiwis for itself: it is licensed under GPL (some discussion of limitations here, (+1), supports images, tables, lists (+3), math is supported with a plugin (+1), and it supports real-time collaboration with remote cursor and selections too (+2). The team behind CKEditor has written an amazing in-depth blog post about how they've implemented it using operational transformations.

However, I was unable to find the required server implementation for the real-time collaboration to work, and all documentation points to their cloud services as a required (and paid) component, even in GPL-compatible projects.

The level of offline support is difficult to determine, but it looks like it's not entirely there (e.g. briefly interrupted connectivity is fine, but there might be dragons if you're offline for days). Half a kiwi, I guess? Same goes for mobile support, work is underway for full support there as well (+0.5). And the last point, CKEditor is surely used in production, but I was unable to find a list of products built with it (+0.5).

A nice editor, but the lack of an available open source collaboration server is a bummer indeed.

Total goodness: 8.5/11 🥝🥝🥝🥝🥝🥝🥝🥝⚪️⚪️ (just realised I'm actually using half a kiwi to represent a whole, hmm...)

Atlaskit Editor

Figure 3: Atlassian's Atlaskit's Editor

About a year or so ago, Atlassian's Atlaskit design system was launched and open sourced, and with it came a lot of goodies, including a fully-fledged production ready editor, the ProseMirror-based Atlaskit Editor. I really feel like this could be the editor we've been searching for! 🤞The editor is licensed under the Apache 2.0 license, which is arguably a very permissive open source license (+1 kiwi). It not only supports images, tables, and lists, it supports them exceptionally well (+3 kiwis)! The table implementation, something sorely missing from the majority of editors out there, is especially exceptionally well done:

Figure 4: Table management in Atlaskit Editor

Math, unfortunately, isn't supported, but since the editor itself is ProseMirror-based, it shouldn't be too difficult to add an existing solution to it (+0.5). It definitely supports real-time collaboration with remote cursors and selections (+2), with one of the best implementation of its UI bits I've ever seen (e.g. remote cursors dimmed if overlapping with your own and other lovely details). Additionally, from my testing offline supports seems to be very sturdy indeed (+1) – this is likely due to the central authority approach that ProseMirror uses for collaboration. Wow, we're already up to 7.5 kiwis, and we're not close to done yet! Onwards! Unfortunately, right about here's where we get a bit of a reality check. Just like with CKEditor, there are no available server bits for real-time collaboration. Bummer. Major bummer. I do wonder if there's something we can do about that... Anyway, if we continue making our way down the ranking criteria, all I see is kiwis: it is absolutely used in production (by millions of users), and it does have mobile support (+2). Just as I was about to declare a winner (with a grand total of 9.5 kiwis) and publish this post, something else came up. Turns out the logos and icons used in the editor, and a lot of the other Atlaskit components, are licensed under the very restrictive ADG license, which states that you cannot use them outside of the Atlassian universe. The icons, in particular, are extensively used and tightly integrated into the editor. Major bummer number 2 and I'm taking that kiwi away. We're back down to 8.5, for those counting, which means the race is now even. And finished.

But what if we could also do something about those icons...

I thought it would be too unfortunate if we were, after all these recent advancements in the state of the art for collaborative text editors, still left with two editors, that despite their amazing features, we simply couldn't use in open source projects – either due to licensing issues, lack of openly available building blocks, or both.

So first I built the server parts (backed by the PostgreSQL, powered by the PubSweet framework and a mix of REST/WebSockets), and the code for that is available here: https://github.com/jure/pubsweet-blogger/blob/master/server/component-atlaskit-collab/src/collab.js. It's quite simple but gets the job done and provides document persistence and a real-time communication channel. It's amazing that due to the vast ProseMirror community, I was able to piece together how this is supposed to work without having an exact interface specification (+1 kiwi).

And then I replaced the icons used by the editor with icons from the Feather set. Specifically, this amounted to replacing three packages, and since the ADG license prohibits any sort of derivative work, among other things, these packages had to be redone from scratch. There's probably some functionality lost due to this, mainly in terms of accessibility, but it provides a good open base for further development. For example, the icons themselves are missing better alternatives in some cases, perhaps if you're looking to contribute to this project, that's a good first issue (+1 kiwi)!

What all this means is that we're now only half a kiwi away from a perfect score (but we can probably sort math), something I did not believe was possible when starting on this journey. It also means we have a clear winner in this battle, and ultimately Atlaskit's Editor was victorious due to the diverse, open and helpful ProseMirror community. Check out the project's docs for instructions on how to start playing around. I do believe that if you're starting a new project and you're considering having real-time collaboration as a feature, this is a great place to start exploring, even with all the caveats.

VIDEOFigure 5: A demo of Atlaskit Editor's real-time editing capabilities

Total (enhanced) goodness: 10.5/11 🥝🥝🥝🥝🥝🥝🥝🥝🥝🥝️⚪️

Honourable mentions

There are two editors I'd like to point out here, even though they don't have ready-to-go real-time collaboration built in right now, as they're both built on top of libraries that have support for it, and adding it is either possible, planned or both.

Figure 6: Wax editor (ProseMirror-based word processor)

First is Wax (named after the ancient wax tablets), the Coko Foundation's editor – nay, word processor – built first on top of the Substance library (repo), and now being rebuilt with ProseMirror (repo), is absolutely packed with features. They're not exactly the same features that we're giving kiwis for, but they're great features nonetheless. Wax, initially built for a book production app, has support for notes (numbered annotations common in books), track changes, find and replace, ornaments, and others. Maybe we could give oranges 🍊for these? In that case, Wax would score 4 kiwis (open license, images, lists, used in production) and a couple of oranges, maybe 4. 🥝🥝🥝🥝🍊🍊🍊🍊. That's a lot of fruit!

Texture

Figure 7: Texture, a Substance-based editor

And last but not least, we have Texture, a very specific, professional (but open source) editor for producing scientific content in JATS format (an XML standard for journal articles). If we do the fruit counting exercise here, we'll almost certainly run out of fingers: Open license, tables, images, deep support for math, references and cross-references, lists, metadata support, the list goes on! 🥝🍊🍎To reiterate, both Wax and Texture are amazing editors, and the only thing that's keeping them from the top kiwi rankings is the lack of real-time collaboration features, which this post is focused on.

Conclusion

If you're building a system in which real-time collaboration in an editor is a key component, you've chosen the right time to do it. In the last year, the landscape has matured and there are almost plug-and-play open solutions out there. With your help, perhaps this time next year we'll have a whole suite of open projects to learn from. And perhaps, just perhaps, we'll also be using tools built on top of these open layers, without even knowing it. 🥝

Further reading

1. Data Laced with History: Causal Trees & Operational CRDTs: http://archagon.net/blog/2018/03/24/data-laced-with-history/#conflict-free-replicated-data-types
2. Lessons learned from creating a rich-text editor with real-time collaboration: https://ckeditor.com/blog/Lessons-learned-from-creating-a-rich-text-editor-with-real-time-collaboration/
3. The Case for HTML Word Processors: https://www.adamhyde.net/html-word-processors/
4. Collaborative Editing in ProseMirror: http://marijnhaverbeke.nl/blog/collaborative-editing.html

via A Case for Spaceships

May 7, 2019 at 09:01PM