Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tldraw #126

Draft
wants to merge 23 commits into
base: main
Choose a base branch
from
Draft

Tldraw #126

wants to merge 23 commits into from

Conversation

robertschnuell
Copy link
Member

@robertschnuell robertschnuell commented Jan 19, 2024

Screenshot 2024-01-19 at 14 17 45

This pull request implements a proof of concept prototype for a tl;draw implementation, storing each sketch's data in the matrix itself. It serves as a replacement for the current /sketch implementation in the medienhaus/spaces software stack, which originally relied on spacedeck for creating collaborative whiteboards. Although we had been working on integrating spacedeck into the medienhaus/spaces frontend for some time, we encountered increasing issues. Eventually, we faced a turning point: either adding functionality to the spacedeck fork or switching to alternative software. The problems were primarily related to the rights management of sketches in spacedeck, making it challenging to provide access to different types of users. Since spacedeck hasn't been maintained for several years, we opted to explore other options, such as tl;draw. Tl;draw is solely a frontend component without its own backend, allowing us to store user-generated data directly in matrix in a ‘human readable’ or let’s say at least a ‘human accessible’ form directly in matrix. External software like spacedeck served as a temporary solution, avoiding the need to rewrite these applications from scratch. With tl;draw, we the best of both sides in hand for an implementation into medienhaus/spaces.

This proof of concept draws inspiration from the tl;draw-yjs example, available at.

Screenshot 2024-01-19 at 14 36 58 _data of a tl;draw sketch shown in element_

implementation

The tl;draw react component requires a store object containing the keys store and schema. The schema contains information specifying the versions for which specific data is stored. The data in store.store consists mainly of canvas data, including user-generated content like shapes and text, along with information about the canvas itself.
To manage this data effectively, we sperate it virtually. User-generated content, such as shapes, is stored in timeline events. While data related to the editor or canvas itself is stored as custom state events (dev.medienhaus.tldraw.store.store , dev.medienhaus.tldraw.store.schema ) , acting as headers for specific matrix data rooms. This separation enables the lazy loading of user-generated content, preventing potential issues with UX.
The tl;draw editor initiation involves creating a specific matrix client dedicated to the room containing data for a particular sketch, avoiding unnecessary calls to the medienhaus/spaces matrix client. Storing data directly in matrix allows us to leverage matrix functionality for user rights management. Core functionality is based on matrix timeline events with the type m.room.message, chosen for visibility in element. The msgtype for shapes and text events is dev.medienhaus.tldraw.store.store. Of any user with edit rights wants to modify this specific content element a thread is created based on that one. So within this we get a version control, which could be useful for future features. But expect that timeline feature it is also needed as user B can’t edit a message of user A. It is only possible to use the thread function to get the feature of modification of an specific content element. User B can delete a message of User A, which makes it easier to remove content elements in an collaborative session and we don’t need any garbage collator in the background anymore as we did it in the medienhaus/cms.

Each shape (,text etc.) content element gets a unique id, by the tl;draw editor when it is created by a user. We need to like those now to the root event_id in the matrix store somehow. As tl;draw is quite picky when it comes to data integrity the only possible way is to add it to the meta key in which we can create an eventId key which got the value of the initiated eventId of this shape in the specific matrix room. With this it is possible to address within matrix directly one specific content element and then gab the latest version of it with the thread function of the matrix-js-sdk.

The tl;draw editor can trigger add, change, remove events based on the user input within the editor. With this functionally we can update the data stores in matrix.
The other way round if a remote change comes inbound the matrix sync triggers and will try to merge this changes in the local data storage of tl;draw which updates then remote data into the locale hold copy of the data which is shown to the user. It is important at this point to point out that in the end the tl;draw editor is just a fancy json viewer/editor. All other functionality (for good reason) is not part of tl;draw and is handled outside of that given component.

todo

This proof of concept prototype does not include a lot of different functionalities as well as still contains many already known bugs. Will work on those from time to time, or at least will document those.

Beside that there are some functionalities which should be included in the near future. On the one hand to show cursor position of other users in realtime. As well as including their usernames. For now the users functionality of the tl;draw editor is not used at all. But those functions can be added in my point of view also after an initial launch.

  • promote users to moderator when inviting them (or toggle switch to decide if they should be able to(read/write))
  • if there is a read-only view: disable all functions if a user is read-only

to be discussed

  • will be added later

Sorry, something went wrong.

@andirueckel andirueckel marked this pull request as ready for review January 19, 2024 13:37
@andirueckel andirueckel marked this pull request as draft January 19, 2024 13:37
fnwbr added 4 commits January 19, 2024 16:22

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber
…ldraw` in this case

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber
…rite or /sketch)

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber
# Conflicts:
#	lib/Matrix.js
Copy link

@github-advanced-security github-advanced-security bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ESLint found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

robertschnuell and others added 4 commits February 2, 2024 14:25

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber
# Conflicts:
#	components/layouts/partials/navigation.js
#	lib/Matrix.js
#	package-lock.json
#	package.json

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber
lib/auth/TldrawAuthProvider.js Fixed Show fixed Hide fixed
pages/draw/MatrixSpecificRoomsProvider.js Fixed Show fixed Hide fixed
pages/draw/MatrixSpecificRoomsProvider.js Fixed Show fixed Hide fixed
pages/draw/MatrixSpecificRoomsProvider.js Fixed Show fixed Hide fixed
pages/draw/MatrixSpecificRoomsProvider.js Fixed Show fixed Hide fixed
pages/draw/editor.js Fixed Show fixed Hide fixed
pages/draw/tldrawMatrix.js Fixed Show fixed Hide fixed
));
setInitialSyncDone(false);
initiallyPopulated.current = false;
}, [roomId]);

Check failure

Code scanning / ESLint

verifies the list of dependencies for Hooks like useEffect and similar Error

React Hook useEffect has missing dependencies: 'setMatrixRoomClient' and 'setStore'. Either include them or remove the dependency array.
const fetchStoreFromTimelineEvents = async (tempStore) => {
const room = matrixRoomClient.getMatrixClient().getRoom(roomId);
if (!room) return;
const events = room?.getLiveTimeline()?.getEvents();

Check failure

Code scanning / ESLint

Disallow unused variables Error

'events' is assigned a value but never used.
pages/draw/[[...roomId]].js Fixed Show fixed Hide fixed
@aofn
Copy link
Member

aofn commented Feb 2, 2024

looks like draw rooms have the wrong template, since they're not being added to the tldraw folder. Haven't check if this is the actual reason, tho, but it's very likely.

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber
pages/draw/editor.js Fixed Show fixed Hide fixed
pages/draw/editor.js Fixed Show fixed Hide fixed
pages/draw/editor.js Fixed Show fixed Hide fixed

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber
pages/_app.js Fixed Show fixed Hide fixed
pages/draw/editor.js Fixed Show fixed Hide fixed
}
}
},
[matrixRoomClient, setStore],

Check failure

Code scanning / ESLint

verifies the list of dependencies for Hooks like useEffect and similar Error

React Hook useCallback has a missing dependency: 'store'. Either include it or remove the dependency array.

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber

Verified

This commit was signed with the committer’s verified signature.
fnwbr Florian Weber
const serviceSpaceId = matrix.serviceSpaces.tldraw;
const spacedeckChildren = matrix.spaces.get(serviceSpaceId)?.children?.filter((child) => child !== 'undefined'); // Filter out any undefined values to ensure 'spacedeckChildren' only contains valid objects

const tldrawMatrix = TldrawMatrixProvider(roomId);

Check failure

Code scanning / ESLint

Require constructor names to begin with a capital letter Error

A function with a name starting with an uppercase letter should only be used as a constructor.
@fnwbr fnwbr mentioned this pull request Mar 19, 2024
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants