Add `heroes` and room summary fields to Sliding Sync `/sync` #17419

MadLittleMods · 2024-07-08T23:41:57Z

Add heroes and room summary fields (joined_count, invited_count) to Sliding Sync /sync

Based on MSC3575: Sliding Sync

Dev notes

Instead of returning stripped state for heroes, we could simply return the User ID's of the heroes like Sync v2 (see m.heroes) and return the full membership event in required_state, see matrix-org/matrix-spec-proposals#3575 (comment)

We don't even have access to heroes if we're the first person on our server to be invited to a remote room over federation, see matrix-org/matrix-spec-proposals#3575 (comment)

Pull Request Checklist

Pull request is based on the develop branch
Pull request includes a changelog file. The entry should:
- Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
- Use markdown where necessary, mostly for code blocks.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
Code style is correct
(run the linters)

synapse/types/handlers/__init__.py

MadLittleMods · 2024-07-08T23:53:32Z

synapse/types/rest/client/__init__.py

@@ -270,7 +267,6 @@ class Filters(RequestBodyModel):
        else:
            ranges: Optional[List[Tuple[conint(ge=0, strict=True), conint(ge=0, strict=True)]]] = None  # type: ignore[valid-type]
        slow_get_all_rooms: Optional[StrictBool] = False
-        include_heroes: Optional[StrictBool] = False


Removed include_heroes since it seems redundant, see matrix-org/matrix-spec-proposals#3575 (comment)

…ttlemods/sliding-sync-heroes

Conflicts: synapse/handlers/sliding_sync.py

Discussion tracking this: matrix-org/matrix-spec-proposals#3575 (comment) It's also hard because we may not have any full membership events when you're the first one on your server to be invited to a new room over federation.

MadLittleMods · 2024-07-09T22:33:33Z

synapse/handlers/sliding_sync.py

@@ -1041,6 +1043,103 @@ async def sort_rooms(
            reverse=True,
        )

+    async def get_current_state_ids_at(


Extracted this logic out into a couple helper functions (get_current_state_ids_at(...)/get_current_state_at(...)) since I now need to do fetch state twice. Once to check if the room name is populated and once for fetching the full state that we need.

MadLittleMods · 2024-07-09T22:35:10Z

synapse/handlers/sliding_sync.py

@@ -1072,7 +1171,7 @@ async def get_room_sync_data(
        # membership. Currently, we have to make all of these optional because
        # `invite`/`knock` rooms only have `stripped_state`. See
        # https://github.com/matrix-org/matrix-spec-proposals/pull/3575#discussion_r1653045932
-        timeline_events: Optional[List[EventBase]] = None
+        timeline_events: List[EventBase] = []


Simplifying things a bit by removing these optionals and the checks necessary to guard from them.

We now just decide whether to serialize them in the response by checking their len(...)

MadLittleMods · 2024-07-09T22:36:29Z

synapse/handlers/sliding_sync.py

+            # TODO: Figure out how to get the membership summary for left/banned rooms
+            room_membership_summary = {}


Not going to address this TODO here.

It's an edge case for left/ban rooms so it doesn't matter that much

Not sure of an efficient way to do this for a long-forgotten left/banned room

synapse/storage/databases/main/roommember.py

See #17419 (comment)

MadLittleMods · 2024-07-09T23:14:37Z

synapse/storage/databases/main/roommember.py

-                    CASE membership WHEN ? THEN 1 WHEN ? THEN 2 ELSE 3 END ASC,
-                    event_id ASC
+                    CASE membership WHEN ? THEN 1 WHEN ? THEN 2 WHEN ? THEN 3 ELSE 4 END ASC,
+                    event_stream_ordering ASC


By changing this now, it will probably cause peoples rooms names to calculate to something different across the board (this function is also used by Sync v2). Are we ok with that? Potentially disruptive if people are really used to their old "flawed" name.

It's already a FIXME comment in extract_heroes_from_room_summary(...) but we still need to pull in the right order from the database.

Related to matrix-org/matrix-spec#1334

Why do we need to change the order now? I think we only need to have a "stable" ordering?

The spec says this:

This should be the first 5 members of the room, ordered by stream ordering, which are joined or invited. The list must never include the client’s own user ID. When no joined or invited members are available, this should consist of the banned and left users.

-- https://spec.matrix.org/v1.10/client-server-api/#_matrixclientv3sync_roomsummary

Hmm, interesting. Can we keep as is for now and open a separate PR / issue to consider this? I'm failing to think through whether this is going to cause some UX issues or not

Reverted in this PR and split out to #17435 ⏩

synapse/storage/databases/main/roommember.py

erikjohnston

Otherwise mostly looks good

erikjohnston · 2024-07-11T12:41:14Z

synapse/handlers/sliding_sync.py

+        if name_event_id is None:
+            hero_user_ids = extract_heroes_from_room_summary(
+                room_membership_summary, me=user.to_string()
+            )


Should we only do this calculation when initial true? And can we not do this after we've already fetched the room name later?

Should we only do this calculation when initial true?

We need to be careful about including these when membership changes as well. Since we can't really tell whether membership has changed without looking, it seems easiest just to fetch it out each time when the room name isn't set and we can have the future "whether this room has been sent down this connection before" tracking mechanism handle whether they need to be sent down.

And can we not do this after we've already fetched the room name later?

We could do either way (or even the third option):

Right now, we first check whether the room name state is set without pulling out the full event for it. We bundle the hero member event fetching along with all of the other full state events we fetch (see meta_room_state)

We could do it the opposite way and wait for the full state to come back (which will include the meta events like m.room.name) and then have another full event fetch for the heroes that we need.

Or we could fetch the heroes in the room every time

I chose the current way (option 1) because we only need a single lookup for full events.

It does suck that we have this dependent waterfall though so option 3 is also appealing and simple.

erikjohnston · 2024-07-11T12:43:18Z

synapse/storage/databases/main/roommember.py

-                    CASE membership WHEN ? THEN 1 WHEN ? THEN 2 ELSE 3 END ASC,
-                    event_id ASC
+                    CASE membership WHEN ? THEN 1 WHEN ? THEN 2 WHEN ? THEN 3 ELSE 4 END ASC,
+                    event_stream_ordering ASC


Why do we need to change the order now? I think we only need to have a "stable" ordering?

Split off from #17419 (comment)

tests/rest/client/test_sync.py

See #17419 (comment)

MadLittleMods · 2024-07-11T19:08:39Z

Thanks for the review @erikjohnston 🦧

The spec specifically mentions `stream_ordering` but that's a Synapse specific concept. In any case, the essence of the spec is basically the first 5 members of the room which `stream_ordering` accomplishes. Split off from #17419 (comment) ## Spec compliance > This should be the first 5 members of the room, **ordered by stream ordering**, which are joined or invited. The list must never include the client’s own user ID. When no joined or invited members are available, this should consist of the banned and left users. > > *-- https://spec.matrix.org/v1.10/client-server-api/#_matrixclientv3sync_roomsummary* Related to matrix-org/matrix-spec#1334

@Michael-Hollister

This security release is to update our locked dependency on Twisted to 24.7.0rc1, which includes a security fix for [CVE-2024-41671 / GHSA-c8m8-j448-xjx7: Disordered HTTP pipeline response in twisted.web, again](GHSA-c8m8-j448-xjx7). Note that this security fix is also available as **Synapse 1.111.1**, which does not include the rest of the changes in Synapse 1.112.0. This issue means that, if multiple HTTP requests are pipelined in the same TCP connection, Synapse can send responses to the wrong HTTP request. If a reverse proxy was configured to use HTTP pipelining, this could result in responses being sent to the wrong user, severely harming confidentiality. With that said, despite being a high severity issue, **we consider it unlikely that Synapse installations will be affected**. The use of HTTP pipelining in this fashion would cause worse performance for clients (request-response latencies would be increased as users' responses would be artificially blocked behind other users' slow requests). Further, Nginx and Haproxy, two common reverse proxies, do not appear to support configuring their upstreams to use HTTP pipelining and thus would not be affected. For both of these reasons, we consider it unlikely that a Synapse deployment would be set up in such a configuration. Despite that, we cannot rule out that some installations may exist with this unusual setup and so we are releasing this security update today. **pip users:** Note that by default, upgrading Synapse using pip will not automatically upgrade Twisted. **Please manually install the new version of Twisted** using `pip install Twisted==24.7.0rc1`. Note also that even the `--upgrade-strategy=eager` flag to `pip install -U matrix-synapse` will not upgrade Twisted to a patched version because it is only a release candidate at this time. - Upgrade locked dependency on Twisted to 24.7.0rc1. ([\#17502](element-hq/synapse#17502)) Please note that this release candidate does not include the security dependency update included in version 1.111.1 as this version was released before 1.111.1. The same security fix can be found in the full release of 1.112.0. - Add to-device extension support to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17416](element-hq/synapse#17416)) - Populate `name`/`avatar` fields in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17418](element-hq/synapse#17418)) - Populate `heroes` and room summary fields (`joined_count`, `invited_count`) in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17419](element-hq/synapse#17419)) - Populate `is_dm` room field in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17429](element-hq/synapse#17429)) - Add room subscriptions to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17432](element-hq/synapse#17432)) - Prepare for authenticated media freeze. ([\#17433](element-hq/synapse#17433)) - Add E2EE extension support to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17454](element-hq/synapse#17454)) - Add configurable option to always include offline users in presence sync results. Contributed by @Michael-Hollister. ([\#17231](element-hq/synapse#17231)) - Fix bug in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint when using room type filters and the user has one or more remote invites. ([\#17434](element-hq/synapse#17434)) - Order `heroes` by `stream_ordering` as the Matrix specification states (applies to `/sync`). ([\#17435](element-hq/synapse#17435)) - Fix rare bug where `/sync` would break for a user when using workers with multiple stream writers. ([\#17438](element-hq/synapse#17438)) - Update the readme image to have a white background, so that it is readable in dark mode. ([\#17387](element-hq/synapse#17387)) - Add Red Hat Enterprise Linux and Rocky Linux 8 and 9 installation instructions. ([\#17423](element-hq/synapse#17423)) - Improve documentation for the [`default_power_level_content_override`](https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html#default_power_level_content_override) config option. ([\#17451](element-hq/synapse#17451)) - Make sure we always use the right logic for enabling the media repo. ([\#17424](element-hq/synapse#17424)) - Fix argument documentation for method `RateLimiter.record_action`. ([\#17426](element-hq/synapse#17426)) - Reduce volume of 'Waiting for current token' logs, which were introduced in v1.109.0. ([\#17428](element-hq/synapse#17428)) - Limit concurrent remote downloads to 6 per IP address, and decrement remote downloads without a content-length from the ratelimiter after the download is complete. ([\#17439](element-hq/synapse#17439)) - Remove unnecessary call to resume producing in fake channel. ([\#17449](element-hq/synapse#17449)) - Update experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint to bump room when it is created. ([\#17453](element-hq/synapse#17453)) - Speed up generating sliding sync responses. ([\#17458](element-hq/synapse#17458)) - Add cache to `get_rooms_for_local_user_where_membership_is` to speed up sliding sync. ([\#17460](element-hq/synapse#17460)) - Speed up fetching room keys from backup. ([\#17461](element-hq/synapse#17461)) - Speed up sorting of the room list in sliding sync. ([\#17468](element-hq/synapse#17468)) - Implement handling of `$ME` as a state key in sliding sync. ([\#17469](element-hq/synapse#17469)) * Bump bytes from 1.6.0 to 1.6.1. ([\#17441](element-hq/synapse#17441)) * Bump hiredis from 2.3.2 to 3.0.0. ([\#17464](element-hq/synapse#17464)) * Bump jsonschema from 4.22.0 to 4.23.0. ([\#17444](element-hq/synapse#17444)) * Bump matrix-org/done-action from 2 to 3. ([\#17440](element-hq/synapse#17440)) * Bump mypy from 1.9.0 to 1.10.1. ([\#17445](element-hq/synapse#17445)) * Bump pyopenssl from 24.1.0 to 24.2.1. ([\#17465](element-hq/synapse#17465)) * Bump ruff from 0.5.0 to 0.5.4. ([\#17466](element-hq/synapse#17466)) * Bump sentry-sdk from 2.6.0 to 2.8.0. ([\#17456](element-hq/synapse#17456)) * Bump sentry-sdk from 2.8.0 to 2.10.0. ([\#17467](element-hq/synapse#17467)) * Bump setuptools from 67.6.0 to 70.0.0. ([\#17448](element-hq/synapse#17448)) * Bump twine from 5.1.0 to 5.1.1. ([\#17443](element-hq/synapse#17443)) * Bump types-jsonschema from 4.22.0.20240610 to 4.23.0.20240712. ([\#17446](element-hq/synapse#17446)) * Bump ulid from 1.1.2 to 1.1.3. ([\#17442](element-hq/synapse#17442)) * Bump zipp from 3.15.0 to 3.19.1. ([\#17427](element-hq/synapse#17427))

@Michael-Hollister

# Synapse 1.112.0 (2024-07-30) This security release is to update our locked dependency on Twisted to 24.7.0rc1, which includes a security fix for [CVE-2024-41671 / GHSA-c8m8-j448-xjx7: Disordered HTTP pipeline response in twisted.web, again](GHSA-c8m8-j448-xjx7). Note that this security fix is also available as **Synapse 1.111.1**, which does not include the rest of the changes in Synapse 1.112.0. This issue means that, if multiple HTTP requests are pipelined in the same TCP connection, Synapse can send responses to the wrong HTTP request. If a reverse proxy was configured to use HTTP pipelining, this could result in responses being sent to the wrong user, severely harming confidentiality. With that said, despite being a high severity issue, **we consider it unlikely that Synapse installations will be affected**. The use of HTTP pipelining in this fashion would cause worse performance for clients (request-response latencies would be increased as users' responses would be artificially blocked behind other users' slow requests). Further, Nginx and Haproxy, two common reverse proxies, do not appear to support configuring their upstreams to use HTTP pipelining and thus would not be affected. For both of these reasons, we consider it unlikely that a Synapse deployment would be set up in such a configuration. Despite that, we cannot rule out that some installations may exist with this unusual setup and so we are releasing this security update today. **pip users:** Note that by default, upgrading Synapse using pip will not automatically upgrade Twisted. **Please manually install the new version of Twisted** using `pip install Twisted==24.7.0rc1`. Note also that even the `--upgrade-strategy=eager` flag to `pip install -U matrix-synapse` will not upgrade Twisted to a patched version because it is only a release candidate at this time. ### Internal Changes - Upgrade locked dependency on Twisted to 24.7.0rc1. ([\#17502](element-hq/synapse#17502)) # Synapse 1.112.0rc1 (2024-07-23) Please note that this release candidate does not include the security dependency update included in version 1.111.1 as this version was released before 1.111.1. The same security fix can be found in the full release of 1.112.0. ### Features - Add to-device extension support to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17416](element-hq/synapse#17416)) - Populate `name`/`avatar` fields in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17418](element-hq/synapse#17418)) - Populate `heroes` and room summary fields (`joined_count`, `invited_count`) in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17419](element-hq/synapse#17419)) - Populate `is_dm` room field in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17429](element-hq/synapse#17429)) - Add room subscriptions to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17432](element-hq/synapse#17432)) - Prepare for authenticated media freeze. ([\#17433](element-hq/synapse#17433)) - Add E2EE extension support to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17454](element-hq/synapse#17454)) ### Bugfixes - Add configurable option to always include offline users in presence sync results. Contributed by @Michael-Hollister. ([\#17231](element-hq/synapse#17231)) - Fix bug in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint when using room type filters and the user has one or more remote invites. ([\#17434](element-hq/synapse#17434)) - Order `heroes` by `stream_ordering` as the Matrix specification states (applies to `/sync`). ([\#17435](element-hq/synapse#17435)) - Fix rare bug where `/sync` would break for a user when using workers with multiple stream writers. ([\#17438](element-hq/synapse#17438)) ### Improved Documentation - Update the readme image to have a white background, so that it is readable in dark mode. ([\#17387](element-hq/synapse#17387)) - Add Red Hat Enterprise Linux and Rocky Linux 8 and 9 installation instructions. ([\#17423](element-hq/synapse#17423)) - Improve documentation for the [`default_power_level_content_override`](https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html#default_power_level_content_override) config option. ([\#17451](element-hq/synapse#17451)) ### Internal Changes - Make sure we always use the right logic for enabling the media repo. ([\#17424](element-hq/synapse#17424)) - Fix argument documentation for method `RateLimiter.record_action`. ([\#17426](element-hq/synapse#17426)) - Reduce volume of 'Waiting for current token' logs, which were introduced in v1.109.0. ([\#17428](element-hq/synapse#17428)) - Limit concurrent remote downloads to 6 per IP address, and decrement remote downloads without a content-length from the ratelimiter after the download is complete. ([\#17439](element-hq/synapse#17439)) - Remove unnecessary call to resume producing in fake channel. ([\#17449](element-hq/synapse#17449)) - Update experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint to bump room when it is created. ([\#17453](element-hq/synapse#17453)) - Speed up generating sliding sync responses. ([\#17458](element-hq/synapse#17458)) - Add cache to `get_rooms_for_local_user_where_membership_is` to speed up sliding sync. ([\#17460](element-hq/synapse#17460)) - Speed up fetching room keys from backup. ([\#17461](element-hq/synapse#17461)) - Speed up sorting of the room list in sliding sync. ([\#17468](element-hq/synapse#17468)) - Implement handling of `$ME` as a state key in sliding sync. ([\#17469](element-hq/synapse#17469)) ### Updates to locked dependencies * Bump bytes from 1.6.0 to 1.6.1. ([\#17441](element-hq/synapse#17441)) * Bump hiredis from 2.3.2 to 3.0.0. ([\#17464](element-hq/synapse#17464)) * Bump jsonschema from 4.22.0 to 4.23.0. ([\#17444](element-hq/synapse#17444)) * Bump matrix-org/done-action from 2 to 3. ([\#17440](element-hq/synapse#17440)) * Bump mypy from 1.9.0 to 1.10.1. ([\#17445](element-hq/synapse#17445)) * Bump pyopenssl from 24.1.0 to 24.2.1. ([\#17465](element-hq/synapse#17465)) * Bump ruff from 0.5.0 to 0.5.4. ([\#17466](element-hq/synapse#17466)) * Bump sentry-sdk from 2.6.0 to 2.8.0. ([\#17456](element-hq/synapse#17456)) * Bump sentry-sdk from 2.8.0 to 2.10.0. ([\#17467](element-hq/synapse#17467)) * Bump setuptools from 67.6.0 to 70.0.0. ([\#17448](element-hq/synapse#17448)) * Bump twine from 5.1.0 to 5.1.1. ([\#17443](element-hq/synapse#17443)) * Bump types-jsonschema from 4.22.0.20240610 to 4.23.0.20240712. ([\#17446](element-hq/synapse#17446)) * Bump ulid from 1.1.2 to 1.1.3. ([\#17442](element-hq/synapse#17442)) * Bump zipp from 3.15.0 to 3.19.1. ([\#17427](element-hq/synapse#17427)) # Synapse 1.111.0 (2024-07-16) No significant changes since 1.111.0rc2. # Synapse 1.111.0rc2 (2024-07-10) ### Bugfixes - Fix bug where using `synapse.app.media_repository` worker configuration would break the new media endpoints. ([\#17420](element-hq/synapse#17420)) ### Improved Documentation - Document the new federation media worker endpoints in the [upgrade notes](https://element-hq.github.io/synapse/v1.111/upgrade.html) and [worker docs](https://element-hq.github.io/synapse/v1.111/workers.html). ([\#17421](element-hq/synapse#17421)) ### Internal Changes - Route authenticated federation media requests to media repository workers in Complement tests. ([\#17422](element-hq/synapse#17422)) # Synapse 1.111.0rc1 (2024-07-09) ### Features - Add `rooms` data to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17320](element-hq/synapse#17320)) - Add `room_types`/`not_room_types` filtering to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17337](element-hq/synapse#17337)) - Return "required state" in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17342](element-hq/synapse#17342)) - Support [MSC3916](https://github.com/matrix-org/matrix-spec-proposals/blob/main/proposals/3916-authentication-for-media.md) by adding [`_matrix/client/v1/media/download`](https://spec.matrix.org/v1.11/client-server-api/#get_matrixclientv1mediadownloadservernamemediaid) endpoint. ([\#17365](element-hq/synapse#17365)) - Support [MSC3916](https://github.com/matrix-org/matrix-spec-proposals/blob/rav/authentication-for-media/proposals/3916-authentication-for-media.md) by adding [`_matrix/client/v1/media/thumbnail`](https://spec.matrix.org/v1.11/client-server-api/#get_matrixclientv1mediathumbnailservernamemediaid), [`_matrix/federation/v1/media/thumbnail`](https://spec.matrix.org/v1.11/server-server-api/#get_matrixfederationv1mediathumbnailmediaid) endpoints and stabilizing the remaining [`_matrix/client/v1/media`](https://spec.matrix.org/v1.11/client-server-api/#get_matrixclientv1mediaconfig) endpoints. ([\#17388](element-hq/synapse#17388)) - Add `rooms.bump_stamp` for easier client-side sorting in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17395](element-hq/synapse#17395)) - Forget all of a user's rooms upon deactivation, preventing local room purges from being blocked on deactivated users. ([\#17400](element-hq/synapse#17400)) - Declare support for [Matrix 1.11](https://matrix.org/blog/2024/06/20/matrix-v1.11-release/). ([\#17403](element-hq/synapse#17403)) - [MSC3861](matrix-org/matrix-spec-proposals#3861): allow overriding the introspection endpoint. ([\#17406](element-hq/synapse#17406)) ### Bugfixes - Fix rare race which caused no new to-device messages to be received from remote server. ([\#17362](element-hq/synapse#17362)) - Fix bug in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint when using an old database. ([\#17398](element-hq/synapse#17398)) ### Improved Documentation - Clarify that `url_preview_url_blacklist` is a usability feature. ([\#17356](element-hq/synapse#17356)) - Fix broken links in README. ([\#17379](element-hq/synapse#17379)) - Clarify that changelog content *and file extension* need to match in order for entries to merge. ([\#17399](element-hq/synapse#17399)) ### Internal Changes - Make the release script create a release branch for Complement as well. ([\#17318](element-hq/synapse#17318)) - Fix uploading packages to PyPi. ([\#17363](element-hq/synapse#17363)) - Add CI check for the README. ([\#17367](element-hq/synapse#17367)) - Fix linting errors from new `ruff` version. ([\#17381](element-hq/synapse#17381), [\#17411](element-hq/synapse#17411)) - Fix building debian packages on non-clean checkouts. ([\#17390](element-hq/synapse#17390)) - Finish up work to allow per-user feature flags. ([\#17392](element-hq/synapse#17392), [\#17410](element-hq/synapse#17410)) - Allow enabling sliding sync per-user. ([\#17393](element-hq/synapse#17393)) ### Updates to locked dependencies * Bump certifi from 2023.7.22 to 2024.7.4. ([\#17404](element-hq/synapse#17404)) * Bump cryptography from 42.0.7 to 42.0.8. ([\#17382](element-hq/synapse#17382)) * Bump ijson from 3.2.3 to 3.3.0. ([\#17413](element-hq/synapse#17413)) * Bump log from 0.4.21 to 0.4.22. ([\#17384](element-hq/synapse#17384)) * Bump mypy-zope from 1.0.4 to 1.0.5. ([\#17414](element-hq/synapse#17414)) * Bump pillow from 10.3.0 to 10.4.0. ([\#17412](element-hq/synapse#17412)) * Bump pydantic from 2.7.1 to 2.8.2. ([\#17415](element-hq/synapse#17415)) * Bump ruff from 0.3.7 to 0.5.0. ([\#17381](element-hq/synapse#17381)) * Bump serde from 1.0.203 to 1.0.204. ([\#17409](element-hq/synapse#17409)) * Bump serde_json from 1.0.117 to 1.0.120. ([\#17385](element-hq/synapse#17385), [\#17408](element-hq/synapse#17408)) * Bump types-setuptools from 69.5.0.20240423 to 70.1.0.20240627. ([\#17380](element-hq/synapse#17380)) # Synapse 1.110.0 (2024-07-03) No significant changes since 1.110.0rc3. # Synapse 1.110.0rc3 (2024-07-02) ### Bugfixes - Fix bug where `/sync` requests could get blocked indefinitely after an upgrade from Synapse versions before v1.109.0. ([\#17386](element-hq/synapse#17386), [\#17391](element-hq/synapse#17391)) ### Internal Changes - Limit size of presence EDUs to 50 entries. ([\#17371](element-hq/synapse#17371)) - Fix building debian package for debian sid. ([\#17389](element-hq/synapse#17389)) # Synapse 1.110.0rc2 (2024-06-26) ### Internal Changes - Fix uploading packages to PyPi. ([\#17363](element-hq/synapse#17363)) # Synapse 1.110.0rc1 (2024-06-26) ### Features - Add initial implementation of an experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17187](element-hq/synapse#17187)) - Add experimental support for [MSC3823](matrix-org/matrix-spec-proposals#3823) - Account suspension. ([\#17255](element-hq/synapse#17255)) - Improve ratelimiting in Synapse. ([\#17256](element-hq/synapse#17256)) - Add support for the unstable [MSC4151](matrix-org/matrix-spec-proposals#4151) report room API. ([\#17270](element-hq/synapse#17270), [\#17296](element-hq/synapse#17296)) - Filter for public and empty rooms added to Admin-API [List Room API](https://element-hq.github.io/synapse/latest/admin_api/rooms.html#list-room-api). ([\#17276](element-hq/synapse#17276)) - Add `is_dm` filtering to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17277](element-hq/synapse#17277)) - Add `is_encrypted` filtering to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17281](element-hq/synapse#17281)) - Include user membership in events served to clients, per [MSC4115](matrix-org/matrix-spec-proposals#4115). ([\#17282](element-hq/synapse#17282)) - Do not require user-interactive authentication for uploading cross-signing keys for the first time, per [MSC3967](matrix-org/matrix-spec-proposals#3967). ([\#17284](element-hq/synapse#17284)) - Add `stream_ordering` sort to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17293](element-hq/synapse#17293)) - `register_new_matrix_user` now supports a --password-file flag, which is useful for scripting. ([\#17294](element-hq/synapse#17294)) - `register_new_matrix_user` now supports a --exists-ok flag to allow registration of users that already exist in the database. This is useful for scripts that bootstrap user accounts with initial passwords. ([\#17304](element-hq/synapse#17304)) - Add support for via query parameter from [MSC4156](matrix-org/matrix-spec-proposals#4156). ([\#17322](element-hq/synapse#17322)) - Add `is_invite` filtering to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17335](element-hq/synapse#17335)) - Support [MSC3916](https://github.com/matrix-org/matrix-spec-proposals/blob/main/proposals/3916-authentication-for-media.md) by adding a federation /download endpoint. ([\#17350](element-hq/synapse#17350)) ### Bugfixes - Fix searching for users with their exact localpart whose ID includes a hyphen. ([\#17254](element-hq/synapse#17254)) - Fix wrong retention policy being used when filtering events. ([\#17272](element-hq/synapse#17272)) - Fix bug where OTKs were not always included in `/sync` response when using workers. ([\#17275](element-hq/synapse#17275)) - Fix a long-standing bug where an invalid 'from' parameter to [`/notifications`](https://spec.matrix.org/v1.10/client-server-api/#get_matrixclientv3notifications) would result in an Internal Server Error. ([\#17283](element-hq/synapse#17283)) - Fix edge case in `/sync` returning the wrong the state when using sharded event persisters. ([\#17295](element-hq/synapse#17295)) - Add initial implementation of an experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17301](element-hq/synapse#17301)) - Fix email notification subject when invited to a space. ([\#17336](element-hq/synapse#17336)) ### Improved Documentation - Add missing quotes for example for `exclude_rooms_from_sync`. ([\#17308](element-hq/synapse#17308)) - Update header in the README to visually fix the the auto-generated table of contents. ([\#17329](element-hq/synapse#17329)) - Fix stale references to the Foundation's Security Disclosure Policy. ([\#17341](element-hq/synapse#17341)) - Add default values for `rc_invites.per_issuer` to docs. ([\#17347](element-hq/synapse#17347)) - Fix an error in the docs for `search_all_users` parameter under `user_directory`. ([\#17348](element-hq/synapse#17348)) ### Internal Changes - Remove unused `expire_access_token` option in the Synapse Docker config file. Contributed by @AaronDewes. ([\#17198](element-hq/synapse#17198)) - Use fully-qualified `PersistedEventPosition` when returning `RoomsForUser` to facilitate proper comparisons and `RoomStreamToken` generation. ([\#17265](element-hq/synapse#17265)) - Add debug logging for when room keys are uploaded, including whether they are replacing other room keys. ([\#17266](element-hq/synapse#17266)) - Handle OTK uploads off master. ([\#17271](element-hq/synapse#17271)) - Don't try and resync devices for remote users whose servers are marked as down. ([\#17273](element-hq/synapse#17273)) - Re-organize Pydantic models and types used in handlers. ([\#17279](element-hq/synapse#17279)) - Expose the worker instance that persisted the event on `event.internal_metadata.instance_name`. ([\#17300](element-hq/synapse#17300)) - Update the README with Element branding, improve headers and fix the #synapse:matrix.org support room link rendering. ([\#17324](element-hq/synapse#17324)) - Change path of the experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync implementation to `/org.matrix.simplified_msc3575/sync` since our simplified API is slightly incompatible with what's in the current MSC. ([\#17331](element-hq/synapse#17331)) - Handle device lists notifications for large accounts more efficiently in worker mode. ([\#17333](element-hq/synapse#17333), [\#17358](element-hq/synapse#17358)) - Do not block event sending/receiving while calculating large event auth chains. ([\#17338](element-hq/synapse#17338)) - Tidy up `parse_integer` docs and call sites to reflect the fact that they require non-negative integers by default, and bring `parse_integer_from_args` default in alignment. Contributed by Denis Kasak (@dkasak). ([\#17339](element-hq/synapse#17339)) ### Updates to locked dependencies * Bump authlib from 1.3.0 to 1.3.1. ([\#17343](element-hq/synapse#17343)) * Bump dawidd6/action-download-artifact from 3.1.4 to 5. ([\#17289](element-hq/synapse#17289)) * Bump dawidd6/action-download-artifact from 5 to 6. ([\#17313](element-hq/synapse#17313)) * Bump docker/build-push-action from 5 to 6. ([\#17312](element-hq/synapse#17312)) * Bump jinja2 from 3.1.3 to 3.1.4. ([\#17287](element-hq/synapse#17287)) * Bump lazy_static from 1.4.0 to 1.5.0. ([\#17355](element-hq/synapse#17355)) * Bump msgpack from 1.0.7 to 1.0.8. ([\#17317](element-hq/synapse#17317)) * Bump netaddr from 1.2.1 to 1.3.0. ([\#17353](element-hq/synapse#17353)) * Bump packaging from 24.0 to 24.1. ([\#17352](element-hq/synapse#17352)) * Bump phonenumbers from 8.13.37 to 8.13.39. ([\#17315](element-hq/synapse#17315)) * Bump regex from 1.10.4 to 1.10.5. ([\#17290](element-hq/synapse#17290)) * Bump requests from 2.31.0 to 2.32.2. ([\#17345](element-hq/synapse#17345)) * Bump sentry-sdk from 2.1.1 to 2.3.1. ([\#17263](element-hq/synapse#17263)) * Bump sentry-sdk from 2.3.1 to 2.6.0. ([\#17351](element-hq/synapse#17351)) * Bump tornado from 6.4 to 6.4.1. ([\#17344](element-hq/synapse#17344)) * Bump mypy from 1.8.0 to 1.9.0. ([\#17297](element-hq/synapse#17297)) * Bump types-jsonschema from 4.21.0.20240311 to 4.22.0.20240610. ([\#17288](element-hq/synapse#17288)) * Bump types-netaddr from 1.2.0.20240219 to 1.3.0.20240530. ([\#17314](element-hq/synapse#17314)) * Bump types-pillow from 10.2.0.20240423 to 10.2.0.20240520. ([\#17285](element-hq/synapse#17285)) * Bump types-pyyaml from 6.0.12.12 to 6.0.12.20240311. ([\#17316](element-hq/synapse#17316)) * Bump typing-extensions from 4.11.0 to 4.12.2. ([\#17354](element-hq/synapse#17354)) * Bump urllib3 from 2.0.7 to 2.2.2. ([\#17346](element-hq/synapse#17346))

MadLittleMods added 5 commits July 8, 2024 15:35

Initial work of adding name/avatar to rooms response

9deb387

Add tests

b0bb37f

Add changelog

b6e36ef

Actually test that invited should see current state

6ef39dd

Start of heroes in Sliding Sync

32c5409

MadLittleMods added the A-Sync label Jul 8, 2024

MadLittleMods commented Jul 8, 2024

View reviewed changes

synapse/types/handlers/__init__.py Outdated Show resolved Hide resolved

MadLittleMods commented Jul 8, 2024

View reviewed changes

MadLittleMods added 2 commits July 8, 2024 18:54

Merge branch 'develop' into madlittlemods/sliding-sync-room-name-avatar

9641ca7

Merge branch 'madlittlemods/sliding-sync-room-name-avatar' into madli…

9692c76

…ttlemods/sliding-sync-heroes

Base automatically changed from madlittlemods/sliding-sync-room-name-avatar to develop July 9, 2024 17:26

MadLittleMods added 4 commits July 9, 2024 12:31

Merge branch 'develop' into madlittlemods/sliding-sync-heroes

ee9114c

Conflicts: synapse/handlers/sliding_sync.py

heroes is not required_state for now

f58d6fc

Discussion tracking this: matrix-org/matrix-spec-proposals#3575 (comment) It's also hard because we may not have any full membership events when you're the first one on your server to be invited to a new room over federation.

Add changelog

82bf80c

Add tests

10f8540

MadLittleMods commented Jul 9, 2024

View reviewed changes

MadLittleMods added 3 commits July 9, 2024 17:41

Remove unncessary check

62925b6

Test joined_count/invited_count

b9f1eb1

Test invite before/after ban

e50bf86

MadLittleMods commented Jul 9, 2024

View reviewed changes

synapse/storage/databases/main/roommember.py Outdated Show resolved Hide resolved

MadLittleMods added 2 commits July 9, 2024 18:08

Sort leave before knock

2fb77b3

See #17419 (comment)

Explain the order

275da50

MadLittleMods commented Jul 9, 2024

View reviewed changes

MadLittleMods added 2 commits July 9, 2024 18:23

Add test to make sure we only return 5 heroes

6060408

Fix lint

91cefaa

MadLittleMods commented Jul 9, 2024

View reviewed changes

synapse/storage/databases/main/roommember.py Outdated Show resolved Hide resolved

MadLittleMods marked this pull request as ready for review July 9, 2024 23:46

MadLittleMods requested a review from a team as a code owner July 9, 2024 23:46

erikjohnston reviewed Jul 11, 2024

View reviewed changes

MadLittleMods requested a review from erikjohnston July 11, 2024 15:14

MadLittleMods added 3 commits July 11, 2024 12:04

Revert heroes order changes

868dcdc

Better comment on what we expect

a4753bf

Merge branch 'develop' into madlittlemods/sliding-sync-heroes

5583ac1

MadLittleMods added a commit that referenced this pull request Jul 11, 2024

Order heroes by stream_ordering

1702b7c

Split off from #17419 (comment)

MadLittleMods mentioned this pull request Jul 11, 2024

Order heroes by stream_ordering (as spec'ed) #17435

Merged

3 tasks

MadLittleMods requested review from erikjohnston and removed request for erikjohnston July 11, 2024 17:12

MadLittleMods commented Jul 11, 2024

View reviewed changes

tests/rest/client/test_sync.py Outdated Show resolved Hide resolved

Have to use basic assertion for now

e2138b7

See #17419 (comment)

erikjohnston approved these changes Jul 11, 2024

View reviewed changes

MadLittleMods merged commit 5a97bbd into develop Jul 11, 2024
39 checks passed

MadLittleMods deleted the madlittlemods/sliding-sync-heroes branch July 11, 2024 19:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `heroes` and room summary fields to Sliding Sync `/sync` #17419

Add `heroes` and room summary fields to Sliding Sync `/sync` #17419

MadLittleMods commented Jul 8, 2024 •

edited

Loading

MadLittleMods Jul 8, 2024

MadLittleMods Jul 9, 2024

MadLittleMods Jul 9, 2024

MadLittleMods Jul 9, 2024 •

edited

Loading

MadLittleMods Jul 9, 2024 •

edited

Loading

erikjohnston Jul 11, 2024

MadLittleMods Jul 11, 2024

erikjohnston Jul 11, 2024

MadLittleMods Jul 11, 2024

erikjohnston left a comment

erikjohnston Jul 11, 2024

MadLittleMods Jul 11, 2024

erikjohnston Jul 11, 2024

MadLittleMods commented Jul 11, 2024

		# TODO: Figure out how to get the membership summary for left/banned rooms
		room_membership_summary = {}

Add heroes and room summary fields to Sliding Sync /sync #17419

Add heroes and room summary fields to Sliding Sync /sync #17419

Conversation

MadLittleMods commented Jul 8, 2024 • edited Loading

Dev notes

Pull Request Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MadLittleMods Jul 9, 2024 • edited Loading

Choose a reason for hiding this comment

MadLittleMods Jul 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

erikjohnston left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MadLittleMods commented Jul 11, 2024

Add `heroes` and room summary fields to Sliding Sync `/sync` #17419

Add `heroes` and room summary fields to Sliding Sync `/sync` #17419

MadLittleMods commented Jul 8, 2024 •

edited

Loading

MadLittleMods Jul 9, 2024 •

edited

Loading

MadLittleMods Jul 9, 2024 •

edited

Loading