Remove experimental MSC2716 implementation to incrementally import history into existing rooms

[MadLittleMods]

Remove experimental MSC2716 implementation to incrementally import history into existing rooms

Context for why we're removing the implementation:

• MSC2716: Incrementally importing history into existing rooms matrix-spec-proposals#2716 (comment)
• MSC2716: Incrementally importing history into existing rooms matrix-spec-proposals#2716 (comment)

Spurred on by wanting to optimize get_backfill_events(...) as part of #15653 without dealing with the MSC2716 logic.

Anyone wanting to continue MSC2716, should also address these leftover tasks: #10737

Closes #10737 in the fact that it is not longer necessary to track those things.

Todo

• Remove database tables -> Gracefully drop MSC2716 related tables (incrementally import history) #15786
  • insertion_events
  • insertion_event_edges
  • insertion_event_extremities
  • batch_events
  • See Remove experimental MSC2716 implementation to incrementally import history into existing rooms #15748 (comment)
• Potentially remove ... argument threading through to everything (like state_event_ids and allow_no_prev_events)
  • We can handle this in future PRs as we see fit
• Remove event.internal_metadata.is_historical and m.historical things

Pull Request Checklist

• Pull request is based on the develop branch
• Pull request includes a changelog file. The entry should:
  • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
  • Use markdown where necessary, mostly for code blocks.
  • End with either a period (.) or an exclamation mark (!).
  • Start with a capital letter.
  • Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
• Pull request includes a sign off
• Code style is correct
  (run the linters)

[MadLittleMods]

Do we need to worry about gracefully dropping database tables for an experimental feature like this?

Relevant tables to drop:

-- Drop tables related to MSC2716 since the implementation is being removed
DROP TABLE insertion_events;
DROP TABLE insertion_event_edges;
DROP TABLE insertion_event_extremities;
DROP TABLE batch_events;

Can we just stop writing to the tables in the Synapse version that this PR ships in, then in the next Synapse version, bump the SCHEMA_COMPAT_VERSION and drop the tables?

[erikjohnston]

I'd say we may as well? We still want to be careful that we don't accidentally break upgrades/downgrades for people that have turned on experimental features.

Easiest way of tracking the task of dropping those tables is to create an issue with the future maintenance label and add it to one of the milestones.

[MadLittleMods]

Created #15786 to track this ⏩

[erikjohnston]

Removing this room version will mean that users won't be able to leave / interact with such rooms. Not sure if that's a problem for this feature or not

[MadLittleMods]

ehh, people are going to get left behind regardless of the time period we give and I don't think this was very widely used. I've created #15785 to track a solution to this problem in general.

[erikjohnston]

Otherwise I think this looks good

[MadLittleMods]

Thanks for the review @erikjohnston 🪹