Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve how we apply database migrations #4009

Merged
merged 4 commits into from
Oct 4, 2022
Merged

Improve how we apply database migrations #4009

merged 4 commits into from
Oct 4, 2022

Conversation

lutter
Copy link
Collaborator

@lutter lutter commented Sep 29, 2022

In the hosted service, restarting a node can take a long time. That can be almost entirely attributed to the fact that we rebuild the fdw mappings between shards on every start, and sometimes nodes have to wait for other activity to stop before that rebuild can happen. We rebuild the mappings on every start since we don't properly propagate whether the schema in a shard was changed to the other shards that import tables from that shard.

With this PR, the fdw mappings are only rebuilt if there were actually schema changes, i.e., in the vast majority of cases the rebuild can be skipped and startups will be much faster.

Since interrupting graph-node startup at the wrong time (right after migrations ran but before remapping is done) can cause fdw mappings to not be updated, graphman now has a command graphman database remap to kick off remapping manually.

It also now has a command graphman database migrate to apply schema migrations, though graph-node still does that on startup.

@lutter lutter force-pushed the lutter/migrations branch from 75dad84 to ef75f31 Compare October 4, 2022 18:51
@lutter lutter merged commit ef75f31 into master Oct 4, 2022
@lutter lutter deleted the lutter/migrations branch October 4, 2022 18:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants