This repository has been archived by the owner on Apr 26, 2024. It is now read-only.
-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Fixup worker doc (again) #8000
Merged
Merged
Fixup worker doc (again) #8000
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
Improve workers docs. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
@@ -23,7 +23,7 @@ The processes communicate with each other via a Synapse-specific protocol called | |||||||||
feeds streams of newly written data between processes so they can be kept in | ||||||||||
sync with the database state. | ||||||||||
|
||||||||||
When configured to do so, Synapse uses a | ||||||||||
When configured to do so, Synapse uses a | ||||||||||
[Redis pub/sub channel](https://redis.io/topics/pubsub) to send the replication | ||||||||||
stream between all configured Synapse processes. Additionally, processes may | ||||||||||
make HTTP requests to each other, primarily for operations which need to wait | ||||||||||
|
@@ -66,23 +66,31 @@ https://hub.docker.com/r/matrixdotorg/synapse/. | |||||||||
|
||||||||||
To make effective use of the workers, you will need to configure an HTTP | ||||||||||
reverse-proxy such as nginx or haproxy, which will direct incoming requests to | ||||||||||
the correct worker, or to the main synapse instance. See | ||||||||||
the correct worker, or to the main synapse instance. See | ||||||||||
[reverse_proxy.md](reverse_proxy.md) for information on setting up a reverse | ||||||||||
proxy. | ||||||||||
|
||||||||||
To enable workers you should create a configuration file for each worker | ||||||||||
process. Each worker configuration file inherits the configuration of the shared | ||||||||||
homeserver configuration file. You can then override configuration specific to | ||||||||||
that worker, e.g. the HTTP listener that it provides (if any); logging | ||||||||||
configuration; etc. You should minimise the number of overrides though to | ||||||||||
maintain a usable config. | ||||||||||
When using workers, each worker process has its own configuration file which | ||||||||||
contains settings specific to that worker, such as the HTTP listener that it | ||||||||||
provides (if any), logging configuration, etc. | ||||||||||
|
||||||||||
Normally, the worker processes are configured to read from a shared | ||||||||||
configuration file as well as the worker-specific configuration files. This | ||||||||||
makes it easier to keep common configuration settings synchronised across all | ||||||||||
the processes. | ||||||||||
|
||||||||||
### Shared Configuration | ||||||||||
The main process is somewhat special in this respect: it does not normally | ||||||||||
need its own configuration file and can take all of its configuration from the | ||||||||||
shared configuration file. | ||||||||||
|
||||||||||
|
||||||||||
### Shared configuration | ||||||||||
|
||||||||||
Normally, only a couple of changes are needed to make an existing configuration | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
file suitable for use with workers. First, you need to enable an "HTTP replication | ||||||||||
listener" for the main process; and secondly, you need to enable redis-based | ||||||||||
replication. For example: | ||||||||||
|
||||||||||
Next you need to add both a HTTP replication listener, used for HTTP requests | ||||||||||
between processes, and redis config to the shared Synapse configuration file | ||||||||||
(`homeserver.yaml`). For example: | ||||||||||
|
||||||||||
```yaml | ||||||||||
# extend the existing `listeners` section. This defines the ports that the | ||||||||||
|
@@ -105,7 +113,7 @@ Under **no circumstances** should the replication listener be exposed to the | |||||||||
public internet; it has no authentication and is unencrypted. | ||||||||||
|
||||||||||
|
||||||||||
### Worker Configuration | ||||||||||
### Worker configuration | ||||||||||
|
||||||||||
In the config file for each worker, you must specify the type of worker | ||||||||||
application (`worker_app`), and you should specify a unqiue name for the worker | ||||||||||
|
@@ -145,6 +153,9 @@ plain HTTP endpoint on port 8083 separately serving various endpoints, e.g. | |||||||||
Obviously you should configure your reverse-proxy to route the relevant | ||||||||||
endpoints to the worker (`localhost:8083` in the above example). | ||||||||||
|
||||||||||
|
||||||||||
### Running Synapse with workers | ||||||||||
|
||||||||||
Finally, you need to start your worker processes. This can be done with either | ||||||||||
`synctl` or your distribution's preferred service manager such as `systemd`. We | ||||||||||
recommend the use of `systemd` where available: for information on setting up | ||||||||||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could keep these headers?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not really sure how they interact with the prose though, since we explain the worker config files outside the worker configuration section. Which feels a bit weird to me?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's fine. the top section introduces the structure; the later sections explain the details of what you need to put in each file.
What I would say is that the "Worker configuration" section goes on to talk about starting synapse, which could do with its own
###
-level heading.