Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Fixup worker doc (again) #8000

Merged
merged 5 commits into from
Aug 6, 2020
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/8000.doc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Improve workers docs.
37 changes: 24 additions & 13 deletions docs/workers.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ The processes communicate with each other via a Synapse-specific protocol called
feeds streams of newly written data between processes so they can be kept in
sync with the database state.

When configured to do so, Synapse uses a
When configured to do so, Synapse uses a
[Redis pub/sub channel](https://redis.io/topics/pubsub) to send the replication
stream between all configured Synapse processes. Additionally, processes may
make HTTP requests to each other, primarily for operations which need to wait
Expand Down Expand Up @@ -66,23 +66,31 @@ https://hub.docker.com/r/matrixdotorg/synapse/.

To make effective use of the workers, you will need to configure an HTTP
reverse-proxy such as nginx or haproxy, which will direct incoming requests to
the correct worker, or to the main synapse instance. See
the correct worker, or to the main synapse instance. See
[reverse_proxy.md](reverse_proxy.md) for information on setting up a reverse
proxy.

To enable workers you should create a configuration file for each worker
process. Each worker configuration file inherits the configuration of the shared
homeserver configuration file. You can then override configuration specific to
that worker, e.g. the HTTP listener that it provides (if any); logging
configuration; etc. You should minimise the number of overrides though to
maintain a usable config.
When using workers, each worker process has its own configuration file which
contains settings specific to that worker, such as the HTTP listener that it
provides (if any), logging configuration, etc.

Normally, the worker processes are configured to read from a shared
configuration file as well as the worker-specific configuration files. This
makes it easier to keep common configuration settings synchronised across all
the processes.

### Shared Configuration
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could keep these headers?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not really sure how they interact with the prose though, since we explain the worker config files outside the worker configuration section. Which feels a bit weird to me?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine. the top section introduces the structure; the later sections explain the details of what you need to put in each file.

What I would say is that the "Worker configuration" section goes on to talk about starting synapse, which could do with its own ###-level heading.

The main process is somewhat special in this respect: it does not normally
need its own configuration file and can take all of its configuration from the
shared configuration file.


### Shared configuration

Normally, only a couple of changes are needed to make an existing configuration
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Normally, only a couple of changes are needed to make an existing configuration
### Shared Configuration
Normally, only a couple of changes are needed to make an existing configuration

file suitable for use with workers. First, you need to enable an "HTTP replication
listener" for the main process; and secondly, you need to enable redis-based
replication. For example:

Next you need to add both a HTTP replication listener, used for HTTP requests
between processes, and redis config to the shared Synapse configuration file
(`homeserver.yaml`). For example:

```yaml
# extend the existing `listeners` section. This defines the ports that the
Expand All @@ -105,7 +113,7 @@ Under **no circumstances** should the replication listener be exposed to the
public internet; it has no authentication and is unencrypted.


### Worker Configuration
### Worker configuration

In the config file for each worker, you must specify the type of worker
application (`worker_app`), and you should specify a unqiue name for the worker
Expand Down Expand Up @@ -145,6 +153,9 @@ plain HTTP endpoint on port 8083 separately serving various endpoints, e.g.
Obviously you should configure your reverse-proxy to route the relevant
endpoints to the worker (`localhost:8083` in the above example).


### Running Synapse with workers

Finally, you need to start your worker processes. This can be done with either
`synctl` or your distribution's preferred service manager such as `systemd`. We
recommend the use of `systemd` where available: for information on setting up
Expand Down