add aliases/lists: when sending to an alias, the message gets deliver…

…ed to all members

the members must currently all be addresses of local accounts.

a message sent to an alias is accepted if at least one of the members accepts
it. if no members accepts it (e.g. due to bad reputation of sender), the
message is rejected.

if a message is submitted to both an alias addresses and to recipients that are
members of the alias in an smtp transaction, the message will be delivered to
such members only once.  the same applies if the address in the message
from-header is the address of a member: that member won't receive the message
(they sent it). this prevents duplicate messages.

aliases have three configuration options:
- PostPublic: whether anyone can send through the alias, or only members.
  members-only lists can be useful inside organizations for internal
  communication. public lists can be useful for support addresses.
- ListMembers: whether members can see the addresses of other members. this can
  be seen in the account web interface. in the future, we could export this in
  other ways, so clients can expand the list.
- AllowMsgFrom: whether messages can be sent through the alias with the alias
  address used in the message from-header. the webmail knows it can use that
  address, and will use it as from-address when replying to a message sent to
  that address.

ideas for the future:
- allow external addresses as members. still with some restrictions, such as
  requiring a valid dkim-signature so delivery has a chance to succeed. will
  also need configuration of an admin that can receive any bounces.
- allow specifying specific members who can sent through the list (instead of
  all members).

for github issue #57 by hmfaysal.
also relevant for #99 by naturalethic.
thanks to damir & marin from sartura for discussing requirements/features.

README.md

@@ -24,10 +24,10 @@ See Quickstart below to get started.
 - Automatic TLS with ACME, for use with Let's Encrypt and other CA's.
 - DANE and MTA-STS for inbound and outbound delivery over SMTP with STARTTLS,
   including REQUIRETLS and with incoming/outgoing TLSRPT reporting.
-- Web admin interface that helps you set up your domains and accounts
-  (instructions to create DNS records, configure
-  SPF/DKIM/DMARC/TLSRPT/MTA-STS), for status information, managing
-  accounts/domains, and modifying the configuration file.
+- Web admin interface that helps you set up your domains, accounts and list
+  aliases (instructions to create DNS records, configure
+  SPF/DKIM/DMARC/TLSRPT/MTA-STS), for status information, and modifying the
+  configuration file.
 - Account autodiscovery (with SRV records, Microsoft-style, Thunderbird-style,
   and Apple device management profiles) for easy account setup (though client
   support is limited).
@@ -135,7 +135,6 @@ https://nlnet.nl/project/Mox/.
 
 ## Roadmap
 
-- Aliases, for delivering to multiple local accounts.
 - Calendaring with CalDAV/iCal
 - More IMAP extensions (PREVIEW, WITHIN, IMPORTANT, COMPRESS=DEFLATE,
   CREATE-SPECIAL-USE, SAVEDATE, UNAUTHENTICATE, REPLACE, QUOTA, NOTIFY,
@@ -145,6 +144,7 @@ https://nlnet.nl/project/Mox/.
 - Forwarding (to an external address)
 - Add special IMAP mailbox ("Queue?") that contains queued but
   undelivered messages, updated with IMAP flags/keywords/tags and message headers.
+- External addresses in aliases/lists.
 - Sieve for filtering (for now see Rulesets in the account config)
 - Autoresponder (out of office/vacation)
 - OAUTH2 support, for single sign on

config/config.go

@@ -273,17 +273,18 @@ type TransportDirect struct {
 }
 
 type Domain struct {
-	Description                string  `sconf:"optional" sconf-doc:"Free-form description of domain."`
-	ClientSettingsDomain       string  `sconf:"optional" sconf-doc:"Hostname for client settings instead of the mail server hostname. E.g. mail.<domain>. For future migration to another mail operator without requiring all clients to update their settings, it is convenient to have client settings that reference a subdomain of the hosted domain instead of the hostname of the server where the mail is currently hosted. If empty, the hostname of the mail server is used for client configurations. Unicode name."`
-	LocalpartCatchallSeparator string  `sconf:"optional" sconf-doc:"If not empty, only the string before the separator is used to for email delivery decisions. For example, if set to \"+\", [email protected] will be delivered to [email protected]."`
-	LocalpartCaseSensitive     bool    `sconf:"optional" sconf-doc:"If set, upper/lower case is relevant for email delivery."`
-	DKIM                       DKIM    `sconf:"optional" sconf-doc:"With DKIM signing, a domain is taking responsibility for (content of) emails it sends, letting receiving mail servers build up a (hopefully positive) reputation of the domain, which can help with mail delivery."`
-	DMARC                      *DMARC  `sconf:"optional" sconf-doc:"With DMARC, a domain publishes, in DNS, a policy on how other mail servers should handle incoming messages with the From-header matching this domain and/or subdomain (depending on the configured alignment). Receiving mail servers use this to build up a reputation of this domain, which can help with mail delivery. A domain can also publish an email address to which reports about DMARC verification results can be sent by verifying mail servers, useful for monitoring. Incoming DMARC reports are automatically parsed, validated, added to metrics and stored in the reporting database for later display in the admin web pages."`
-	MTASTS                     *MTASTS `sconf:"optional" sconf-doc:"MTA-STS is a mechanism that allows publishing a policy with requirements for WebPKI-verified SMTP STARTTLS connections for email delivered to a domain. Existence of a policy is announced in a DNS TXT record (often unprotected/unverified, MTA-STS's weak spot). If a policy exists, it is fetched with a WebPKI-verified HTTPS request. The policy can indicate that WebPKI-verified SMTP STARTTLS is required, and which MX hosts (optionally with a wildcard pattern) are allowd. MX hosts to deliver to are still taken from DNS (again, not necessarily protected/verified), but messages will only be delivered to domains matching the MX hosts from the published policy. Mail servers look up the MTA-STS policy when first delivering to a domain, then keep a cached copy, periodically checking the DNS record if a new policy is available, and fetching and caching it if so. To update a policy, first serve a new policy with an updated policy ID, then update the DNS record (not the other way around). To remove an enforced policy, publish an updated policy with mode \"none\" for a long enough period so all cached policies have been refreshed (taking DNS TTL and policy max age into account), then remove the policy from DNS, wait for TTL to expire, and stop serving the policy."`
-	TLSRPT                     *TLSRPT `sconf:"optional" sconf-doc:"With TLSRPT a domain specifies in DNS where reports about encountered SMTP TLS behaviour should be sent. Useful for monitoring. Incoming TLS reports are automatically parsed, validated, added to metrics and stored in the reporting database for later display in the admin web pages."`
-	Routes                     []Route `sconf:"optional" sconf-doc:"Routes for delivering outgoing messages through the queue. Each delivery attempt evaluates account routes, these domain routes and finally global routes. The transport of the first matching route is used in the delivery attempt. If no routes match, which is the default with no configured routes, messages are delivered directly from the queue."`
-
-	Domain                  dns.Domain `sconf:"-" json:"-"`
+	Description                string           `sconf:"optional" sconf-doc:"Free-form description of domain."`
+	ClientSettingsDomain       string           `sconf:"optional" sconf-doc:"Hostname for client settings instead of the mail server hostname. E.g. mail.<domain>. For future migration to another mail operator without requiring all clients to update their settings, it is convenient to have client settings that reference a subdomain of the hosted domain instead of the hostname of the server where the mail is currently hosted. If empty, the hostname of the mail server is used for client configurations. Unicode name."`
+	LocalpartCatchallSeparator string           `sconf:"optional" sconf-doc:"If not empty, only the string before the separator is used to for email delivery decisions. For example, if set to \"+\", [email protected] will be delivered to [email protected]."`
+	LocalpartCaseSensitive     bool             `sconf:"optional" sconf-doc:"If set, upper/lower case is relevant for email delivery."`
+	DKIM                       DKIM             `sconf:"optional" sconf-doc:"With DKIM signing, a domain is taking responsibility for (content of) emails it sends, letting receiving mail servers build up a (hopefully positive) reputation of the domain, which can help with mail delivery."`
+	DMARC                      *DMARC           `sconf:"optional" sconf-doc:"With DMARC, a domain publishes, in DNS, a policy on how other mail servers should handle incoming messages with the From-header matching this domain and/or subdomain (depending on the configured alignment). Receiving mail servers use this to build up a reputation of this domain, which can help with mail delivery. A domain can also publish an email address to which reports about DMARC verification results can be sent by verifying mail servers, useful for monitoring. Incoming DMARC reports are automatically parsed, validated, added to metrics and stored in the reporting database for later display in the admin web pages."`
+	MTASTS                     *MTASTS          `sconf:"optional" sconf-doc:"MTA-STS is a mechanism that allows publishing a policy with requirements for WebPKI-verified SMTP STARTTLS connections for email delivered to a domain. Existence of a policy is announced in a DNS TXT record (often unprotected/unverified, MTA-STS's weak spot). If a policy exists, it is fetched with a WebPKI-verified HTTPS request. The policy can indicate that WebPKI-verified SMTP STARTTLS is required, and which MX hosts (optionally with a wildcard pattern) are allowd. MX hosts to deliver to are still taken from DNS (again, not necessarily protected/verified), but messages will only be delivered to domains matching the MX hosts from the published policy. Mail servers look up the MTA-STS policy when first delivering to a domain, then keep a cached copy, periodically checking the DNS record if a new policy is available, and fetching and caching it if so. To update a policy, first serve a new policy with an updated policy ID, then update the DNS record (not the other way around). To remove an enforced policy, publish an updated policy with mode \"none\" for a long enough period so all cached policies have been refreshed (taking DNS TTL and policy max age into account), then remove the policy from DNS, wait for TTL to expire, and stop serving the policy."`
+	TLSRPT                     *TLSRPT          `sconf:"optional" sconf-doc:"With TLSRPT a domain specifies in DNS where reports about encountered SMTP TLS behaviour should be sent. Useful for monitoring. Incoming TLS reports are automatically parsed, validated, added to metrics and stored in the reporting database for later display in the admin web pages."`
+	Routes                     []Route          `sconf:"optional" sconf-doc:"Routes for delivering outgoing messages through the queue. Each delivery attempt evaluates account routes, these domain routes and finally global routes. The transport of the first matching route is used in the delivery attempt. If no routes match, which is the default with no configured routes, messages are delivered directly from the queue."`
+	Aliases                    map[string]Alias `sconf:"optional" sconf-doc:"Aliases that cause messages to be delivered to one or more locally configured addresses. Keys are localparts (encoded, as they appear in email addresses)."`
+
+	Domain                  dns.Domain `sconf:"-"`
 	ClientSettingsDNSDomain dns.Domain `sconf:"-" json:"-"`
 
 	// Set when DMARC and TLSRPT (when set) has an address with different domain (we're
@@ -292,6 +293,27 @@ type Domain struct {
 	ReportsOnly bool `sconf:"-" json:"-"`
 }
 
+// todo: allow external addresses as members of aliases. we would add messages for them to the queue for outgoing delivery. we should require an admin addresses to which delivery failures will be delivered (locally, and to use in smtp mail from, so dsns go there). also take care to evaluate smtputf8 (if external address requires utf8 and incoming transaction didn't).
+// todo: as alternative to PostPublic, allow specifying a list of addresses (dmarc-like verified) that are (the only addresses) allowed to post to the list. if msgfrom is an external address, require a valid dkim signature to prevent dmarc-policy-related issues when delivering to remote members.
+// todo: add option to require messages sent to an alias have that alias as From or Reply-To address?
+
+type Alias struct {
+	Addresses    []string `sconf-doc:"Expanded addresses to deliver to. These must currently be of addresses of local accounts. To prevent duplicate messages, a member address that is also an explicit recipient in the SMTP transaction will only have the message delivered once. If the address in the message From header is a member, that member also won't receive the message."`
+	PostPublic   bool     `sconf:"optional" sconf-doc:"If true, anyone can send messages to the list. Otherwise only members, based on message From address, which is assumed to be DMARC-like-verified."`
+	ListMembers  bool     `sconf:"optional" sconf-doc:"If true, members can see addresses of members."`
+	AllowMsgFrom bool     `sconf:"optional" sconf-doc:"If true, members are allowed to send messages with this alias address in the message From header."`
+
+	LocalpartStr    string         `sconf:"-"` // In encoded form.
+	Domain          dns.Domain     `sconf:"-"`
+	ParsedAddresses []AliasAddress `sconf:"-"` // Matches addresses.
+}
+
+type AliasAddress struct {
+	Address     smtp.Address // Parsed address.
+	AccountName string       // Looked up.
+	Destination Destination  // Belonging to address.
+}
+
 type DMARC struct {
 	Localpart string `sconf-doc:"Address-part before the @ that accepts DMARC reports. Must be non-internationalized. Recommended value: dmarc-reports."`
 	Domain    string `sconf:"optional" sconf-doc:"Alternative domain for reporting address, for incoming reports. Typically empty, causing the domain wherein this config exists to be used. Can be used to receive reports for domains that aren't fully hosted on this server. Configure such a domain as a hosted domain without making all the DNS changes, and configure this field with a domain that is fully hosted on this server, so the localpart and the domain of this field form a reporting address. Then only update the DMARC DNS record for the not fully hosted domain, ensuring the reporting address is specified in its \"rua\" field as shown in the suggested DNS settings. Unicode name."`
@@ -412,6 +434,13 @@ type Account struct {
 	NeutralMailbox             *regexp.Regexp `sconf:"-" json:"-"`
 	NotJunkMailbox             *regexp.Regexp `sconf:"-" json:"-"`
 	ParsedFromIDLoginAddresses []smtp.Address `sconf:"-" json:"-"`
+	Aliases                    []AddressAlias `sconf:"-"`
+}
+
+type AddressAlias struct {
+	SubscriptionAddress string
+	Alias               Alias    // Without members.
+	MemberAddresses     []string // Only if allowed to see.
 }
 
 type JunkFilter struct {

config/doc.go

@@ -913,6 +913,31 @@ See https://pkg.go.dev/github.com/mjl-/sconf for details.
 					MinimumAttempts: 0
 					Transport:
 
+			# Aliases that cause messages to be delivered to one or more locally configured
+			# addresses. Keys are localparts (encoded, as they appear in email addresses).
+			# (optional)
+			Aliases:
+				x:
+
+					# Expanded addresses to deliver to. These must currently be of addresses of local
+					# accounts. To prevent duplicate messages, a member address that is also an
+					# explicit recipient in the SMTP transaction will only have the message delivered
+					# once. If the address in the message From header is a member, that member also
+					# won't receive the message.
+					Addresses:
+						-
+
+					# If true, anyone can send messages to the list. Otherwise only members, based on
+					# message From address, which is assumed to be DMARC-like-verified. (optional)
+					PostPublic: false
+
+					# If true, members can see addresses of members. (optional)
+					ListMembers: false
+
+					# If true, members are allowed to send messages with this alias address in the
+					# message From header. (optional)
+					AllowMsgFrom: false
+
 	# Accounts represent mox users, each with a password and email address(es) to
 	# which email can be delivered (possibly at different domains). Each account has
 	# its own on-disk directory holding its messages and index database. An account

ctl.go

@@ -9,6 +9,7 @@ import (
 	"io"
 	"log"
 	"log/slog"
+	"maps"
 	"net"
 	"os"
 	"path/filepath"
@@ -20,6 +21,7 @@ import (
 
 	"github.com/mjl-/bstore"
 
+	"github.com/mjl-/mox/config"
 	"github.com/mjl-/mox/dns"
 	"github.com/mjl-/mox/message"
 	"github.com/mjl-/mox/metrics"
@@ -1017,6 +1019,165 @@ func servectlcmd(ctx context.Context, ctl *ctl, shutdown func()) {
 		ctl.xcheck(err, "removing address")
 		ctl.xwriteok()
 
+	case "aliaslist":
+		/* protocol:
+		> "aliaslist"
+		> domain
+		< "ok" or error
+		< stream
+		*/
+		domain := ctl.xread()
+		d, err := dns.ParseDomain(domain)
+		ctl.xcheck(err, "parsing domain")
+		dc, ok := mox.Conf.Domain(d)
+		if !ok {
+			ctl.xcheck(errors.New("no such domain"), "listing aliases")
+		}
+		ctl.xwriteok()
+		w := ctl.writer()
+		for _, a := range dc.Aliases {
+			lp, err := smtp.ParseLocalpart(a.LocalpartStr)
+			ctl.xcheck(err, "parsing alias localpart")
+			fmt.Fprintln(w, smtp.NewAddress(lp, a.Domain).Pack(true))
+		}
+		w.xclose()
+
+	case "aliasprint":
+		/* protocol:
+		> "aliasprint"
+		> address
+		< "ok" or error
+		< stream
+		*/
+		address := ctl.xread()
+		_, alias, ok := mox.Conf.AccountDestination(address)
+		if !ok {
+			ctl.xcheck(errors.New("no such address"), "looking up alias")
+		} else if alias == nil {
+			ctl.xcheck(errors.New("address not an alias"), "looking up alias")
+		}
+		ctl.xwriteok()
+		w := ctl.writer()
+		fmt.Fprintf(w, "# postpublic %v\n", alias.PostPublic)
+		fmt.Fprintf(w, "# listmembers %v\n", alias.ListMembers)
+		fmt.Fprintf(w, "# allowmsgfrom %v\n", alias.AllowMsgFrom)
+		fmt.Fprintln(w, "# members:")
+		for _, a := range alias.Addresses {
+			fmt.Fprintln(w, a)
+		}
+		w.xclose()
+
+	case "aliasadd":
+		/* protocol:
+		> "aliasadd"
+		> address
+		> json alias
+		< "ok" or error
+		*/
+		address := ctl.xread()
+		line := ctl.xread()
+		addr, err := smtp.ParseAddress(address)
+		ctl.xcheck(err, "parsing address")
+		var alias config.Alias
+		xparseJSON(ctl, line, &alias)
+		err = mox.AliasAdd(ctx, addr, alias)
+		ctl.xcheck(err, "adding alias")
+		ctl.xwriteok()
+
+	case "aliasupdate":
+		/* protocol:
+		> "aliasupdate"
+		> alias
+		> "true" or "false" for postpublic
+		> "true" or "false" for listmembers
+		> "true" or "false" for allowmsgfrom
+		< "ok" or error
+		*/
+		address := ctl.xread()
+		postpublic := ctl.xread()
+		listmembers := ctl.xread()
+		allowmsgfrom := ctl.xread()
+		addr, err := smtp.ParseAddress(address)
+		ctl.xcheck(err, "parsing address")
+		err = mox.DomainSave(ctx, addr.Domain.Name(), func(d *config.Domain) error {
+			a, ok := d.Aliases[addr.Localpart.String()]
+			if !ok {
+				return fmt.Errorf("alias does not exist")
+			}
+
+			switch postpublic {
+			case "false":
+				a.PostPublic = false
+			case "true":
+				a.PostPublic = true
+			}
+			switch listmembers {
+			case "false":
+				a.ListMembers = false
+			case "true":
+				a.ListMembers = true
+			}
+			switch allowmsgfrom {
+			case "false":
+				a.AllowMsgFrom = false
+			case "true":
+				a.AllowMsgFrom = true
+			}
+
+			d.Aliases = maps.Clone(d.Aliases)
+			d.Aliases[addr.Localpart.String()] = a
+			return nil
+		})
+		ctl.xcheck(err, "saving alias")
+		ctl.xwriteok()
+
+	case "aliasrm":
+		/* protocol:
+		> "aliasrm"
+		> alias
+		< "ok" or error
+		*/
+		address := ctl.xread()
+		addr, err := smtp.ParseAddress(address)
+		ctl.xcheck(err, "parsing address")
+		err = mox.AliasRemove(ctx, addr)
+		ctl.xcheck(err, "removing alias")
+		ctl.xwriteok()
+
+	case "aliasaddaddr":
+		/* protocol:
+		> "aliasaddaddr"
+		> alias
+		> addresses as json
+		< "ok" or error
+		*/
+		address := ctl.xread()
+		line := ctl.xread()
+		addr, err := smtp.ParseAddress(address)
+		ctl.xcheck(err, "parsing address")
+		var addresses []string
+		xparseJSON(ctl, line, &addresses)
+		err = mox.AliasAddressesAdd(ctx, addr, addresses)
+		ctl.xcheck(err, "adding addresses to alias")
+		ctl.xwriteok()
+
+	case "aliasrmaddr":
+		/* protocol:
+		> "aliasrmaddr"
+		> alias
+		> addresses as json
+		< "ok" or error
+		*/
+		address := ctl.xread()
+		line := ctl.xread()
+		addr, err := smtp.ParseAddress(address)
+		ctl.xcheck(err, "parsing address")
+		var addresses []string
+		xparseJSON(ctl, line, &addresses)
+		err = mox.AliasAddressesRemove(ctx, addr, addresses)
+		ctl.xcheck(err, "removing addresses to alias")
+		ctl.xwriteok()
+
 	case "loglevels":
 		/* protocol:
 		> "loglevels"