RFC: Meta Transaction Implementation

[muharem]

Summary

This document proposes several potential solutions for implementing Meta Transactions for FRAME, accompanied by draft implementations. The first solution relies solely on a runtime’s general transaction and its transaction extensions, also known as signed extensions. In contrast, the second and third solutions introduce a new pallet call. In these solutions, validation pipeline for a meta transaction is abstracted via transaction extensions or a new contract/trait and can be defined through the pallet's configuration.

Motivation

The concept of Meta Transaction is well-established in the Ethereum ecosystem, referring to a transaction authorised by one party (signer) and executed by an untrusted third party that covers the transaction fees (relayer). This concept proves useful in scenarios where the signer lacks the assets to cover the fee or lacks the incentive to do so.
Examples include (source: #266):

• dApp covering transaction fees for its users;
• proxy accounts lacking balance;
• transaction fees paid in any asset when a signer incentivizes a relayer to cover the fee, as demonstrated by a transaction such as batch([sendCustomToken(relayerAddr, amount), doTheActualWork()]);
• signer delegating voting power without covering the transaction fee.

The solution must ensure that a relayer can execute only actions authorized by the signer and is protected from attacks such as replay attacks. Since it does concerns the transaction layer, it must be aligned with the Extrinsic Horizon (#2415).

Additionally, an important consideration is the complexity of implementation and integration for clients. Furthermore, this proposal addresses the issue where a signer lacks an on-chain account, preventing the storage of information such as nonce (an issue initially raised and discussed here - #266).

The initial solution, which relies solely on extending the extension type of a general transaction, has sparked some scepticism regarding its complexity. Therefore, alternative solutions are presented here, along with draft implementations.

Proposed Solutions

Solution 1: Meta Transaction as Extension to General Transaction.

Draft Implementation: #3712

The first solution extends the runtime’s transaction extension type with two new extensions. Additionally, it introduces a new context type for transaction extension to share a relevant information down the pipeline.

TxExtension = (VerifyRelayerSig, VerifyAccountSig, MetaTxExtension);
MetaTxExtension = (CheckNonZeroSender, CheckSpecVersion, ..., RelayTransactionPayment);

The first two extensions support meta transactions; they are optional and no-op if a signature is not provided. The last extension, responsible for fee charging, now includes an optional user data field - tx_relayer, which sets a restriction on who has to cover the fee. If left unset, it operates as before, with the origin covering the fee. If set to AnyRelayer or Relayer(account), the context must carry the account of the relayer from whom the fee is charged.

The signer constructs a meta transaction similarly to a general transaction, signs it, and shares it with the world. It should include the extensions starting from CheckNonZeroSender, including specifications, genesis, and nonce checks.

type MetaTx = (Account, Call, MetaTxExtension, Signature);

The relayer can read the transaction and evaluate it. It constructs a regular transaction using the data provided by the signer and fulfilling two missing extensions. The VerifyAccountSig extension requires a signer's account and signature to verify the call and the subsequent extension, utilizing a new concept of inherent implications (doc), to set an origin as signed by the signer. The VerifyRelayerSig similarly verifies the relayer's signature and alters the context to set the relayer account id for later fee charging.

let final_tx = (Call, VerifyRelayerSig::from(relayer_sig) + VerifyAccountSig::from(signer_sig) + MetaTxExtension)

Pros/Cons

I find this solution to be most appealing because it introduces minimal entropy and complexity and leverages existing formats and contracts. This should lead to simpler client integration. Additionally, meta transactions implemented in this way can inherit features introduced later for general transactions, such as origin authorization with advanced cryptography.

Solution 2.1: Meta Transactions as a specialised Pallet's Call with Transaction Extensions

Draft Implementation: #4122

Another approach involves passing information about meta transactions, such as the target call, requirements (e.g., nonce check), and the signer's signature, as arguments to a specialised pallet's call. In this approach, the runtime extrinsic's transaction extensions remain unchanged. The relayer signs a transaction targeting the call with information from the signer as an argument.
Since the call must perform the same checks for nonce, chain's genesis, mortality, etc., the pallets require a configurable means to do so. This can be abstracted using the same transaction extension trait and leveraging the same types like CheckNonce.

Pros/Cons

Upon reviewing the draft implementation, one can observe that a significant portion of functionality for validating the extrinsic and handling the dispatch is duplicated at the pallet’s call level. This results in increased complexity and necessitates custom solutions for integration. Additionally, some new features might require integration at both levels.

Utilising the existing transaction extension trait for checks in the pallet has the advantage of avoiding duplication of functionality but introduces new utilisation for that contract, potentially affecting its flexibility and increasing maintenance complexity.

However, this solution may facilitate the implementation of the atomic multi-origin transaction described in this issue - #266.

Solution 2.2: Meta Transactions as a specialised Pallet's Call with New Contract

A third potential solution mirrors the approach of Solution 2.1 but employs a custom contract/trait instead of transaction extension.

Pros/Cons

This solution inherits considerations from Solution 2.1, with the distinction that transaction extensions are exempt from this new custom use. However, functionalities like nonce checks to validate transactions will need to be implemented for a different contract.

The issue of a non-existent signer account and the chain's inability to track its nonce can be addressed across all three solutions by introducing an additional extension that creates an account for the signer by incrementing the provider reference and, for example, deducting a deposit from a relayer.

FROM HERE AND BELOW UPDATED ON 2 MAY 08:37 UTC

Conclusion on this RFC in the comment - #4123 (comment)

[joepetrowski]

I find this solution to be most appealing because it introduces minimal entropy and complexity and leverages existing formats and contracts. This should lead to simpler client integration. Additionally, meta transactions implemented in this way can inherit features introduced later for general transactions, such as origin authorization with advanced cryptography.

I share this sentiment, but would be good to get some feedback from UX/wallet developers. I also have the impression that (a) the pallet version could probably be delivered sooner, and (b) both solutions could co-exist. If option 2 can be available sooner and unlocks highly demanded features, that could be a compelling reason to implement it.

@TarikGul @Tbaut @antonkhvorov

[Tbaut]

Pinging @josepot as well, since this will mostly be abstracted away by the api layer, and Dapps will only minimally have to change things. In terms of demand from the ecosystem, I haven't personally heard loud voices about it, I'll ask around. If anything, this signal could help between choosing the solution that is quicker, or more flexible.

[josepot]

I share this sentiment, but would be good to get some feedback from UX/wallet developers.

IMO solution 2.x is preferable for the simple reason that it doesn't imply adding support to new signed-extensions. Also, because IMO option 1 strikes to me as a premature abstraction. Is this really a cross-cutting concern? Because if it isn't, then option 1 shouldn't be the way to go. I'm actually a bit on the fence myself, but I'm a bit more inclined to think that this is not a cross-cutting concern.

This should lead to simpler client integration

As of today that really isn't the case. Supporting new signed-extensions means ensuring that all client libraries (polkadot-api, PJS, subxt, etc) release a new version that adds support for the signed-extensions, while also ensuring that all browser-extensions, wallets, signers, etc also support the new signed-extensions.

Solution 2.x, on the other hand, shouldn't imply releasing a new library version (at least in the case of Polkadot-API) and just providing some helpers/sugar for well-known patterns once we have figured them out. With option 2.x we would be able to start using it and iterating on it right away, without making any changes into the "core" of the client libraries, because the "sugar" is always something that we can add later, once we have discovered the best DX patterns.

both solutions could co-exist

I actually hope this never happens. I mean, if it happens temporarily on a test-network due to the fact that we are still iterating on this, because we are trying to figure out which one is the best solution, then that's fine, of course. However, at some point we should pick a winner and stick to it.

since this will mostly be abstracted away by the api layer, and Dapps will only minimally have to change things

Again, I'm afraid that this could lead to premature abstractions. In the end, the DApp will have to -regardless of which option we go with- be responsible for passing the relayer context for the transaction. So, I'm not sure that there is a whole lot that can be abstracted away in here.

[joepetrowski]

As a result of RFC-84, the signed extensions will eventually change. If people don't think that both solutions should co-exist, we should pick the better option with the knowledge that client libraries will need to adapt to it later.

[josepot]

I've been thinking a bit about what the best API would be for leveraging "meta transactions", and unless I misunderstood something (which is quite possible), I think that solution 2.x also happens to compose better. In the sense that it would be possible to create batched transactions where certain parts of the batch have a relayer context, while other parts of the batch have a different relayer context, which TBH I think that it could be quite interesting both from a DX perspective and also because it could enable some interesting use-cases.

Did I misunderstand something? WDYT?

[joepetrowski]

In the sense that it would be possible to create batched transactions where certain parts of the batch have a relayer context, while other parts of the batch have a different relayer context, which TBH I think that it could be quite interesting both from a DX perspective and also because it could enable some interesting use-cases.

That's a great point, hadn't thought of that.

[muharem]

@josepot makes sense to me, thanks.

[georgepisaltu]

I would favor a 2.x approach.

I posted a more in-depth opinion in a previous comment on the topic, but to summarize:

• Extensions are best utilized when all transactions need to run that check, bonus if the extension takes its input automatically from the chain and the user doesn't have to worry about it (e.g. CheckTxVersion)
• The extension model is more complex than the extrinsic model, with more pitfalls (like the fact that a post_dispatch failure can invalidate an entire block, something that the user must take into account before deploying an extension that will run for all transactions)
• @josepot 's point about batching and tx composing is great; there's a lot more infrastructure in substrate around extrinsics than extensions
• The pallet/extrinsic approach is more familiar and easier to understand and use for users
• Not related only to this meta transaction case but generally, if the logic is constrained in a pallet extrinsic, users who don't use it don't have to understand it or interact with it, which is not true for the extension case, where the extension is always part of the pipeline and users need to be aware of how it works with various input

The main downside is that runtime devs need to hide more complexity under the pallet hood, but that's a compromise I'd like to make in order to improve UX. Speaking of UX, for a 2.2 approach we could even have runtime APIs to do the signatures for a non-standard way to do the nonce check.

I have a veeery slight preference for 2.2 (I think it's the same as what I'm saying here) because I think we don't need to bring the whole extension pipeline to make it functional. If we could get away with not using it, we wouldn't be constrained by the SignedExtension/TransactionExtension interface either, which is fairly complex.

[muharem]

It appears that we have reached a consensus to proceed with options 2.x. Allow me to summarize why I believe it is more sensible to pursue this approach, particularly option 2.2.

While the integration (code integration specifically) of option 1 still seem to me straightforward for clients who already familiar with the contract, the rollout process is considerably complex. All clients would need to upgrade to the new transaction extension and be prepared with the release. Failure to do so would result in broken transaction submissions for clients, making this solution less flexible for experiments and future updates. This, in my view, is the primary drawback of option 1.

With option 2.2, as opposed to 2.1, we can avoid the unnecessary inheritance of the transaction extension contract, which extends beyond what is required. Additionally, we can provide adapters to utilize transaction extension implementations within the new contract for meta transactions if necessary.

[muharem]

#4122

I've updated the draft version of option 2.1 and it's ready for review. (CI fails due to a style issue in the crate, unrelated to the PR; I need to wait until the base branch is updated.) Even though the CIs are red, the code compiles and tests pass.

I attempted to draft a new contract instead of using TransactionExtension and realized that I would need to copy almost everything. I believe we can use it and leverage existing types. In the kitchensink runtime example, you'll see that we use six existing types of TransactionExtension. I think that's essentially all we'll need in the first production version.

One issue I've discovered with our solution is that the chain metadata won't contain the identifiers for meta tx extensions, and clients won't be able to rely on them as they do with transaction extensions. Our metadata schema has a dedicated item for transactions, where the identifier for extensions is included. Since our meta transaction does not have such and is passed as a call's argument, its extension will be described as a regular type without its identifier, but with the full type path. I believe this is minor for now; the transaction extension set won't change frequently, and we can provide documentation for the type with the list of extensions and the correct order.