xcm-executor: take transport fee from transferred assets if necessary

[acatangiu]

Description

Sending XCM messages to other chains requires paying a "transport fee". This can be paid either:

• from origin local account if jit_withdraw = true,
• taken from Holding register otherwise.

This currently works for following hops/scenarios:

1. On destination no transport fee needed (only sending costs, not receiving),
2. Local/originating chain: just set JIT=true and fee will be paid from signed account,
3. Intermediary hops - only if intermediary is acting as reserve between two untrusted chains (aka only for DepositReserveAsset instruction) - this was fixed in xcm-executor: DepositReserveAsset charges delivery fees from inner assets #3142

But now we're seeing more complex asset transfers that are mixing reserve transfers with teleports depending on the involved chains.

Example

E.g. transferring DOT between Relay and parachain, but through AH (using AH instead of the Relay chain as parachain's DOT reserve).

In the Parachain --1--> AssetHub --2--> Relay scenario, DOT has to be reserve-withdrawn in leg 1, then teleported in leg 2.
On the intermediary hop (AssetHub), InitiateTeleport fails to send onward message because of missing transport fees. We also can't rely on jit_withdraw because the original origin is lost on the way, and even if it weren't we can't rely on the user having funded accounts on each hop along the way.

Solution/Changes

• Charge the transport fee in the executor from the transferred assets (if available),
• Only charge from transferred assets if JIT_WITHDRAW was not set,
• Only charge from transferred assets if unless using XCMv5 PayFees where we do not have this problem.

Testing

Added regression tests in emulated transfers.

Fixes #4832
Fixes #6637

[xlc]

we shouldn't touch xcm-executor without open discussions (i.e. RFC process)
we are going to remote JIT mode so we shouldn't touch it
there needs a way for user to limit max fee so we can't do this

[acatangiu]

we shouldn't touch xcm-executor without open discussions (i.e. RFC process)

I agree we can go through RFC process for this (I assumed it is ok even without RFC since it enables new scenarios without breaking any existing ones).

there needs a way for user to limit max fee so we can't do this

I don't agree, the proposed change is only used in scenarios not possible before (nobody is doing this yet) - when they'll try to do it, there are only two options (in current XCM) or three with XCMv5:

1. (without this PR) transfers will fail somewhere along the way - leaving all funds stuck/trapped in some intermediary hop
2. (with this PR) transfers work, fees (on intermediary hop only) are taken from transferred assets, there is no way to limit max fee in the XCM program, but with the dry-run API, the fee can be inspected beforehand (shown in Transfer UI) and either accepted by the user or not.
3. wait for XCMv5, use new PayFees - requires all chains involved to be upgraded to XCMv5 (so realistically another year from now 😢 )

From a practical point of view, option 2 will enable needed scenarios in the Community while still being able to control fee UX from the higher levels (UI).

[xlc]

The issue of adding new features without RFC is that we can't easily change/remove it in future without a breaking change. So we need to get it right. We have observed such case in past and I want to make sure we don't repeat the same mistake.

I get we want to get features out and don't want to wait forever (everything it is taking too long and we need to address it somehow). However, I need to point it out with the option 2, there is a non zero chance people will lose funds in corner cases. e.g. if the fee payment mechanism is powered by a swap action and there could be someone doing market manipulation to take nearly all the transferred amount as fee and steal funds. This is a security risk and much worse than just have people funds stuck in limbo, which can be rescued via governance action.

Such risk maybe tolerable (e.g. the hop/dest chain are not using swap for fee payment) but I need to make sure all the concerns are discussed and addressed.

[acatangiu]

closing this - will be cleanly supported using #5876 and #5420 in XCMv5

[acatangiu]

@xlc @seadanda @franciscoaguirre

Using existing XCMv3 and XCMv4, we currently do not support DOT (or KSM for Kusama) transfers from a Parachain to the Relay Chain while using AH as reserve. We need this PR to make it possible. See #6637

XCMv5 supports it nicely.

I want your input considering the XCMv5 timeline (released in SDK this year, integrated in Relay and Sys Chains Q1 2025), and the Minimal Relay migration.

We have three options:

1. require everyone to upgrade to XCMv5 before migration (so that they can switch to AH as DOT/KSM reserve before migration),
2. have everyone keep DOT/KSM reserve on Relay, but coordinate that they switch their reserve (chain, tooling and everything) atomically with migration,
3. merge this PR, get it integrated in fellowship runtimes and parachains can switch to AH as DOT/KSM reserve without needing to also upgrade to XCMv5.

Note: option 2 is practically impossible, bound to fail in practice, so either 1 or 3.

WDYT?

[xlc]

Upgrade to XCM v5 requires upgrade to latest polkadot-sdk, which could be a lot of work for some teams that are many versions behind. So I can feel there will be a lot of push-back from teams about option 1.
I guess option 3 it is.

[franciscoaguirre]

Looks good, I would just rename everywhere to delivery_fees since it seems to be the winning term and we shouldn't use multiple terms

[yrong]

Thanks, should work for us!

[paritytech-cmd-bot-polkadot-sdk]

Created backport PR for stable2407:

• [stable2407] Backport #4834 #6810 with remaining conflicts!

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin backport-4834-to-stable2407
git worktree add --checkout .worktree/backport-4834-to-stable2407 backport-4834-to-stable2407
cd .worktree/backport-4834-to-stable2407
git reset --hard HEAD^
git cherry-pick -x e79fd2bb9926be7d94be0c002676fca57b6116bf
git push --force-with-lease

[paritytech-cmd-bot-polkadot-sdk]

Created backport PR for stable2409:

• [stable2409] Backport #4834 #6811 with remaining conflicts!

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin backport-4834-to-stable2409
git worktree add --checkout .worktree/backport-4834-to-stable2409 backport-4834-to-stable2409
cd .worktree/backport-4834-to-stable2409
git reset --hard HEAD^
git cherry-pick -x e79fd2bb9926be7d94be0c002676fca57b6116bf
git push --force-with-lease

[paritytech-cmd-bot-polkadot-sdk]

Successfully created backport PR for stable2412:

• [stable2412] Backport #4834 #6812