Add out-of-gas recovery logic to Wasm Hooks

[nicolaslara]

Background

Relayers batch IBC messages in an SDK multi-message. If one of these messages contains a wasm-hook execution (or lifecycle callback) it will execute a contract. Executing a contract means the expected gas of an IBC message execution is no longer "constant", but may vary depending on the contract's implementation. As a consequence, calling a contract with excesive gas consumption may fail the whole TX batch.

This can lead to a temporary DoS for message transfers and puts the onus on the relayers to manage it. We should ensure that either:

• There is a hard-cap on the gas that a wasm-hooks contract can use
• Messages fail independently (feat: nonatomic multimsg cosmos/cosmos-sdk#13911)
• Gas can be adequately predicted

or a combination of the above.

Suggested Design

There are still open design questions here. My opinion is that improving multi-message and ensuring that relayers can deal with rogue packets is the right solution here,

Acceptance Criteria

• Only offending transactions should fail and others shouldn't be affected
• There is no expense risk for relayers on high gas

[nicolaslara]

@ValarDragon Added this ticket to track the out-of-gas issue. Can you add your notes from the call here when you get a chance?

[nicolaslara]

This is an example from the ibc codebase on how to use a custom gas meter for the contract calls: https://github.com/cosmos/ibc-go/blob/main/modules/core/keeper/msg_server.go#L451