Fixed GrandpaConsensusLogReader::find_scheduled_change

[svyatonik]

The issue is here:

		// find the first consensus digest with the right ID which converts to
		// the right kind of consensus log.
		digest
			.convert_first(|log| log.consensus_try_to(&GRANDPA_ENGINE_ID))
			.and_then(|log| match log {
				ConsensusLog::ScheduledChange(change) => Some(change),
				_ => None,
			})

Correct code should be like that (see https://github.com/paritytech/parity-bridges-common/blob/ff8840b1ad043c5ae90a527e46c0dbd217e1b436/primitives/header-chain/src/lib.rs#L84-L93):

	let id = OpaqueDigestItemId::Consensus(&GRANDPA_ENGINE_ID);

	let filter_log = |log: ConsensusLog<H::Number>| match log {
		ConsensusLog::ScheduledChange(change) => Some(change),
		_ => None,
	};

	// find the first consensus digest with the right ID which converts to
	// the right kind of consensus log.
	header.digest().convert_first(|l| l.try_to(id).and_then(filter_log))

As you see, the incorrect code gets first GRANDPA digest item and tries to parse it as ScheduledChange.
The correct code tries to find first GRANDPA digest item which is ScheduledChange.

The issue is that on Kusama there are mandatory headers with digest: [Consensus(GRANDPA_ENGINE_ID, ConsensusLog::OnDisabled(_)), Consensus(GRANDPA_ENGINE_ID, ConsensusLog::ScheduledChange(_))]. So the incorrect code finds first item (OnDisabled) and ignores the actual ScheduledChange. IIUC this OnDisabled happens when validator is slashed (so staking involved), so we haven't seen it on Rialto/Millau/Rococo/Westend.

What happens:

1. relayer: scans Kusama headers for those with ScheduledChange, but they are not detected => it skips such headers and proceeds to next mandatiry. Next mandatory is signed by different authority set => it fails to optimize justification (because it removes invalid signatures and all signatures are invalid here)
2. what I've tried: I've fixed the relayer code and submitted such header to pallet_bridge_grandpa. I thought it should have solved our issue, but...
3. the on-chain code: has received that header and applied incorrect code => so it has not found the authorities set change => the set has stayed the same. So now it does not accept next mandatory header, because it is signed by the different authorities set.

The only possible fix is to upgrade bridge hub runtime. We can't do set_storage to fix teh authorities set, because SafeCallFilter supports limited number of storage entries that can be modified this way.

[Polkadot-Forum]

This pull request has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/polkadot-kusama-bridge/2971/45

[acatangiu]

This needed a prdoc instead of "silent"

[svyatonik]

This needed a prdoc instead of "silent"

I've added it for 4209, but it is in a release branch, so copied here: #4239

[kianenigma]

so we haven't seen it on Rialto/Millau/Rococo/Westend.

We can use the pallet-root-offences that have been built for more or less this purpose to make sure on all testnets we have regularly slash a random set of validators. Even if not with this pallet, I think, we can easily scarp together the code to do this.