Isolate onchain data sources from accessing and modifying offchain entities

[leoyvens]

The current implementation prevents offchain data sources prevents them from accessing the store at all. However onchain data sources are still able to read and even update entities created by offchain data sources.

The fix is to properly implement causality region isolation. Entity tables need a column that tracks their causality region. Also the in-memory representation of data sources needs to be aware of its own causality region. See #3072 for background on the design of causality region isolation.

[leoyvens]

Problem statement

For indexing results to be reproducible given a set of available files, regardless of the order in which the files are found, it is necessary to isolate the effects of processing offchain data sources. An onchain data source cannot read or update entities created by an offchain data source, and an offchain data source cannot read or update an entity created by another offchain data source.

Causality regions help us, because by assigning all onchain data sources the same causality region and each offchain data source their own causality region, we can restate the goal as: A data source cannot read or update entities from other causality regions.

The isolation rule can be further refined as read restriction and a write restriction.

• Read restriction: An attempted read from a different causality region returns null.
• Write restriction: An an attempted update or overwrite to an entity of a different causality region is a conflict which must raise an error.

This issue is about implementing both restrictions.

Implementation plan

• 1. Enforce the entities manifest field is enforced on file data sources (fix(offchain): Enforce entities list on offchain data sources #4147)
  This way we know which entity types actually need causality region tracking, which reduces the performance costs of checking causality regions, on the principle of "you don't pay for what you don't use".
• 2. Add a causality region column to entity tables which are used by file data sources (Add causality region column and implement isolation rules #4162)
• 3. Write the correct causality region to entities written to those tables. See FIXME(#3770) on the code left by Add causality region column and implement isolation rules #4162.
• 4. Implement the read restriction.
• 5. Implement the write restriction, the simplest way to do this is to always create the exclusion constraint for tables which have a causality region column.
• 6. End-to-end tests for all the permutations of reading and updating from onchain/offchain and offchain/offchain.

The details on 3 and 4 are not clear yet, what would the best way be to get the causality region value to the store? Maybe add it to the EntityKey?

[leoyvens]

Done.