Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

enable firehose eth block ingestor #4204

Merged
merged 1 commit into from
Nov 30, 2022
Merged

enable firehose eth block ingestor #4204

merged 1 commit into from
Nov 30, 2022

Conversation

mangas
Copy link
Contributor

@mangas mangas commented Nov 25, 2022 •
edited
Loading

Fixes #4059

@mangas mangas force-pushed the filipe/firehose-block-ingestor branch from 19292ba to 45c5c94 Compare November 25, 2022 11:35
@mangas mangas marked this pull request as ready for review November 25, 2022 11:35
@mangas mangas force-pushed the filipe/firehose-block-ingestor branch 2 times, most recently from 1b64757 to 957aaae Compare November 25, 2022 11:38
maoueh
maoueh reviewed Nov 25, 2022
View reviewed changes
node/src/main.rs Outdated Show resolved Hide resolved
@mangas mangas force-pushed the filipe/firehose-block-ingestor branch 2 times, most recently from 2a7d69d to bf2f3a5 Compare November 28, 2022 09:49
@mangas mangas requested a review from leoyvens November 28, 2022 18:20
@maoueh
Copy link
Contributor

maoueh commented Nov 28, 2022

Also, I don't see in the PR where we handle the current EthereumBlockIngestor upsert full block block logic. Is it scheduled for another follow-up PR?

@maoueh
Copy link
Contributor

maoueh commented Nov 29, 2022

For now, no receipts will be written when using a Firehose Block Ingestor on Ethereum, it's not really required anyway since Firehose is able to stream that information directly, so there is no point in saving that information right now.

Only draw back is that switching back to RPC provider is not possible without manual intervetion on the database by removing the accumulated blocks so far before switching back to RPC.

maoueh
maoueh reviewed Nov 29, 2022
View reviewed changes
node/src/main.rs Outdated Show resolved Hide resolved
@mangas mangas force-pushed the filipe/firehose-block-ingestor branch 2 times, most recently from 57158f7 to 2b552ee Compare November 30, 2022 09:41
@leoyvens
Copy link
Collaborator

Code looks safe to merge. But I'll let @maoueh, who did the in-depth review, give the green check.

@mangas mangas force-pushed the filipe/firehose-block-ingestor branch from efaf8cf to 0e09870 Compare November 30, 2022 13:53
@mangas mangas force-pushed the filipe/firehose-block-ingestor branch from 0e09870 to c0cd2f8 Compare November 30, 2022 14:14
@mangas mangas merged commit be4782f into master Nov 30, 2022
@mangas mangas deleted the filipe/firehose-block-ingestor branch November 30, 2022 15:35
@mangas mangas mentioned this pull request Nov 30, 2022
Merged
vutran1710 added a commit to KyberNetwork/graph-node that referenced this pull request Feb 1, 2023
@vutran1710 @dependabot
@neysofu @lutter @saihaj @azf20 @tilacog @mangas @evaporei @leoyvens @maoueh @islishude @dotansimha @mudabbirk @Theodus @rattrap @jeffywu @sduchesneau
Sync fork (#37)
f3ecf35 
* build(deps): bump blake3 from 1.3.1 to 1.3.2 (graphprotocol#4194)

Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3) from 1.3.1 to 1.3.2.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases)
- [Commits](BLAKE3-team/BLAKE3@1.3.1...1.3.2)

---
updated-dependencies:
- dependency-name: blake3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump cid from 0.8.6 to 0.9.0 (graphprotocol#4193)

Bumps [cid](https://github.com/multiformats/rust-cid) from 0.8.6 to 0.9.0.
- [Release notes](https://github.com/multiformats/rust-cid/releases)
- [Commits](multiformats/rust-cid@v0.8.6...v0.9.0)

---
updated-dependencies:
- dependency-name: cid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump indexmap from 1.9.1 to 1.9.2 (graphprotocol#4192)

Bumps [indexmap](https://github.com/bluss/indexmap) from 1.9.1 to 1.9.2.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Changelog](https://github.com/bluss/indexmap/blob/master/RELEASES.md)
- [Commits](indexmap-rs/indexmap@1.9.1...1.9.2)

---
updated-dependencies:
- dependency-name: indexmap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* all: fix some Clippy warnings (graphprotocol#4178)

* node, store: Add `graphman stats target`

* store: Make resolve_column_names return SqlName

* node, store: Allow analyzing all tables with `graphman analyze`

* node, store: Add `graphman stats set-target`

* store/postgres: insist on a database with C locale

* store: remove unused lazy_static! (graphprotocol#4200)

* Update docker-compose.yml

* Update docker-compose.yml

* Fix: run locale check after migrations (graphprotocol#4201)

* store/postgres: run locale check after migrations

* store/postgres: place succes log message after locale check

* Update docker-compose.yml

* Update docker-compose.yml

* ci: fix Postgres' locale in CI

* integration-tests: createdb with C locale

Co-authored-by: tilacog <[email protected]>
Co-authored-by: Adam Fuller <[email protected]>
Co-authored-by: Filippo Costa <[email protected]>
Co-authored-by: Filippo Costa <[email protected]>

* update locale for parallel tests (graphprotocol#4205)

* store: Unfail subgraph status properly after rewind

This is being done by changing `revert_subgraph_errors` to change both
the `failed` and `status` columns in the case of both being `failed`.

Without this change, rewinding failed subgraphs past the failed
block would result in the `failed` status staying the same.

* tests: Test that rewind resets subgraph health

Ports the `fatal-error` test from the legacy test suite to the new test suite,
and extends it to check that the subgraph health is reset to `Healthy` after a rewind.

* enable firehose eth block ingestor (graphprotocol#4204)

* firehose ingestor transforms (graphprotocol#4216)

* Updated `Substreams` to latest version of Protobuf definition and activated `ProductionMode` by default (graphprotocol#4211)

The production mode is required to benefits from automatic backprocessing and downloading of block scoped data message as they are produced. This will drastically improve the ingestion speed of a substreams (time to gather some metrics!).

Updated also the instructions to re-generate the `substreams.proto` file with more instructions of how we do it.

Co-authored-by: Matthieu Vachon <[email protected]>

* Remove unused deps from `graph-server-metrics` (graphprotocol#4224)

* server-metrics: remove unused deps

* metrics-server: remove MetricsServer trait

* build(deps): bump openssl from 0.10.42 to 0.10.43 (graphprotocol#4208)

Bumps [openssl](https://github.com/sfackler/rust-openssl) from 0.10.42 to 0.10.43.
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](sfackler/rust-openssl@openssl-v0.10.42...openssl-v0.10.43)

---
updated-dependencies:
- dependency-name: openssl
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* The `SubstreamsBlockStream` was using `request.clone()` side stepping latest cursor value (graphprotocol#4228)

The `request.clone()` does not correctly use the `latest_cursor` value which is the valid up to date in memory cursor to use on re-connection. This led to poisining error in `graph-node` where the same block was processed multiple time because the cursor was not correctly used.

Fixed by moving the request creation directly where it's needed which will use the correct up to date `latest_cursor` value now.

Co-authored-by: Matthieu Vachon <[email protected]>

* fail when ens rainbow not present (graphprotocol#4219)

* docker: use exec form for CMD directive (graphprotocol#4217)

* graph, graphql: expose metrics for GraphQL validations failures (graphprotocol#4230)

* `EmptyNodeCapabilities` for chains that don't discriminate nodes (graphprotocol#3985)

* chain(all), graph: EmptyNodeCapabilities

* chain-ethereum: fix PartialOrd for NodeCapabilities

The Ord and PartialOrd implementations of NodeCapabilities currently
break trait rules: `NodeCapabilities {true, false}` is both greater
than and less than (at the same time) `NodeCapabilities {false, true}`.
It'd be nice to get this fixed before we get some obscure bug during
comparisons, or a panic due to std assuming transitivity during
sorts.

* build(deps): bump express in /tests/integration-tests (graphprotocol#4235)

Bumps [express](https://github.com/expressjs/express) from 4.17.1 to 4.18.2.
- [Release notes](https://github.com/expressjs/express/releases)
- [Changelog](https://github.com/expressjs/express/blob/master/History.md)
- [Commits](expressjs/express@4.17.1...4.18.2)

---
updated-dependencies:
- dependency-name: express
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build: remove cargo-chef, revert to stable rust (graphprotocol#4187)

* build(deps): bump base64 from 0.13.1 to 0.20.0 (graphprotocol#4241)

Bumps [base64](https://github.com/marshallpierce/rust-base64) from 0.13.1 to 0.20.0.
- [Release notes](https://github.com/marshallpierce/rust-base64/releases)
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md)
- [Commits](marshallpierce/rust-base64@v0.13.1...v0.20.0)

---
updated-dependencies:
- dependency-name: base64
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* graph, node: Move regex into the graph crate

* node, store: Parse index definitions

* node, store: Allow suppressing standard indexes in 'graphman index list'

* graph, node: Colorize the output of 'graphman index list'

* node, store: Use index::Method when creating manual indexes

* node, store: Add SQL printing to `graphman index list`

* node: Fix output from `graphman index list`

* node, store: Parse the index method using FromStr

* graphql: Simplify the arguments for prefetch::fetch

* store: Simplify arguments for Layout.query

* graph, graphql, store: Pass trace flag for query

* all: Allow tracing GraphQL queries via HTTP

* graph, graphql: Include query_id and variables in trace

* graph, node: Print execution errors from `graphman query`

* graph, graphql: Include block number in query trace

* build(deps): bump openssl from 0.10.43 to 0.10.45 (graphprotocol#4267)

Bumps [openssl](https://github.com/sfackler/rust-openssl) from 0.10.43 to 0.10.45.
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](sfackler/rust-openssl@openssl-v0.10.43...openssl-v0.10.45)

---
updated-dependencies:
- dependency-name: openssl
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump proc-macro2 from 1.0.47 to 1.0.49 (graphprotocol#4259)

Bumps [proc-macro2](https://github.com/dtolnay/proc-macro2) from 1.0.47 to 1.0.49.
- [Release notes](https://github.com/dtolnay/proc-macro2/releases)
- [Commits](dtolnay/proc-macro2@1.0.47...1.0.49)

---
updated-dependencies:
- dependency-name: proc-macro2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump quote from 1.0.20 to 1.0.23 (graphprotocol#4258)

Bumps [quote](https://github.com/dtolnay/quote) from 1.0.20 to 1.0.23.
- [Release notes](https://github.com/dtolnay/quote/releases)
- [Commits](dtolnay/quote@1.0.20...1.0.23)

---
updated-dependencies:
- dependency-name: quote
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump anyhow from 1.0.66 to 1.0.68 (graphprotocol#4257)

Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.66 to 1.0.68.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](dtolnay/anyhow@1.0.66...1.0.68)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump semver from 1.0.14 to 1.0.16 (graphprotocol#4256)

Bumps [semver](https://github.com/dtolnay/semver) from 1.0.14 to 1.0.16.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](dtolnay/semver@1.0.14...1.0.16)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump num_cpus from 1.14.0 to 1.15.0 (graphprotocol#4265)

Bumps [num_cpus](https://github.com/seanmonstar/num_cpus) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/seanmonstar/num_cpus/releases)
- [Changelog](https://github.com/seanmonstar/num_cpus/blob/master/CHANGELOG.md)
- [Commits](seanmonstar/num_cpus@v1.14.0...v1.15.0)

---
updated-dependencies:
- dependency-name: num_cpus
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump termcolor from 1.1.2 to 1.1.3 (graphprotocol#4264)

Bumps [termcolor](https://github.com/BurntSushi/termcolor) from 1.1.2 to 1.1.3.
- [Release notes](https://github.com/BurntSushi/termcolor/releases)
- [Commits](BurntSushi/termcolor@1.1.2...1.1.3)

---
updated-dependencies:
- dependency-name: termcolor
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Updated link to graph-cli networks list link (graphprotocol#4246)

* Update protobuf-related dependencies (graphprotocol#4272)

* ci: run code coverage stats every 3 days (graphprotocol#4188)

* fix cloud build (graphprotocol#4279)

* Fix attestability of ValueParseError, EnumCoercionError, ScalarCoercionError (graphprotocol#4278)

* Fix attestability of ValueParseError

* Fix attestability of EnumCoercionError, ScalarCoercionError

* build(deps): bump git-testament from 0.2.0 to 0.2.2 (graphprotocol#4275)

Bumps [git-testament](https://github.com/kinnison/git-testament) from 0.2.0 to 0.2.2.
- [Release notes](https://github.com/kinnison/git-testament/releases)
- [Commits](kinnison/git-testament@0.2.0...0.2.2)

---
updated-dependencies:
- dependency-name: git-testament
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Update ipfs api (graphprotocol#4283)

* build(deps): bump blake3 from 1.3.2 to 1.3.3 (graphprotocol#4209)

Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3) from 1.3.2 to 1.3.3.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases)
- [Commits](BLAKE3-team/BLAKE3@1.3.2...1.3.3)

---
updated-dependencies:
- dependency-name: blake3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: Include graft ipfs lookup for raw_yaml in infinite retries (graphprotocol#4284)

* substreams: Fix spurious builds (graphprotocol#4286)

* graphql: Allow gt/lt comparisons with `Bytes` fields

Fixes graphprotocol#4282

* graph, store: Avoid a bind variable for fulltext queries

We control the string we pass as the language, and therefore do not need to
use a bind variable. We just need to make sure we are passing a valid SQL
string constant

* store: Properly account for fulltext columns in inserts

Fixes graphprotocol#2330

* docker: adds support for ipfs auth urls (graphprotocol#4252)

* `v0.29.0` release notes, `NEWS.md` and Cargo package version (graphprotocol#4298)

* Release v0.29.0

* news: release v0.29.0

* build(deps): bump proc-macro2 from 1.0.49 to 1.0.50 (graphprotocol#4295)

Bumps [proc-macro2](https://github.com/dtolnay/proc-macro2) from 1.0.49 to 1.0.50.
- [Release notes](https://github.com/dtolnay/proc-macro2/releases)
- [Commits](dtolnay/proc-macro2@1.0.49...1.0.50)

---
updated-dependencies:
- dependency-name: proc-macro2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump cid from 0.9.0 to 0.10.1 (graphprotocol#4289)

Bumps [cid](https://github.com/multiformats/rust-cid) from 0.9.0 to 0.10.1.
- [Release notes](https://github.com/multiformats/rust-cid/releases)
- [Changelog](https://github.com/multiformats/rust-cid/blob/master/CHANGELOG.md)
- [Commits](multiformats/rust-cid@v0.9.0...v0.10.1)

---
updated-dependencies:
- dependency-name: cid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Cache recent blocks in memory to reduce load on the db (graphprotocol#4215)

* store: cache recent blocks in memory

* store: refactor RecentBlocksCache's Inner methods

* store: fix ancestor/child naming in RecentBlocksCache

* store: use a BTreeMap for RecentCachedBlocks

This commit significatly alters the design of RecentCachedBlocks. The
most prominent changes are:

1. We don't require a `.set_chain_head` call anymore. Block insertion
   and chain head update attempt are now the same thing.
2. We don't evict all items in the cache anymore every time the chain
   head advances.
3. Unlike the previous data structure, we are now limited to storing a
   contiguous range of blocks in the cache. This is not really a
   drawback (as the cache contents will usually be identical, i.e. the
   last N blocks before the chain head), but it's worth pointing out.

* rust: upgrade to 1.66

* store: readability improvements to RecentBlocksCache

* store: fix RecentBlocksCache for NEAR

This commit removes the requirement of continuous block number ranges
for RecentBlocksCache. Two reasons for that:

- NEAR doesn't have continuous block number ranges, so the previous
  cache design would have limited effectiveness for NEAR chains.
- Parent hash comparisons are actually enough to uphold all invariants,
  so block number checks were unnecessary in the first place.

The actual code changes are small and mostly limited to `insert_block`.

* build(deps): bump tower-test from `c9d84cd` to `b01bb12` (graphprotocol#4296)

Bumps [tower-test](https://github.com/tower-rs/tower) from `c9d84cd` to `b01bb12`.
- [Release notes](https://github.com/tower-rs/tower/releases)
- [Commits](tower-rs/tower@c9d84cd...b01bb12)

---
updated-dependencies:
- dependency-name: tower-test
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump prost from 0.11.5 to 0.11.6 (graphprotocol#4297)

Bumps [prost](https://github.com/tokio-rs/prost) from 0.11.5 to 0.11.6.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](tokio-rs/prost@v0.11.5...v0.11.6)

---
updated-dependencies:
- dependency-name: prost
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump termcolor from 1.1.3 to 1.2.0 (graphprotocol#4294)

Bumps [termcolor](https://github.com/BurntSushi/termcolor) from 1.1.3 to 1.2.0.
- [Release notes](https://github.com/BurntSushi/termcolor/releases)
- [Commits](BurntSushi/termcolor@1.1.3...1.2.0)

---
updated-dependencies:
- dependency-name: termcolor
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump atomic_refcell from 0.1.8 to 0.1.9 (graphprotocol#4290)

Bumps [atomic_refcell](https://github.com/bholley/atomic_refcell) from 0.1.8 to 0.1.9.
- [Release notes](https://github.com/bholley/atomic_refcell/releases)
- [Commits](https://github.com/bholley/atomic_refcell/commits)

---
updated-dependencies:
- dependency-name: atomic_refcell
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump bumpalo from 3.7.0 to 3.12.0 (graphprotocol#4306)

* Env. vars. for forking (graphprotocol#4308)

* adding env params for forking opts

* adding documentation

Co-authored-by: Jeff Wu <[email protected]>

* build(deps): bump prost-types from 0.11.5 to 0.11.6 (graphprotocol#4312)

* build(deps): bump git-testament from 0.2.2 to 0.2.4 (graphprotocol#4313)

* Add causality region column and implement isolation rules (graphprotocol#4162)

* feature(offchain): Add `causality_region` column to entity tables

For now this just tracks the tables that need the column and adds the
column to the DDL, but still unconditionally inserts 0. Inserting the
correct causality region is follow up work.

* store: Move `has_causality_region` to manifest, rename to `entities_with_causality_region`

* *: Add `causality_region` to EntityKey

The tricky part was changing `get_many` to return the entity key.

* store: Insert the causality region

* store: Read isolation between causality regions

It was just necessary to make sure that `find` and `find_many` use
the causality region in their where clause.

* fix: Fix release build

* provider: Make stop idempotent

Callers wanted that anyways, and it helps tests.

* tests: Refactor file ds test to use events

* tests: Test conflict between onchain and offchain

* tests: Test conflict between offchain and offchain

* test: Fix unit test

* tests: Improve tests and comments to address review

* fix: Change migration to add column 'if not exists'

* store: Add a materialized view 'info.chain_sizes'

* config: Bump default ipfs timeout to 60 seconds (graphprotocol#4324)

It used to be that 30 seconds was sufficient, but now we see ipfs
requests taking minutes to find a file in the DHT. So 60 seconds
seems like a reasonable step.

* strip all null bytes from utf8 strings that come from substreams, like it's done on other sources (graphprotocol#4328)

* fix test

* add protobuf

* fix database setup for unit test on kyber-ci

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Filippo Costa <[email protected]>
Co-authored-by: David Lutterkort <[email protected]>
Co-authored-by: Saihajpreet Singh <[email protected]>
Co-authored-by: Adam Fuller <[email protected]>
Co-authored-by: Tiago Guimarães <[email protected]>
Co-authored-by: Filippo Costa <[email protected]>
Co-authored-by: Filippo Costa <[email protected]>
Co-authored-by: Filipe Azevedo <[email protected]>
Co-authored-by: Eva Pace <[email protected]>
Co-authored-by: Leonardo Yvens <[email protected]>
Co-authored-by: Matthieu Vachon <[email protected]>
Co-authored-by: Matthieu Vachon <[email protected]>
Co-authored-by: Matthieu Vachon <[email protected]>
Co-authored-by: Shude Li <[email protected]>
Co-authored-by: Dotan Simha <[email protected]>
Co-authored-by: mudabbirk <[email protected]>
Co-authored-by: Theo Butler <[email protected]>
Co-authored-by: Leo <[email protected]>
Co-authored-by: Jeff Wu <[email protected]>
Co-authored-by: Stéphane Duchesneau <[email protected]>
Co-authored-by: Vu Tran <[email protected]>
Co-authored-by: Vu Tran <[email protected]>
@paymog
Copy link

paymog commented Feb 15, 2023

For now, no receipts will be written when using a Firehose Block Ingestor on Ethereum, it's not really required anyway since Firehose is able to stream that information directly, so there is no point in saving that information right now.

Only draw back is that switching back to RPC provider is not possible without manual intervetion on the database by removing the accumulated blocks so far before switching back to RPC.

would it be good to add a graphman command to make it easy to roll back to rpc only providers? is it the chainX.blocks table that needs to be wiped during a rollback?

@paymog paymog mentioned this pull request Feb 15, 2023
Merged
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@maoueh maoueh maoueh approved these changes

@leoyvens leoyvens Awaiting requested review from leoyvens

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add a Firehose Block Ingestor for Ethereum
4 participants
@mangas @maoueh @leoyvens @paymog