Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(runner): Enable starting testnet follower from state-sync #105

Merged
merged 3 commits into from
Mar 15, 2023
Merged

feat(runner): Enable starting testnet follower from state-sync #105

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
39be721
refactor(runner): reorder testnet chain setup
mhofman Mar 14, 2023
ff4ecc3
feat(runner): add --use-state-sync testnet option
mhofman Mar 14, 2023
8665c4e
fixup! feat(runner): add --use-state-sync testnet option
mhofman Mar 15, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions runner/lib/main.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,9 @@ const main = async (progName, rawArgs, powers) => {
/** @type {string} */
let testnetOrigin;

/** @type {boolean | undefined} */
let useStateSync;

const profile = argv.profile == null ? 'local' : argv.profile;

switch (profile) {
Expand All @@ -291,6 +294,7 @@ const main = async (progName, rawArgs, powers) => {
case 'stage':
makeTasks = makeTestnetTasks;
testnetOrigin = argv.testnetOrigin || `https://${profile}.agoric.net`;
useStateSync = argv.useStateSync;
break;
default:
throw new Error(`Unexpected profile option: ${profile}`);
Expand Down Expand Up @@ -382,6 +386,7 @@ const main = async (progName, rawArgs, powers) => {
metadata: {
profile,
testnetOrigin,
useStateSync,
...envInfo,
testData: argv.testData,
},
Expand Down Expand Up @@ -961,6 +966,7 @@ const main = async (progName, rawArgs, powers) => {
chainOnly: globalChainOnly,
withMonitor,
testnetOrigin,
useStateSync,
};
logPerfEvent('setup-tasks-start', setupConfig);
({ chainStorageLocation, clientStorageLocation } =
Expand Down
1 change: 1 addition & 0 deletions runner/lib/stats/types.d.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,7 @@ export interface StageStats extends StageStatsInitData {
export interface RunMetadata {
readonly profile: string;
readonly testnetOrigin?: string;
readonly useStateSync?: boolean | undefined;
readonly agChainCosmosVersion?: unknown;
readonly testData?: unknown;
}
Expand Down
58 changes: 49 additions & 9 deletions runner/lib/tasks/testnet.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ export const makeTasks = ({
/** @type {Record<string, string>} */
const additionChainEnv = {};

/** @param {import("./types.js").TaskBaseOptions & {config?: {reset?: boolean, chainOnly?: boolean, withMonitor?: boolean, testnetOrigin?: string}}} options */
/** @param {import("./types.js").TaskBaseOptions & {config?: {reset?: boolean, chainOnly?: boolean, withMonitor?: boolean, testnetOrigin?: string, useStateSync?: boolean}}} options */
const setupTasks = async ({
stdout,
stderr,
Expand All @@ -135,6 +135,7 @@ export const makeTasks = ({
chainOnly,
withMonitor = true,
testnetOrigin: testnetOriginOption,
useStateSync,
} = {},
}) => {
const { console, stdio } = getConsoleAndStdio(
Expand Down Expand Up @@ -182,9 +183,6 @@ export const makeTasks = ({
const genesisPath = joinPath(chainStateDir, 'config', 'genesis.json');

if (!(await fsExists(genesisPath))) {
console.log('Fetching genesis');
const genesis = await fetchAsJSON(`${testnetOrigin}/genesis.json`);

await childProcessDone(
printerSpawn(
sdkBinaries.cosmosChain,
Expand All @@ -193,11 +191,6 @@ export const makeTasks = ({
),
);

fs.writeFile(
joinPath(chainStateDir, 'config', 'genesis.json'),
JSON.stringify(genesis),
);

await childProcessDone(
printerSpawn(
sdkBinaries.cosmosChain,
Expand Down Expand Up @@ -227,6 +220,53 @@ export const makeTasks = ({
configP2p.seeds = seeds.join(',');
configP2p.addr_book_strict = false;
delete config.log_level;

if (!useStateSync) {
console.log('Fetching genesis');
const genesis = await fetchAsJSON(`${testnetOrigin}/genesis.json`);

fs.writeFile(
joinPath(chainStateDir, 'config', 'genesis.json'),
JSON.stringify(genesis),
);
} else {
console.log('Fetching state-sync info');
/** @type {any} */
const currentBlockInfo = await fetchAsJSON(
`http://${rpcAddrs[0]}/block`,
);

// `trustHeight` is the block height considered as the "root of trust"
// for state-sync. The node will attempt to find a snapshot offered for
// a block at or after this height, and will validate that block's hash
// using a light client with the configured RPC servers.
// We want to use a block height recent enough, but for which a snapshot
// exists since then.
const stateSyncInterval =
Number(process.env.AG_SETUP_COSMOS_STATE_SYNC_INTERVAL) || 2000;
const trustHeight = Math.max(
1,
Number(currentBlockInfo.result.block.header.height) -
stateSyncInterval,
Comment on lines +249 to +250
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this calculation work even if the height is much greater than the stateSyncInterval, and the interval is a fairly large number? I don't know the semantics of what you're trying to do here; it would be worth a comment.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pretty standard state-sync, which requires a "root of trust". This is basically configuring state-sync saying we'll trust any block after the given height (and corresponding hash). The goal is to give something recent enough so that it doesn't need to verify too many blocks hashes with the light client, but old enough that there exists a state-sync snapshot taken after that block height.

Most state-sync instructions/scripts out there basically do this "latest block height - some offset" math without giving much explanation as to why. This is what I've gathered from understanding how state-sync works.

I'll add a comment adding an explanation.

);

/** @type {any} */
const trustedBlockInfo = await fetchAsJSON(
`http://${rpcAddrs[0]}/block?height=${trustHeight}`,
);
const trustHash = trustedBlockInfo.result.block_id.hash;

const configStatesync = /** @type {import('@iarna/toml').JsonMap} */ (
config.statesync
);
configStatesync.enable = true;
configStatesync.rpc_servers = rpcAddrs
.map((host) => `http://${host}`)
.join(',');
configStatesync.trust_height = trustHeight;
configStatesync.trust_hash = trustHash;
}

await fs.writeFile(configPath, TOML.stringify(config));
}

Expand Down