Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Frequent query failures on Decentralized Network #3839

Closed
lowjiansheng opened this issue Aug 15, 2022 · 4 comments
Closed

Frequent query failures on Decentralized Network #3839

lowjiansheng opened this issue Aug 15, 2022 · 4 comments
Labels
Stale

Comments

@lowjiansheng
Copy link

lowjiansheng commented Aug 15, 2022 •
edited
Loading

Do you want to request a feature or report a bug? BUG

What is the current behavior?
Subgraph ID: 6tGbL7WBx287EZwGUvvcQdL6m67JGMJrma3JSTtt5SV7
When querying a particular subgraph, we're encountering occasional server error messages. It does not happen all the time, only on occasional queries.

Error message:

returned error 502 Bad Gateway: \n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>502 Server Error</title>\n<script src=\"/cdn-cgi/apps/head/a-7iaLT0SRdb_kvxGKIYLrjkGzU.js\"></script></head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>\n<h2></h2>\n<script defer src=\"https://static.cloudflareinsights.com/beacon.min.js/v652eace1692a40cfa3763df669d7439c1639079717194\" integrity=\"sha512-Gi7xpJR8tSkrpF7aordPZQlW2DLtzUlZcumS8dMQjwDHEnw9I7ZLyiOj/6tZStRBGtGgN6ceN6cMH8z7etPGlw==\" data-cf-beacon='{\"rayId\":\"(censored)\",\"token\":\"(censored)\",\"version\":\"2022.6.0\",\"si\":100}' crossorigin=\"anonymous\"></script>\n</body></html>\n

Querying on Postman and Playgrounds encounter the same errors, albeit a slightly different error message.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem.
Postman query: https://thegraph.com/explorer/subgraph?id=6tGbL7WBx287EZwGUvvcQdL6m67JGMJrma3JSTtt5SV7&view=Playground

image

https://thegraph.com/explorer/subgraph?id=6tGbL7WBx287EZwGUvvcQdL6m67JGMJrma3JSTtt5SV7&view=Playground
I'm only seeing this error when querying this particular subgraph. I've yet to see this error on other subgraphs that we’ve deployed to the decentralized network. I’ve tried to make the same query on the playground and am getting a Failed to fetch error. Occasionally, I’m getting a “No indexer available” error too.

I've lowered the batch size (list of MarketIDs passed in as parameters), but the errors still occur occasionally. I can't conclusively say if lowering the batch size reduces the frequency of errors.

Example query body:

{
  markets(subgraphError: allow, orderBy:id, orderDirection: asc, where:{id_in:["0x158079ee67fce2f58472a96584a73c7ab9ac95c1", "0x39aa39c021dfbae8fac545936693ac917d5e7563", "0x80a2ae356fc9ef4305676f7a3e2ed04e12c33946", "0x95b4ef2869ebd94beb4eee400a99824bf5dc325b", "0xb3319f5d18bc0d84dd1b4825dcde5d5f7266d407", "0x4b0181102a0112a2ef11abee5563bb4a3176c9d7", "0x35a18000230da775cac24873d00ff85bccded550", "0x7713dd9ca933848f6819f38b8352d9a15ea73f67", "0x041171993284df560249b57358f931d9eb7b925d", "0x12392f67bdf24fae0af363c24ac620a2f67dad86", "0x4ddc2d193948926d02f9b1fe9e1daa0718270ed5", "0x5d3a536e4d6dbd6114cc1ead35777bab948e3643", "0x6c8c6b02e7b2be14d4fa6022dfd6d75921d90e4e", "0x70e36f6bf80a52b3b46b3af8e106cc0ed743e8e4", "0xc11b1268c1a384e55c48c2391d8d480264a3a7f4", "0xccf4429db6322d5c611ee964527d42e5d685dd6a", "0xe65cdb6479bac1e22340e4e755fae7e509ecd06c", "0xf5dce57282a584d2746faf1593d3121fcac444dc", "0xf650c3d88d12db855b8bf7d11be6c55a4e07dcc9", "0xface851a4921ce59e912d19329929ce6da6eb0c7"]}) {
    id
    dailySnapshots(first: 1, orderBy: timestamp, orderDirection: desc, where:{timestamp_lt: 1656720000}) {
        id
        market {
          id
           inputToken {
            id
            decimals
          }
          outputToken {
            id
            decimals
          }
        }
        timestamp
        rates {
          id
          rate
          type
          side
        }
        totalValueLockedUSD
        totalDepositBalanceUSD
        totalBorrowBalanceUSD
        dailyBorrowUSD
        dailyDepositUSD
        dailyLiquidateUSD
        dailyRepayUSD
        dailyWithdrawUSD
        cumulativeSupplySideRevenueUSD
        dailySupplySideRevenueUSD
        cumulativeProtocolSideRevenueUSD
        dailyProtocolSideRevenueUSD
        cumulativeTotalRevenueUSD
        dailyTotalRevenueUSD
        inputTokenBalance
        inputTokenPriceUSD
        outputTokenSupply
        outputTokenPriceUSD
      }
  }
}

What is the expected behavior?
No errors and values returned.
@lowjiansheng lowjiansheng changed the title Query failures on Decentralized Network Frequent query failures on Decentralized Network Aug 15, 2022
@azf20
Copy link
Contributor

azf20 commented Aug 17, 2022

Hi! I think this is therefore a subgraph / indexer specific issue on the network - I expect that this error is a timeout from the Indexer or the Gateway. There are a reasonably large number of entities in the subgraph, but I think there are lots of prior values of the snapshots, which likely slow the query down. I think indexers could likely benefit from applying the account-like optimisation here, but interested in @lutter's thoughts

@lutter
Copy link
Collaborator

lutter commented Aug 18, 2022

I just ran this query against the hosted service (by using the URL https://api.thegraph.com/subgraphs/id/Qme5mFGEqDg1KJ8PeaWj7vPDccTpSeHGPz9Zr8T4zAqJ3F)

The first time, the query took 20s, subsequent executions, even with small changes in the timestamp, take < 1s.

I think it is likely that the failure is due to a timeout caused by the indexer taking a long time to respond to the query.

@github-actions
Copy link

Looks like this issue has been open for 6 months with no activity. Is it still relevant? If not, please remember to close it.

@github-actions github-actions bot added the Stale label Feb 15, 2023
@azf20
Copy link
Contributor

azf20 commented Mar 6, 2023

Closing this issue 🙏

@azf20 azf20 closed this as completed Mar 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Stale
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@lutter @azf20 @lowjiansheng