playwright.helper.Error: Navigation failed because page crashed! #4549

yshalsager · 2020-11-29T18:39:37Z

I am running Python 3.8.5 on Ubuntu 20.04.1 and playwright==0.162.1.
I have a pretty complicated script that navigates through many pages of a given website. After navigating for a while (with goto method) playwright suddenly returns playwright.helper.Error: Navigation failed because page crashed! and skip to the next page then does the same thing. This happens randomly. There's no certain way to replicate it. Some of my script tests were totally fine (more than 200 pages navigation per run on both Chrome and Firefox non-headless mode).

Traceback with playwright debugging (some info are removed)

  pw:api   "load" event fired +0ms
Traceback (most recent call last):
  File "/home/x/y/z/a/pages/mobile/somepage.py", line 66, in create
    await self.open(self.url)
  File "/home/x/y/a/common/base_page.py", line 25, in open
    await browser.goto(url)
  File "/home/x/y/venv/lib/python3.8/site-packages/playwright/async_api.py", line 4037, in goto
    await self._impl_obj.goto(
  File "/home/x/y/venv/lib/python3.8/site-packages/playwright/page.py", line 451, in goto
    return await self._main_frame.goto(**locals_to_params(locals()))
  File "/home/x/y/venv/lib/python3.8/site-packages/playwright/frame.py", line 116, in goto
    await self._channel.send("goto", locals_to_params(locals()))
  File "/home/x/y/venv/lib/python3.8/site-packages/playwright/connection.py", line 39, in send
    result = await callback.future
playwright.helper.Error: Navigation failed because page crashed!
  pw:api   retrieving innerHTML from "html" +556ms
  pw:api   selector resolved to <html lang="en">…</html> +6ms
(node:50631) UnhandledPromiseRejectionWarning: Error [ERR_STREAM_DESTROYED]: Cannot call write after a stream was destroyed
    at doWrite (_stream_writable.js:399:19)
    at writeOrBuffer (_stream_writable.js:387:5)
    at Socket.Writable.write (_stream_writable.js:318:11)
    at Transport.send (/snapshot/playwright-cli/node_modules/playwright/lib/protocol/transport.js:38:25)
    at DispatcherConnection.onmessage (/snapshot/playwright-cli/lib/driver.js:49:61)
    at DispatcherConnection.sendMessageToClient (/snapshot/playwright-cli/node_modules/playwright/lib/dispatchers/dispatcher.js:129:14)
    at WorkerDispatcher._dispatchEvent (/snapshot/playwright-cli/node_modules/playwright/lib/dispatchers/dispatcher.js:70:26)
    at CRServiceWorker.<anonymous> (/snapshot/playwright-cli/node_modules/playwright/lib/dispatchers/pageDispatcher.js:213:58)
    at CRServiceWorker.emit (events.js:315:20)
    at CRBrowser._onDetachedFromTarget (/snapshot/playwright-cli/node_modules/playwright/lib/server/chromium/crBrowser.js:190:27)

common/base_page.py open function is nothing special.

    async def open(url):
        browser = await Browser.get_browser()
        await browser.goto(url)
        await browser.waitForSelector('body')

If any other info is needed please let me know.

The text was updated successfully, but these errors were encountered:

yeducom · 2020-11-30T10:51:22Z

mac FileNotFoundError: [Errno 2] No such file or directory: '/Users/apple/Library/Python/3.8/lib/python/site-packages/playwright/driver/playwright-cli'

kumaraditya303 · 2020-11-30T10:54:02Z

@yeducom What is your Mac Version ?

yeducom · 2020-11-30T10:55:36Z

11.0.1

kumaraditya303 · 2020-11-30T11:05:43Z

11.0.1

@yeducom Currently MacOS Big Sur isn't supported See microsoft/playwright-python#325

kumaraditya303 · 2020-11-30T11:11:13Z

I am running Python 3.8.5 on Ubuntu 20.04.1 and playwright==0.162.1.
I have a pretty complicated script that navigates through many pages of a given website. After navigating for a while (with goto method) playwright suddenly returns playwright.helper.Error: Navigation failed because page crashed! and skip to the next page then does the same thing. This happens randomly. There's no certain way to replicate it. Some of my script tests were totally fine (more than 200 pages navigation per run on both Chrome and Firefox non-headless mode).

Traceback with playwright debugging (some info are removed)

  pw:api   "load" event fired +0ms
Traceback (most recent call last):
  File "/home/x/y/z/a/pages/mobile/somepage.py", line 66, in create
    await self.open(self.url)
  File "/home/x/y/a/common/base_page.py", line 25, in open
    await browser.goto(url)
  File "/home/x/y/venv/lib/python3.8/site-packages/playwright/async_api.py", line 4037, in goto
    await self._impl_obj.goto(
  File "/home/x/y/venv/lib/python3.8/site-packages/playwright/page.py", line 451, in goto
    return await self._main_frame.goto(**locals_to_params(locals()))
  File "/home/x/y/venv/lib/python3.8/site-packages/playwright/frame.py", line 116, in goto
    await self._channel.send("goto", locals_to_params(locals()))
  File "/home/x/y/venv/lib/python3.8/site-packages/playwright/connection.py", line 39, in send
    result = await callback.future
playwright.helper.Error: Navigation failed because page crashed!
  pw:api   retrieving innerHTML from "html" +556ms
  pw:api   selector resolved to <html lang="en">…</html> +6ms
(node:50631) UnhandledPromiseRejectionWarning: Error [ERR_STREAM_DESTROYED]: Cannot call write after a stream was destroyed
    at doWrite (_stream_writable.js:399:19)
    at writeOrBuffer (_stream_writable.js:387:5)
    at Socket.Writable.write (_stream_writable.js:318:11)
    at Transport.send (/snapshot/playwright-cli/node_modules/playwright/lib/protocol/transport.js:38:25)
    at DispatcherConnection.onmessage (/snapshot/playwright-cli/lib/driver.js:49:61)
    at DispatcherConnection.sendMessageToClient (/snapshot/playwright-cli/node_modules/playwright/lib/dispatchers/dispatcher.js:129:14)
    at WorkerDispatcher._dispatchEvent (/snapshot/playwright-cli/node_modules/playwright/lib/dispatchers/dispatcher.js:70:26)
    at CRServiceWorker.<anonymous> (/snapshot/playwright-cli/node_modules/playwright/lib/dispatchers/pageDispatcher.js:213:58)
    at CRServiceWorker.emit (events.js:315:20)
    at CRBrowser._onDetachedFromTarget (/snapshot/playwright-cli/node_modules/playwright/lib/server/chromium/crBrowser.js:190:27)

common/base_page.py open function is nothing special.

    async def open(url):
        browser = await Browser.get_browser()
        await browser.goto(url)
        await browser.waitForSelector('body')

If any other info is needed please let me know.

There isn't any method get_browser() in API of playwright

yshalsager · 2020-11-30T16:24:05Z

I am running Python 3.8.5 on Ubuntu 20.04.1 and playwright==0.162.1.
I have a pretty complicated script that navigates through many pages of a given website. After navigating for a while (with goto method) playwright suddenly returns playwright.helper.Error: Navigation failed because page crashed! and skip to the next page then does the same thing. This happens randomly. There's no certain way to replicate it. Some of my script tests were totally fine (more than 200 pages navigation per run on both Chrome and Firefox non-headless mode).
Traceback with playwright debugging (some info are removed)

  pw:api   "load" event fired +0ms
Traceback (most recent call last):
  File "/home/x/y/z/a/pages/mobile/somepage.py", line 66, in create
    await self.open(self.url)
  File "/home/x/y/a/common/base_page.py", line 25, in open
    await browser.goto(url)
  File "/home/x/y/venv/lib/python3.8/site-packages/playwright/async_api.py", line 4037, in goto
    await self._impl_obj.goto(
  File "/home/x/y/venv/lib/python3.8/site-packages/playwright/page.py", line 451, in goto
    return await self._main_frame.goto(**locals_to_params(locals()))
  File "/home/x/y/venv/lib/python3.8/site-packages/playwright/frame.py", line 116, in goto
    await self._channel.send("goto", locals_to_params(locals()))
  File "/home/x/y/venv/lib/python3.8/site-packages/playwright/connection.py", line 39, in send
    result = await callback.future
playwright.helper.Error: Navigation failed because page crashed!
  pw:api   retrieving innerHTML from "html" +556ms
  pw:api   selector resolved to <html lang="en">…</html> +6ms
(node:50631) UnhandledPromiseRejectionWarning: Error [ERR_STREAM_DESTROYED]: Cannot call write after a stream was destroyed
    at doWrite (_stream_writable.js:399:19)
    at writeOrBuffer (_stream_writable.js:387:5)
    at Socket.Writable.write (_stream_writable.js:318:11)
    at Transport.send (/snapshot/playwright-cli/node_modules/playwright/lib/protocol/transport.js:38:25)
    at DispatcherConnection.onmessage (/snapshot/playwright-cli/lib/driver.js:49:61)
    at DispatcherConnection.sendMessageToClient (/snapshot/playwright-cli/node_modules/playwright/lib/dispatchers/dispatcher.js:129:14)
    at WorkerDispatcher._dispatchEvent (/snapshot/playwright-cli/node_modules/playwright/lib/dispatchers/dispatcher.js:70:26)
    at CRServiceWorker.<anonymous> (/snapshot/playwright-cli/node_modules/playwright/lib/dispatchers/pageDispatcher.js:213:58)
    at CRServiceWorker.emit (events.js:315:20)
    at CRBrowser._onDetachedFromTarget (/snapshot/playwright-cli/node_modules/playwright/lib/server/chromium/crBrowser.js:190:27)

common/base_page.py open function is nothing special.

    async def open(url):
        browser = await Browser.get_browser()
        await browser.goto(url)
        await browser.waitForSelector('body')

If any other info is needed please let me know.

There isn't any method get_browser() in API of playwright

Sorry, I forgot to mention it.
It's nothing more than a method for returning the current BrowserType instance created with launch method.

pavelfeldman · 2020-11-30T17:15:47Z

It looks like you are hitting a renderer crash (sad tab), so it would be great if there was a repro so that we could narrow it down. Also, any reason you navigate the same page so many times, could you share your use case?

yshalsager · 2020-11-30T17:28:10Z

It looks like you are hitting a renderer crash (sad tab), so it would be great if there was a repro so that we could narrow it down. Also, any reason you navigate the same page so many times, could you share your use case?

Basically, the script is testing some online store coupons and have to go through a large list of products (1500+).
Each product needs at least 3 navigation steps (product page - checkout - cart).

pavelfeldman · 2020-11-30T17:43:20Z

You are hitting the out-of-memory error due to collected state in the context. You should use new context for each product to make tests isolated and more real. I.e. browser.newContext followed by a context.newPage, then your navigation steps. Don't forget to close the context (or page) after each test. That would isolate tests and would make sure state does not collect over time. If you need to sign it only once for that, there are ways to inject cookies / storage in the new context for each test so that you did not need to log in over and over.

We currently retain all the resource handles within the context, that adds over time. We'll fix it upstream, but you could switch to a context-per-test for now to work around it, it is how it is supposed to be anyways.

yshalsager · 2020-12-01T18:35:36Z

You are hitting the out-of-memory error due to collected state in the context. You should use new context for each product to make tests isolated and more real. I.e. browser.newContext followed by a context.newPage, then your navigation steps. Don't forget to close the context (or page) after each test. That would isolate tests and would make sure state does not collect over time. If you need to sign it only once for that, there are ways to inject cookies / storage in the new context for each test so that you did not need to log in over and over.

We currently retain all the resource handles within the context, that adds over time. We'll fix it upstream, but you could switch to a context-per-test for now to work around it, it is how it is supposed to be anyways.

Thanks for the clarification, I will update my code as soon as possible and see if everything works as it should.

yshalsager · 2020-12-06T10:52:07Z

You are hitting the out-of-memory error due to collected state in the context. You should use new context for each product to make tests isolated and more real. I.e. browser.newContext followed by a context.newPage, then your navigation steps. Don't forget to close the context (or page) after each test. That would isolate tests and would make sure state does not collect over time. If you need to sign it only once for that, there are ways to inject cookies / storage in the new context for each test so that you did not need to log in over and over.

We currently retain all the resource handles within the context, that adds over time. We'll fix it upstream, but you could switch to a context-per-test for now to work around it, it is how it is supposed to be anyways.

Done updating my code, and it's working perfectly now. I have run a test with more than 1500 products at night and nothing went wrong thankfully.

pavelfeldman · 2021-01-18T17:33:35Z

Yay!

pavelfeldman transferred this issue from microsoft/playwright-python Nov 30, 2020

pavelfeldman added the P2-bug label Nov 30, 2020

jperl mentioned this issue Dec 29, 2020

[BUG] Navigation failed because page crashed! #4851

Closed

pavelfeldman closed this as completed Jan 18, 2021

pavelfeldman mentioned this issue Jan 18, 2021

[REGRESSION]: Memory leak related to route interception #4511

Closed

jfp1992 mentioned this issue Oct 3, 2022

[BUG] Memory increases when same context is used #6319

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

playwright.helper.Error: Navigation failed because page crashed! #4549

playwright.helper.Error: Navigation failed because page crashed! #4549

yshalsager commented Nov 29, 2020

yeducom commented Nov 30, 2020

kumaraditya303 commented Nov 30, 2020 via email •

edited

Loading

yeducom commented Nov 30, 2020

kumaraditya303 commented Nov 30, 2020

kumaraditya303 commented Nov 30, 2020

yshalsager commented Nov 30, 2020

pavelfeldman commented Nov 30, 2020

yshalsager commented Nov 30, 2020

pavelfeldman commented Nov 30, 2020

yshalsager commented Dec 1, 2020

yshalsager commented Dec 6, 2020

pavelfeldman commented Jan 18, 2021

playwright.helper.Error: Navigation failed because page crashed! #4549

playwright.helper.Error: Navigation failed because page crashed! #4549

Comments

yshalsager commented Nov 29, 2020

yeducom commented Nov 30, 2020

kumaraditya303 commented Nov 30, 2020 via email • edited Loading

yeducom commented Nov 30, 2020

kumaraditya303 commented Nov 30, 2020

kumaraditya303 commented Nov 30, 2020

yshalsager commented Nov 30, 2020

pavelfeldman commented Nov 30, 2020

yshalsager commented Nov 30, 2020

pavelfeldman commented Nov 30, 2020

yshalsager commented Dec 1, 2020

yshalsager commented Dec 6, 2020

pavelfeldman commented Jan 18, 2021

kumaraditya303 commented Nov 30, 2020 via email •

edited

Loading