Skip to content
This repository has been archived by the owner on Nov 10, 2017. It is now read-only.

Timeout value too low and no useful error message #25

Open
mattblaha opened this issue Jan 7, 2017 · 1 comment
Open

Timeout value too low and no useful error message #25

mattblaha opened this issue Jan 7, 2017 · 1 comment

Comments

@mattblaha
Copy link
Contributor

I was running into the same "Unable to access" errors other users have reported. I eventually realized that the 1000 ms timeout value in phantom-scrape.js was just too low for any page I was hitting and my internet connection.

I'd like to bump the timeout value to at least 5000 ms, and add a handler so the app returns a more useful message.

@mattblaha
Copy link
Contributor Author

I realized this patch can sometimes cause a pretty serious problem. Since the main webapp reads stdout from phantom-scrape.js, writing the error message with console.log() can cause the main app to read invalid JSON. One unimportant resource timing out in a large page will cause an error in the webapp even if everything else works and the results are what we want. The valid JSON will be there, but the parser won't be able to handle the error message printed above it.

I switched it to write to stderr, which fixes the problem, though I'm not sure this is the best way to do that.

It's the best way I could find after reading this discussion:

ariya/phantomjs#10150

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant