Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logs report a warning: Spider.make_requests_from_url method is deprecated... #181

Open
LittleHello opened this issue Dec 11, 2020 · 2 comments

Comments

@LittleHello
Copy link

Environments : scrapy-redis 0.6.8, Scrapy 2.4.1, Python 3.8.5
When running the spider, the logs report a warning : Spider.make_requests_from_url method is deprecated: it will be removed and not be called by the default Spider.start_requests method in future Scrapy releases. Please override Spider.start_requests method instead.
Warning is from line 66 in https://github.com/scrapy/scrapy/blob/master/scrapy/spiders/__init__.py, the reason may be in https://github.com/rmax/scrapy-redis/blob/master/src/scrapy_redis/spiders.py, method named make_request_from_data in line 121.
Can this warning be fixed?

@nieweiming
Copy link
Contributor

use make_request_from_data function and override it;

@rmax
Copy link
Owner

rmax commented Mar 25, 2021

As @nieweiming said, you can avoid the warning for your project by overriding make_request_from_data and instead of calling make_requests_from_url just create the Request object directly.

That could be a good PR too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants