Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make only a single set of recentchanges API calls for incremental updates #30

Open
kevinlul opened this issue Apr 11, 2024 · 0 comments

Comments

@kevinlul
Copy link
Contributor

Currently, the incremental update script is invoked once per automatically-updated category.

url = "https://yugipedia.com/api.php?format=json&formatversion=2&action=query&generator=recentchanges&prop=revisions%7Ccategories&grctype=new%7Cedit%7Ccategorize&grctoponly=true&grclimit=50&grcnamespace=0&grcdir=newer&rvprop=content&cllimit=max"
url += f"&clcategories=Category:{sys.argv[1]}"

The recentchanges generator actually returns all changed pages, not just the one specified by clcategories, which only controls the categories that annotate each returned page. This parameter supports multiple categories combined with the | operator like the other parameters. Therefore, all categories to be automatically-updated can be retrieved in one Python script execution and one set of recentchanges API calls, as long as the output is written out to the correct directory based on the category.

There is maximum number of joined categories supported by clcategories. Pages may be in multiple categories and the categories array in the response should reflect this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant