Check / update data for indoor swimming pools

[joergreichert]

Official data

https://www.leipzig.de/freizeit-kultur-und-tourismus/sport/sportstaetten/schwimmhallen

JSON data

scraped via yarn run scrape-swimming-pools

Scraped output

public/data/leipzig-swimming-pools.json

Transformed output

no additional transformation needed

Referenced in code

src/components/Map.js
showMarker -> swimming_pools

App

[alex-fdias]

Hi!

I cannot fully understand why executing the script "src/scrapers/leipzig-swimming-pools.js" leads to writing an empty JSON file ("public/data/leipzig-swimming-pools.json").

If I understood correctly, the script "leipzig-swimming-pools.js" uses the module "scrape-it" to retrieve the data on swimming pools from two URLs, and these data would be retrieved via AJAX requests. However, I checked for AJAX requests using the Developer Tools of Chrome (Network tab -> Fetch/XHR) and the data retrieved via AJAX requests has nothing to do with swimming pool data (it's the Werbung of the "Stadtliche Karriere" that shows at the bottom on the right). Also, I checked the URLs HTML source code and the lists of Schwimmhallen/Freibäder are hard coded anyway, though the information shown on the webpages is not structured/as detailed as in the file "public/data/leipzig-swimming-pools.json" of this repository.

My guess is that the webpages corresponding to the two URLs were changed or the existing data ("public/data/leipzig-swimming-pools.json") was obtained in another way (OpenStreetMap?).

[joergreichert]

leipzig-swimming-pools.js uses the npm library https://www.npmjs.com/package/scrape-it to scrape the swimming pools data from https://www.leipzig.de/freizeit-kultur-und-tourismus/sport/sportstaetten/freibaeder by select each entry on that site and also follow for each the hyperlink to the details page to scrape the details data as well. Obviously they redesigned the web page, so the matchers don't work anymore and thus produce an empty JSON file.

[joergreichert]

I've fixed the scraper: https://github.com/CodeforLeipzig/leipzigmaps/blob/main/src/scrapers/leipzig-swimming-pools.js

https://github.com/CodeforLeipzig/leipzigmaps/blob/main/test/scrapers/leipzig-swimming-pools-test.js can be used via node ./test/scrapers/leipzig-swimming-pools-test.js to test the scraper for exactly one entry.