PMTiles + enhancements to vector tile data sources in Rapid #1016
Conversation
Also fixes a few bugs in the CustomData layer where it wasn't computing the dataID correctly This could lead to a bunch of features all ending up with a single broken id like `undefined`
1. It was only using the `entity.tags` anyway 2. This allows us to label things that aren't entities. 3. Now we add labels to custom data if we can (e.g. if it has a `name` property)
- Make sure that custom data can be hovered, selected (this was not working because of undefined dataID) - The current way to bind data to a feature is to call `feature.setData(dataID, data)` - There was still some old code that was expecting each data to have a `.id` property. This is not a safe assumption, should instead be using the `feature.data` / `feature.dataID` accessors - Fixed the data ids for the custom data layer.. All features here can just use `__featurehash__.toString()` as their identifier
Going to try to establish some naming conventions, what we did before was chaos - Use `loadTiles()` to fetch more tiles This function uses context.projection to know what tiles to load - Use `getData()` (and similar) to return loaded data This function uses MapSystem.extent() to know what tiles to load There is an unfortunate mismatch here and that's because the sdk Tiler accepts a Projection, and the projection has the dimensions built into it, and this is kind of weird IMO. (we will need to change this once we implement map rotation) But the good news is that every other place that needs to tell services to load tiles and return data can avoid looking up this stuff and passing the params around.
- Now it creates caches "per-zoom", because vector tiles can return different data at different zooms - Does a decent enough job of stitching geometries together across tile boundaries (still some edge cases but it is mostly good) - Rewinds the geojson so pixi will display it properly - Breaks multi part features into single part features so we can work with them in the editor
|
Nice! When will this make it to http://rapideditor.org ? |
Sometime before end of August! 👍 |
|
FYI I tested the feature for an upcomming dataset which will be released officially soon which will include all parking spots in Berlin. We are considering how to migrate this data to OSM. Rapid could be a huge help for this. Here is the sample dataset in Rapid https://rapideditor.org/edit#background=Berlin-2022&data=https://atlas-tiles.s3.eu-central-1.amazonaws.com/berlin-parking-polygons-euvm.pmtiles&datasets=fbRoads,msBuildings&disable_features=boundaries&map=19.76/52.49806/13.34018 (also a preview of the pmtiles and a preview along side our processed OSM parking data) The thing I am missing most right now in order to do anything with this feature in terms of mapping is a way to get the data in OSM. Something like a "create Geometry as OSM Polygon". And, of course, we would have to prepare the data to translate the existing tags to OSM tags in order to also have useful tags to keep (see also #545). |
|
Thanks @tordans - this is really great to see it in action! @cbeddow recently floated an idea like - if someone's data includes a property like For anyone following the issue, this is what the parking zones look like - it's a real wealth of information! : 
|
I like that idea. I think I would then keep the original attributes and only add the It would also make sure that not all data is added "blindly" but only such data that was crafted in the right way. Which I think is a good protection since one has to know a what to do in order to prepare the data and during that process there is enough time to think about licencing.
I would appreciate some formality here. Maybe the "add to map" button is only present when an What would the format of { "properties": { "osm_keys": "key=value\nkey2=value2" } } |
|
One more thing about the "add data to map" flow. A secondary use case would be to update or merge data with an existing OSM object. In the case of the parking datasets, we might have an existing parking polygon in OSM but want to update the tags with what we have in the external data. Maybe there is room for a feature where I select both the OSM and the external Data and instead of the "add shape with tags to map" button there is a "merge |
|
I'm looking at PMTiles for OSMUK local chapter. I can see some cases where this would benefit us too, including the merge suggestion in the last comment. |
|
One caveat I want to point out - for tiled vector sources in general, of which PMTiles overlays are one - is we need to be careful about how overzooming is handled. If the mapper is viewing Rapid at neighborhood scale and seeing z15 tiles, then the vector tile (MVT) geometry should be quite accurate. But if the vector tile datasource only contains zoom levels several before the viewed map (example: only tiles up to z10 or z11) then the geometry shown in Rapid may stray far from the intended positions. This is equivalent to viewing satellite imagery as an overlay that has a limited zoom level and looks blurry when zoomed in. For raster tiles, the blur makes the problem obvious to the mapper, but I'm not sure how this would look in Rapid for vector. If anyone is using tippecanoe to work with data in Rapid it would be helpful to show a demo of tilesets with fewer zoom levels. |
This is a good point - it's something I have noticed when using the example overture buildings tiles:
This data is sourced from OSM, so people would expect it to match exactly, but the cyan buildings have clearly lost some precision. This is because the tiles are baked at zoom 13, so they have an effective resolution of only about 3/4 meter.
tile id = "1294,2802,13"
tile extent = geoLatToMeters( 49.32512199104001 - 49.29647160265807) geoLonToMeters ( -123.0908203125 - -123.134765625, 49.31) spatial resolution is: meters / 4096 (the tile resolution) 3178.70490492619 / 4096 |
tl;dr
This PR includes a massive refresh of the vector tile support in Rapid, and adds support for Protomaps PMTiles URLs!
Why
Overture Map Foundation has recently released a whole lot of data under a permissive license. Yay! 🎉
Like everyone else, we're really interested to see how this data looks on a map.
Shout out to @jenningsanderson and @bdon for building these demos that do exactly that:
Both of these demos are powered by Protomaps PMTiles - a newish single-file format for storing vector tiles in the cloud. It's like Mapbox Vector Tiles (MVT), but a map client can use HTTP Range Requests to get just the data you need from the archive.
This PR adds support for .pmtiles sources in our "Custom Map Data" layers - this is the part of the Rapid code responsible for rendering custom .geojson, .kml, .gpx, files - and also connecting to vector tile sources of data.
👉 Importantly - the data is view-only for now.
This PR is not a solution for ingesting third party data into OSM, or bring-your-own-data, or anything like that.
You can connect to a PMTiles URL and look at what's in it, that's all!
Here are some .pmtiles URLs to try (no guarantees!):
Overture Places around Newark, NJ
I also did some much-needed refresh of Rapid's vector tile code. It's a lot more performant now, and it does a better job stitching together features across tile boundaries. Take a peek into your favorite vector tile map!
OSM Americana around Newark, NJ
What next
Many things that we can ticket separately