Parquet/no-build-tool java profile usability improvements #914
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
msbarry
changed the title
Full logs: https://github.com/onthegomap/planetiler/actions/runs/9465725196 |
|
bdon
added a commit
to protomaps/basemaps
that referenced
this pull request
Jun 13, 2024
Since onthegomap/planetiler#914 the registerSourceHandler is called implicitly
bdon
added a commit
to protomaps/basemaps
that referenced
this pull request
Jun 13, 2024
* Fix build Since onthegomap/planetiler#914 the registerSourceHandler is called implicitly * fix build: explicit source handling * rename processFeature -> processOsm
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Improvements and bug fixes found while using planetiler no-build-tool java profiles in https://github.com/onthegomap/planetiler-examples
improvements:
--tests spec.yamlto run the profile against a set of example test case featuresfeatures.anyGeometry(layer)to FeatureCollector API that creates a line, point, or polygon vector tile feature based on the geometry type of the source featureinheritAttrsFromSourceandinheritAttrsFromSourceWithMinzoomto FeatureCollector APIForwardingProfilehandlers also return a boolean expression fromfilter()method that limits the features they are called with (and skip entire parquet files that are not of interest)Expression.matchSourceandExpression.matchSourceLayerboolean expressions that generate optimized indexes for matching features at runtimeExpression.matchFieldandExpression.matchAnyhandle structured attributes, for example:matchField("names.primary", "Massachusetts Turnpike")ForwardingProfileskip or include only certain layers automatically with--only-layersand--exclude-layersoption.PlanetilerRunner#runso that profiles don't need to handle themOSM_ATTRIBUTIONconstant toProfilewith default recommended openstreetmap attributionnameimplementation toProfileso profiles only need to implementprocessFeaturemethod - and they can use lambda method syntaxDistributorutility when reading parquet files to spread features from final row groups across other threads as they finishfixes:
setAttrWithMinSizework correctly for partial-length line featuresStruct#rawValuewas including nested structs for maps and lists, changed to return raw objects