Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crosswalk between openEO API and OCG API - Coverages #502

Draft
wants to merge 8 commits into
base: ogcapi-processes
Choose a base branch
from

Conversation

m-mohr
Copy link
Member

@m-mohr m-mohr commented Jul 3, 2023

No description provided.


### API description

- openEO: `n/a` (?)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way of linking to OpenAPI documents in openEO and OAC1 is the same, it's just that OAC1 pre-defines a path for it and requires the links, which are optional in openEO.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So in openEO there is no predefined link to the OpenAPI docs? How can a client discover the link?
(feel free to edit the document yourself anyway, of course)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In both specs, you use the relation types service-desc and service-doc. I think in both cases they can point anywhere, but OGC APIs propose to use /api as endpoint.

Copy link
Member Author

@m-mohr m-mohr Jul 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(It's better to first discuss, come to a common understanding and update afterwards :-) )


- openEO:
- [`GET /collections/{collectionId}`](https://api.openeo.org/1.2.0/#tag/EO-Data-Discovery/operation/describe-collection) (required)
- [`GET /collections/{collectionId}/queryables`](https://api.openeo.org/1.2.0/#tag/EO-Data-Discovery/operation/list-collection-queryables) (required?)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's optional.


### Tiles

- openEO: `n/a`
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

openEO allows it through secondary web services, similar to OAC1.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I mention that, but I'll make it clearer by replacing the n/a also.

thanks to the [`/service_types`](https://api.openeo.org/1.2.0/#tag/Capabilities/operation/list-service-types) resource.


### Media encoding
Copy link
Member Author

@m-mohr m-mohr Jul 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure whether comparing these two is correct. Let's discuss tomorrow.


## Crosswalk

*Executive summary (tl;dr)*
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For me an important details is missing here. Can OGC API - Coverages be combined with the openEO API? It seems they have a common baseline and it could just be additional endpoints (mostly /collections/{collectionId}/coverage/...).

Due to the openEO WCPS driver, we could translate openEO to WCPS, right? It seems vice-versa is not easily possible (waiting for an answer from Peter Baumann).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are probably a couple of assumptions to be made regarding the cubes' dimensions, which makes openEO/OGC not 100% compatible, but probably cover most of the "real" coverages that are there anyway: I mean that there is not necessarily a 1:1 agreement and directional alignment between the dimensions of a cube and the CRS' axes (imagine I don't know, an oblique plane/tile in a 3D fishtank), which is a case that the OGC Coverage model can describe, while I am not sure about this with openEO. ?

Also, since the datacube STAC extension merges a coverage's domain and range spaces into the same "dimensions" container, there is the little "pain" to define the way to identify which dimensions of a datacube are what (spatial, temporal, other, band, etc), based on fragile assumptions on axis/dimensions labels?

Translating a (subset of) (the most basic, like filtering, scaling, etc) openEO processes to WCPS is feasible (are you talking about the openEO Spring driver?).

The other way around is probably also feasible, with the additional step of requiring to identify the category of the dimensions, as openEO processes are much more strongly typed in this sense, WCPS treats all dimensions equally and operations on them do not "discriminate".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess openEO could describe the 3D fishtank if there's a CRS for it?

I agree that the STAC data cube extension is a bit messy with regards to the type, but it's specified. type = spatial (with axis), type = temporal, type = ... (e.g. bands). At least internally it should be clear. "other" only really shows up if you add a "user-defined" dimension in the process graph.

I'm not specifically talking about any implementation, more what theoretically can be translated or whether there are any roadblocks.

openEO processes only really "discriminate" spatial axes as x and y need usually be used together, but I'm not sure how WCPS handles the special case of spatial?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess openEO could describe the 3D fishtank if there's a CRS for it?

Yes, but how can the orientation of the cube dimensions be specified? Even in the simple/common case when the cube is "aligned" with the CRS axes: we should not rely on the labels of the dimensions to infer its "direction". OGC uses the "offset vectors" for this.

I agree that the STAC data cube extension is a bit messy with regards to the type, but it's specified. type = spatial (with axis), type = temporal, type = ... (e.g. bands). At least internally it should be clear. "other" only really shows up if you add a "user-defined" dimension in the process graph.

I don't think it is messy, it is clear! Only I personally prefer the distinction that the OGC Coverages do between domain and range, it makes it even clearer.

openEO processes only really "discriminate" spatial axes as x and y need usually be used together, but I'm not sure how WCPS handles the special case of spatial?

Afaik WCPS does not handle any special spatial (nor temporal) case: all operators are completely agnostic of the type of dimensions they operate onto.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dimension labels in openEO have an order, but otherwise there's nothing like that, indeed.

Afaik WCPS does not handle any special spatial (nor temporal) case

Well, it does, right? bands (range) seems separate from spatial/temporal (domain), while in openEO it's just all a dimension. What's so special about bands?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I was talking indeed about a Coverage dimensions: no special case there, but yes, bands indeed are treated separately. To me it makes a whole lot of sense to have a domain, a "space", (or space-time) where the measurements lie, and then you have the data-space, which is what are the measurements about.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, it's good to understand this, thanks.

@m-mohr
Copy link
Member Author

m-mohr commented Jul 3, 2023

Thanks for the comparisong @pierocampa.

I've added a couple of comments above. I think an important question to answer here is how could openEO be combined with Coverages and/or WCPS. What would someone need to do to expose Coverages endpoints in openEO? Could WCPS and openEO processes somehow be translated?

@m-mohr
Copy link
Member Author

m-mohr commented Jul 3, 2023

An additional point that, I think, is important to add: The actual definitions of datacubes, dimensions etc. How do they compare? Can they be mapped 1:1? Are there any conflicts?

Edited API/service description comments.
@pierocampa
Copy link
Member

An additional point that, I think, is important to add: The actual definitions of datacubes, dimensions etc. How do they compare? Can they be mapped 1:1? Are there any conflicts?

Right that one. As I was mentioning in the other comment, this is probably the key and most painful point to be solved to reach compatibility (and happiness)!

so we can generally compare it with STAC [collections](https://github.com/radiantearth/stac-spec/blob/master/collection-spec/collection-spec.md),
especially [datacubes](https://openeo.org/documentation/1.0/datacubes.html).

### OGC
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For better understanding, it might help to give examples here so that we can more easily map between openEO and Coverages.

I'm trying to map right now, but I'm unsure:

  • collection = collection?
  • coverage/feature = data cube?
  • range = dimension?
  • domain = dimension labels?

Copy link
Member

@pierocampa pierocampa Jul 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mapping is more like this:

  • collection = collection
  • coverage/feature = data cube / item / collection
  • range set = the actual data (pixels, voxels, n-xels?!)
  • range type = the description of the data space (bands) = dimensions::bands
  • domain set = dimensions::spatial, dimensions::temporal, dimensions::other

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(anyway I'm a bit confused here by your mapping: STAC datacube can also be a STAC collection, right?)

Copy link
Member Author

@m-mohr m-mohr Jul 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I'd keep STAC aside for now, it's just the metadata, but doesn't really care in this context (I think) as we should just map the coverage and data cube concepts.

Your list makes sense to me, I'm just confused about the range type. What happend with the range type if the data is non-spectral? And why is other a domain set then?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Range type is just metadata about the data: it says what is it (the label given, the unit of measure, a link to a formal description, etc (a SWE record). It is separated from the domain set because the coverage model separates them! The where and the what, as I said in the previous comment.

And this is because the coverage is defined as a function that links every point in the domain space to a point in the range space, like saying: (y_1*, y_2*, y_3*) = f(x_1*, x_2*). You are essentially asking why having the vector of dimensions x_i and the vector of dependent variables y_i not merged together. That's the way I see it!

Copy link
Member Author

@m-mohr m-mohr Jul 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I just think that dimensions of type other in openEO are often more a range rather than a domain. It could for example be a data cube x,y,t,indices where indices is of type other and contains ndvi, evi, ndsi etc. Then I think it should be a range in WCPS, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The indices should go in the coverage range, exactly. Though this is not always the case! I modeled 5D marrays for climate indices projections with "model" and "scenario" as additional "other" dimensions (then x,y,t) (see an example here): I would not want these "other" dimensions to go into a coverage range, but rather stay as dimensions (domain). In the range I might want to keep the values only, or maybe I could put value and uncertainty, etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants