Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update geostationary projection to allow clean description of newer generation satellites #258

Closed
erget opened this issue Apr 1, 2020 · 30 comments · Fixed by #259
Closed

Update geostationary projection to allow clean description of newer generation satellites #258

erget opened this issue Apr 1, 2020 · 30 comments · Fixed by #259
Assignees
@JonathanGregory
Labels
enhancement Proposals to add new capabilities, improve existing ones in the conventions, improve style or format

Comments

@erget
Copy link
Member

erget commented Apr 1, 2020 •
edited
Loading

Moderator

@JonathanGregory

Moderator Status Review [last updated: YY/MM/DD]

Brief comment on current status, update periodically

Requirement Summary

The geostationary projection should be compatible with all generations of satellites that are used to generate data products from geostationary orbit.

Technical Proposal Summary

The current wording of the geostationary projection uses the concept of sweep_angle_axis and fixed_angle_axis. This is applicable for when instruments sweep their view along a given axis of a data product, whereas the other axis of the product is fixed. An example of this is the Meteosat mission, whose satellites' imagers "sweep" north as the satellite spins about its fixed axis.

This is no longer an applicable description of how newer satellites work. GOES-17, for example, uses a different approach to scans, as will Meteosat Third Generation. These satellites are three-axis stabilised and don't have a fixed axis.

As in practice, sweep_angle_axis and fixed_angle_axis refer to the elevation and azimuth, independent of the instrument's operating principles, I propose adding the capability of having a more general description of the image which does not rely on the underlying scan mechanism. In the end, the grid definition is a mathematical construct and the user is interested in understanding the grid in order to recover Earth coordinates from it; the instrument's function is likely interesting but not important in order to understand the geophysical observation.

Benefits

Users and producers of geostationary data products.

Status Quo

It is currently possible to encode data using the CF Conventions with the attributes described in Appendix F, but this can be confusing, especially for data producers, when spin and sweep aren't actually related to the instrument in question.

Detailed Proposal

It might be a bit tricky to get this right so I'm open to suggestions. I'd like to wait to create a PR until I receive feedback from the community. In a nutshell I propose:

  • Clarifying the notes related to spin and sweep in order to specify the generations of satellites they apply to. I might also include some language describing what they actually meant in the context of those satellites, some of which will soon be considered legacy.
  • Add the possibility of using elevation and azimuth rather than spin and sweep axes, whilst stating that the data producer must specify either one of elevation/azimuth or one of spin/sweep axis.

This would not break existing data, and for future data it would be a bit clearer. However, it makes the projection a little bit more involved, as we would have 2 sets of attributes that would essentially be identical.
@erget erget added the enhancement Proposals to add new capabilities, improve existing ones in the conventions, improve style or format label Apr 1, 2020
@JimBiardCics
Copy link
Contributor

@erget This sounds like an excellent change to me.

@JonathanGregory
Copy link
Contributor

I'm not familiar with this projection, but in principle I think your proposal is sensible. There isn't a problem in having alternative ways to do things if there are new things which need to be described. Thanks.

@cf-metadata-list
Copy link

cf-metadata-list commented Apr 1, 2020 via email

@JimBiardCics
Copy link
Contributor

Randy, what you say is quite correct. Do you have a suggestion for better names that don't assume too much?

@rhorne99
Copy link

rhorne99 commented Apr 1, 2020 via email •
edited by erget
Loading

@erget
Copy link
Member Author

erget commented Apr 2, 2020

Hi @rhorne99 you're right, there's nothing that prevents a 3-axis stabilised satellite from behaving like Meteosat 1st or 2nd generation.

Meteosat 3rd generation deviates from the concept of previous generations by scanning in out of alignment with the image's axes. And to make matters more interesting this misalignment is different throughout the full disc scan - it's skewed in opposite directions in opposing hemispheres. This picture illustrates how it works:
FCI scan principle

The result of this is that the position within the data array doesn't necessarily map to a position on the Earth in the same way that it did before. I think you put it well when you said

the location of data points in the projection vary as a function of how the imaging instrument works

In this configuration there isn't really a fixed axis at all. This is why we'd like to be able to signify which axes give elevation and azimuth angles. Independently of how the instrument is scanning, then, a rectified product would be well-described using these attributes.

If this approach makes sense to somebody who's not me I'll put together a pull request showing the exact changes proposed :)

@cf-metadata-list
Copy link

cf-metadata-list commented Apr 2, 2020 via email

@erget
Copy link
Member Author

erget commented Apr 3, 2020

Hi Randy,

Concerning your questions:

  1. As at EUMETSAT we are essentially using sweep_angle_axis as synonymous with the axis signifying azimuth angle and fixed_angle_axis as synonymous with the axis signifying elevation, we'd like to be able to specify azimuth_axis and fixed_angle_axis instead of sweep_angle_axis and fixed_angle_axis. This more general definition would more accurately reflect our understanding of how our instrument works.
  2. Yes, the Meteosat Third Generation image products will still have North, South, East, and West as the image outer boundaries - we scan tilted, but full disk images won't be tilted.

Daniel

@cf-metadata-list
Copy link

cf-metadata-list commented Apr 3, 2020 via email

@steingod
Copy link

Although primarily working with polar orbiters this suggested change make sense to me. I fully acknowledge the challenge identified by Randy, but think that shouldn't prevent this update and I think the proposal as stated also acknowledge this.

@erget
Copy link
Member Author

erget commented Apr 14, 2020

Hi all,

Thanks very much for your inputs! A few points to respond to:

From Randy:

if the current geostationary projection parameters are deprecated, I would be very surprised if the products for the yet to be launched GOES-R series satellites (T and U) will reflect any changes you are proposing.

I understand, this is an issue that's present with all of the major satellite data producers. I do not intend to deprecate anything, but rather to provide clarifications so that implementing things are easier.

@steingod thanks for chiming in as well, it's very much appreciated.

TLDR

I myself have sought further clarification of the issue with one of our optical imager engineers @ameraner who has been kind enough to delve deep into the details of how sweep_angle_axis and fixed_angle_axis are used. I think that the issue can in fact be resolved not by adding additional attributes, but rather by clarifying in the notes how these axes are used. PROJ probably has the best documentation on this - concise and understandable - and I believe that CF would profit from similar clarity in the notes.

I'll propose specific changes to the notes in a TBW PR, but surgery on the Conventions needs care and so I'm summarising the specifics of the issue below.

Nuts and bolts

As nicely described by PROJ the geostationary projection works using a gimbal model of an observation point's viewing angle, like thus:
Conceptual gimbal used by Meteosat

Some terms:
Sweep-angle axis: This is the outer-gimbal axis, the one whose axis is aligned with the "base" in the figure. If you imagine this as the Earth, its equator always describes the same great circle.
Fixed-angle axis: This is the inner-gimbal axis, the one whose axis is attached to the "equator" of the outer-gimbal circle. Therefore its "equator" describes a different great circle depending on the position of the outer gimbal.

As you can see, the definitions could be considered counterintuitive but they're widely adopted and there's no reason to (try to) change them.

The important thing to note is that even if you know the azimuth and elevation of a given point, you also have to know which axis is aligned with the conceptual outer and inner gimbal. Order of operations matters in this case. This contradicts what I was asserting earlier and has led to implementation difficulties, not because the attributes are incorrect, but rather because they require some mental gymnastics to parse.

The solution is to clarify a few things in the notes:

  • How the axes correspond to a gimbal view, as this has confused developers
  • That the y-axis (in existing implementations, and probably for the foreseeable future) aligns with N/S and the x-axis aligns with E/W. This is true in practice but is not specified in our description.

@ameraner
Copy link

Hi all,

thanks @erget for trying to clarify this issue! It's causing some confusion in the community, as it's a tricky thing to understand. I'm fully in line with your last recap.

As it's already been mentioned here, imho one thing that each description should really clarify (including the one from PROJ), is that this gimbal scanning geometry we are describing for the geostationary projection is independent of the actual scanning geometry of the actual instrument in orbit. It rather describes how the image is aquired by a fictitious, simplified, two-axis gimbal imaging instrument in an idealized geostationary position.

Elaborating on this, for future newbies like me delving into this issue:
Higher-lever geostationary imagery (e.g. SEVIRI L1.5, FCI L1c, and GOES-R L1b) is rectified to a specified regular azimuth/elevation grid. With this, each image pixel has a pair of azimuth and elevation angles, that define how this idealized gimbal needs to be rotated to reach that specific pointing towards Earth. The sweep-angle axis describes which convention is used to define these azimuth/elevation rotations (either the "standing" gimbal as in the image above, or a turned-over version). The origin of the reference frame used to define these angles (i.e. the center of the gimbal and the intersection of the two rotation axes), is fixed and placed on an ideal satellite position. With this, each pixel of the rectified image is always at the same location on Earth, independently of the instrument scanning pattern, and the wobbling and drifting of the satellite in the non-perfect and perturbed geostationary orbit. This ideal satellite position is specified by the other variables of the geostationary projection.

@rhorne99
Copy link

rhorne99 commented Apr 14, 2020 via email

@erget erget mentioned this issue Apr 21, 2020
Merged
@erget
Copy link
Member Author

erget commented Apr 29, 2020

Dear geostationary friends,

I've now completed a proposal text, found in #259 . It reflects the results of this discussion and would be happy to discuss your approval or criticism.

At this note, now that we have a concrete proposal on the table, who would be willing to moderate this discussion?

Cheers,
Daniel

@erget
Copy link
Member Author

erget commented May 6, 2020

@JimBiardCics you've chimed in a few times on this discussion - would you be willing to moderate the proposal? We now have a concrete text on the table for potential approval.

@erget
Copy link
Member Author

erget commented May 7, 2020

@JonathanGregory I think @JimBiardCics might be out of office - would you be interested in moderating this discussion, now that the proposed text change is on the table?

@JonathanGregory
Copy link
Contributor

Yes I could try to moderate it. I'd certainly be a neutral moderator since I'm quite ignorant of this! I've just read it and I have a few questions of clarification:

  • You have a typo thsi.

  • It would be helpful to provide a diagram to show the two circles and the two angles. The text description is pretty good, but a diagram would be useful too. I haven't really grasped why one axis is called sweep and one fixed, and why they aren't the other way round, since the fixed-angle axis is the one which moves and the sweep-angle axis is stationary. I might understand with the aid of a diagram.

  • Adding the Earth major axis to the perspective point height gives the distance to the centre of the Earth only if you're above the equator, I think. Otherwise a different radius applies.

  • I don't understand sweep_angle_axis. Couldn't this axis be pointing in any direction (in the rotating frame of the Earth)? How do we interpret x and y? Could it be z? Why not some arbitrary direction?

  • For fixed_angle_axis, what does "opposite" mean?

Everyone is invited to comment on the proposal text in #259. Please post your comments here and not in the pull request (following our GitHub guidelines). Thanks.

@erget
Copy link
Member Author

erget commented May 7, 2020

Thanks @JonathanGregory for jumping in! Good points, I will address these before the weekend.

@erget
Copy link
Member Author

erget commented May 8, 2020

Hi @JonathanGregory , answers inline below.

Yes I could try to moderate it. I'd certainly be a neutral moderator since I'm quite ignorant of this! I've just read it and I have a few questions of clarification:

  • You have a typo thsi.

Fixed in ee79789.

  • It would be helpful to provide a diagram to show the two circles and the two angles. The text description is pretty good, but a diagram would be useful too. I haven't really grasped why one axis is called sweep and one fixed, and why they aren't the other way round, since the fixed-angle axis is the one which moves and the sweep-angle axis is stationary. I might understand with the aid of a diagram.

Excellent question - I also think that the terms are confusing, and don't know anybody who thinks that they're straightforward. Even a diagram doesn't really explain this for me; it seems that one must simply memorise it.

I did consider inserting a diagram but in the whole of the CF Conventions there are only tables and code samples! That is the reason that in the proposed text I link to further explanatory information and a nice diagram on the PROJ website (in fact I stole the diagram of choice and used it in a previous comment).

In order to keep the document's style consistent I would prefer to use this approach, as the linked document is short and explains it as well as one can expect. It would be a small matter to introduce a diagram (or to steal the existing one) but would be a departure from the document's current style. Would you agree to leaving that part of the text as-is, sans diagram?

  • Adding the Earth major axis to the perspective point height gives the distance to the centre of the Earth only if you're above the equator, I think. Otherwise a different radius applies.

This is correct. I've struck that line in a471a2e - as we have not limited this grid mapping to views over the equator thus far, it would potentially break backwards compatibility to do that now, and the explanation becomes clunky if we try to account for all cases.

  • I don't understand sweep_angle_axis. Couldn't this axis be pointing in any direction (in the rotating frame of the Earth)? How do we interpret x and y? Could it be z? Why not some arbitrary direction?

Yes. Taken by themselves, x and y are ambiguous, although there is no z here. That's why in the proposed text I've narrowed it down to align x and y with the Earth's axes. This wasn't specified before but the grid mapping would never have worked otherwise because there's no way to specify them, so the assumption was already implicit. Thus I think that this does not break backwards compatibility; it simply fills a gap in the description.

  • For fixed_angle_axis, what does "opposite" mean?

Meaning if one of the axes is x, the other must be y, and vice versa. I've clarified this in c54b94e.

Everyone is invited to comment on the proposal text in #259. Please post your comments here and not in the pull request (following our GitHub guidelines). Thanks.

@JonathanGregory
Copy link
Contributor

Dear Daniel @erget

Thanks. Now I understand much better. That leads to some more questions, but they're at the next order of small quantities.

  • Since you too find the terms not obvious, maybe it would help the reader to acknowledge this, for instance by appending a sentence after "so that order of operations is essential in achieving accurate results", such as "(The two axes are conventionally called the sweep-angle and fixed-angle axes; we adhere to this terminology, although many find these terms rather confusing.)" Then at least the reader won't be surprised to be confused.

  • The diagram in the PROJ document is helpful and it's fine to leave it there. I didn't look at it the first time because it sounded as though it was just about the software. Could you rephrase this sentence as "Explanatory diagrams for the projection may be found on the PROJ website, as well as notes on using the PROJ software for computing the mapping"?

  • The sweep_angle_axis and fixed_angle_axis attributes should be added to Table F.1; perspective_point_height is already there. In the table, you can state their two possible values and that each is optional if the other is present.

  • In general CF attributes with a prescribed list of possible values are regarded as case-insensitive, for ease of use of the convention, since insisting on either upper or lower case will probably mean some people will write incorrect files. I think we may have a general statement about that somewhere. If your x and y are case-insensitive, it would be helpful to say so.

Cheers

Jonathan

@JonathanGregory
Copy link
Contributor

I should just clarify that not all attributes are case-insensitive; units and standard_name are case-sensitive, for instance.

@erget
Copy link
Member Author

erget commented May 11, 2020 •
edited
Loading

@JonathanGregory thanks for these improvements!

Since you too find the terms not obvious, maybe it would help the reader to acknowledge this, for instance by appending a sentence after "so that order of operations is essential in achieving accurate results", such as "(The two axes are conventionally called the sweep-angle and fixed-angle axes; we adhere to this terminology, although many find these terms rather confusing.)" Then at least the reader won't be surprised to be confused.

I've phrased this as follows in cfb37b1:

The two axes are conventionally called the sweep-angle and fixed-angle axes; we adhere to this terminology, although some find these terms confusing, for the sake of interoperability with existing implementations.

The diagram in the PROJ document is helpful and it's fine to leave it there. I didn't look at it the first time because it sounded as though it was just about the software. Could you rephrase this sentence as "Explanatory diagrams for the projection may be found on the PROJ website, as well as notes on using the PROJ software for computing the mapping"?

Done in 0fa8f39.

The sweep_angle_axis and fixed_angle_axis attributes should be added to Table F.1; perspective_point_height is already there. In the table, you can state their two possible values and that each is optional if the other is present.

Done in dd75623.

In general CF attributes with a prescribed list of possible values are regarded as case-insensitive, for ease of use of the convention, since insisting on either upper or lower case will probably mean some people will write incorrect files. I think we may have a general statement about that somewhere. If your x and y are case-insensitive, it would be helpful to say so.

Done in 9511395.

Best regards,
Daniel

@davidhassell davidhassell mentioned this issue May 11, 2020
Closed
@TomLav
Copy link

TomLav commented May 11, 2020

Hei @erget and all,

(reposting here instead of #259 , sorry for the mess):

I happen to have spent the week-end on this projection, browsing through the history, and trying to understand what we were encoding in the CF object. I received great help from the Pytroll community.

I propose a small clarification (against the current 1.9 doc) to:

In the case of this projection, the projection coordinates in this projection are directly related to the scanning angle of the satellite instrument.

First, we have "this projection" twice.

Second, "related to the scanning angle". Aren't they the scanning angle themselves, can we drop "related to"?

Third, it seems to me that this projection introduces a mismatch between what is stored in the CF file (the angles in radians) and what "mainstream" projection libraries (Proj) use (distance from origin in meters). So that the wording "the projection coordinates" could be confusing.

Thus, I humbly suggest:

In the case of this projection, the x and y coordinates stored in the file are directly the scanning angles of the satellite instrument.

I also suggest a sentence to be added in the bullet points, to warn the non-expert user, along the lines of:

Note that the CF convention stores the scanning angles (with canonical unit gradients) as the coordinates for this projection, while the popular library PROJ uses distance from origin (meters).

I understand the implementation in PROJ shouldn't be driving the CF convention (and it didn't), but I feel we could be explicit for the non-expert users (after all, AFAIK, the geostationary projection is the only one in CF where CF stores something different that what PROJ returns/expects).

@erget
Copy link
Member Author

erget commented May 12, 2020

Hej @TomLav ,

No worries about the "mess" - it's no bother to anyone and every community has its own way of doing things, so we're all located somewhere along the same learning curve ;)

Concerning the clarifications (1) and (2) you propose, I've implemented some changes in bc93215.

Concerning (3) (units) I am not familiar with any products that actually use distance from origin. Do you have an example? We had some discussion in the past about this because the units used in CF implied that meters were used, whereas in fact the units were angular. This is the reason why projection_x_coordinate is allowed, but deprecated - using that attribute would signify linear distance, whereas projection_x_angular_coordinate signifies angular distance.

In any case my opinion is that there's enough text concerning angular vs linear coordinates in the new draft that we risk beating a dead horse by adding more on this topic. Having reviewed the PROJ docs, I also don't see the use of linear distance from origin, but I am happy to be corrected on this if I have overlooked something.

@TomLav TomLav mentioned this issue May 12, 2020
Open
@JonathanGregory
Copy link
Contributor

@TomLav has moved the point he made to cf-convention/discuss#49. Hence it is no longer an unanswered question on this issue. I believe that there are no outstanding questions at the moment.

@erget erget linked a pull request May 15, 2020 that will close this issue
Clarify geostationary projection items #259
Merged
@JonathanGregory
Copy link
Contributor

This enhancement (Update geostationary projection) will be agreed tomorrow (Tue 2nd June) if no further difficulties are raised.

@JonathanGregory
Copy link
Contributor

This enhancement (update geostationary projection) is now agreed. Thank you, Daniel @erget. The conformance document is unaffected, and Daniel is already listed as an author, but I think the history.adoc needs a comment about this issue, which I don't see in the pull request - is it there? If I'm correct about that, please could you add it to the PR, which I will then merge. Thanks.

@erget
Copy link
Member Author

erget commented Jun 2, 2020

@JonathanGregory thanks a bunch, I've updated the PR to include changes to history.adoc and resolved all merge conflicts so that it can be merged to master.

@JonathanGregory
Copy link
Contributor

Thanks @erget. I have merged the PR. That concludes this issue. I don't know about deleting the source branch. I presume that's your own private affair. I was following your helpful instructions in CONTRIBUTING.md. 😄

@erget
Copy link
Member Author

erget commented Jun 2, 2020

@JonathanGregory 😆 yeah at the time that I inserted that update I wasn't sure if we'd be using the most GitHub-esque way of doing Pull Requests, which is via forks of repositories, as we're doing now. In such cases that falls to the proposer, but some groups prefer workflows that use the same repository to hold all branches. It's a cultural issue and I feel that we've established we use the multi-repository workflow in CF so that's an item that should be updated in CONTRIBUTING! I hope the rest of the instructions were helpful nonetheless :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@JonathanGregory JonathanGregory

Labels
enhancement Proposals to add new capabilities, improve existing ones in the conventions, improve style or format
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Clarify geostationary projection items
8 participants
@erget @steingod @JonathanGregory @TomLav @JimBiardCics @cf-metadata-list @ameraner @rhorne99