-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support OME-zarr labels layers #45
Comments
Another example of showing / hiding / coloring labels: https://downloads.openmicroscopy.org/presentations/2020/community-call-2020-10-29/OMERO%20Plus,%20PathViewer%20and%20NGFF%202020-10-27.mp4 |
Ah, I wasn't aware that the labels data were dense arrays. What are the non-zero elements for the labels (value wise)? I think we should be able to reuse the loader modules (to fetch data tiles), but I think transparaent for 0 values will require a new layer (or shaders at minimum) perhaps @ilan-gold has some input. |
I think we use a deck.gl polygon layer for the segmentation in vitessce because the data are in a columnar format (not dense pixel arrays). Both @keller-mark and @ilan-gold have more experience with writing custom layers in deck.gl, and hopefully they have some ideas of how to do something similar for dense array masks With that said, I like the idea of having different layers because then we can render UI elements based on different |
Yes we use selectable polygon layers from deck.gl which requires pre-processing the data to get polygons but likely will not require any sort of pyramid/multi-resolution handling. We could certainly write shaders to handle this concept of value-per-color, raster/pixel-based-storage, segmentation mask, especially if this is the direction OMERO is headed - it would be very straightforward to do and could scale very well to lots of different labels if what I have in my head is right for the implementation. I agree @manzt that we should be able to reuse the loaders for it - we would change the layer (or really just the shaders) used by |
See Will's link above (https://github.com/ome/omero-ms-zarr/blob/master/spec.md#image-label-metadata). The OME-Zarr specification uses (dense) zarr arrays for masks. |
Ilan and I talked about this today and I think we have an idea for implementation (likely a new layer with custom shaders). I would be useful to have an example dataset with these arrays to work with. I can't seem to find the an image with "image-label" metadata? Is this different from "masks"? I've tried opening the examples in ome/ome-zarr-py#30 with the $ napari https://s3.embassy.ebi.ac.uk/idr/zarr/v0.1/6001240.zarr # just opens image layers |
Yeah, I had to ask around to find an image with labels. (we are planning a directory page to list all our ome-zarr images soon).
The presence of labels is specified by the existence of https://s3.embassy.ebi.ac.uk/idr/zarr/v0.1/6001247.zarr/labels/.zattrs Labels are multi-resolution arrays (same as pixel data) but tiles may be missing if they are all zeros to save space when you have sparsely-populated labels. |
Hey @manzt |
Label layer support still requires shader changes. With Viv migrating to deck.gl 9, contributing directly to Vizarr might not be ideal due to potential breaking changes. I'm focused on wrapping up my PhD and may not have the time to push this feature forward, but I'm happy to support the work. That said, I'd say it's better to focus on a feature like this after we've been able to upgrade Viv/Deck.gl. |
Hey @manzt The major question for me is: How large of an undertaking do you expect adding such label layer shader changes would be? Any pointers & support from your side on how to best go about it would be appreciated. And thanks also for the context on the upcoming migration. Is there a potential timeline for the Viv/deck.gl change? Is that a major tricky migration or something coming in the next few weeks to months? I'd be happy to coordinate with you and @zonia3000 from our team on how we could contribute to the label layer shader work if that makes sense. Sonia has already been looking into the vizarr code a little bit in the context of enabling us to use the vizarr viewer in the Fractal platform (see https://github.com/fractal-analytics-platform/fractal-vizarr-viewer). It has been impressive to see what vizarr can do out-of-the-box! 👏🏻 |
It will happen within the next two months or so, I imagine, but it's tough to say. We discovered bugs in their codebase and may continue to do so. So this means we can't just do it so quickly.
It should not be too hard; for an example, see vitessce |
Thanks @ilan-gold that's very useful to hear. Fingers crossed for the deck.gl change then! 🤞🏻 |
Hi there, if we understand correctly the deck.gl 9 update for viv has progressed. Would it be a good time for resuming the discussion about labels (cc @manzt)? We at the Fractal team (cc @zonia3000 @jluethi) would be interested in exploring this issue. We understand the basic structure of OME-Zarr images, and we have example data with labels (on top of the many public examples), but we would likely need some advice/support re: integration in vizarr. In the current discussion, the most specific plan seems to be #45 (comment) by @ilan-gold - is this something relevant at this point? It'd be great if we could get some more details. To get started, we would try to implement the core functionality only - namely we would postpone:
Any feedback is welcome - thank you! |
Thanks for keeping the conversation going and for opening this thread! My PhD defense is scheduled for next week, so I won’t be able to contribute actively until after that. I appreciate your understanding and look forward to diving into this discussion once I’m available. |
No worries, and all the best for your PhD defense! |
It would be great to support ome-zarr labels data.
This is a zarr image where values of 0 are transparent (no label) and colour of each label may be specified by metadata:
https://github.com/ome/omero-ms-zarr/blob/master/spec.md#image-label-metadata
I can imagine that loading the zarr data into new layers won't need much new code (happy to have a stab at this), but I don't know how to render particular pixel values with pre-defined colours?
It's important that the labels layer is transparent so that the underlying image is visible, so I believe they would have to be part of the same Deck.gl layer?
See ome-zarr labels support in napari, added via ome-zarr-py plugin: ome/ome-zarr-py#30
These labels layers may be part of the same ome-zarr that contains image layers (see spec) but it's possible that you might load image layers from one source and labels from another.
Having just been reading #32, I can imagine that an API for showing/hiding or setting colors on masks would be very cool. Similar to what you're doing in vitessce http://vitessce.io/?dataset=linnarsson-2018
cc @joshmoore
The text was updated successfully, but these errors were encountered: