Last row/column fails when writing chunks

[niklasmueboe]

I have an issue when generating Arrays where the last column or row does not fill the full chunk size.

As an example (mostly taken from the documentation directly)

let store = Arc::new(FilesystemStore::new("/path/to/store")?);

let array = ArrayBuilder::new(
    vec![3, 4], // array shape
    DataType::Float32,
    vec![2, 2].try_into()?, // regular chunk shape
    FillValue::from(0),
)
.build(store.clone(), "/array")?;
array.store_metadata()?;

// this works
array.store_chunk_ndarray::<f32, _>(&[0, 0], ndarray::array![[-1.1, -1.2], [-2.1, -2.2]])?;

// this fails
array.store_chunk_ndarray::<f32, _>(&[1, 1], ndarray::array![[-0.1, -0.2]])?;

The Chunk with Index (1,1) should only have shape (1,2) in this case. However when providing a correctly sized array the operation fails with the error that a (2,2) array was expected.

Or did I misunderstand how to assign into chunks directly?

[LDeakin]

With a regular chunk grid, all chunks must have the same size, even those that extend beyond the end of the array.

There are a few options here, but just always using store_array_subset_ndarray is probably easiest. I'll improve the docs for this.

[LDeakin]

FYI this is something that has irked me in the spec of Zarr V3, because it is a waste of bytes. I would like to have a regular-like chunk grid where the straddling chunks can be smaller.

A rectangular chunk grid can support this but it is overcomplicated for this common use case, and AFAIK is not supported by other implementations yet.

[niklasmueboe]

Ok, thanks for the clear up.

I'll give the subset approach a shot.

[LDeakin]

I've added some additional docs in #89; I hope that helps. store_chunk_subset_ndarray is also relevant to your specific example.