Support for non-NOR (or subtly different) storage styles

[chrysn]

Currently, embedded-storage contains the NOR traits and the byte-addressed Storage (which is risky to use as it has unbounded RMW impact on system failure).

I'd like to track the need and requirements for other storage kinds in this issue:

• Block storage := storage with a single page size that can be overwritten arbitrarily.

  This can be expressed as NorFlash (by setting WRITE_SIZE = ERASE_SIZE), but that is not ideal because a) it necessitates an extra erase call, and b) it may amplify writes to the backend due to the requriement that erased areas are all ones.

  If we prefer not to have a dedicated trait, these downsides could also mitigated by having a write_page function that acts on the erase size, is provided to run an erase and a page write, but may be implemented more efficiently by just issuing an "overwrite page" command. Such implementations may still need to costily implement the erase command with a overwrite-with-all-ones, but users that "think" block-wise (or users with highly generic code that detect that WRITE_SIZE = ERASE_SIZE) can avoid calling that ever.

  SD cards fall in this category.

• Byte storage := block storage with a size of 1.

  This could freeload on whatever we do with block storage.

  EEPROM falls in this category.

• Flash memories with limited multi-write.

  Some chips such as the EFR32XG1 allow words to be written only a limited number of times (eg. twice per 16-bit word as in EFR32XG1, nWRITE=2 times per 32-bit word as in nRF5340 and nRF52840, or nWRITE,BLOCK=181 times per erase page as in the nRF52832)

  This could be expressed in a parametrized NorFlashLimitedMultiwrite trait; the downside being that using that trait on a MultiwriteNorFlash would likely mean going through a newtype: while all arbitrarily multiwrite flashes do satisfy any NorFlashLimitedMultiwrite constraints, chances are nobody will impl<const NWRITE: usize> NorFlashLimitedMultiwrite<NWRITE> for TheirType {}.

• Flash memories with error correction.

  While those look like they fit the NOR API, there are two stages of not fitting in:

  • Some (like STM32H5) have relatively transparent error handling.

    These can be expressed with the NOR API, but can't do MultiwriteNorFlash (which is convenient because they couldn't uphold its guarantees in the power loss case).

    They might profit from extra APIs that indicate ECC failure, but can reasonably implement the regular interface by either panicking (it's an error the user has not anticipated) or even hard faulting (which AIU is the default behavior unless ECC errors are handled).

  • Some (like LPC55xx as explained in their knowledge base) have an erase state that is not ffffffff. It may be possible for those to emulate the NOR flash API, but only at relatively high cost of doing extra checks for every read.

    As the data sheet UM11126 is unavailable and none of the descriptions are fully clear, I'd hold off proposing traits for that until anyone around has actually worked with this flash to confirm.

• Memories that are known to go bad (even when not being written to during power loss), and need bad blocks handling.

  Never worked with those so can't tell.

[Sympatron]

Byte storages like EEPROMs can just implement Storage directly. You don't have to jump through hoops like with NOR flashes there.

Block storage might warrant a new trait IMO, but I don't know what the methods and especially their semantics and guarantees should look like.

Regarding flash memories with limited multi-write: I think the actual flash storage driver implementation could return an error if the maximum amount of writes is reached. We could add a NorFlashErrorKind::TooManyWrites for this case. We could also add a method write_limit() -> Option<u32> { None } to MultiwriteNorFlash so consumers of MultiwriteNorFlash could try to respect those limits.

We could also add a NorFlashErrorKind::EccFailed kind for flash with error correction.

I agree that cases like the LPC55xx should be ignored until someone has an actual use case for them.

I don't know any example of "Memories that are known to go bad". What kind of memory technology do you mean?

[chrysn]

When EEPROMs use Storage, they don't pass along their guarantee that a write during power loss will only affect the current byte. A consumer (eg. file system) would need to demand from the type system that the input is a Storage, and from the provider (eg. the developer assembling the components) that the storage has an error behavior limited to the bytes being written -- not an unheard of design, but I hope we can do better.

Excessive multiwrites do not necessarily result in a detectable error. I don't have test data to prove it, and the data sheets just say "it's not allowed" without telling what can go wrong, but I expect that at least the word being written can have bit flips, possibly even the whole erase unit, so I don't think EAFTP will cut it here. The author of the file system / database / config store will need that information to make the right choices, and ideally at compile time. A const MAX_WRITES_PER_WRITE_UNIT: usize; would be more usable than a write_limit function and could be added to the MultiwriteNorFlash trait with a default of usize::MAX.

EccFailed sounds like a good addition -- I completely missed that even reads are already fallible.

As for memories that are known to go bad: NAND flash came up in recent matrix discussions as an example of a memory where the user needs to know how it typically fails in order to set up the right badblocks strategy.

[Sympatron]

I believe there are also page based EEPROMs so I don't know if we can assume all EEPROMs to have this guarantee of only writing one byte at a time. That being said I think what you are missing is a trait representing a block storage device that has the same guarantees as the NorFlash trait but without requiring an erase to set 0 to 1. Is that correct? Would this trait even need an erase? And if it does, what state would the data be guaranteed to be afterwards?

Limited multiwrites could also be expressed with an associated const, but I would prefer it to be Option<usize> personally to make the distinction between limited and unlimited writes more clear.

I don't have any experience with NAND so I can't really comment on this. Maybe this should wait for a specific use case?

[chrysn]

Discussion on RIOT matrix just showed another case of non-0xff reset state (similar to the LPC5xx) in the STM32L0, and a user of that is running into trouble with a flash storage database.