Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement crypto byte array newtypes in term of a shared type #3684

Merged
merged 30 commits into from
Mar 19, 2024

Conversation

davxy
Copy link
Member

@davxy davxy commented Mar 13, 2024

Introduces CryptoBytes type defined as:

pub struct CryptoBytes<const N: usize, Tag = ()>(pub [u8; N], PhantomData<fn() -> Tag>);

The type implements a bunch of methods and traits which are typically expected from a byte array newtype
(NOTE: some of the methods and trait implementations IMO are a bit redundant, but I decided to maintain them all to not change too much stuff in this PR)

It also introduces two (generic) typical consumers of CryptoBytes: PublicBytes and SignatureBytes.

pub struct PublicTag;
pub PublicBytes<const N: usize, CryptoTag> = CryptoBytes<N, (PublicTag, CryptoTag)>;

pub struct SignatureTag;
pub SignatureBytes<const N: usize, CryptoTag> = CryptoBytes<N, (SignatureTag, CryptoTag)>;

Both of them use a tag to differentiate the two types at a higher level. Downstream specializations will further specialize using a dedicated crypto tag. For example in ECDSA:

pub struct EcdsaTag;

pub type Public = PublicBytes<PUBLIC_KEY_SERIALIZED_SIZE, EcdsaTag>;
pub type Signature = PublicBytes<PUBLIC_KEY_SERIALIZED_SIZE, EcdsaTag>;

Overall we have a cleaner and most importantly consistent code for all the types involved

All these details are opaque to the end user which can use Public and Signature for the cryptos as before

@davxy davxy added R0-silent Changes should not be mentioned in any release notes I4-refactor Code needs refactoring. labels Mar 13, 2024
@davxy davxy self-assigned this Mar 13, 2024
@paritytech paritytech deleted a comment from paritytech-cicd-pr Mar 13, 2024
@paritytech paritytech deleted a comment from paritytech-cicd-pr Mar 13, 2024
@paritytech paritytech deleted a comment from paritytech-cicd-pr Mar 15, 2024
@paritytech paritytech deleted a comment from paritytech-cicd-pr Mar 15, 2024
@paritytech paritytech deleted a comment from paritytech-cicd-pr Mar 15, 2024
@paritytech paritytech deleted a comment from paritytech-cicd-pr Mar 15, 2024
@davxy davxy marked this pull request as ready for review March 15, 2024 16:34
@davxy davxy requested review from athei, a team and koute as code owners March 15, 2024 16:34
@davxy davxy mentioned this pull request Mar 17, 2024
Copy link
Contributor

@michalkucharczyk michalkucharczyk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! like the idea.

}
}

impl<const N: usize, T> UncheckedFrom<[u8; N]> for CryptoBytes<N, T> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: do we need unchecked_from?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that the original intent of UncheckedFrom was to make explicit the fact that the construction of the object is performed without actually validating the content (e.g. in case of EC based crypto that the value represents a valid point on the used curve).

While this may have sense, should be said that the same types were exposing constructors like CryptoBytes::from_raw([u8; N]) or ByteArray::from_raw([u8; N]) where the docs explicitly states that the check is not performed.

IMHO may be sufficient to document that the constructors from a raw byte array doesn't ensure validity of the bytes when these are then tried to be deserialized into the higher level object.

What could go wrong if you use a bad byte array:I've not removed it yet (as other traits and redundant stuff) to keep this PR as minimal as possible

  • Signatures: nothing. The verification simply fails.
  • Public keys: if used to verify something. It will fail.

So, in both cases, the failure is postponed to the usage stage instead of the construction phase.

May instead have sense to remove the From<[u8; 32]>? And keep UncheckedFrom only?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May instead have sense to remove the From<[u8; 32]>? And keep UncheckedFrom only?

I see your point, it makes sense. Being explicit here is better. On the other hand from is tempting as it is shorter.

But still it is a nit, cleanup could be done in some follow-up.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May instead have sense to remove the From<[u8; 32]>? And keep UncheckedFrom only?

Hmmm... it could make sense, but I guess maybe only if we remove all of the other easy ways to make a T in an unchecked fashion?

Also, in my refactoring PR I'm using these From impls to convert between raw arrays when passing these types through the FFI boundary between the runtime and the host. So if we remove UncheckedFrom I'll have to add another wrapper strategy type just for passing these using the UncheckedFrom. Not a big deal, but it is extra code that will have to be added.

Copy link
Member Author

@davxy davxy Mar 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd also like to add that the wrapped type is (and was) public. So maybe having this "UncheckedFrom" doesn't really add anything really valuable? Mostly syntactic sugar.

As said I leave it untouched for now

pub struct SignatureTag;

/// Generic encoded signature.
pub type SignatureBytes<const N: usize, SubTag> = CryptoBytes<N, (SignatureTag, SubTag)>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: it is all mostly wrappers, but maybe you could add some tests? However I am not 100% they are actually necessary. Feel free to ignore.

Copy link
Contributor

@koute koute left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea!

substrate/primitives/core/src/crypto_bytes.rs Outdated Show resolved Hide resolved
substrate/primitives/core/src/crypto_bytes.rs Show resolved Hide resolved
substrate/primitives/core/src/sr25519.rs Outdated Show resolved Hide resolved
substrate/primitives/core/src/crypto_bytes.rs Show resolved Hide resolved
///
/// The tag `T` is held in a `PhantomData<fn() ->T>`, a trick allowing
/// `CryptoBytes` to be `Send` and `Sync` regardless of `T` properties
/// ([ref](https://doc.rust-lang.org/nomicon/phantom-data.html#table-of-phantomdata-patterns)).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice :)

@davxy davxy added this pull request to the merge queue Mar 19, 2024
Merged via the queue into master with commit 1e9fd23 Mar 19, 2024
133 of 134 checks passed
@davxy davxy deleted the davxy-byte-array-refactory branch March 19, 2024 16:57
dharjeezy pushed a commit to dharjeezy/polkadot-sdk that referenced this pull request Mar 24, 2024
…tech#3684)

Introduces `CryptoBytes` type defined as:

```rust
pub struct CryptoBytes<const N: usize, Tag = ()>(pub [u8; N], PhantomData<fn() -> Tag>);
```

The type implements a bunch of methods and traits which are typically
expected from a byte array newtype
(NOTE: some of the methods and trait implementations IMO are a bit
redundant, but I decided to maintain them all to not change too much
stuff in this PR)

It also introduces two (generic) typical consumers of `CryptoBytes`:
`PublicBytes` and `SignatureBytes`.

```rust
pub struct PublicTag;
pub PublicBytes<const N: usize, CryptoTag> = CryptoBytes<N, (PublicTag, CryptoTag)>;

pub struct SignatureTag;
pub SignatureBytes<const N: usize, CryptoTag> = CryptoBytes<N, (SignatureTag, CryptoTag)>;
```

Both of them use a tag to differentiate the two types at a higher level.
Downstream specializations will further specialize using a dedicated
crypto tag. For example in ECDSA:


```rust
pub struct EcdsaTag;

pub type Public = PublicBytes<PUBLIC_KEY_SERIALIZED_SIZE, EcdsaTag>;
pub type Signature = PublicBytes<PUBLIC_KEY_SERIALIZED_SIZE, EcdsaTag>;
```

Overall we have a cleaner and most importantly **consistent** code for
all the types involved

All these details are opaque to the end user which can use `Public` and
`Signature` for the cryptos as before
github-merge-queue bot pushed a commit that referenced this pull request Mar 25, 2024
…ature Bytes (#3806)

Another simple refactory to prune some duplicate code

Follow up of: #3684
dharjeezy pushed a commit to dharjeezy/polkadot-sdk that referenced this pull request Apr 9, 2024
…ature Bytes (paritytech#3806)

Another simple refactory to prune some duplicate code

Follow up of: paritytech#3684
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
I4-refactor Code needs refactoring. R0-silent Changes should not be mentioned in any release notes
Projects
Status: done
Development

Successfully merging this pull request may close these issues.

5 participants