Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unbox first level of options when serializing JSON #598

Merged
merged 8 commits into from
Jul 11, 2024
Merged

Conversation

lihaoyi
Copy link
Member

@lihaoyi lihaoyi commented Jul 5, 2024

Fixes #528

This PR makes Option[T]s serialize to null or unboxed ts by default. Together with the default configuration of serializeDefaults = false, this allows None fields on case classes to be omitted when serializing case classes. Even with serializeDefaults = true, it still serializes Option[T] as null or unboxed t. Either way is much more in line with standard REST API and JSON schema practices than the current serialization of Option[T] as zero-or-one-element-arrays [] or [t]

Before After
None [] null (or nothing, if a case class field with serializeDefaults = false
Some(1) [1] 1
Some(None) [[]] [null]
Some(Some(1)) [[1]] [1]
etc.

Despite the convenience, this does mean that there are certain data structures that do not get round tripped. e.g. a field null: Option[T] would get serialized to null, and deserialized as None. This is a tradeoff, but given the rarity of nulls in Scala codebases, and the intuitive expectations of how Options should behave, it seems a reasonable tradeoff.

This PR does make an effort to support nested options: Some(None) is serialized as [null], while Some(Some(t)) is serialized as [t]. This manual boxing allows nested Options to be preserved during round-trip read/write, rather than being flattened out to a single top-level None. These nested options typically do not appear in REST APIs or JSON schemas, and so the choice to preserve round-trip-ability should not affect compatibility with public APIs

This is a breaking change that will need to go into uPickle 4.x. For backwards compatibility, and for migration purposes, the new serialization format is controlled under a flag optionsAsNulls = true. Users who really need the full round-trip preservation of Scala data structures, or who want to preserve compatibility with existing systems, can create a custom config with optionsAsNulls = false. Given the change in the serialization format, I haven't found a way to make uPickle read both old and new formats during the transition, but users can continue to use uPickle 4.x with optionsAsNulls = false indefinitely if they want to preserve compatibility with the old serialization format

For users that want to enable serializeDefaults = true, we should be able to allow serializeDefaults to be configurable on a per-field basis to allow it to be disabled just for the fields typed as Options, to continue eliding them while serializing other default values. Doing that can be done as a follow up

@lihaoyi lihaoyi merged commit bb0430b into main Jul 11, 2024
8 checks passed
@lihaoyi lihaoyi deleted the unboxed-options branch July 11, 2024 03:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make Option[T] in case classes un-boxed by default
2 participants