-
Notifications
You must be signed in to change notification settings - Fork 590
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tracking: jsonb operations #7714
Comments
…7977) Part of the `jsonb` type support (preview all on [this branch](https://github.com/risingwavelabs/risingwave/compare/XJ-jsonb-WIP-2?expand=1)): * introduce `Scalar` and `ScalarRef` **(this PR)** * Introduce `ArrayBuilder` and `Array` * Introduce `DataType::Jsonb` * Add more expressions #7714 `serde_json` is chosen instead of `simd_json` for the following reasons: * Better interoperability with other libraries (e.g. `postgres_types`). * Despite the name `BorrowedValue`, it is not `Copy` and not suitable for `JsonbRef`. * `simd_json` uses `HashMap` but `serde_json` defaults to `BTreeMap`, which has a deterministic ordering and string representation. Approved-By: TennyZhuang
Part of the `jsonb` type support (preview all on [this branch](https://github.com/risingwavelabs/risingwave/compare/XJ-jsonb-WIP-2?expand=1)): * #7977 * Introduce `ArrayBuilder` and `Array` **(this PR)** * Introduce `DataType::Jsonb` * Add more expressions #7714 In this PR: * The in-memory layout of `JsonbArray` is simply a `Vec` or roots of json trees. This is easier to operate but do have space for optimization. * The protobuf layout is the same as `bytea` variable length array, with each element storing its value encoding. In case we switch to a newer protobuf layout, a new ArrayType enum value can be introduced without affecting parts other than `to_protobuf` and `from_protobuf`. * Refactored `VarSizedValueReader` from returning `RefItem` into accepting `&mut ArrayBuilder`. This is to use `JsonbArrayBuilder::append_move` on the deserialized `OwnedItem`. It is impossible to get a `JsonbArray::RefItem` from `&[u8]`. * Blanket implementation for `arrow` / `HashKeySerDe` / `RandValue`. Approved-By: BugenZhao
Part of the `jsonb` type support: * #7977 * #7986 * Introduce `DataType::Jsonb` **(this PR)** * Add more expressions #7714 In this PR: * Add `DataType::Jsonb`. * Support constructing it from string and displaying it as string. * Add e2e tests for `insert` and `select`. Also tested for the following but not added in CI: * prepared statement with BINARY format * kafka+json source with a `jsonb` field Approved-By: BugenZhao
Removing the milestone as the remaining operators are optional / long-term. |
Also, We might need another expression to optimize the |
Opened a dedicated issue: #17617 |
What's still missing? Looks mostly finished |
PostgreSQL 16 (and maybe the upcoming 17) is still actively adding more JSON expressions. But I am okay with either close this or keep it open. We have no plan to add those proactively in the near future. |
Listing all jsonb operations from PostgreSQL, and which ones to implement now, soon, or until requested.
Basic
General for all types:
IS [NOT] NULL
/CASE WHEN
/COALESCE
/ etcEq / Hash / Ord
IS [NOT] DISTINCT
/NULLIF
/IN
/ etcCast to simple types
to_jsonb
.Accessing array / object
-> int
/-> varchar
: to access member asjsonb
or cast to bool / number->> int
/->> varchar
: to access member as string. (Note that cast to string is DIFFERENT. See example below.)#> varchar[]
/#>> varchar[]
feat(expr): support#>
and#>>
operator for extracting jsonb at a path #13110jsonb_extract_path
/jsonb_extract_path_text
: feat(expr): supportjsonb_extract_path(_text)
function #13143Informative / Debugging
jsonb_array_length
jsonb_pretty
feat(expr): addjsonb_pretty
function #13050jsonb_typeof
Predicate
?
/?|
/?&
: object key lookup@>
/<@
: containment feat(expr): support jsonb@>
,<@
,?
,?|
and?&
operator #13056Construction / Mutation
jsonb || jsonb
→jsonb
#12180-
array index /-
object key (also array string element) /-
object keys /#-
path feat(expr): support jsonb-
and#-
operator #13118jsonb_strip_nulls
to_jsonb
#12834: basis of many operations below.jsonb_build_array
/jsonb_build_object
#13016jsonb_object
: only when values are also strings. feat(expr): addjsonb_object
function #13036Subscripting
j[index]
/j[key]
and use#>
/#>>
/->
/->>
instead.Aggregation
bool_and
,bool_or
,jsonb_agg
,jsonb_object_agg
#9452Table Function (aka Set Returning Function)
Advanced
jsonb_populate_record
/jsonb_populate_recordset
/jsonb_to_record
/jsonb_to_recordset
jsonb_set
/jsonb_set_lax
feat(expr): implementjsonb_set
function #17103jsonb_insert
row_to_json
/array_to_json
jsonpath type
Additional Details
While SQL array is 1-indexed, JSON array as part of SQL is still 0-indexed.
Our goal is to ingest JSON data and then extract strongly-typed columns from it. So operations on constructing / mutating / comparison of JSON are not of interest right now.
Dedicated string member access operator vs Cast to string
Operations on object keys also work for array string elements
(TODO)
The text was updated successfully, but these errors were encountered: