Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: binding struct values when they are parsed as row #2371

Merged
merged 47 commits into from
May 12, 2022

Conversation

cykbls01
Copy link
Contributor

@cykbls01 cykbls01 commented May 8, 2022

What's changed and what's your intention?

PLEASE DO NOT LEAVE THIS EMPTY !!!

  1. binding struct values when they are parsed as row.
  2. add struct array unit test in values and insert executor.

Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests

Refer to a related PR or issue link (optional)

#581

@codecov
Copy link

codecov bot commented May 8, 2022

Codecov Report

Merging #2371 (b1b4902) into main (74e17ca) will increase coverage by 0.03%.
The diff coverage is 79.59%.

@@            Coverage Diff             @@
##             main    #2371      +/-   ##
==========================================
+ Coverage   71.31%   71.35%   +0.03%     
==========================================
  Files         688      688              
  Lines       88254    88392     +138     
==========================================
+ Hits        62936    63069     +133     
- Misses      25318    25323       +5     
Flag Coverage Δ
rust 71.35% <79.59%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/common/src/array/list_array.rs 88.28% <0.00%> (-0.61%) ⬇️
src/expr/src/expr/expr_literal.rs 90.74% <ø> (ø)
src/frontend/src/binder/expr/value.rs 92.52% <ø> (ø)
src/common/src/types/mod.rs 65.08% <40.00%> (+0.86%) ⬆️
src/common/src/types/scalar_impl.rs 57.40% <46.66%> (-0.17%) ⬇️
src/frontend/src/binder/values.rs 92.68% <79.16%> (-5.60%) ⬇️
src/batch/src/executor2/insert.rs 83.53% <100.00%> (+3.68%) ⬆️
src/batch/src/executor2/values.rs 95.45% <100.00%> (+0.89%) ⬆️
src/common/src/array/struct_array.rs 84.49% <100.00%> (+4.06%) ⬆️
src/frontend/src/binder/expr/mod.rs 88.21% <100.00%> (+0.04%) ⬆️
... and 8 more

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

src/common/src/array/struct_array.rs Outdated Show resolved Hide resolved
src/common/src/types/mod.rs Outdated Show resolved Hide resolved
src/common/src/array/struct_array.rs Outdated Show resolved Hide resolved
src/common/src/array/list_array.rs Outdated Show resolved Hide resolved
src/frontend/src/binder/values.rs Outdated Show resolved Hide resolved
src/frontend/src/binder/values.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@neverchanje neverchanje left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm but you should fix the comments after this pr or in this pr

select * from s where s.v3 = (1,2,(1,2,3));
logical_plan: |
LogicalProject { exprs: [$1, $2, $3], expr_alias: [v1, v2, v3] }
LogicalFilter { predicate: ($3 = {Some(Int32(1)), Some(Int32(2)), Some(Struct(StructValue { fields: [Some(Int32(1)), Some(Int32(2)), Some(Int32(3))] }))}:Struct { fields: [Int32, Int32, Struct { fields: [Int32, Int32, Int32] }] }) }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why the test format of StructValue remains the verbose form even after you impl fmt::Display in concise form?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe because the display of Datum is like these Some(...), so this looks like complex.

@@ -322,6 +322,15 @@ for_all_scalar_variants! { scalar_impl_enum }
pub type Datum = Option<ScalarImpl>;
pub type DatumRef<'a> = Option<ScalarRefImpl<'a>>;

pub fn get_data_type_from_datum(datum: &Datum) -> Result<DataType> {
match datum {
None => Err(internal_error(
Copy link
Contributor

@neverchanje neverchanje May 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the data type of a literal should be unknown until it's inferred by some means, think of such case:

postgres=# create type foo as (v1 int, v2 int);                                                                                           CREATE TYPE
postgres=# create table t (v1 foo);
CREATE TABLE
postgres=# SELECT * FROM t WHERE v1 > (1, null);
 v1
----
(0 rows)

How can we determine the type of (1, null) until it's inferred from v1? (given the rule that v1 and (1, null) must be the same type, so the null's type must be int).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see.., will fix in follow-up issue.

@cykbls01 cykbls01 marked this pull request as ready for review May 12, 2022 11:39
@cykbls01 cykbls01 merged commit c3594bd into main May 12, 2022
@cykbls01 cykbls01 deleted the feat/insert_struct_data branch May 12, 2022 11:39
let data_type = match self {
ScalarImpl::Int16(_) => DataType::Int16,
ScalarImpl::Int32(_) => DataType::Int32,
ScalarImpl::Int64(_) => DataType::Int64,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC timestamp with time zone is also stored as ScalarImpl::Int64 but has DataType::Timestampz.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for mention, I will try to fix in follow pr.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just bringing it up. So that we are aware of this tricky issue in type system and can see which design is better. I do need see an immediate way to "fix" - in the current design, mapping from ScalarImpl to DataType is not unique; alternatively, we may add a chrono wrapper for timestampz.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me see, I found that acutally I don't need this function, I refactor this part #2508.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants