Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Coercion from Dictionary(UInt32, Boolean) to Boolean #12511

Open
samuelcolvin opened this issue Sep 17, 2024 · 3 comments
Open

Coercion from Dictionary(UInt32, Boolean) to Boolean #12511

samuelcolvin opened this issue Sep 17, 2024 · 3 comments
Labels
enhancement New feature or request

Comments

@samuelcolvin
Copy link
Contributor

samuelcolvin commented Sep 17, 2024

I'm trying to run a query like

select count(*) from records where json_data_colum ? 'foo' and true

With datafusion and getting an error:

type_coercion caused by Error during planning: Cannot infer common argument type for logical boolean operation Dictionary(UInt32, Boolean) AND Boolean

(note in our case json_data_colum is a dictionary column containing JSON strings, so json_data_colum ? 'foo' returns a Dictionary(UInt32, Boolean))

This is related to #12382 where @adriangb fixed the case of filtering on a dictionary column.

I'll fix this specific case in https://github.com/datafusion-contrib/datafusion-functions-json, but I guess this should work.

@samuelcolvin samuelcolvin added the bug Something isn't working label Sep 17, 2024
@samuelcolvin samuelcolvin changed the title Coercion from Dictionary(UInt32, Boolean) Boolean Coercion from Dictionary(UInt32, Boolean) to Boolean Sep 17, 2024
@tustvold tustvold added enhancement New feature or request and removed bug Something isn't working labels Sep 17, 2024
@alamb
Copy link
Contributor

alamb commented Sep 17, 2024

I think type coercion is handled in DataFusion -- do you mind if I move this ticket to the datafusion repo?

@adriangb
Copy link
Contributor

Please go ahead 😄

FWIW I think this is another case where the argument could be made that the producer of said array should do the coercion because a dict of bools never makes sense to create in the first place.

@alamb alamb transferred this issue from apache/arrow-rs Sep 17, 2024
@alamb
Copy link
Contributor

alamb commented Sep 17, 2024

I moved the ticket -- and I agree I would suggest spending time updating the producer to avoid Dict(bool) rather than having DataFUsion do the coercion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants