Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(query): improve Query UX #450

Merged
merged 12 commits into from
Jul 26, 2022
Merged

feat(query): improve Query UX #450

merged 12 commits into from
Jul 26, 2022

Conversation

bednar
Copy link
Contributor

@bednar bednar commented Jun 3, 2022

Closes #445
Closes #446

Proposed Changes

  1. query() function returns TableList instead of List['FluxTable'], query_csv() function returns CSVIterator instead of csv.reader => we are able to add additional functions for better procession of the query results
  2. Query results can be easily serialised into list of values and JSON:
    image
  3. CSV query results can be easily serialised into list of values and optionally spit out the headers:
    image
  4. query_data_frame() function contains warning and updated doc about using pivot() function to optimal shaping of query results
    image
    from influxdb_client import InfluxDBClient
    
    with InfluxDBClient(url="http://localhost:8086", token="my-token", org="my-org", debug=False) as client:
        # query to data_frame
        data_frame = client.query_api().query_data_frame('''from(bucket:"my-bucket") |> range(start: -10m)''')
    
        print(data_frame.to_string())
    /influxdb-client-python/influxdb_client/client/warnings.py:31: MissingPivotFunction: The query doesn't contains the pivot() function.
    
    The result will not be shaped to optimal processing by pandas.DataFrame. Use the pivot() function by:
    
        from(bucket:"my-bucket") |> range(start: -10m) |> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value")
    
    You can disable this warning by:
        import warnings
        from influxdb_client.client.warnings import MissingPivotFunction
    
        warnings.simplefilter("ignore", MissingPivotFunction)
    
    For more info see:
        - https://docs.influxdata.com/resources/videos/pivots-in-flux/
        - https://docs.influxdata.com/flux/latest/stdlib/universe/pivot/
        - https://docs.influxdata.com/flux/latest/stdlib/influxdata/influxdb/schema/fieldsascols/
    
        result  table                           _start                            _stop                            _time  _value       _field    _measurement  location
    0  _result      0 2022-06-16 12:17:07.240362+00:00 2022-06-16 12:27:07.240362+00:00 2022-06-16 12:24:17.125455+00:00    24.3  temperature  my_measurement  New York
    1  _result      1 2022-06-16 12:17:07.240362+00:00 2022-06-16 12:27:07.240362+00:00 2022-06-16 12:24:17.125455+00:00    25.3  temperature  my_measurement    Prague
    
    

Checklist

  • CHANGELOG.md updated
  • Rebased/mergeable
  • A test has been added if appropriate
  • pytest tests completes successfully
  • Commit messages are conventional
  • Sign CLA (if not already signed)

@codecov-commenter
Copy link

codecov-commenter commented Jun 3, 2022

Codecov Report

Merging #450 (58d349f) into master (465476b) will increase coverage by 0.04%.
The diff coverage is 95.08%.

@@            Coverage Diff             @@
##           master     #450      +/-   ##
==========================================
+ Coverage   89.35%   89.39%   +0.04%     
==========================================
  Files          38       39       +1     
  Lines        3260     3301      +41     
==========================================
+ Hits         2913     2951      +38     
- Misses        347      350       +3     
Impacted Files Coverage Δ
influxdb_client/client/flux_table.py 90.32% <90.62%> (+0.15%) ⬆️
influxdb_client/client/_base.py 97.85% <100.00%> (+<0.01%) ⬆️
influxdb_client/client/flux_csv_parser.py 98.29% <100.00%> (ø)
influxdb_client/client/invokable_scripts_api.py 48.57% <100.00%> (ø)
influxdb_client/client/query_api.py 100.00% <100.00%> (ø)
influxdb_client/client/query_api_async.py 100.00% <100.00%> (ø)
influxdb_client/client/warnings.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 465476b...58d349f. Read the comment docs.

@bednar bednar changed the title feat(query): iterate over query results as a JSON, as a collection of values feat(query): improve Query UX Jun 16, 2022
@bednar bednar marked this pull request as ready for review June 16, 2022 12:40
@bednar
Copy link
Contributor Author

bednar commented Jul 17, 2022

Hi @Anaisdg, @Jayclifford345,

Can you take a look to this PR as a solution for #445 and #446?

Regards

@bednar bednar requested a review from powersj July 18, 2022 12:56
@Anaisdg
Copy link

Anaisdg commented Jul 18, 2022

@bednar It looks really great! Thank you so much!

@bednar
Copy link
Contributor Author

bednar commented Jul 18, 2022

@Anaisdg thanks

@powersj
Copy link
Contributor

powersj commented Jul 20, 2022

query() function returns TableList instead of List['FluxTable']
query_csv() function returns CSVIterator instead of csv.reader

I haven't started digging into queries with the library. However, do either of these break backward compatibility?

@bednar
Copy link
Contributor Author

bednar commented Jul 21, 2022

query() function returns TableList instead of List['FluxTable']
query_csv() function returns CSVIterator instead of csv.reader

I haven't started digging into queries with the library. However, do either of these break backward compatibility?

It is backward compatible change because the TableList extends List['FluxTable'] and CSVIterator extends Iterator[List[str]].

Copy link
Contributor

@powersj powersj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is backward compatible change

great, should this be part of the release this week then?

@bednar
Copy link
Contributor Author

bednar commented Jul 26, 2022

It is backward compatible change

great, should this be part of the release this week then?

Yes, it will be part of it.

@bednar bednar merged commit 940d5d3 into master Jul 26, 2022
@bednar bednar deleted the query_helpers branch July 26, 2022 04:17
@bednar bednar added this to the 1.31.0 milestone Jul 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants