Skip to content

Commit

Permalink
Merge branch 'main' of github.com:apache/datafusion into dev/xinli/po…
Browse files Browse the repository at this point in the history
…sition-udf
  • Loading branch information
xinlifoobar committed Jul 5, 2024
2 parents 9a9ae70 + b46d5b7 commit e18f6f6
Show file tree
Hide file tree
Showing 76 changed files with 2,401 additions and 978 deletions.
69 changes: 0 additions & 69 deletions .pre-commit-config.yaml

This file was deleted.

9 changes: 3 additions & 6 deletions datafusion-examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,9 @@ cd datafusion
# Download test data
git submodule update --init

# Run the `csv_sql` example:
# Run the `dataframe` example:
# ... use the equivalent for other examples
cargo run --example csv_sql
cargo run --example dataframe
```

## Single Process
Expand All @@ -47,10 +47,8 @@ cargo run --example csv_sql
- [`advanced_udf.rs`](examples/advanced_udf.rs): Define and invoke a more complicated User Defined Scalar Function (UDF)
- [`advanced_udwf.rs`](examples/advanced_udwf.rs): Define and invoke a more complicated User Defined Window Function (UDWF)
- [`advanced_parquet_index.rs`](examples/advanced_parquet_index.rs): Creates a detailed secondary index that covers the contents of several parquet files
- [`avro_sql.rs`](examples/avro_sql.rs): Build and run a query plan from a SQL statement against a local AVRO file
- [`catalog.rs`](examples/catalog.rs): Register the table into a custom catalog
- [`composed_extension_codec`](examples/composed_extension_codec.rs): Example of using multiple extension codecs for serialization / deserialization
- [`csv_sql.rs`](examples/csv_sql.rs): Build and run a query plan from a SQL statement against a local CSV file
- [`csv_sql_streaming.rs`](examples/csv_sql_streaming.rs): Build and run a streaming query plan from a SQL statement against a local CSV file
- [`custom_datasource.rs`](examples/custom_datasource.rs): Run queries against a custom datasource (TableProvider)
- [`dataframe-to-s3.rs`](examples/external_dependency/dataframe-to-s3.rs): Run a query using a DataFrame against a parquet file from s3 and writing back to s3
Expand All @@ -66,12 +64,11 @@ cargo run --example csv_sql
- [`memtable.rs`](examples/memtable.rs): Create an query data in memory using SQL and `RecordBatch`es
- [`optimizer_rule.rs`](examples/optimizer_rule.rs): Use a custom OptimizerRule to replace certain predicates
- [`parquet_index.rs`](examples/parquet_index.rs): Create an secondary index over several parquet files and use it to speed up queries
- [`parquet_sql.rs`](examples/parquet_sql.rs): Build and run a query plan from a SQL statement against a local Parquet file
- [`parquet_sql_multiple_files.rs`](examples/parquet_sql_multiple_files.rs): Build and run a query plan from a SQL statement against multiple local Parquet files
- [`parquet_exec_visitor.rs`](examples/parquet_exec_visitor.rs): Extract statistics by visiting an ExecutionPlan after execution
- [`parse_sql_expr.rs`](examples/parse_sql_expr.rs): Parse SQL text into Datafusion `Expr`.
- [`plan_to_sql.rs`](examples/plan_to_sql.rs): Generate SQL from Datafusion `Expr` and `LogicalPlan`
- [`pruning.rs`](examples/parquet_sql.rs): Use pruning to rule out files based on statistics
- [`pruning.rs`](examples/pruning.rs): Use pruning to rule out files based on statistics
- [`query-aws-s3.rs`](examples/external_dependency/query-aws-s3.rs): Configure `object_store` and run a query against files stored in AWS S3
- [`query-http-csv.rs`](examples/query-http-csv.rs): Configure `object_store` and run a query against files vi HTTP
- [`regexp.rs`](examples/regexp.rs): Examples of using regular expression functions
Expand Down
51 changes: 0 additions & 51 deletions datafusion-examples/examples/avro_sql.rs

This file was deleted.

70 changes: 0 additions & 70 deletions datafusion-examples/examples/csv_sql.rs

This file was deleted.

51 changes: 0 additions & 51 deletions datafusion-examples/examples/parquet_sql.rs

This file was deleted.

Loading

0 comments on commit e18f6f6

Please sign in to comment.