Code

Add rag enrichment and ingestion.#33413

Code

Open

claudevdm wants to merge 11 commits intoapache:masterapache/beam:masterfrom claudevdm:add-rag-ingestion-enrichmentclaudevdm/beam:add-rag-ingestion-enrichmentCopy head branch name to clipboard

+1,720-7

Commits

Commits on Dec 18, 2024

• Add base VectorDatabaseTransform.

  claudevdm

  committed
  76475e2

• Add BigQueryVectorWriterConfig.

  claudevdm

  committed
  a22a48c

Commits on Dec 19, 2024

• Add BigQueryVectorWriterConfig tests.

  

  Claude

  authored and

  claudevdm

  committed
  153e48e

• Allow overriding joinfn and custom types from EnrichmentSourceHandler.

  claudevdm

  committed
  311d1c7

• Add BigQueryVectorSearchEnrichmentHandler.

  claudevdm

  committed
  3774dca

• Fix streaming test.

  claudevdm

  committed
  589023f

• Add licence.

  claudevdm

  committed
  3044d1d

• Fix project in vector search test and pydocs.

  claudevdm

  committed
  1ba9ee9

Commits on Dec 20, 2024

• Fix bigquery streaming test.

  claudevdm

  committed
  2385897

Commits on Dec 23, 2024

• Resolve open comments. Also fix batching logic when metadata restrictions are applied.

  claudevdm

  committed
  358440d

• Merge branch 'master' into add-rag-ingestion-enrichment

  claudevdm

  committed
  e2274a8