Configurable databases?

[andrewluetgers]

Very cool looking project! How about configurable databases? I am partial to LanceDB https://lancedb.github.io/lancedb/#documentation-guide for vector and if I wanted to use a different sql server? I think it would be great to have these options configurable.

[iwilltry42]

Thanks for the feedback!
We currently only support the SQLite and Chromem combo because it's the only truly embedded combination of databases with no external dependencies.
Implementing new vectorstores should be fairly straightforward using the interface definition: https://github.com/gptscript-ai/knowledge/blob/main/pkg/vectorstore/vectorstores.go#L7

For LanceDB specifically I don't see any Go bindings, so that may be a bit more work.

We'd happily accept a PR :)

[iwilltry42]

@andrewluetgers FYI: the internal interfaces are quite a bit cleaner now and recently we added the following options:

• Index: sqlite, postgres
• Vector: chromem-go, sqlite-vec, pgvector

They're configurable via the KNOW_INDEX_DSN and KNOW_VECTOR_DSN environment variables. Documentation is not available yet, but there's some info in the PRs: #150 & #146