Add QueryWithNegative

[erikdubbelboer]

QueryWithNegative allows you to pass a string that will be excluded from the search results.

There are tree ways to implement this:

1. Find the document matching the normal query. Then re-order the result by multiplying the similarity by the dot product with the negative vector and some constant.
2. Find the document matching the normal query. Exclude documents where the dot product with the negative vector are above a constant.
3. The simpler method I implemented which just subtracts the negative vector from the positive one and re-normalizes the result.

I have done some simple tests and the results look good.

I'm not sure if the extra function is a nice API. It could also be added as extra argument to Query. Or maybe Query should get a struct for it's argument as the number of arguments seems to keep increasing.

[philippgille]

That's an interesting feature!

3. The simpler method I implemented which just subtracts the negative vector from the positive one and re-normalizes the result.

I have done some simple tests and the results look good.

Did you test with positive and negative queries that were similar (e.g. a use case where a result is likely to appear but you want to exclude), as well also where they were entirely different (e.g. for use cases where some result is already unlikely, but you want to be even more certain to exclude it)?

I'm worried that the resulting vector C=A-B might be too different from A in some cases. E.g.

What about implementing all three and letting the user choose between them?

I'm not sure if the extra function is a nice API. It could also be added as extra argument to Query. Or maybe Query should get a struct for it's argument as the number of arguments seems to keep increasing.

WDYT about keeping the Query for now (for backward compatibility and catering to simple / most common use cases), but introducing a QueryWithOptions that already takes a struct as parameter with the where, whereDocument and negativeQuery fields?

Related to above proposal with three implementations, one field could also be an enum for picking the way to exclude the negative query.

---

Maybe this PR could just introduce the QueryWithOptions and the enum with two values (unspecified, which validates to an error, and the substract exclusion), and follow-up PRs can introduce the other two ways?

[erikdubbelboer]

I rewrote it to QueryWithOptions. Can you please have a look.

[philippgille]

Thanks for the rewrite, plus adding the other two modes! 👍

Please consider my new comments more as questions for now, not yet as requests for change.

Regarding the reorder mode, if we don't find a good solution, one option can be to exclude that mode for now, and add it in a follow-up PR. That way users can already benefit from the simpler modes and it gives more time to discuss the reorder.

[erikdubbelboer]

I'm not using this feature in production yet. I'm still playing around with it, trying to improve it. Since embedding models can't encode something being negative, I'm currently using a fast model like Phi3 to split a search query users write into a positive and a negative part (see below). And then passing these parts to my index in chromem to find the results.

Break down the following search query in it's positive and negative parts. For example for
```
football not manager
```
you should return:
```
positive: football
negative: manager
```

Or another example, for:
```
village builder but not idle
```
you should return:
```
positive: village builder
negative: idle
```

Here is the query:
```
...
```

[philippgille]

Thank you for the changes/improvements! I should be able to review again before the end of the week.

[philippgille]

Approving already. One comment is just a nitpick, and the other more a question, but I think I answered it already by running with different Ollama versions and getting different embeddings with each. You can address the comments if you have time, but otherwise no need and I'll merge next week.

[philippgille]

I'm getting different numbers for all prompts, but it seems to change with the Ollama version? I ran this, then updated Ollama and ran it again, and got different numbers both times (also different between the two runs). I didn't expect this, but maybe it's normal? Just to double check, can you run with the latest v0.1.44 please? (There's a v0.1.45 but it's pre-release).

---

On my side I'm running the code on an x86_64 Intel CPU, Fedora 39 (Kernel 6.8).

Ran with:

test code

package chromem

import (
	"context"
	"fmt"
	"os"
	"testing"
)

func TestFoo(t *testing.T) {
	e := NewEmbeddingFuncOllama("nomic-embed-text", os.Getenv("OLLAMA_HOST"))

	ee := func(s string) []float32 {
		v, err := e(context.Background(), s)
		if err != nil {
			panic(err)
		}
		return v
	}

	testEmbeddings := map[string][]float32{
		"search_document: Village Builder Game": ee("search_document: Village Builder Game"),
		"search_document: Town Craft Idle Game": ee("search_document: Town Craft Idle Game"),
		"search_document: Some Idle Game":       ee("search_document: Some Idle Game"),
		"search_query: town":                    ee("search_query: town"),
		"search_query: idle":                    ee("search_query: idle"),
	}

	fmt.Printf("%#v\n", testEmbeddings)
}

And then: go test -v -run TestFoo -count 1 .

[erikdubbelboer]

Yeah embeddings were kinda broken in some versions of Ollama. See: ollama/ollama#3777 and ollama/ollama#3053. I used v0.1.31 to generated them originally. I have re-generated them with v0.1.45 now, which changes them slightly again 🤷‍♂️

[philippgille]

Thanks again for this new feature! And sorry that I was slow with reviewing!