Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using an or expression with a knn expression raises a ResponseError with a Syntax error #557

Open
RogerThomas opened this issue Sep 5, 2023 · 0 comments

Comments

@RogerThomas
Copy link

RogerThomas commented Sep 5, 2023

I am trying to use the tags feature along with KNN vector search, so for example, a query searching for all albums where

(Genre:rock AND Decade: 60s) OR (Genre:rock AND Decade: 70s)

Along with a KNN vector search.

Querying on just the expression above is fine, however when I combine this with a KNN search, I get the following error:

redis.exceptions.ResponseError: Syntax error at offset 84 near >[

Code to reproduce

import numpy as np
from redis_om import Field, JsonModel, VectorFieldOptions, Migrator, KNNExpression


class Album(JsonModel):
    title: str = Field(primary_key=True)
    tags: str = Field(index=True)
    title_embeddings: list[float] = Field(
        index=True,
        vector_options=VectorFieldOptions.flat(
            type=VectorFieldOptions.TYPE.FLOAT32,
            dimension=2,
            distance_metric=VectorFieldOptions.DISTANCE_METRIC.COSINE,
        ),
    )


def embedding_function(text: str) -> [float]:
    """Mock embedding fnction, just counts vowels/consonants and normalises"""
    vowels = "aeiouAEIOU"
    num_vowels = sum((char in vowels) for char in text)
    num_consonants = len(text) - num_vowels
    x = num_vowels / len(text)
    y = num_consonants / len(text)
    norm = (x * x + y * y) ** 0.5
    xdim = x / norm
    ydim = y / norm
    vector = [round(xdim, 9), round(ydim, 9)]
    return vector


def main():
    Migrator().run()
    album_tuples = [
        ("The Dark Side Of The Moon", ["Genre:prog-rock", "Decade:70s"]),
        ("Rumours", ["Genre:rock", "Decade:70s"]),
        ("Abbey Road", ["Genre:rock", "Decade:60s"]),
    ]
    for title, tags in album_tuples:
        album = Album(
            title=title,
            tags="|".join(tags),
            title_embeddings=embedding_function(title),
        )
        album.save()

    expression = (Album.tags == "Genre:rock|Decade:70s") | (Album.tags == "Genre:rock|Decade:60s")
    knn_expression = KNNExpression(
        k=3,
        vector_field=Album.__fields__["title_embeddings"],
        reference_vector=np.array(embedding_function("Hotel California"), dtype="f4").tobytes()
    )
    # ---------------------------------------------------------------------------------------------
    # Query 1, Query on Tags only
    # ---------------------------------------------------------------------------------------------
    print("\nRunning Query 1, Tags only")
    albums = Album.find(expression).all()
    for album in albums:
        print(album)

    # ---------------------------------------------------------------------------------------------
    # Query 2, Query on KNN only
    # ---------------------------------------------------------------------------------------------
    print("\nRunning Query 2, KNN only")
    albums = Album.find(knn=knn_expression).all()
    for album in albums:
        print(album)

    # ---------------------------------------------------------------------------------------------
    # Query 2, Query on Tags and KNN
    # ---------------------------------------------------------------------------------------------
    print("\nRunning Query 2, KNN only")
    albums = Album.find(expression, knn=knn_expression).all()
    for album in albums:
        print(album)


if __name__ == "__main__":
    main()

This will fail on this line

    albums = Album.find(expression, knn=knn_expression).all()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant