Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot create HashModel containing a list to do knn #544

Open
GDegrove opened this issue Jul 25, 2023 · 4 comments
Open

Cannot create HashModel containing a list to do knn #544

GDegrove opened this issue Jul 25, 2023 · 4 comments

Comments

@GDegrove
Copy link

GDegrove commented Jul 25, 2023

We are currently trying to get Redis-om to work with redis-search for knn search on a HashModel.

Our model is the following:

class Document(HashModel):

    vector: float = Field(
        vector_options=VectorFieldOptions.flat(
            type=VectorFieldOptions.TYPE.FLOAT32,
            dimension=512,
            distance_metric=VectorFieldOptions.DISTANCE_METRIC.COSINE
        )
    )

When using the type float we can go into that line of code, that creates the correct vector type. However, a vector is actually NOT a float, so any creation of the model will fail.

Transforming the model to a valid model, we see that we need to use:

class Document(HashModel):

    vector: list[float] = Field(
        vector_options=VectorFieldOptions.flat(
            type=VectorFieldOptions.TYPE.FLOAT32,
            dimension=512,
            distance_metric=VectorFieldOptions.DISTANCE_METRIC.COSINE
        )
    )

That would allow us to create the correct VECTOR and use it. However, it seems that there's a check in __init__ that prevent any list to be a field in the HashModel

However, when creating the schema, there's a specific clause to check that the subscribed type, is supported if this is a list or a tuple that we are clearly in.

The question is:

How can we create a HashModel with vector list[float] to be able to create a knn_vector query?
Is it a bug where we need to skip the list and check that actually we support list and tuples in HashModel? Or should we use any other container to let us do the vector query?

Thanks already for the information!

@bearrito
Copy link

bearrito commented Jul 25, 2023

Are lists supported in a HashModels? I always assumed based on docs they are not.
https://github.com/redis/redis-om-python/blob/main/docs/models.md#hashmodel-vs-jsonmodel

@GDegrove
Copy link
Author

Are lists supported in a HashModels? I always assumed based on docs they are not. https://github.com/redis/redis-om-python/blob/main/docs/models.md#hashmodel-vs-jsonmodel

Thanks for the information.
I saw that piece of documentation, but the code seems to indicate otherwise (and there are comment in the code that also indicate otherwise). Especially since a Vector that you can create in a HashModel is by definition a list of float/int/decimal.
But maybe there's no way to support Vector in HashModel, and so we may have to document that limitation. For example the Vector we create is of dimension 512. That mesa 512 float in a list/array like data type.

That's why I'm asking if there's a bug or a limitation that should be documented.

@GDegrove
Copy link
Author

Another solution would be to let bytes as a valid vector type (or maybe the only vector type) so we can create a Vector field of type bytes

@RogerThomas
Copy link

I'm also trying to get vector search working with redis-om, is this possible at the moment?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants