Skip to content

problem with delete or update existing records on WeaviateIndex store #1636

Open
@vincetrep

Description

@vincetrep

Initial Checks

  • I have read and followed the docs and still think this is a bug

Description

I'm seeing an odd behavior which may be a bug on the WeaviateIndex implementation:
I have a document with subindices.
Expected Behavior
When that document is saved in the Weaviate store and i then try to save it again, should I expect the record to be updated for the same docarrayid?
Actual Behavior
What happens is that the parent index e.g. RecipeIndex doesn't get duplicated. However, all the subindices get created again and they all have the same parent_id. My expectation would be to update the children classes like a RecipeIndex_name that have the same parent_id as the RecipeIndex class I am updating get updated and not duplicated.

###Update
There is the same issue when using store.delitem => only the parent record is deleted. Then subindices are still there after the delete operation.

Example Code

import docarray
from docarray.documents import TextDoc
class RecipeRecord(BaseDoc):
    recipe_id: str
    name: DocList[TextDoc]

from docarray.index.backends.weaviate import WeaviateDocumentIndex
import weaviate

host="http://localhost:9999"
dbconfig = WeaviateDocumentIndex.DBConfig(
    host=host,
    index_name='RecipeIndex'
)  # Replace with your endpoint)

store = WeaviateDocumentIndex[RecipeRecord](db_config=dbconfig)

doc = RecipeRecord(
id='id123'
recipe_id='id123',
name = DocList[TextDoc]([TextDoc(text='test name',embedding=None)])
)

docList = DocList[RecipeRecord]()
docList.append(doc)

store.index(docList)
#upon running the index again, there will be partial duplication
store.index(docList)

Python, Pydantic & OS Version

docarray 0.33

Affected Components

Metadata

Metadata

Assignees

Type

No type

Projects

Status

Todo/Bug

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions