Releases: weaviate/weaviate
v1.25.2 - Property Length Tracker Fix, Context-aware wand Fix, Replication and HNSW tombstone deletion Fixes, Google, OpenAI and OctoAI module headers Fixes
Breaking Changes
none
New Features
none
Fixes
- Do not recover from panics in integration tests by @dirkkul in #4970
- Support Implicit Tenant Activation (targeting
v1.25.1
) by @etiennedi in #4826 - RAFT: avoid rebuilding GQL twice by @moogacs in #4969
- fix: add update class description by @moogacs in #4982
- Refactor cluster client/server RPC implementation by @reyreaud-l in #4983
- Fixed documentation reference in meta.go by @Maurits-de-Groot in #4990
- RAFT: MT use RLock for reads by @moogacs in #4991
- Fix extracting near object vector for named vector classes by @tsmith023 in #4994
- fix: response of replication.CommitReplication() bubble up error by @moogacs in #5001
- various fixes to indexing queue by @asdine in #4854
- More accurate module usage telemetry data by @antas-marcin in #4996
- Remove superfluous
findEntrypoint
in delete logic by @etiennedi in #5010 - Fix broken
index.Contains
logic in flat index by @etiennedi in #4971 - RAFT: always load db on start for snapshot by @moogacs in #5014
- fix: local CommitReplication response if local shard Not found by @moogacs in #5017
- replication: replica fail early on getters with no-retry errors by @moogacs in #5031
- Fix race in tombstone cleanup for
stable/v1.24
by @trengrj in #5030 - Fix handling of OpenAI headers by @antas-marcin in #4997
- Add support for X-Google-Vertex-Api-Key header in multi2vec-palm module by @antas-marcin in #5033
- Keep track of deleted tombstones across commit logs by @trengrj in #5028
- Delete bloom tmp files before compaction by @amourao in #4992
- Introduce Tombstone Cleanup Limits by @etiennedi in #5012
- chore wrapped allow list by @jeroiraz in #5008
- Proplen tracker fix by @donomii in #5022
- backup:scheduler: compare and error based on major versions only by @antas-marcin in #5039
- chore: context-aware wand by @jeroiraz in #4907
- Replication: bubble up error on replication writes if shard init errors by @moogacs in #5044
- Fix Azure OpenAI batch endpoint handling by @dirkkul in #5035
- refactor cluster implementation to have clear separation of raft/schema by @reyreaud-l in #4993
- Cluster package housekeeping by @reyreaud-l in #5045
- Fix handling of OctoAI headers by @antas-marcin in #5050
- Support auto tenant creation for single object inserts by @parkerduckworth in #5047
- finish refactor of raft & cluster config by @reyreaud-l in #5053
- Fix HNSW visited list leak and improve error handling of already deleted nodes by @etiennedi in #5013
New Contributors
- @Maurits-de-Groot made their first contribution in #4990
Full Changelog: v1.25.1...v1.25.2
v1.24.15 - Property Length Tracker Fix, Context-aware wand Fix, Replication and HNSW tombstone deletion Fixes
Breaking Changes
none
New Features
none
Fixes
- Do not recover from panics in integration tests by @dirkkul in #4970
- Fixed documentation reference in meta.go by @Maurits-de-Groot in #4990
- Fix extracting near object vector for named vector classes by @tsmith023 in #4994
- fix: response of replication.CommitReplication() bubble up error by @moogacs in #5001
- various fixes to indexing queue by @asdine in #4854
- More accurate module usage telemetry data by @antas-marcin in #4996
- Remove superfluous
findEntrypoint
in delete logic by @etiennedi in #5010 - Fix broken
index.Contains
logic in flat index by @etiennedi in #4971 - fix: local CommitReplication response if local shard Not found by @moogacs in #5017
- replication: replica fail early on getters with no-retry errors by @moogacs in #5031
- Fix race in tombstone cleanup for
stable/v1.24
by @trengrj in #5030 - Fix handling of OpenAI headers by @antas-marcin in #4997
- Add support for X-Google-Vertex-Api-Key header in multi2vec-palm module by @antas-marcin in #5033
- Keep track of deleted tombstones across commit logs by @trengrj in #5028
- Delete bloom tmp files before compaction by @amourao in #4992
- Introduce Tombstone Cleanup Limits by @etiennedi in #5012
- chore wrapped allow list by @jeroiraz in #5008
- Proplen tracker fix by @donomii in #5022
- chore: context-aware wand by @jeroiraz in #4907
New Contributors
- @Maurits-de-Groot made their first contribution in #4990
Full Changelog: v1.24.14...v1.24.15
v1.23.16- Replication and Backup related Fixes
Breaking Changes
none
New Features
none
Fixes
- fix: response of replication.CommitReplication() bubble up error by @moogacs in #5001
- fix: local CommitReplication response if local shard Not found by @moogacs in #5017
- replication: replica fail early on getters with no-retry errors by @moogacs in #5031
- backup:scheduler: compare and error based on major versions only by @antas-marcin in #5039
Full Changelog: v1.23.15...v1.23.16
v1.25.1 - Schema V2 Fixes, AWS modules Fixes, Google Gemini 1.5 models support, Overwriting an object which is locally deleted Fix
Breaking Changes
none
New Features
none
Fixes
- fix prevent empty segment generation by @jeroiraz in #4893
- raft: refactor db opening & logging by @moogacs in #4875
- raft: always apply local schema before db by @moogacs in #4876
- raft: open db on service.Open instead on apply by @moogacs in #4878
- raft: remove schemaOnly and rename lastAppliedIndexOnStart by @moogacs in #4882
- improve log messages to apply to also dump cmd related output by @reyreaud-l in #4895
- Ensure
classCache
is added to the context before validate objects call by @tsmith023 in #4902 - Fix batch delete proto Java option setting by @antas-marcin in #4903
- Octoai header fixes + image generation by @dirkkul in #4906
- chore: warn with older partially written compacted segments by @jeroiraz in #4908
- Re-add schemaOnly parameter when applying raft log entries and ensure we reloadDB on catching up by @reyreaud-l in #4897
- Support raft rf scale (part 2) by @aliszka in #4913
- RAFT: fix data race on registering call backs and atomic lastAppliedI… by @moogacs in #4914
- RAFT: support back RF scale increase by @moogacs in #4856
- Replace namedLocks with KeyLocker by @aliszka in #4919
- Fix generative-aws module, add support for all Bedrock models by @antas-marcin in #4918
- Migrate text2vec-huggingface module to batching by @dirkkul in #4894
- backup:scheduler: compare and error based on major versions only by @moogacs in #4925
- Improve error messages on failed get/init shard by @aliszka in #4923
- fix race in tombstone cleanup by @asdine in #4910
- fix: skipping deleted nodes when finding entry point for node by @jeroiraz in #4807
- Add support for Google Gemini 1.5 models by @antas-marcin in #4929
- Remove unsearchable properties by @donomii in #4930
- Fix cluster remote aggregate hybrid neartext by @donomii in #4926
- Update VoyageAI vectorizer available model names by @fzowl in #4933
- Optimize locking for HNSW flat search by @trengrj in #4938
- Improved error message when autotenant expected to work with single object insertion by @parkerduckworth in #4948
- fix: special case when overwriting object which is locally deleted by @jeroiraz in #4950
- Do not validate the model for openai with a different base url by @dirkkul in #4951
- RAFT: fix race between apply for classes in the schema and report ready only after db reloaded by @moogacs in #4946
- Fix govulnchecks by @antas-marcin in #4958
- Add retry on failure in CI for tests instability by @antas-marcin in #4961
- fix deadlock by avoiding recursive locking in meta_class by @reyreaud-l in #4959
- Add support for separate headers for Google Studio and Vertex AI by @antas-marcin in #4962
- add missing wait for schema version on vector.PutObject by @moogacs in #4963
Full Changelog: v1.25.0...v1.25.1
v1.24.14 - Overwriting an object which is locally deleted Fix, Google Studio and Vertex separate headers support
Breaking Changes
none
New Features
none
Fixes
- Update VoyageAI vectorizer available model names by @fzowl in #4933
- fix: special case when overwriting object which is locally deleted by @jeroiraz in #4950
- Fix govulnchecks by @antas-marcin in #4958
- Add support for separate headers for Google Studio and Vertex AI by @antas-marcin in #4962
Full Changelog: v1.24.13...v1.24.14
v1.23.15 - Overwriting an object which is locally deleted Fix
Breaking Changes
none
New Features
none
Fixes
- fix: special case when overwriting object which is locally deleted by @jeroiraz in #4950
- Fix govulnchecks by @antas-marcin in #4958
Full Changelog: v1.23.14...v1.23.15
v1.24.13 - Empty segment generation Fix, Bedrock generative and Gemini 1.5 models support
Breaking Changes
none
New Features
none
Fixes
- remove debug logs by @etiennedi in #4889
- fix: max length field validation during object marshalling by @jeroiraz in #4877
- fix prevent empty segment generation by @jeroiraz in #4893
- chore: warn with older partially written compacted segments by @jeroiraz in #4908
- Fix generative-aws module, add support for all Bedrock models by @antas-marcin in #4918
- backup:scheduler: compare and error based on major versions only by @moogacs in #4925
- fix: skipping deleted nodes when finding entry point for node by @jeroiraz in #4807
- Add support for Google Gemini 1.5 models by @antas-marcin in #4929
Full Changelog: v1.24.12...v1.24.13
v1.25.0 - RAFT-based Schema, Batch Vectorization, Hybrid Search Improvements, Implicit Tenant Creation, Dynamic Index Switching
Breaking Changes
none
New Features
RAFT-based Schema
We're excited to announce the release of our RAFT-based schema! With this, Weaviate now supports concurrent schema updates, eliminating bottlenecks and significantly improving performance in large-scale and dynamic settings.
- Update schema manager interface to expose schema version (Part 1) by @reyreaud-l in #4659
- Update schema manager interface to expose schema version (Part 2) by @reyreaud-l in #4660
- Implement schema updates using RAFT consensus by @redouan-rhazouani in #3944
- Rebuild GQL just once when reloading local db by @redouan-rhazouani in #4670
- Update local_dev script RAFT_JOIN config to bootstrap cluster correctly by @moogacs in #4676
- Refactor consensus dev script on nodes name weaviate-* by @moogacs in #4677
- Update object interface to include version by @reyreaud-l in #4675
- RAFT store Apply tests calling mocks expectations assertion by @moogacs in #4684
- Save copy of the current RAFT schema into the old format by @moogacs in #4679
- Trigger schema callbacks on AddProperty only on success by @moogacs in #4685
- Check error on apply to avoid panics by @moogacs in #4687
- Add schema version to batch write client and server by @reyreaud-l in #4681
- Added store-specific statistics to existing RAFT statistics by @nathanwilk7 in #4689
- Client rpc reuse the same conn by @moogacs in #4686
- Update batch delete with schema version by @reyreaud-l in #4690
- Schema v2 props upsert by @aliszka in #4680
- Autoschema optimizations - cached class schema by @aliszka in #4662
- RAFT store.Service.Ready is flaky in cluster/store/store_test.go, poll for 2s instead of 1s wait to improve robustness by @nathanwilk7 in #4706
- Implement schema querying based on a specific version for enhanced version control by @redouan-rhazouani in #4693
- Update put, merge and add ref with schema version by @reyreaud-l in #4704
- Update UpdateShardStatus to use schema version by @reyreaud-l in #4713
- Fix nil ptr panic when closing RAFT leader client by @reyreaud-l in #4723
- RAFT GRPC: tweak the client service config by @moogacs in #4715
- Add query shard tenant and update usages by @reyreaud-l in #4727
- Enable cancelation for HTTP replication requests and reduce timeout for faster node failure detection by @redouan-rhazouani in #4730
- Schema v2 idempotent tenants by @aliszka in #4699
- Make batch operation retrieve and pass schema version to client by @reyreaud-l in #4738
- Update auto schema to return the schema version by @reyreaud-l in #4742
- Refactor usage of index.getOrInitLocalShard() to the minimum by @redouan-rhazouani in #4744
- Schema V2 writes with schema version by @aliszka in #4745
- Allow migration from RAFT to take as much time as necessary by @redouan-rhazouani in #4710
- Add missing CloudService to ClusterService rename by @aliszka in #4747
- Update RAFT subsystem to use the same log format by @redouan-rhazouani in #4746
- Convert
TenantExists
to handle eventual consistency by @tsmith023 in #4750 - Add an external gRPC method for getting tenant information by @tsmith023 in #4741
- update object endpoints to fetch and propagate schema version by @reyreaud-l in #4752
- Replace slog with logrus for logs consistency by @moogacs in #4753
- Add action logrus field to RAFT logs by @moogacs in #4755
- Autoschema fix to correctly return max schemaVersion by @aliszka in #4757
- Make default CLUSTER_HOSTNAME consistent between memberlist and RAFT by @moogacs in #4758
- Cluster log formatting by @nathanwilk7 in #4759
- Handle possible nil classes in autoSchema by @moogacs in #4761
- Refresh class's schema when props added by autoschema by @aliszka in #4762
- Specialize voter nodes for metadata storage by @redouan-rhazouani in #4734
- Update the schema version parsing to not always return 0 by @reyreaud-l in #4767
- Refactor leader error distinguish between election and network issues by @moogacs in #4784
- Update query shard tenant to get multiple shards at once by @moogacs in #4731
- Avoid schema updating race with DB update by @reyreaud-l in #4769
- Backup restore: return err class exists if it does by @moogacs in #4792
- Make coordinator node wait for schema version if changed on validate step by @reyreaud-l in #4794
- Add RAFT_GRPC_MESSAGE_MAX_SIZE Environment Variable to set maximum GRPC message size for RAFT by @nathanwilk7 in #4799
- Only involve leader for tenant status when necessary by @etiennedi in #4803
- Shift add tenant to the RAFT leader by @reyreaud-l in #4801
- Fix RAFT shutdown, force it for dependencies and convert to enterrors.GoWrapper by @moogacs in #4810
- Allow Leader QueryShardingState type by @moogacs in #4817
- Update replica usecase to wait for EC on writes with schema version by @reyreaud-l in #4814
- Schema v2 prevent shutdown by @aliszka in #4821
- Aggregate querying the leader for classes with variadic by @moogacs in #4787
- Observe durations of schema reads and writes for local and leader reads by @etiennedi in #4844
- Reserve RAFT (all casing permutations) as a class name by @moogacs in #4874
- Make bootstrap exit early on RAFT store reporting ready by @reyreaud-l in #4871
Batch Vectorization
Avoid vectorization API rate-limiting and enjoy faster insertion.
- Batch vectorization openai by @dirkkul in #4384
- Factor out batch vectorization and add Cohere support by @dirkkul in #4515
- Factor out batching in modules by @dirkkul in #4564
- Vectorizer batching for VoyageAI by @dirkkul in #4578
Dynamic Index Switching
Automatically switch vector index types to achieve peak performance and efficiency.
- Dynamic vector index type by @abdelr in #4350
- Fix merge conflict on schema validation of dynamic index by @trengrj in #4829
- Adding more tests around the dynamic index and fixing an existing bug by @abdelr in #4836
Implicit Tenant Creation
Create tenants on the fly by simply including the tenant name during batch inserts.
- Create nonexistent tenants in batch by default by @parkerduckworth in #3741
- Support auto-tenant toggling on MT-enabled classes by @parkerduckworth in #4813
Hybrid Search Improvements
- Add nearvector and neartext to hybrid search by @donomii in #4462
- Batch vectorization with custom rate limits by @dirkkul in #4546
- Add groupby to hybrid search and bm25f, add moveto/movefrom etc to aggregate hybrid search by @donomii in #4477
- Add tests, improve parameter extraction in hybrid aggregate by @donomii in #4809
Other
v1.24.12 - LSM Segment Size limit setting, text2vec-aws module Bedrock models support Fix
Breaking Changes
none
New Features
none
Fixes
- fix: vector validation in batch operation by @jeroiraz in #4858
- Introduce optional LSM Segment Size limit by @etiennedi in #4872
- Fix text2vec-aws module Bedrock support by @antas-marcin in #4881
- Allow setting HNSW Max log size ("condensing limit") dynamically by @etiennedi in #4884
- Add support for Amazon Titan Text Embeddings V2 model by @antas-marcin in #4886
Full Changelog: v1.24.11...v1.24.12
v1.24.11 - Vector search with named vectors Fix, improved validation when using nearObject and nearVector searches
Breaking Changes
none
New Features
none
Fixes
- Fix macOS monitor tests by @antas-marcin in #4708
- Add latest tag to docker image only when it's a tag build by @antas-marcin in #4732
- Remove unintended debug logs by @etiennedi in #4766
- Improve input validation of ID in nearObject and vector in nearVector by @tsmith023 in #4782
- Fix named vector groupby non-search aggregate queries by @tsmith023 in #4783
- Add recovery wrappers for GRPC APIs by @dirkkul in #4780
- index queue: add logs by @asdine in #4842
- Revert "index queue: add logs" by @asdine in #4850
- Fix an issue introduced in
v1.24.9
where HNSW commit logs are no longer optimized by @etiennedi in #4853 - Fix vector search with named vectors by @antas-marcin in #4857
Full Changelog: v1.24.10...v1.24.11