Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(deps): add postgresql 17 #68

Merged
merged 3 commits into from
Nov 21, 2024

Conversation

vector-kerr
Copy link
Contributor

Greetings!

This PR adds CNPG@17 to the list of build versions.

I've submitted this PR directly as I could not find any contribution or submission guidelines. If you have a process to follow, I'm happy to oblige!

Kind regards,
@vector-kerr

Signed-off-by: Daniel 'Vector' Kerr <[email protected]>
@bo0tzz
Copy link
Member

bo0tzz commented Nov 8, 2024

Totally forgot about this, thank you!

…into chore/postgresql17

Signed-off-by: Daniel 'Vector' Kerr <[email protected]>
@vector-kerr
Copy link
Contributor Author

Just saw #69 has gone through, I've resolved the merge and pushed that through, should be good now.

@bo0tzz
Copy link
Member

bo0tzz commented Nov 15, 2024

Way ahead of me, thanks! Turns out the upstream .deb artifact naming has changed, so that's another thing for me to fix.

@bo0tzz
Copy link
Member

bo0tzz commented Nov 15, 2024

Hmm on closer look, it seems like the 0.3.0 release just doesn't have the same .deb artifact for pg 17 as is present for the other versions? @gaocegege do you have more insight into what's happened there?

@vector-kerr
Copy link
Contributor Author

Yikes. Seems like I'm just causing trouble now. :P

@bo0tzz
Copy link
Member

bo0tzz commented Nov 15, 2024

Not causing it, just bringing it to light ;) You're all good.

@vector-kerr
Copy link
Contributor Author

Looks like it might not have been in the build matrix for the workflow.

It seems like v17 build was introduced last month (<-- workflow for first successful run) - my guess would be that a rebuild for the tagged release hasn't happened since.

@bo0tzz
Copy link
Member

bo0tzz commented Nov 16, 2024

What confuses me is that there are -some- pg17 artifacts on the release.

@vector-kerr
Copy link
Contributor Author

What confuses me is that there are -some- pg17 artifacts on the release.

Oh, that is interesting.

On the v0.3.0 release page a bunch of the assets are dated June 20th (for my timezone) which matches the initial workflow run for that version, and then others are showing as ~1-2 weeks ago on November 5th, 6th, 8th, ...

Based on the dates and looking through all of the workflow runs, it looks like the PG17 packages might have been generated as part of this (failed) Release for Enterprise run. This seems to include the assets that we see deb packages for (public, vectors, extensions, ...) but doesn't seem to include the main package.

Seems like (whether it's intentional or not), the Release for Enterprise workflow is building + publishing + clobbering assets in the release, but that doesn't include the main deb package. As a result, all of the main deb packages that are in the release were built on the 20th of June as part of the standard 'Release' workflow, and a bunch of other deb packages have been (re-)built and attached to the v0.3.0 release as part of the separate Release for Enterprise workflow, as recently as about a week ago.

Not knowing a lot about the release mechanism in the pgvecto.rs repo, I would imagine that re-running the 'Release' workflow for the v0.3.0 tag might be sufficient to generate the main deb package for PG17 which would then allow this PR to build successfully, if that's something that the repo owners are happy to do.

@VoVAllen
Copy link
Member

Hi, thank you all for your contributions! The reason why we didn't release 0.3 and 0.4 in one complete workflow is that we didn't achieve enough improvements over the previous version. And the upgrade path is hard to maintain to make sure everything works (like the immich update from v0.1 to v0.2 of pgvecto.rs). So we deliberately did not do a lot of promotion because we feel that version 0.2 is sufficient for the vast majority of existing users. For v0.4, we introduced rabitq support, which showed some improvements over existing indexing methods like HNSW. But we didn't stop there, we think the algorithm is suitable for the native Postgres storage engine instead of a separate vector engine.

Then we started the VectorChord project as a successor to pgvecto.rs. It's based on the vector type of pgvector, so users can enjoy the compatibility with major cloud vendors. And they can choose the better indexing provided by VectorChord. Also, native postgres storage makes backup and restore much easier with existing pg ecosystems.

So we're aiming to make VectorChord the official successor of pgvecto.rs at the end of this year, when we'll migrate most of the features of pgvecto.rs to it (>2000dim vector support, fp16/int8 support). And for the pg17 support of pgvecto.rs, we will add the official release next week, but we strongly recommend users not to migrate to v0.4 if the current version works well. The main concern is that it's hard for us to maintain index compatibility across refactorings, and users won't feel a major achievement by upgrading to new versions.

@VoVAllen
Copy link
Member

We're still happy to maintain the cnpg image and the release of pgvecto.rs.

@bo0tzz
Copy link
Member

bo0tzz commented Nov 16, 2024

Sounds good, thank you! I'll keep this open for when the pg17 support is out. I'm hoping to get around to setting up the new cnpg image for VectorChord sometime this week 🙂

@bo0tzz
Copy link
Member

bo0tzz commented Nov 21, 2024

@VoVAllen just to clarify for me, are we expecting pg17 support for pgvecto.rs 0.3.0 as well?

@VoVAllen
Copy link
Member

@bo0tzz We can add pg17 support, but might be under a different version such as 0.3.1 with the exact same code, so that we don't need to re-release 0.3.0 again. Does it work?

@bo0tzz
Copy link
Member

bo0tzz commented Nov 21, 2024

@VoVAllen no worries, I don't need the support, I was just asking so I know in what order to build things in this repo.

@bo0tzz bo0tzz enabled auto-merge November 21, 2024 16:08
@bo0tzz bo0tzz merged commit 08e6ff7 into tensorchord:main Nov 21, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants