Expected Behavior I should be able to select other PgIndexTypes such as BRIN or GIN.

Current Behavior PgIndexTypes HNSW and IVFFLAT only supports dimensions at max 2000

Context I was storing some question and answer pairs information and tried to save it into the postgres vector store and it easily hits the limit.

Comment From: tzolov

@kevintanhongann , thanks for raising this.

Can you please point us to a documentation/reference showing PGVector support for BRIN or GIN? The PGVector reference lists only the HNSW and IVFFLAT as supported types. Also there is a related open issue on the PGVector GH page related to the max 2000 limit.

Perhaps until this is resolved you can try some of the other VectorStore integration. I have not tested yet the max dimension limits per store. Please share your findings.

Comment From: kevintanhongann

Sorry, perhaps I might have mistaken. Is there something that I can do to workaround the limitation?

Not sure why Copilot came up with the suggestion that it should support BRIN and GIN. Maybe it assumed that it's still postgres at the end of the day. https://www.postgresql.org/docs/current/indexes-types.html

On Sun, Mar 3, 2024, 3:24 PM Christian Tzolov @.***> wrote:

@kevintanhongann https://github.com/kevintanhongann , thanks for raising this.

Can you please point us to a documentation/reference showing PGVector support for BRIN or GIN? The PGVector reference https://github.com/pgvector/pgvector lists only the HNSW and IVFFLAT as supported types. Also there is a related open issue https://github.com/pgvector/pgvector/issues/461 on the PGVector GH page related to the max 2000 limit.

Perhaps until this is resolved you can try some of the other VectorStore integration https://docs.spring.io/spring-ai/reference/0.8-SNAPSHOT/api/vectordbs.html#_available_implementations . I have not tested yet the max dimension limits per store. Please share your findings.

— Reply to this email directly, view it on GitHub https://github.com/spring-projects/spring-ai/issues/385#issuecomment-1975071918, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACXY4ITXCJKDFA2PCNKUC3YWLFZJAVCNFSM6AAAAABEDT4FPGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZVGA3TCOJRHA . You are receiving this because you were mentioned.Message ID: @.***>

Comment From: tzolov

@kevintanhongann please see my comment here: https://github.com/spring-projects/spring-ai/issues/383#issuecomment-1975117368 I'm going to class this issue as not relevant for Spring AI.

Comment From: tzolov

This is a PGVector limitation. It is not relevant to Spring AI.