Comments (3)
Thank you @kernelsoe! Such an API indeed makes sense. We can implement it at least slightly more efficiently than the remove
& add
combo, by reusing the same memory buffers under the hood. However, generally, remove
is discouraged. As part of any operation or standalone.
How many vectors do you generally deal with?
from usearch.
@ashvardanian For my use case, I'm trying to load and add a few hundred user notes from sqlite into usearch on app startup without remove op then as the user edits 1 or reflecting multiple notes, I'm reindexing by removing and adding all with the same key again.
Doing this combined operation for multiple times in 1 function call would sometimes throw error, so I did something like below for now 😄:
for (const payload of payloads) {
await remove()...
await timeout(300)
}
for (const payload of payloads) {
await add()...
await timeout(300)
}
May be I should use batch op?
from usearch.
Aside: There is a sqlite plugin sqlite-vss that adds vector search, but unfortunately uses FAISS Flat not HNSW. In addition, updates are not supported, so it seems the issue is elsewhere. I think pg-vector rolled their own.
While I wish this was inside sqlite, I am looking forward to using it side by side like you are.
from usearch.
Related Issues (20)
- Bug: crash when hardware concurrency is exceeded HOT 5
- Bug: index.search returns invalid keys when k > index size HOT 5
- Bug: Deadlock in concurrent update()s HOT 5
- Bug: Replacing initial entry affects visibility of other entries HOT 2
- Feature: Cross compilation of sqlite extension for ios and android for react native apps HOT 2
- Bug: Issues index dtype=i8 with Inner Product Metrics HOT 27
- Feature parity between GoLang and C HOT 1
- Feature: Java search API extension to batch search and ANN.
- Bug: Segfault when dimensions of added vector don't add up (Rust) HOT 8
- Bug: Failed to run c++ examples. HOT 2
- Bug: Arm64 versions starting at v10.0 and up give the error Fatal Python error: Illegal instruction HOT 3
- Low index performance after `clear()` HOT 2
- Bug: Syntax Error with Jest in ESM HOT 3
- Bug: Rust build does not use simsimd (`index.hardware_acceleration()` reports `serial`) HOT 2
- Bug: cannot open old database (created with 2.9.2) with new version (2.12.0) HOT 4
- Feature: adding `py.typed` metadata to `python/usearch` HOT 1
- Bug: npm package does not support esm in nodejs project. HOT 3
- "usearch_sqlite" binary for Windows HOT 1
- Bug: Rust test_add_remove_vector fails on main-dev HOT 2
- Docs: The usearch.h exact_search documentation doesn't match the function arguments
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from usearch.