The indexing module for search-index
fergiemcdowall / search-index-adder Goto Github PK
View Code? Open in Web Editor NEWThe indexing module for search-index
License: MIT License
The indexing module for search-index
License: MIT License
The indexing module for search-index
I'm opening this issue to discuss possible optimisations, either in the code or in how users operate their indices. This is following on from fergiemcdowall/search-index#261
This is just my first batch of ideas - they might be terrible or wrong. Please feel free to criticise them, suggest better alternatives, etc.
I think speed reductions will come mainly from reducing the number of db operations.
It'd be nice to be able to use e9980a1 without pointing at my fork.
I'm getting this when finishing an indexing
/home/.../node_modules/search-index/lib/siUtil.js:33
siOptions.log.info('closed...')
^
TypeError: Cannot read property 'info' of undefined
at /home/.../node_modules/search-index/lib/siUtil.js:33:21
at /home/.../node_modules/levelup/lib/levelup.js:141:18
at /home/.../node_modules/abstract-leveldown/abstract-leveldown.js:65:7
Apart from the code itself, is there a description of the algorithm written up anywhere?
as per search-index
At present doc vectors are split up and distributed around the index in sorted term vector arrays. This makes it easy to find out which documents have the highest term frequency for a given document. However, in cases where there are two or more search terms this approach alone becomes computationally less efficient since all term vector arrays have to be traversed to find out scores.
It would be better if the document vector was saved on a per-document basis in the index. This would give the following advantages:
DELETE-DOCUMENT
entries to be removed, since all index entries for a document can be derived from its vectorA declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.