Comments (1)
Thanks for the report but I'm afraid it's not a bug...
- You are not actually measuring query performance in this test. You are timing
how long it takes CQEngine to return a ResultSet, but queries are evaluated
*lazily* so you need to actually iterate through the results to make CQEngine
do any work.
Here you see a slight increase in the time it takes CQEngine to return a
ResultSet when you have an index in place, because CQEngine is doing some quick
calculations to work out if it will use the index or not. If there are no
indexes available then it doesn't need to make this calculation. But this would
be a negligible computation in normal query processing.
- I don't know if you actually ran this test with COUNT = 20 as in the source
code, but 20 objects is way too few for a benchmark. I assume you tested with
more objects and I tested with 10,000 and 100,000 objects anyway.
- All of your 'between' queries match 50% of the collection. Indexes can make
finding a needle in a haystack a fast operation. But here, you are selecting
half of the haystack :)
By default indexes like NavigableIndex are optimized for finding needles in the
haystack - the buckets are very small so that fine-grained queries can be
evaluated without filtering.
If we have a collection of 10,000 objects, and your typical queries will match
5,000 objects, then actually it means the query will span 5,000 buckets, and
there is some overhead in jumping between the buckets.
So you should configure the index with a Quantizer to reduce the number of
buckets. This will improve performance for coarse-grained queries (and also
reduce memory overhead), but it will reduce performance for fine-grained
queries. So it's about tuning the index based on the granularity of your
expected or typical queries.
I ran the test, with:
----------------------
COUNT = 100000;
...
items.addIndex(NavigableIndex.withQuantizerOnAttribute(
IntegerQuantizer.withCompressionFactor(1000),
Item._x
));
...
Iterables.count(items.retrieve(q[j])); // <- iterates the ResultSet
----------------------
Results were (taking output lines 38 and 41 as examples):
38: Q0: 3167000 // 'equal' query, without NavigableIndex
38: Q0: 246000 // 'equal' query, with NavigableIndex, 13 times faster
41: Q3: 7127000 // 'between' query, without NavigableIndex
41: Q3: 1731000 // 'between' query, with NavigableIndex, 4 times faster
I'll close this issue as Invalid, please don't take offence at that but I don't
think this is a bug. If you find other issues let me know as the more eyes on
the code the better. The discussion forum is also available:
https://groups.google.com/forum/#!forum/cqengine-discuss
Original comment by [email protected]
on 30 May 2014 at 11:51
- Changed state: Invalid
from cqengine.
Related Issues (20)
- Support STRICT_REPLACEMENT in ObjectLockingIndexedCollection/ConcurrentIndexedCollection
- TransactionalIndexedCollection support for UniqueIndex is broken
- Support for attribute accessors for Java record type HOT 1
- Kryo generates InaccessibleObjectException - Make Dependencies "Java 17 compatible"
- Bug in AttributeOrdersComparator HOT 5
- Offheap cache visibility HOT 1
- Support for ANTLR 4.10.1 HOT 12
- Does it support SQL statistics queries HOT 3
- Thread blocked when using Materialized View Sort For the Longest Prefix for CSV that contain more than 11 K rows HOT 3
- Error while parsing SQL NOT LIKE query HOT 1
- Error while parsing SQL LIKE '%%%' query
- java.lang.IllegalStateException thrown in the getGenericType(Member member) method of AttributeBytecodeGenerator.java
- CQNGrammar Sources missing from master HOT 1
- What is use of sqlite-jdbc in cqengine HOT 2
- Index updated even though objectStore is not modified during IndexedCollection add HOT 2
- Does CQEngine compatible with Java 17 HOT 2
- Issues with Offheap Concurrent Index collection clear()
- Getting below error on jdk17.0.8.8.1 HOT 1
- Is CQEngine officially dead? HOT 6
- com/googlecode/concurrenttrees/radix/node/NodeFactory
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cqengine.