Comments (11)
This is an enhancement request not a defect.
Original comment by [email protected]
on 25 Oct 2012 at 2:48
from cqengine.
This is a good question. And quite a tricky one. On one hand, handling nulls
automatically would be helpful in many cases. On the other hand, handling nulls
automatically can have a big impact on performance.
The easy way to handle potential nulls in your data right now, is to check for
null in your implementation of attributes. Essentially you need to COALESCE
nulls to a value of your choosing, as you would do in SQL.
For example, here's a null-safe NAME attribute for a Car:
public static final Attribute<Car, String> NAME = new SimpleAttribute<Car, String>("name") {
public String getValue(Car car) {
String name = car.name;
return name == null ? "" : name;
}
};
You would only do this on fields which you know might contain nulls, for
performance reasons (see below).
Currently CQEngine follows the convention of the Java Collections Framework, by
throwing NullPointerException if null is supplied but is not expected. Some
collections in the JDK support nulls, some do not. For example
HashMap.put(null, null) will work, but ConcurrentHashMap.put(null, null) will
throw NullPointerException.
Checking for nulls can have an impact on performance in code which operates in
tight loops, and so I think the choice of null handling strategy needs to be a
user's decision.
In CQEngine, if attributes checked for nulls by default, it would have a big
impact on the performance of queries on attributes for which you haven't added
any indexes (CQEngine would need to read attributes and check for null values
in every object). For attributes on which you have added indexes, it wouldn't
make such a difference however (i.e. only when objects are added to the index
for the first time). Also, when indexing primitive fields, of course there can
never be an issue with nulls so it wouldn't make sense to check for nulls on
attributes on those fields.
Personally I don't like the extra lines of code required to implement the
null-safe attribute above. I will think about it and maybe look into adding an
additional type of attribute that would simplify it. For example there could be
a SimpleNullSafeAttribute or something. In the meantime the example above will
work for fields which might contain nulls.
Thanks for the report :)
Original comment by [email protected]
on 25 Oct 2012 at 11:04
- Added labels: Type-Enhancement
- Removed labels: Type-Defect
from cqengine.
Original comment by [email protected]
on 29 Oct 2012 at 1:50
- Changed state: Started
from cqengine.
Fixed in 1.0.0.
CQEngine now has more explicit support for attributes to indicate that "this
attribute does not apply to this object". CQEngine 1.0.0 supports new
"Nullable" attributes:
- http://cqengine.googlecode.com/svn/cqengine/javadoc/apidocs/com/googlecode/cqengine/attribute/SimpleNullableAttribute.html
- http://cqengine.googlecode.com/svn/cqengine/javadoc/apidocs/com/googlecode/cqengine/attribute/MultiValueNullableAttribute.html
In CQEngine 0.9.1, this was actually possible using MultiValueAttribute, where
if the attribute did not apply to a given object, MultiValueAttribute could
return an empty list (i.e. the empty set) as the value for that object.
CQEngine would then not add the object to any indexes built on that attribute.
So the engine supported this behaviour, but it wasn't very clear how to use it
before.
Here's a null-safe NAME attribute for a car based on the new
SimpleNullableAttribute in CQEngine 1.0.0:
public static final Attribute<Car, String> NAME = new SimpleNullableAttribute<Car, String>("name") {
public String getValue(Car car) { return car.name; }
};
So the code is simple again. 1.0.0 is available in the Downloads tab, and is
queued for sync with Maven central, it should appear there in a few hours.
Original comment by [email protected]
on 29 Oct 2012 at 9:59
- Changed state: Fixed
from cqengine.
This still doesn't work for me, using CQEngine:
<dependency>
<groupId>com.googlecode.cqengine</groupId>
<artifactId>cqengine</artifactId>
<version>1.2.6</version>
</dependency>
Property:
public static final Attribute<ConsumerLoad, String> ADDRESS2 = new
SimpleNullableAttribute<ConsumerLoad, String>("ADDRESS2") {
public String getValue(ConsumerLoad consumerload) {
String address2 = consumerload.address2;
return address2 == null ? "" : address2;
}
};
Error and dumps:
and(equal(ConsumerLoad.ADDRESS1, 5651 N ARLINGTON BLVD),
equal(ConsumerLoad.ADDRESS2, null))
-Update failed, load id 200061 : null
java.lang.NullPointerException
at com.googlecode.cqengine.query.simple.Equal.calcHashCode(Equal.java:79)
at com.googlecode.cqengine.query.simple.SimpleQuery.hashCode(SimpleQuery.java:102)
at java.util.AbstractList.hashCode(Unknown Source)
at com.googlecode.cqengine.query.logical.And.calcHashCode(And.java:76)
at com.googlecode.cqengine.query.logical.LogicalQuery.hashCode(LogicalQuery.java:110)
at java.util.concurrent.ConcurrentHashMap.get(Unknown Source)
at com.googlecode.cqengine.engine.impl.QueryEngineImpl.retrieveRecursive(QueryEngineImpl.java:301)
at com.googlecode.cqengine.engine.impl.QueryEngineImpl.retrieve(QueryEngineImpl.java:241)
at com.googlecode.cqengine.collection.impl.ConcurrentIndexedCollection.retrieve(ConcurrentIndexedCollection.java:79)
at com.palmcoastdata.cdb.update.ConsumerUpdate.update(ConsumerUpdate.java:166)
Original comment by [email protected]
on 5 Feb 2014 at 3:36
from cqengine.
public static final Attribute<ConsumerLoad, String> ADDRESS2 = new
SimpleNullableAttribute<ConsumerLoad, String>("ADDRESS2") {
public String getValue(ConsumerLoad consumerload) { return consumerload.address2; }
};
..Also does not work.
Original comment by [email protected]
on 5 Feb 2014 at 3:50
from cqengine.
Hi clint317,
You need to rephrase your query using 'has' instead:
http://cqengine.googlecode.com/svn/cqengine/javadoc/apidocs/com/googlecode/cqeng
ine/query/simple/Has.html
Replace: equal(ConsumerLoad.ADDRESS2, null)
With: not(has(ConsumerLoad.ADDRESS2))
..and it should work.
'has' in a CQEngine query is equivalent to 'IS NOT NULL' in an SQL query. It
means "does the object have a value for this attribute".
So 'not(has)' is the same as SQL 'IS NULL'.
Null values won't be stored in CQEngine indexes by default, so see the JavaDoc
for 'has' above, for guidance on accelerating these queries. That is, you can
add a StandingQueryIndex to accelerate it (if you wish).
Original comment by [email protected]
on 5 Feb 2014 at 5:21
from cqengine.
I don't want to assert that it has a value or doesn't have a value. I want to
assert that the property's value is equal to a given value.
select * where ConsumerLoad.ADDRESS2 = 'value';
--'value' may be null or defined
select * where ConsumerLoad.ADDRESS2 is null;
equal(ConsumerLoad.ADDRESS2, '123 Main St') - works
equal(ConsumerLoad.ADDRESS2, null) does not work
has(ConsumerLoad.ADDRESS2) - only test if it HAS a value and not if the value
matches something
Original comment by [email protected]
on 6 Feb 2014 at 1:36
from cqengine.
Null is not a value. Null is the absence of a value.
Your SQL statement..
select * where ConsumerLoad.ADDRESS2 = 'value';
..will not work as you expect if you substitute null for 'value'. It will not
match any rows. In SQL you should never see or use "WHERE column = null"
because it evaluates to false. See
http://stackoverflow.com/questions/3777230/is-there-any-difference-between-is-nu
ll-and-null
SQL has a special syntax for testing if a column does not have a value: "IS
NULL". CQEngine has a special syntax also: has, and not(has).
So you need to use 'has' if you want to test for null in a regular field (or
test for no values in a multi-valued field), or use 'equal' if you want to test
equality of non-null values. It is fairly consistent with the design of SQL.
Original comment by [email protected]
on 6 Feb 2014 at 2:30
from cqengine.
Ok, so what your saying is that 'equal()' is EXACTLY like SQL limitations on
null. I know how to use SQL and I'm aware you can't use x=null in a select. I
was under the impression that SimpleNullableAttribute meant I could use
'equal()' to test a value if the value might be null.
In other words there is no way to use CQEngine to do something like:
equal(ConsumerLoad.ADDRESS2, null);
Unless you want to use 'has()', or 'not(has())' to simply test for nulls and
only for nulls.
And if you use:
equal(ConsumerLoad.ADDRESS2, variable);
Equal() will throw an exception if variable is null even if the property is
SimpleNullableAttribute.
I can't even use:
in(ConsumerLoad.ADDRESS1, null, address.getAddress1());
If I make the property a MultiValueNullableAttribute.
In fact the ONLY way (least possible lines of code) to test a property against
a value that might or not be null is basically converting nulls to empty
strings:
public static final Attribute<ConsumerLoad, String> ADDRESS2 = new
SimpleNullableAttribute<ConsumerLoad, String>("ADDRESS2") {
public String getValue(ConsumerLoad consumerload) { return consumerload.address2==null?"":consumerload.address2; }
};
And:
equal(ConsumerLoad.ADDRESS2,
address.getAddress2()==null?"":address.getAddress2());
Ok, got it. Thank you.
Original comment by [email protected]
on 6 Feb 2014 at 7:30
from cqengine.
May I remind you that CQEngine is free of charge! So if you want things
improved, just supply a patch. Politely please! Take a look at the QueryFactory
class for examples of how queries are created.
If you want a combination of equal() and has(), just add a method like this to
your application:
public static <O, A> Query<O> nullSafeEqual(Attribute<O, A> attribute, A attributeValue) {
return attributeValue == null ? not(has(attribute)) : equal(attribute, attributeValue);
}
Original comment by [email protected]
on 6 Feb 2014 at 9:44
from cqengine.
Related Issues (20)
- Support STRICT_REPLACEMENT in ObjectLockingIndexedCollection/ConcurrentIndexedCollection
- TransactionalIndexedCollection support for UniqueIndex is broken
- Support for attribute accessors for Java record type HOT 1
- Kryo generates InaccessibleObjectException - Make Dependencies "Java 17 compatible"
- Bug in AttributeOrdersComparator HOT 5
- Offheap cache visibility HOT 1
- Support for ANTLR 4.10.1 HOT 12
- Does it support SQL statistics queries HOT 3
- Thread blocked when using Materialized View Sort For the Longest Prefix for CSV that contain more than 11 K rows HOT 3
- Error while parsing SQL NOT LIKE query HOT 1
- Error while parsing SQL LIKE '%%%' query
- java.lang.IllegalStateException thrown in the getGenericType(Member member) method of AttributeBytecodeGenerator.java
- CQNGrammar Sources missing from master HOT 1
- What is use of sqlite-jdbc in cqengine HOT 2
- Index updated even though objectStore is not modified during IndexedCollection add HOT 2
- Does CQEngine compatible with Java 17 HOT 2
- Issues with Offheap Concurrent Index collection clear()
- Getting below error on jdk17.0.8.8.1 HOT 1
- Is CQEngine officially dead? HOT 6
- com/googlecode/concurrenttrees/radix/node/NodeFactory
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cqengine.