speedment / jpa-streamer Goto Github PK
View Code? Open in Web Editor NEWJPAstreamer is a lightweight library for expressing JPA queries as Java Streams
License: GNU Lesser General Public License v2.1
JPAstreamer is a lightweight library for expressing JPA queries as Java Streams
License: GNU Lesser General Public License v2.1
If at least one of the following conditions are true:
Then, the longest consecutive string of intermediate operations starting from the back that only affects the order of elements (i.e. Sorted) can be removed.
We need to set up a CI-process via GH Actions to keep track off the current build status.
Here is a guide that explains how that is done:
https://docs.github.com/en/actions/building-and-testing-code-with-continuous-integration/setting-up-continuous-integration-using-github-actions
As JPAstreamer makes it easy to stream any database table there are many operations which can be added that results in a non-optimized stream. That is for example the case when accessing fields that depend on other tables. See for example:
long count = jpaStreamer.stream(Film.class)
.filter(f -> f.getActors()
.stream()
.filter(Actor$.lastName.startsWith("A"))
.count() > 0)
.count();
Here, the predicate on the fourth row (actors last names starts with A) should optimally be included as a where-clause in the original Join. As of now, we have to materialize every actor even though we are only interested in a subset of them.
It would be good to find a way to optimize similar queries.
Reproduce:
cd jpa-streamer-test
mvn clean install -e
Produces:
Caused by: java.lang.ClassNotFoundException: com.speedment.jpastreamer.field.ComparableField
at java.net.URLClassLoader.findClass (URLClassLoader.java:471)
at java.lang.ClassLoader.loadClass (ClassLoader.java:588)
at java.lang.ClassLoader.loadClass (ClassLoader.java:521)
at com.speedment.jpastreamer.fieldgenerator.standard.internal.InternalFieldGeneratorProcessor.referenceType (InternalFieldGeneratorProcessor.java:236)
at com.speedment.jpastreamer.fieldgenerator.standard.internal.InternalFieldGeneratorProcessor.addFieldToClass (InternalFieldGeneratorProcessor.java:145)
at com.speedment.jpastreamer.fieldgenerator.standard.internal.InternalFieldGeneratorProcessor.lambda$generatedEntity$3 (InternalFieldGeneratorProcessor.java:126)
at java.lang.Iterable.forEach (Iterable.java:75)
at com.speedment.jpastreamer.fieldgenerator.standard.internal.InternalFieldGeneratorProcessor.generatedEntity (InternalFieldGeneratorProcessor.java:125)
at com.speedment.jpastreamer.fieldgenerator.standard.internal.InternalFieldGeneratorProcessor.generateFields (InternalFieldGeneratorProcessor.java:108)
at com.speedment.jpastreamer.fieldgenerator.standard.internal.InternalFieldGeneratorProcessor.lambda$process$1 (InternalFieldGeneratorProcessor.java:75)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept (ForEachOps.java:183)
at java.util.stream.ReferencePipeline$2$1.accept (ReferencePipeline.java:177)
at java.util.Iterator.forEachRemaining (Iterator.java:133)
I ran into a troublesome issue when trying to build the demo repository. It contains three entities; Film
, Actor
and Language
. Film
has an attribute named language of type Language
as shown below:
@Entity
@Table(name = "film", schema = "sakila")
public class Film implements Serializable {
/...
@ManyToOne
@JoinColumn(name="language_id", nullable = false)
private Language language;
// ... getters and setters
}
When running mvn clean install
, the field-generator throws the following exception:
com.speedment.jpastreamer.fieldgenerator.exception.FieldGeneratorProcessorException: Type with name com.speedment.jpastreamer.test.model.Language was not found.
This is thrown as com.speedment.jpastreamer.test.model.Language
has not yet been compiled, hence cannot be found by the class parser. This means we are left with a bootstrap issue. We need a compiled version of the JPA Entity-model before we run the annotation-processor, yet the application logic relies on the output of the annotation-processor and cannot be compiled until we have generated the JPAstreamer metamodel.
If run under the module system, the rootfactory cannot load services because it does not declare uses
on any and all modules. We need to provide some kind of "proxy" to do this.
The following order shall be used:
Provider
Some terminal operations can be expressed as a combination of multiple intermediate operations, making them easier to represent in SQL:
AnyMatch
.anyMatch(p)
can be expressed as .filter(p).limit(1).count() > 0)
NoneMatch
.noneMatch(p)
can be expressed as .filter(p).limit(1).count() = 0
Some terminal operations can be enhanced with certain intermediate operations in order to optimize the SQL query that is being executed:
FindFirst
.findFirst()
can be expressed as .limit(1).findFirst()
to restrict the size of the ResultSet
FindAny
can be expressed as .unordered().limit(1).findAny()
Starting from the end, the longest string of Intermediate operators that preserve the Size property can safely be removed.
Skip and limit intermediate operations can be directly merged into the JPA Query when they are in the following forms:
.skip(x)
-> OFFSET x
.limit(x)
-> LIMIT x
.skip(x).limit(y)
-> OFFSET x LIMIT y
.limit(x).skip(y)
should be rendered to LIMIT x
as the semantics are not the same as .skip(x).limit(y)
Due to the way JPA Criteria works, we can only fully optimize count terminators if all intermediate operations can be merged into the JPA Criteria. If there are leftover intermediate operations after all optimizations are performed, we can either ignore the leftover operations or count termination has to be performed on the JVM side.
Otherwise, the EntityManagerFactory
might leave resources open.
remove annotation dependency
Swap order of D and V parameters to conform to AttributeConverter
Remove speedment dependencies
The amount of unit tests we currently have is quite low. Postponing this could create a big backlog.
Modules:
As of now, the annotation processor is tightly integrated with the implementation of fieldgenerator-standard. It would be beneficiary if we separated the annotation processor from the fieldgenerator implementation to easily plug-in alternative implementations of the generator, e.g. fieldgenerator-enterprise.
Applications that want to make use of JPA Streamer should have a single dependency that they can use to pull in the most relevant modules:
<dependency>
<groupId>com.speedment.jpastreamer</groupId>
<artifactId>jpastreamer-core</artifactId>
<version>0.1.0</version>
</dependency>
If entity has columns using the same simple name, as:
@Column(name = "release_date", nullable = false, columnDefinition = "DATE")
@Temporal(TemporalType.DATE)
private java.util.Date releaseDate;
@Column(name = "release_time", nullable = false, columnDefinition = "DATE")
private java.sql.Date releaseTime;
The generated code cannot be compiled as the reference type is not rendered properly.
public static final Date> releaseDate = ComparableField.create(
Film.class,
"release_date",
Film::getReleaseDate,
false
);
What I know so far is that this error cannot be derived to the Speedment generator nor the parsing of types.
Anonymous Lambdas should be moved down the order as much as possible to allow as much as possible of the stream to be executed on the database side.
The current fields need to change to fit JPA. For example,
static <ENTITY, D, V extends Comparable<? super V>>
ComparableField<ENTITY, D, V> create(
ColumnIdentifier<ENTITY> identifier,
ReferenceGetter<ENTITY, V> getter,
ReferenceSetter<ENTITY, V> setter,
TypeMapper<D, V> typeMapper,
boolean unique) {
return new ComparableFieldImpl<>(
identifier, getter, setter, typeMapper, unique
);
}
should be
static <ENTITY, D, V extends Comparable<? super V>>
ComparableField<ENTITY, D, V> create(
Class<ENTITY> root, // Changed
ReferenceGetter<ENTITY, V> getter,
ReferenceSetter<ENTITY, V> setter,
Class<AttributeConverter<V, D>> attributeConverter, // Changed
boolean unique) {
return new ComparableFieldImpl<>(
root, getter, setter, attributeConverter, unique
);
}
Exception in thread "main" java.lang.UnsupportedOperationException
at com.speedment.jpastreamer.builder.standard.internal.StreamBuilder.mapToInt(StreamBuilder.java:84)
at com.speedment.jpastreamer.autoclose.standard.internal.AutoClosingStream.mapToInt(AutoClosingStream.java:64)
at com.speedment.jpastreamer.demo.internal.MapToInt.main(MapToInt.java:20)
This has been moved to a separate repository.
Nice if we can un-depend on this legacy library.
Ideally, we should remove speedment dependencies.
module jpastreamer.field {
requires com.speedment.common.invariant;
requires transitive java.persistence;
requires transitive com.speedment.common.tuple;
requires transitive com.speedment.common.function;
requires transitive com.speedment.common.annotation;
requires transitive com.speedment.runtime.config;
requires transitive com.speedment.runtime.compute;
requires transitive com.speedment.runtime.typemapper;
exports com.speedment.jpastreamer.field;
exports com.speedment.jpastreamer.field.collector;
exports com.speedment.jpastreamer.field.comparator;
exports com.speedment.jpastreamer.field.exception;
exports com.speedment.jpastreamer.field.expression;
exports com.speedment.jpastreamer.field.method;
exports com.speedment.jpastreamer.field.predicate;
exports com.speedment.jpastreamer.field.predicate.trait;
exports com.speedment.jpastreamer.field.trait;
exports com.speedment.jpastreamer.field.util;
}
Filter intermediate operations can be merged into JPA queries if they use Speedment Predicates to describe filter conditions.
Depends on #10
Intermediate operations appearing in sequential order that are of the same type can be squashed into a single operation, making them easier to optimize later.
Examples:
.skip(5).skip(5)
can be squashed into .skip(10)
.filter(_Film.filmId.between(1, 100)).filter(_Film.filmId.greaterThan(100))
can be squashed into .filter(_Film.filmId.between(1, 100).and(_Film.filmId.greaterThan(100))
We need a way to map Speedment Predicates to JPA Predicates. Here's a list of Speedment Predicate types for a easier tracking:
Constants
ALWAYS_TRUE / ALWAYS_FALSE
Reference
IS_NULL / IS_NOT_NULL
Comparable
EQUAL / NOT_EQUAL
GREATER_THAN
GREATER_OR_EQUAL
LESS_THAN
LESS_OR_EQUAL
BETWEEN / NOT_BETWEEN
IN / NOT_IN
String
EQUAL_IGNORE_CASE / NOT_EQUAL_IGNORE_CASE
STARTS_WITH / NOT_STARTS_WITH
STARTS_WITH_IGNORE_CASE / NOT_STARTS_WITH_IGNORE_CASE
ENDS_WITH / NOT_ENDS_WITH
ENDS_WITH_IGNORE_CASE / NOT_ENDS_WITH_IGNORE_CASE
CONTAINS / NOT_CONTAINS
CONTAINS_IGNORE_CASE / NOT_CONTAINS_IGNORE_CASE
IS_EMPTY / IS_NOT_EMPTY
We need a way to guarantee certain pre-optimizers (operation reordering) will run before others. The pre-optimizer factory should be rewritten to support prioritization.
There are several advanced topics covered in the Speedment manual which are yet to be covered in the JPAstreamer user guide:
<dependency>
<groupId>com.speedment.jpastreamer</groupId>
<artifactId>application</artifactId>
<version>${jpa-streamer.version}</version>
</dependency>
We have earlier decided to write the documentation in AsciiDocs and version handle it through GitHub. The documents are then rendered with Antora and hosted on Github Pages.
The Antora build process is later to be automated.
Currently all Java Collections are rendered to ReferenceField
and Timestamps are generated as ComparableField
. These Fields contain methods which may not be meaningful in the given context, hence we should implement specialized Field to better suit the properties of the column e.g. CollectionField
.
This might be related to #19 because not all modules are defined
Field predicates could implement HasArg0<T0>
,HasArg1<T1>
and HasArg2<T2>
instead.
interface HasArg0<T0> {
T0 get0();
}
Retaining the Tuple getter names would require minimum effort.
JPA's CriteriaBuilder::between
method is inclusive on both ends (c >= x AND c <= y
) and it seems that this behaviour can't be modified. This can cause problems when mapping our Predicates that contain an inclusion strategy different from the one JPA provides.
The between
method accepts Comparable
objects to determine the start and end of the range. While we can mimic different inclusions for Number
types by incrementing/decrementing the provided values, we can't really account for all possible Comparable
types that might be provided.
But do not implement AutoCloseable
autoclose and more
Field such as ComparableField<ENTITY, D, V>
and ReferenceField<ENTITY, D, V>
require the database column type as an argument. We should investigate if this information is required for JPAStreamer.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.