oliver-loeffler / image-registration Goto Github PK

All transforms in alignment package take a collection of displacements and return a collection of displacements. Unlike thise in distortions package. Here each transform takes a single displacement and returns a single displacement.

This does not feel good and straight forward. Ideally transforms in both package would be structured in same way. For distortions, there is a class AffineTransformCorrection which is a bi-function which takes a predicate to Displacement and a collection of displacements.
A similar way could be used for alignments OR distortion transforms should work like alignment functions.

The question is:

• either Function<Collection,Collection>
• or Function<Displacement,Displacement>

Example from SimpleRigidTransform.java:

@Override
public Collection<Displacement> apply(Collection<Displacement> d) {
	return d.stream()
		 .map(mappper())
		 .collect(Collectors.toList());
}
	
private Function<Displacement,Displacement> mappper() {
	return source -> Displacement.from(source, 
			source.getXd() - this.getTranslationX() + source.getY() * this.getRotation(), 
			source.getYd() - this.getTranslationY() - source.getX() * this.getRotation());
	

Example in AffineTransformCorrection.java:

public class AffineTransformCorrection implements 
    BiFunction<AffineTransform, Collection<Displacement>, Collection<Displacement>> {
  
    @Override
    public Collection<Displacement> apply(AffineTransform t, Collection<Displacement> u) {
       return u.stream()
                    .map(t)
                    .collect(Collectors.toList());
    }
}

Written posts, blog entries or tweets will look more appealing when there is a nice preview image.

Review project and library naming - right now 'image-registration-image-registration' reads kind of weird.

Make a bundle (JavaDoc, Examples, Package Description) and publish this to Maven Central.

Add a non-linear alignment function and compare linear-alignment vs. non-linear alignment in terms of precision and performance.

Switch from Travis-CI to Github actions.

• Update README.md with class and interface hierarchy
• Update how data processing works for FirstOrder
• Complete JAVADOC with examples
• Update code examples in README.md

Idea is to break the project into multiple artifacts to slim down the dependencies.
Currently 4 matrix libraries are references. This will hold back users to use image-registration.

Instead, provide one image-registration JAR which contains the API and all domain types. Then, provide separate artifacts as per solver implementation. So the user can take the main-JAR and add the desired solver dependency. On other hand, one could add the main-JAR and implement its own solver.

Following approach is thinkable:

• Create multi-build project
• sub project: solver-api (not a public jar)
• sub project: image-registration (depends on solver-api, public jar)
• implement classes for solver service discovery
• implement (a) lazy service discovery (first come, first serve) and (b) service discovery using class name. It should be possible to use different solver implementations per test
• sub project: JAMA solver (depends on solver-api, public jar)
• sub project: LA4J solver (depends on solver-api, public jar)
• sub project: EJML solver (depends on solver-api, public jar)
• sub project: Apache Math Commons 3 solver (depends on solver-api, public jar)

• multiple displacements along a straight line in either X or Y direction
• multiple displacements at a single point

These cases usually end up with singular matrix exceptions or errors but basically a handling is possible.

Ideally this condition is detected before an exception occurs, so that the error handler in AffineTransformCalculation can be eliminated later on.

Underlying matrix libraries shall be exchangeable. There are different libraries available, such as La4J, Ejml, JAMA or Apache Math Commons. All of these provide means to solve systems of linear equations.

It would be of advantage, if the user would be able to opt-in for the library of choice.

Consider a more generic approach to decouple matrix computation from image-registration API and abstracttion.

The AffineTransformCalculation function centers the given data before doing any calculations. There are cases, where this centering is not wanted. To cover those cases, try following approach:

• remove the centering code from AffineTransformCalculation and instead compose it with a variant of net.raumzeitfalle.registration.alignment.TranslateToCenter.
• A new version of TranslateToCenter is needed, as the center calculation should be controlled by a Predicate<Displacement>.

Move sources to Java 11 and keep binaries compatible with Java 8.
Don't create a module yet.

Using the option mvn --no-transfer-progress will reduce clutter from Travis CI log output.

Either finish implementation or throw code away. Technically the similarity transform can be implemented on-top of the AffineTransform with appropriate limitations set.

JBLAS is another great LA library for Java.
Add an interface module to utilize JBLAS.

http://jblas.org/

No class in net.raumzeitfalle.registration.displacement has a Javadoc annotation.
As these classes are the base classes, add javadoc.

Add support for image distortions of higher (polynomial) order.

Requirements:

• support for 2nd order including cross-terms kn*X*Y, 2nd order means up to km*X^2*Y^2
• model shall be parametrical, so that each coefficient can be toggled independently (e.g. equation such as t + a*X + b*Y + c*X^2 + d*X*Y + e*X^2*Y + f * X*Y^2 + g*X^2 * Y^2 etc.). Each coefficient in {a,b,c,d,e,f,g} should be independently configured to be enabled/disabled.
• Ideally a builder is there to create the configuration.
• The model should determine best fit conditions independently depending on data Orientation (Dimension) and SpatialDistribution.
• Ideally, model should be expandable to higher orders beyond 2nd order - but right now I have no good idea how to do that.

Currently numerics are untested - demonstrate correctness of numerics with tests and examples for each transform type.

Modularization can be beneficial. At least, identify all possible illegal access pattern and references. The whole library should be moved into one module.