tarantool / cartridge-springdata Goto Github PK
View Code? Open in Web Editor NEWSpring Data Tarantool
License: Other
Spring Data Tarantool
License: Other
Partial search does not work with multipart index.
Since null is serialized to box.NULL.
repro:
Optional<?> findResponse = repository.findById(
ClientChannelId.builder()
.mdmId(1)
.build());
or
Optional<?> findResponse = repository.findById(
ClientChannelId.builder()
.mdmId(1)
.channelType(null)
.build());
org.springframework.dao.DataRetrievalFailureException: InnerErrorMessage:
str: SelectError: Supplied key type of part 1 does not match index part type: expected string
There is a line that this is a temporary hack, but in order not to lose and forget, I will add a ticket for this problem
Create a test that shows the mapping behavior of the Date data type from java to tarantool. What type of data is the conversion expected to be? (Number by my testing)
@Schema(description = "Some date")
private Date date;
And this look strange:
It is also necessary to discuss why some converters are in cartridge-java, and some must be related to spring converters. This is necessary in order to understand which pipeline the date passes from spring to tarantool, or from carg-java to tarantool.
See the description of reactive repositories for Mongo https://docs.spring.io/spring-data/mongodb/docs/current/reference/html/#mongo.reactive.repositories
Since the nature and API of the cartridge-java
driver is asynchronous, this task should not require much effort.
Reproducer for this, try convert string from tarantool to LocalDate
Create a new project with an example and add a link to it to the README.
References:
https://www.baeldung.com/spring-data-mongodb-tutorial
https://github.com/pacphi/spring-boot-mongodb-example
https://www.concretepage.com/spring-5/spring-data-mongotemplate
If there is no ID annotation in the tuple entity, then an incomprehensible error:
@Tuple("entity")
public class Entity {
// @Id
// private Integer id;
2021-03-09 11:31:19.670 WARN 46239 --- [ restartedMain] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'vetController' defined in file [/Users/artyom.dubinin/tarantool/spring-petclinic/target/classes/org/springframework/samples/petclinic/vet/VetController.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'vetRepository' defined in org.springframework.samples.petclinic.vet.VetRepository defined in @EnableTarantoolRepositories declared on TarantoolConfiguration: Invocation of init method failed; nested exception is java.lang.NullPointerException
2021-03-09 11:31:21.793 INFO 46239 --- [ restartedMain] o.apache.catalina.core.StandardService : Stopping service [Tomcat]
2021-03-09 11:31:21.813 INFO 46239 --- [ restartedMain] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2021-03-09 11:31:21.859 ERROR 46239 --- [ restartedMain] o.s.boot.SpringApplication : Application run failed
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'vetController' defined in file [/Users/artyom.dubinin/tarantool/spring-petclinic/target/classes/org/springframework/samples/petclinic/vet/VetController.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'vetRepository' defined in org.springframework.samples.petclinic.vet.VetRepository defined in @EnableTarantoolRepositories declared on TarantoolConfiguration: Invocation of init method failed; nested exception is java.lang.NullPointerException
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:800) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1356) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1206) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:571) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:531) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:944) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:923) ~[spring-context-5.3.3.jar:5.3.3]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:588) ~[spring-context-5.3.3.jar:5.3.3]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:144) ~[spring-boot-2.4.2.jar:2.4.2]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:767) [spring-boot-2.4.2.jar:2.4.2]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:759) [spring-boot-2.4.2.jar:2.4.2]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:426) [spring-boot-2.4.2.jar:2.4.2]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:326) [spring-boot-2.4.2.jar:2.4.2]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1311) [spring-boot-2.4.2.jar:2.4.2]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1300) [spring-boot-2.4.2.jar:2.4.2]
at org.springframework.samples.petclinic.PetClinicApplication.main(PetClinicApplication.java:32) [classes/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_282]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_282]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_282]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_282]
at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49) [spring-boot-devtools-2.4.2.jar:2.4.2]
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'vetRepository' defined in org.springframework.samples.petclinic.vet.VetRepository defined in @EnableTarantoolRepositories declared on TarantoolConfiguration: Invocation of init method failed; nested exception is java.lang.NullPointerException
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1788) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:609) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:531) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1380) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1300) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ~[spring-beans-5.3.3.jar:5.3.3]
... 25 common frames omitted
Caused by: java.lang.NullPointerException: null
at org.springframework.data.tarantool.repository.support.MappingTarantoolEntityInformation.<init>(MappingTarantoolEntityInformation.java:20) ~[spring-data-tarantool-0.3.3.jar:na]
at org.springframework.data.tarantool.repository.support.TarantoolRepositoryFactory.getEntityInformation(TarantoolRepositoryFactory.java:37) ~[spring-data-tarantool-0.3.3.jar:na]
at org.springframework.data.tarantool.repository.support.TarantoolRepositoryFactory.getTargetRepository(TarantoolRepositoryFactory.java:44) ~[spring-data-tarantool-0.3.3.jar:na]
at org.springframework.data.repository.core.support.RepositoryFactorySupport.getRepository(RepositoryFactorySupport.java:281) ~[spring-data-commons-2.4.3.jar:2.4.3]
at org.springframework.data.repository.core.support.RepositoryFactoryBeanSupport.lambda$afterPropertiesSet$6(RepositoryFactoryBeanSupport.java:326) ~[spring-data-commons-2.4.3.jar:2.4.3]
at org.springframework.data.util.Lazy.getNullable(Lazy.java:230) ~[spring-data-commons-2.4.3.jar:2.4.3]
at org.springframework.data.util.Lazy.get(Lazy.java:114) ~[spring-data-commons-2.4.3.jar:2.4.3]
at org.springframework.data.repository.core.support.RepositoryFactoryBeanSupport.afterPropertiesSet(RepositoryFactoryBeanSupport.java:329) ~[spring-data-commons-2.4.3.jar:2.4.3]
at org.springframework.data.tarantool.repository.support.TarantoolRepositoryFactoryBean.afterPropertiesSet(TarantoolRepositoryFactoryBean.java:60) ~[spring-data-tarantool-0.3.3.jar:na]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1847) ~[spring-beans-5.3.3.jar:5.3.3]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1784) ~[spring-beans-5.3.3.jar:5.3.3]
... 36 common frames omitted
Process finished with exit code 1
Currently, the driver does not support automatic updating the schema.
Currently, only the simple ID types (containing only one field) are supported, but Tarantool offers multipart keys, which must be supported.
Use static analysis tools similar to cartridge-java project.
lombok seems to be too controversial to be transitive dependency for a general purpose library.
The fix for this issue may also deal with #32 and #28.
Please check cases there.
Scenario:
function test()
return {
testString = "hello world",
testInt = 123,
}
end
public Class TestEntity {
String testString ;
Integer testInt;
}
// ...
TestEntity t = tarantoolTemplate.call("test");
This is due to retrieving the field name using the wrong method getName
, see
getFieldName
must be used instead.
Some kind of shared resource access occurs when using the default tuple deserialization. Probably the message mapper is configured and then shared between different threads executing requests.
Possible root cause:
Internal modification of the used MessagePack mapper (DefaultMessagePackMapper)
Integration tests do not execute:
See any worklflow 'Build' step log. For example:
https://github.com/tarantool/cartridge-springdata/pull/21/checks#step:5:4026
...
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.035 s - in org.springframework.data.tarantool.repository.support.RepositoryIntegrationTest
...
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 207.488 s - in org.springframework.data.tarantool.core.TarantoolTemplateTest
At the moment, if we need to work with several tarantool instances at once, we have to configure and create service beans ourselves.
@Configuration
@PropertySource(value = "classpath:application-tarantool.properties", encoding = "UTF-8")
@EnableTarantoolRepositories(basePackageClasses = UserTarantoolRepository.class)
public class TarantoolConfiguration extends AbstractTarantoolDataConfiguration {
@Value("${tarantool.nodes}")
private String nodes;
@Value("${tarantool.username}")
private String username;
@Value("${tarantool.password}")
private String password;
@Override
@Bean("tarantoolClusterAddressProvider")
public TarantoolClusterAddressProvider tarantoolClusterAddressProvider() {
List<TarantoolServerAddress> addressList = new ArrayList<>();
for (String address : nodes.split(",")) {
String[] parts = address.split(":");
String host = parts[0];
int port = Integer.parseInt(parts[1]);
addressList.add(new TarantoolServerAddress(host, port));
}
return () -> addressList;
}
@Override
protected void configureClientConfig(TarantoolClientConfig.Builder builder) {
builder.withCredentials(new SimpleTarantoolCredentials(username, password));
}
}
I propose to add the ability to configure the application through a set of properties and assign the logic of their interpretation and application, for example, to AbstractTarantoolDataConfiguration.
For example, we can add support for properties such as
spring.data.tarantool.nodes=localhost:3301,172.1.0.2:3301,172.1.0.3:3303 (default - 127.0.0.1:3301)
spring.data.tarantool.username=admin (default - "guest")
spring.data.tarantool.password=passw0rd (default - "")
spring.data.tarantool.connection-selection-strategy=RoundRobinStrategy (default - "ParallelRoundRobinStrategy")
...
Need to add tests on cases:
Cover custom bucket id in readme.
The derived queries are described here: https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#jpa.query-methods.query-creation
Currently only the queries of "declared" type are supported (via the @query annotation and specification of the stored functoin name).
Drived queries support may rely on the driver's Conditions
class: https://github.com/tarantool/cartridge-java/blob/master/src/main/java/io/tarantool/driver/api/conditions/Conditions.java
Given:
A entity with a nested object which may be null
Then
The mapping read converter threats this property as a map and passes null
to the custom type conversion, and there's a missing nullability check.
Reference:
See the last run results
https://github.com/tarantool/cartridge-springdata/actions/runs/664124454
public interface PetRepository extends TarantoolRepository<Pet, Integer> {
/**
* Retrieve all {@link PetType}s from the data store.
* @return a Collection of {@link PetType}s.
*/
// @Query("SELECT ptype FROM PetType ptype ORDER BY ptype.name")
// @Transactional(readOnly = true)
@Query(function = "find_pet_types")
List<PetType> findPetTypes();
local function find_pet_types()
return crud.select("types")
end
2021-03-09 11:38:19.759 ERROR 46506 --- [nio-8080-exec-8] o.a.c.c.C.[.[.[/].[dispatcherServlet]: Servlet.service() for
servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is
org.springframework.core.convert.ConversionFailedException: Failed to convert from type [java.util.ArrayList<?>] to type
[@org.springframework.data.tarantool.repository.Query java.util.List<org.springframework.samples.petclinic.owner.PetType>] for value '[cat, dog, lizard, snake, bird, hamster]';
nested exception is org.springframework.core.convert.ConverterNotFoundException: No converter found capable of
converting from type [org.springframework.samples.petclinic.owner.Pet] to type
[@org.springframework.data.tarantool.repository.Query org.springframework.samples.petclinic.owner.PetType]] with root
cause
org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type
[org.springframework.samples.petclinic.owner.Pet] to type [@org.springframework.data.tarantool.repository.Query org.springframework.samples.petclinic.owner.PetType]
Incoming data, which is not a map data structure, is trying to convert to a map data type
public Optional<CustomObject> updateCrossReferences(long code, List<InnerObject> innerObjects, LocalDateTime creationDateTime, final Map<String, String> headers) {
Map<String, String> credentials = prepareTdgCredentials(headers);
var args = new HashMap<>();
args.put("index", "code_index");
args.put("value", code);
args.put("innerObjects", innerObjects);
args.put("creationDateTime", creationDateTime);
var customObjects = repository.callService("update", args, credentials);
}
"timestamp":"2021-10-27T16:00:19.105+00:00","status":500,"error":"Internal Server Error","trace":"io.tarantool.driver.mappers.MessagePackObjectMapperException: ObjectConverter for type class InnerObject is not found
For example, Spring Data module for Cassandra has customConversions
bean too. The result of using both Tarantool and Cassandra modules is ambiguity and wrong conversions of Java types to types supported by the driver.
Spring Data has its own hierarchy of exceptions converted from the underlying data layer exceptions. We need to classify the driver exceptions and rewrite DefaultTarantoolExceptionTranslator
.
Analysis needed
The connector normally writes Double data of 0.0
or 0
, but there is a problem to read it back.
Probably cartridge-java converts the number 0.0
from Tarantool to Integer
, and then the spring-data converter cannot find a way to convert Integer
to Double
Example from tdg:
{
"name" : "baseCapitalAmountCur",
"type" : [ "null", "double" ]
}
private Double baseCapitalAmountCur;
2021-11-01 14:25:43.256 ERROR 3916 --- [nio-8080-exec-3] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.dao.DataRetrievalFailureException: org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [java.lang.Integer] to type [java.lang.Double]; nested exception is org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [java.lang.Integer] to type [java.lang.Double]] with root cause
org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [java.lang.Integer] to type [java.lang.Double]
at org.springframework.core.convert.support.GenericConversionService.handleConverterNotFound(GenericConversionService.java:322) ~[spring-core-5.3.8.jar:5.3.8]
at org.springframework.core.convert.support.GenericConversionService.convert(GenericConversionService.java:195) ~[spring-core-5.3.8.jar:5.3.8]
at org.springframework.core.convert.support.GenericConversionService.convert(GenericConversionService.java:175) ~[spring-core-5.3.8.jar:5.3.8]
at org.springframework.data.mapping.model.ConvertingPropertyAccessor.convertIfNecessary(ConvertingPropertyAccessor.java:120) ~[spring-data-commons-2.5.1.jar:2.5.1]
at org.springframework.data.mapping.model.ConvertingPropertyAccessor.setProperty(ConvertingPropertyAccessor.java:63) ~[spring-data-commons-2.5.1.jar:2.5.1]
at org.springframework.data.tarantool.core.convert.MappingTarantoolReadConverter$TarantoolPropertyValueProvider.lambda$convertCustomType$0(MappingTarantoolReadConverter.java:277) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.mapping.model.BasicPersistentEntity.doWithProperties(BasicPersistentEntity.java:360) ~[spring-data-commons-2.5.1.jar:2.5.1]
at org.springframework.data.tarantool.core.convert.MappingTarantoolReadConverter$TarantoolPropertyValueProvider.convertCustomType(MappingTarantoolReadConverter.java:269) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.convert.MappingTarantoolReadConverter$TarantoolPropertyValueProvider.convertIfNeeded(MappingTarantoolReadConverter.java:321) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.convert.MappingTarantoolReadConverter$TarantoolPropertyValueProvider.readValue(MappingTarantoolReadConverter.java:238) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.convert.MappingTarantoolReadConverter$TarantoolPropertyValueProvider.access$100(MappingTarantoolReadConverter.java:168) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.convert.MappingTarantoolReadConverter$TarantoolPropertyValueProvider$1.getPropertyValue(MappingTarantoolReadConverter.java:262) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.convert.MappingTarantoolReadConverter$TarantoolPropertyValueProvider$1.getPropertyValue(MappingTarantoolReadConverter.java:258) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.convert.MappingTarantoolReadConverter$TarantoolPropertyValueProvider.lambda$convertCustomType$0(MappingTarantoolReadConverter.java:273) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.mapping.model.BasicPersistentEntity.doWithProperties(BasicPersistentEntity.java:360) ~[spring-data-commons-2.5.1.jar:2.5.1]
at org.springframework.data.tarantool.core.convert.MappingTarantoolReadConverter$TarantoolPropertyValueProvider.convertCustomType(MappingTarantoolReadConverter.java:269) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.convert.MappingTarantoolReadConverter$TarantoolPropertyValueProvider.convertIfNeeded(MappingTarantoolReadConverter.java:321) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.convert.MappingTarantoolReadConverter$TarantoolPropertyValueProvider.readValue(MappingTarantoolReadConverter.java:238) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.convert.MappingTarantoolReadConverter$TarantoolPropertyValueProvider.getPropertyValue(MappingTarantoolReadConverter.java:216) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.convert.MappingTarantoolReadConverter.setProperty(MappingTarantoolReadConverter.java:156) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.convert.MappingTarantoolReadConverter.lambda$convertProperties$0(MappingTarantoolReadConverter.java:142) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.mapping.model.BasicPersistentEntity.doWithProperties(BasicPersistentEntity.java:360) ~[spring-data-commons-2.5.1.jar:2.5.1]
at org.springframework.data.tarantool.core.convert.MappingTarantoolReadConverter.convertProperties(MappingTarantoolReadConverter.java:138) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.convert.MappingTarantoolReadConverter.read(MappingTarantoolReadConverter.java:113) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.convert.MappingTarantoolConverter.read(MappingTarantoolConverter.java:73) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.BaseTarantoolTemplate.mapToEntity(BaseTarantoolTemplate.java:385) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.BaseTarantoolTemplate.lambda$null$21(BaseTarantoolTemplate.java:361) ~[spring-data-tarantool-0.4.2.jar:na]
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[na:na]
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133) ~[na:na]
at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[na:na]
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[na:na]
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[na:na]
at org.springframework.data.tarantool.core.BaseTarantoolTemplate.lambda$getListValueConverter$2f00f61b$1(BaseTarantoolTemplate.java:362) ~[spring-data-tarantool-0.4.2.jar:na]
at io.tarantool.driver.core.SingleValueCallResultImpl.<init>(SingleValueCallResultImpl.java:32) ~[cartridge-driver-0.6.0.jar:na]
at io.tarantool.driver.mappers.SingleValueCallResultConverter.fromValue(SingleValueCallResultConverter.java:25) ~[cartridge-driver-0.6.0.jar:na]
at io.tarantool.driver.mappers.SingleValueCallResultConverter.fromValue(SingleValueCallResultConverter.java:13) ~[cartridge-driver-0.6.0.jar:na]
at io.tarantool.driver.mappers.DefaultMessagePackMapper.fromValue(DefaultMessagePackMapper.java:109) ~[cartridge-driver-0.6.0.jar:na]
at io.tarantool.driver.mappers.DefaultMessagePackMapper.fromValue(DefaultMessagePackMapper.java:89) ~[cartridge-driver-0.6.0.jar:na]
at io.tarantool.driver.mappers.AbstractResultMapper.fromValue(AbstractResultMapper.java:34) ~[cartridge-driver-0.6.0.jar:na]
at io.tarantool.driver.handlers.TarantoolResponseHandler.channelRead0(TarantoolResponseHandler.java:51) ~[cartridge-driver-0.6.0.jar:na]
at io.tarantool.driver.handlers.TarantoolResponseHandler.channelRead0(TarantoolResponseHandler.java:23) ~[cartridge-driver-0.6.0.jar:na]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]
During call of spring-data function deleteByAuthKey() of next repository:
@Repository
public interface PortfolioTarantoolRepository extends TarantoolRepository<PortfolioEntity, String> {
@Query(function = "delete_by_auth_key")
Optional<PortfolioEntity> deleteByAuthKey(String authKey);
}
when Tarantool unavailable (service started with wrong Tarantool address)
we don't receive any exception/errors in logs.
The following converters are necessary, since MessagePack reduces the numbers to the smallest value, and there is no double
type in Tarantool 1.10:
Description:
If call/callForList methods are called with non-entity return type and invoke a stored function which returns a list of tuples, the result deserialization fails with an error like:
org.springframework.dao.DataRetrievalFailureException: io.tarantool.driver.mappers.MessagePackValueMapperException: ValueConverter for type class org.msgpack.value.impl.ImmutableArrayValueImpl is not found; nested exception is io.tarantool.driver.mappers.MessagePackValueMapperException: ValueConverter for type class org.msgpack.value.impl.ImmutableArrayValueImpl is not found
If the stored function returns a list of tables (maps), the methods work normally. But calling tomap
on each tuple may cause serious performance problems.
I will be extremely appreciate in case you change this library to spring-boot starter with Autoconfiguration
, spring.factories
and start using @ConditionOn...
for more convenient overriding beans
It is hard to find the error, as there may be strange behavior, for example, returning empty value for a tuple or NPE
Consider the following example:
public interface ProfileRepository extends TarantoolRepository<Profile, Integer> {
@Query(function = "put_profiles")
void putProfiles(List<Profile> profiles)
}
Currently invocation of this method fails with the following exception:
io.tarantool.driver.mappers.MessagePackObjectMapperException: ObjectConverter for class x.x.x.Profile is not found
This happens due to the absence of correct entity mapping for call request parameters, the mapping is performed only on the driver level. It is necessary to add the correct entity mapping via the Spring Data facilities.
java.sql.* may be used in projects formerly using a JDBC driver and supporting these types may reduce the code changes when moving to the Cartridge Spring Data module.
Currently, nested structures may be passed only as maps (despite whether the structure is a simple object or an entity). This may be not efficient in case if the structures are merged as a result of a map-reduce operation resembling a "join", especially in the case of collections of structures, since it requires using "tomap" operation on each tuple corresponding to a nested structure.
We need to provide a way for deserializing nested structures from tuples.
Possible solution:
Allow specifying the serialized source type for the object fields that contain nested structures with some kind of annotation (e.g. @Field(serialize = TarantoolSerializationType.TUPLE)
)
I suggest the following convention:
Tests names should be correspond the next pattern:
test_[methodName/scenarioName]_should[Action]_if[Condition]
[methodName/scenarioName]_ - is optional.
if[Condition] - is optional. You can omit this if the execution is straightforward.
Also, inside the body of the test there should be comments of the following format:
//given
[some code that defines conditions]
//when
[some code that executes the test case]
//then
[some code with assertions]
Example:
If we have method like this:
public String hello(String name) {
return "Hello, " + name;
}
Test for this method:
@Test
public void test_hello_shouldReturnHelloWithName() {
//given
String name = "John";
//when
String welcome = hello(name);
//then
assertThat(welcome).isEqualTo("Hello, John");
}
If we have method like this:
public String helloWithCondition(@Nullable String name) {
if (name == null) {
return "Hello Stranger";
}
return "Hello, " + name;
}
Test for this method can be like this:
@Test
public void test_helloWithCondition_shouldReturnHelloStranger_ifNameIsNull() {
//given
String name = null;
//when
String welcome = helloWithCondition(name);
//then
assertThat(welcome).isEqualTo("Hello, Stranger");
}
It's good when this pattern can be included into test method name but it's not always possible.
When the description is too large then test case can contain additional info in comment.
Also consider this points:
Добрый день. Средствами cartridge-springdata можно как-то сложить данные в бинарном виде? Пробовал менять поле на varbinary, говорит, что не может строку преобразовать в varbinary, хотя в сущности поле определено с типом byte[]. Пробовал в тарантуле поле оставлять строковым, а в сущности byte[], тоже не работает. В тестах репозитория не нашел примеров с вставкой бинарных данных в тарантул средствами java
To reproduce the problem, I took an existing test by adding a byte [] field to the entity
@Test
public void testSave() {
BookTranslation translation = BookTranslation.builder()
.bookId(2)
.language("Russian")
.edition(22)
.translator("Ivan Ivanov")
.comments("Some translation")
.bytesString("Hello".getBytes())
.build();
BookTranslation newTranslation = bookTranslationRepository.save(translation);
assertThat(newTranslation).isEqualTo(translation);
}
diff --git a/src/test/java/org/springframework/data/tarantool/entities/BookTranslation.java b/src/test/java/org/springframework/data/tarantool/entities/BookTranslation.java
index 7ea062d..6e573a7 100644
--- a/src/test/java/org/springframework/data/tarantool/entities/BookTranslation.java
+++ b/src/test/java/org/springframework/data/tarantool/entities/BookTranslation.java
@@ -33,4 +33,6 @@ public class BookTranslation {
private String translator;
private String comments;
+
+ private byte[] bytesString;
}
diff --git a/src/test/java/org/springframework/data/tarantool/repository/support/CompositePkIntegrationTest.java b/src/test/java/org/springframework/data/tarantool/repository/support/CompositePkIntegrationTest.java
index 1b42bd1..b7c69ea 100644
--- a/src/test/java/org/springframework/data/tarantool/repository/support/CompositePkIntegrationTest.java
+++ b/src/test/java/org/springframework/data/tarantool/repository/support/CompositePkIntegrationTest.java
@@ -62,6 +62,7 @@ class CompositePkIntegrationTest extends BaseIntegrationTest {
.edition(22)
.translator("Ivan Ivanov")
.comments("Some translation")
+ .bytesString("Hello".getBytes())
.build();
BookTranslation newTranslation = bookTranslationRepository.save(translation);
assertThat(newTranslation).isEqualTo(translation);
diff --git a/src/test/resources/cartridge/app/roles/api_storage.lua b/src/test/resources/cartridge/app/roles/api_storage.lua
index cdc7fa6..3f1ad7f 100644
--- a/src/test/resources/cartridge/app/roles/api_storage.lua
+++ b/src/test/resources/cartridge/app/roles/api_storage.lua
@@ -86,6 +86,7 @@ local function init_space()
{ name = 'edition', type = 'integer' },
{ name = 'translator', type = 'string' },
{ name = 'comments', type = 'string', is_nullable = true },
+ { name = 'bytesString', type = 'string', is_nullable = true }
},
if_not_exists = true,
}
Reproducer showed that data is written to tarantool normally, since messagePack binary
is processed as a string in a tarantool. But we cannot get them back to java, since messagePack String
comes and we are trying to convert it tobyte []
The problem is that byte [] is collection, and the code asks us to return list
The workaround was to use a custom mapper:
MessagePackMapper defaultMapper =
DefaultMessagePackMapperFactory.getInstance().defaultComplexTypesMapper();
defaultMapper.registerValueConverter(
ImmutableStringValueImpl.class, List.class, object -> {
final List<Byte> list = new ArrayList<>();
for (byte b : object.toString().getBytes()) {
list.add(b);
}
return list;
});
Also globally the problem may be related to this: tarantool/tarantool#1629
We need tests convention. For tests consistency and more readability.
Example failing tests (Book
is an entity, Address
and BookNonEntity
are non-entities):
@Test
public void testFunctionReturningNonEntityAndAcceptingNonEntity() {
List<BookNonEntity> byIssuer = tarantoolOperations.callForList("find_book_by_address",
new Address[]{Address.builder().city("Riga").street("Brivibas").number(13).build()}, BookNonEntity.class);
assertTrue(byIssuer != null && byIssuer.size() > 0);
}
@Test
public void testFunctionReturningNonEntityAndAcceptingEntity() {
List<BookNonEntity> byIssuer = tarantoolOperations.callForList("find_book_by_book",
new Book[]{book}, BookNonEntity.class);
assertTrue(byIssuer != null && byIssuer.size() > 0);
}
These tests are failing with an error like:
io.tarantool.driver.mappers.MessagePackObjectMapperException: ObjectConverter for type class org.springframework.data.tarantool.entities.Book is not found
spring-data-tarantool should lazily fetch data from Tarantool cluster to prevent huge results from crud.select or crud.pairs.
Currently, converters for fields use the converter stack without checking the property class.
We need using getObject(fieldName, objectClass)
if it possible to convert with property class.
Please support PagingAndSortingRepository#findAll(Pageable pageable) operation.
function create_or_update_etalon(args)
...
return repository.put(space_name, ...)
end
@Query(function = "call_service")
void callService(String name, Map<?, ?> args, Map<?, ?> options);
org.springframework.data.mapping.MappingException: Cannot read from object of type class java.util.ArrayList
at org.springframework.data.tarantool.core.convert.MappingTarantoolReadConverter.read(MappingTarantoolReadConverter.java:106) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.convert.MappingTarantoolConverter.read(MappingTarantoolConverter.java:73) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.BaseTarantoolTemplate.mapToEntity(BaseTarantoolTemplate.java:385) ~[spring-data-tarantool-0.4.2.jar:na]
at org.springframework.data.tarantool.core.TarantoolTemplate.lambda$null$1(TarantoolTemplate.java:87) ~[spring-data-tarantool-0.4.2.jar:na]
at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:642) ~[na:na]
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[na:na]
at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073) ~[na:na]
at io.tarantool.driver.handlers.TarantoolResponseHandler.channelRead0(TarantoolResponseHandler.java:51) ~[cartridge-driver-0.6.0.jar:na]
at io.tarantool.driver.handlers.TarantoolResponseHandler.channelRead0(TarantoolResponseHandler.java:23) ~[cartridge-driver-0.6.0.jar:na]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]
but if we remove return ...
in lua, then everything will be ok
Given this entity:
@Data
@Builder
@EqualsAndHashCode
@Tuple("customers")
public class Customer {
@Id
private UUID uuid;
private String name;
private List<String> tags;
}
and this space:
local customers = box.schema.space.create(
'customers',
{
format = {
{name = 'uuid', type = 'uuid'},
{name = 'bucket_id', type = 'unsigned'},
{name = 'name', type = 'string'},
{name = 'tags', type = 'array'},
},
if_not_exists = true,
}
)
customers:create_index('uuid', {
parts = {'uuid'},
if_not_exists = true,
})
customers:create_index('bucket_id', {
parts = {'bucket_id'},
unique = false,
if_not_exists = true,
})
it is impossible to save the entity:
@Autowired
TarantoolOperations tarantoolOperations;
private Customer vasya = Customer.builder().uuid(UUID.randomUUID()).name("Vasya").tags(Arrays.asList("one", "two")).build();
@Test
void test() {
tarantoolOperations.save(vasya, Customer.class);
}
The following exception is occurring:
io.tarantool.driver.mappers.MessagePackValueMapperException: ValueConverter for type class org.msgpack.value.impl.ImmutableArrayValueImpl is not found
Module version: 0.3.2
Steps to reproduce:
local function update_tuples_crud(space_name, key, operations)
local result, err = crud.update(space_name, key, operations)
if (err ~= nil) then
return nil, err
end
end
tarantoolOperations.call("update_tuples_crud", parameters.toArray(new Object[0]), AgentEntity.class);
Actual behavior:
The call fails with a NPE
Expected behavior:
The call is successful
Currently, there are some cases when the user needs to have the bucket_id
field in mind and specify either a null
or the right value for it (using Conditions.startAfter()
when using the TarantoolOperations.find(), for example), although the bucket_id may not be specified in the entities and does not appear anywhere in the user data.
The bucket_id usage cases must be considered and the user must be kept from accidentally having to use it if it is not intended.
Links to telegram chats have 'tarNAtool' typo and point nowhere:
See:
Feel free to join the Tarantool community chat in Telegram (or its counterpart in Russian) if you have any questions about Tarantool database or Spring Data Tarantool.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.