Depending how, where, and when I run the suite of unit tests I often get some set of the same errors. When examining the errors they all have the same common trait: there is an SQL query that is sorting on some date/time field. Because of the speed of the machine and the lack of resolution in Java date/time values some operations appear as the executed exactly at the same time even though executed sequentially. The unit tests depend on explicit ordering as the assert
statements are fetching and comparing to specific values in the returned result sets. The SQL query when seeing exactly the same ordering
is free to return the values in any order thus the errors.
There are 4 main tests that exhibit this behavior and they are:
- AdhocSubProcessTest - see (1) below
- HistoricProcessInstanceTest - see (2) below
- HistoricTaskQueryEscapeClauseTest - see (3) -> (12) below; only annotated a few as the pattern became clear
- FullHistoryTest - see (13) below
I see 3 possible solutions:
- always run on a slow machine :)
- add sleep statements to force different dates/time values so the ordering is correct
- use
contains
type test logic instead of comparing to specific values in the returned result set list
I lean towards the last option as we really aren't testing the ability of SQL engines to sort the data but rather the presence or absence of specific records.
Comments?
I'll provide a pull request when a decision is made.
Thanks
Details
testSimpleCompletionCondition(org.activiti.engine.test.bpmn.subprocess.adhoc.AdhocSubProcessTest)
junit.framework.ComparisonFailure: expected:<subProcessTask[]> but was:<subProcessTask[2]>
at org.activiti.engine.test.bpmn.subprocess.adhoc.AdhocSubProcessTest.testSimpleCompletionCondition(AdhocSubProcessTest.java:112)
The query includes: .orderByHistoricTaskInstanceEndTime()
The returned data values are:
2016-10-18 06:19:28.716
2016-10-18 06:19:28.716
2016-10-18 06:19:28.732
(2)
testHistoricProcessInstanceSorting(org.activiti.engine.test.history.HistoricProcessInstanceTest)
junit.framework.ComparisonFailure: expected:<15[898]> but was:<15[903]>
at org.activiti.engine.test.history.HistoricProcessInstanceTest.testHistoricProcessInstanceSorting(HistoricProcessInstanceTest.java:353)
The query includes: .orderByProcessInstanceEndTime()
(3)
testQueryByTaskNameLikeIgnoreCase(org.activiti.standalone.escapeclause.HistoricTaskQueryEscapeClauseTest)
junit.framework.ComparisonFailure: expected:<1[1]> but was:<1[6]>
at org.activiti.standalone.escapeclause.HistoricTaskQueryEscapeClauseTest.testQueryByTaskNameLikeIgnoreCase(HistoricTaskQueryEscapeClauseTest.java:269)
The query includes: .orderByHistoricTaskInstanceStartTime()
The returned date values are:
2016-10-18 06:21:19.996
2016-10-18 06:21:19.996
(4)
testQueryByProcessDefinitionNameLike(org.activiti.standalone.escapeclause.HistoricTaskQueryEscapeClauseTest)
junit.framework.ComparisonFailure: expected:<1[1]> but was:<1[6]>
at org.activiti.standalone.escapeclause.HistoricTaskQueryEscapeClauseTest.testQueryByProcessDefinitionNameLike(HistoricTaskQueryEscapeClauseTest.java:148)
which has: .orderByHistoricTaskInstanceStartTime()
(5)
testQueryByTaskDescriptionLike(org.activiti.standalone.escapeclause.HistoricTaskQueryEscapeClauseTest)
junit.framework.ComparisonFailure: expected:<1[1]> but was:<1[6]>
at org.activiti.standalone.escapeclause.HistoricTaskQueryEscapeClauseTest.testQueryByTaskDescriptionLike(HistoricTaskQueryEscapeClauseTest.java:296)
which has: .orderByHistoricTaskInstanceStartTime()
(6)
testQueryByTaskNameLike(org.activiti.standalone.escapeclause.HistoricTaskQueryEscapeClauseTest)
(7)
testQueryByTaskAssigneeLike(org.activiti.standalone.escapeclause.HistoricTaskQueryEscapeClauseTest)
(8)
testQueryLikeByQueryVariableValue(org.activiti.standalone.escapeclause.HistoricTaskQueryEscapeClauseTest)
(9)
testQueryLikeIgnoreCaseByQueryVariableValue(org.activiti.standalone.escapeclause.HistoricTaskQueryEscapeClauseTest)
(10)
testQueryByProcessInstanceBusinessKeyLikeIgnoreCase(org.activiti.standalone.escapeclause.HistoricTaskQueryEscapeClauseTest)
(11)
testQueryByTenantIdLike(org.activiti.standalone.escapeclause.HistoricTaskQueryEscapeClauseTest)
(12)
testQueryByProcessDefinitionNameLike(org.activiti.standalone.escapeclause.TaskQueryEscapeClauseTest)
(13)
testHistoricVariableRemovedWhenRuntimeVariableIsRemoved(org.activiti.standalone.history.FullHistoryTest)
junit.framework.AssertionFailedError
at org.activiti.standalone.history.FullHistoryTest.testHistoricVariableRemovedWhenRuntimeVariableIsRemoved(FullHistoryTest.java:1392)
which has: .orderByTime()