improve logging for WorkflowRest Service

overtake the testmode and default from the MartyMail Plugin

Currently the date format feature replacing dynamic text strings of the AbstractPlugin did not provide a way to set the locale for date strings.
This results in situation where a date format will be performed with the servers locale. This can be different from the model language.
We need to provide a locale into the format string?

I think that worflow Plugins / EJBs which are using the REQUIRES_NEW Transaction
type:

@TransactionAttribute(value = TransactionAttributeType.REQUIRES_NEW)

run into a problem during runtime if the database type of an mysql database is
set to INNODB. We have this issue in a project where a timeout exception occures
after a plugin have tried to start a new transaction context:

Exception [TOPLINK-4002](Oracle TopLink Essentials - 2.1 %28Build b60e-fcs
%2812/23/2008%29%29): oracle.toplink.essentials.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: Lock wait timeout exceeded; try
restarting transaction
Error Code: 1205
Call: UPDATE SEQUENCE SET SEQ_COUNT = SEQ_COUNT + ? WHERE SEQ_NAME = ?
bind => [50, SEQ_GEN]
Query: DataModifyQuery()
at
oracle.toplink.essentials.exceptions.DatabaseException.sqlException(DatabaseException.java:311)
at
oracle.toplink.essentials.internal.databaseaccess.DatabaseAccessor.executeDirectNoSelect(DatabaseAccessor.java:654)
at
oracle.toplink.essentials.internal.databaseaccess.DatabaseAccessor.executeNoSelect(DatabaseAccessor.java:703)
at
oracle.toplink.essentials.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:492)
at
oracle.toplink.essentials.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:452)
at
oracle.toplink.essentials.internal.sessions.AbstractSession.executeCall(AbstractSession.java:690)

..
...
....

So after all it seems to be a reproducible issue.

Situation:

create a EJB which uses TransactionAttributeType.REQUIRES_NEW for a method and
let this method access a workitem which is already opend in the surronding process.
For example simply let a Plugin class invoice such an EJB to lookup some data on
the same workitem.

This should lead into the exception on a mysql InnoDB but not on mysql MyISAM

Provide a mechanism to test workflow logig based on a REST Client.
The test suite should be usable together with junit.

http://stackoverflow.com/questions/19336380/what-do-i-wrong-while-deleting-an-entity-mysqlintegrityconstraintviolationexcep

metod replacedynamicvalues should log warnings if text format is wrong fomated

We should think about to consolidate fieldnames.
E.g. txtworkflowstatus -> something like '$status' or 'imixs.status' or 'workflow.status'

To avoid side effects we can simply implement an EJB Interceptor for the EntityService which manage the mapping from the old field names to the new one.
for all finder methods and also in case of a save method call.
using a migration-flag the interceptor can also be aware of the current migration status of a WorkItem.

we have an error convering the old format in case of the following situation:

Old Format=[, 06.03.2014 10:29:11 : Info liegt vor von namcurrenteditor., 06.03.2014 10:29:56 : Bestätigung liegt vor von namcurrenteditor.]

Test Szenario for JUnit Test: TestHistoryPlugin

list.add("");
    list.add("14.02.2012 10:35:06 : In Abstimmung mit Ralph Soika.");
    list.add("14.02.2012 10:39:22 : Gespeichert von Ralph Soika.");
    list.add("14.02.2012 15:55:51 : Gespeichert von Ralph Soika.");


    list.add("06.03.2014 10:29:11 : Info liegt vor von <username>namcurrenteditor</username>.");
    list.add("06.03.2014 10:29:56 : Bestätigung liegt vor von <username>namcurrenteditor</username>.");

    ....

Test Class has a typo in Name "TestHisortryPlugin" -> "TestHistoryPlugin"

Migrate Lucene Plugin into the imixs-adaptors project.
So no external dependencies are necessary

add some improvments.......

We migrate from https://java.net/projects/imixs-workflow

if content contains no contenttype encoding than we need to set it

e.g. client encoding or UTF-8 if undefined

the property 'txtWorkflowPluginLog' controlled by the WorkflowKernel did not make any sense.
No useful information.
Better way improve logging in general and write workflow log into separate log file like discussed earlier.

I notice that the current implementation did not compare the userid case sensitive.
The reason seems to be the method queryOptimier() which generates an SQL IN
Operator:

... where access807.value IN ('','http://Max-Mustermann.myopenid.com')

And the IN Operator did not compare strings in case sensetive mode.
In different the method load(String id) and the included method
isCallerReader() did compare the userid case sensitive.

This leads into a problem. For example a the call of findAllEntities() did
return a list of workitems whehre the readaccess compare was not case sensitive.
Next if the code calls the method load() this will be case sensitive. But now
the same workitem will not be returned.!

So in the moment it is very important that the user id is case sensitive.

I am not sure how to fix this issue? Maybe there is a change to change the
optimizeQuery() method and change the IN Operator to force a case sensitive compare.

need more logging information for perfomance analysis in log level FINE

the apt file for the mailPlugin (mailplugin.apt) should be updated with the new configuration settings!
also html support need to be documented

the i:header tag need a new attribute:

tinymcedefaultcss=".."

If defined add the follwoing script:
content_css : "#{facesContext.externalContext.requestContextPath}/layout/css/tinymce.css",

in some strange cases the main navigation did not work (maybe if session is invalid)

I've found a bug in the entityservice in the addindex method.
If you have already created some workitems with additional read and write access defined in the model
and you add an index with the addindex method of the entityservice you will get foreign key constraint
errors like this:

Cannot delete or update a parent row: a foreign key constraint fails (imixs_db_pool.entity_writeaccess, CONSTRAINT FK_ENTITY_WRITEACCESS_writeAccessList_ID FOREIGN KEY (writeAccessList_ID) REFERENCES writeaccess (ID)) Error Code: 1451 Call: DELETE FROM WRITEACCESS WHERE (ID = ?) bind => [1 parameter bound] Query: DeleteObjectQuery(org.imixs.workflow.jee.jpa.WriteAccess@214f32ce)

The reason for that seems to be in the updateWriteAccessList method at line 0968. manager.remove(aItem);
The list will be cleared afterwards at line 0969. aEntity.getWriteAccessList().clear(); but there it seems to be to late.
The writeaccess is already removed before the reference in the entity_writeaccess table is removed so the exception is
thrown.
I then backuped the writeaccess and entity_writeaccess table and truncated them so I thought it should work but
it didn't the next exception was:

Cannot delete or update a parent row: a foreign key constraint fails (imixs_db_pool.entity_textitem, CONSTRAINT FK_ENTITY_TEXTITEM_textItems_ID FOREIGN KEY (textItems_ID) REFERENCES textitem (ID)) Error Code: 1451 Call: DELETE FROM TEXTITEM WHERE (ID = ?) bind => [1 parameter bound] Query: DeleteObjectQuery(org.imixs.workflow.jee.jpa.TextItem@1cac871e)

which I first couldn't understand, but then I got it, it tries to update all the entities and add the index,
so it creates the textitem for my new index and somehow it tries to deleteit here, don't know what
is going on maybe I don't understand the code of the EntityService enough.

A workaround which worked for me is to delete all Entities and then add the index, afterwards I back uped the entities.
The only problem with the soltion is, that the new index will be only applied for new entites which during development
shouldn't be a problem, but if you want to develop it further after a productive time it would be a real problem.

If the report plugin adds a new report into the blob workitem the blob workitem need to be save in a new transaction context to avoid conflicts with later plugins operating on the same workitem!
Example: ReportPlugin adds a pdf file, DMSPlugin trys to update the read/write access of the BlobWorkitem

The property 'txtworkflowactivitylog' contrlled by the Kernel did not provide a sufficient information.

Current Format:

[30.07.2014 11:16:13|2000.20:=2010, 30.07.2014 11:16:17|2010.10:=2010, 30.07.2014 11:16:20|2010.10:=2010]

Better Format:

timestamp|model-version|1000.10|1000|userid|
timestamp|model-version|1000.20|1010|userid|
timestamp|model-version|1010.10|1010|userid|comment

The comment is an optional entry. For example can be configured in the workflow result field of an activity

 start-measuring01

Result:

 timestamp|model-version|1010.300|1010|userid|start-measuring01

combinations:

 timestamp|model-version|1010.300|1010|userid|start-measuring01,stop-measuring02

currently it is not possible to change the encoding/charset used by the mailPlugin

We can make this configurable with the property service ejb.

Possilbe new properties:

mail.htmlCharSet= for html mails (default = text/html; charset=ISO-8859-1)
mail.textCharSet= for text mails (default = text/plain; charset=ISO-8859-7)

The intervalplugin can manage a schedule date

Field: keyIntervall - options: weekly, yearly, monthly

Logic: if datDate in the past, and a scheduled activity is processed, the plugin will adjust the datDate field

In addition we can define a field-name-pattern to use the plugin for different date fields:

datDate -> keyIntervaldatDate
datDueDate -> keyIntervalDatDueDate

The Plugin is only used for scheduled events!

We should think about a simplifcation of the XML representation of an workitem

old:

new:

system-de-0.0.1

but after all this seams not to have much effect. much work for noting :-?

26 chars to 21

= 13
name=""> = 8

If we find a way to remove the xmlns:xsi namespace this would have more effect!

We should provide a way to setDefaultOperator.

Define a new imixs.properties entry for that

public void setDefaultOperator(QueryParser.Operator op)
Sets the boolean operator of the QueryParser. In default mode (OR_OPERATOR) terms without any modifiers are considered optional: for example capital of Hungary is equal to capital OR of OR Hungary.

Line Number 405

To provide a way to control the content of a workitem we can choose a new solution via the RulePlugin.
In general the RulePlugin should not be able to manipulate workitems directly. But what we can do is provide a way to control the ActivityResult. For Example a RulePlugin could add a tag

<space>xxx</space>

This tag can later be evaluated by the TeamPlugin.

So the idea is to allow the rule plugin to define ActivityResult tags. The control of those tags is manged by the corresponding plugins. = separation of concerns.

http://en.wikipedia.org/wiki/Separation_of_concerns

if a span tag for imixstooltip contains nested html elements these will not be shown in the tooltip box.
Reason: the method text() need to be replaced by html() in the imixs layout tooltip script

provide a way to get the count of entities for a specific query

We need an extention to allow setting a different LockFactory - for NFS Storage the SimpleLockFactory is recommanded

Also Upgrade to Version 4

new tag:

<propertyValue>key</propertyValue>

extend the date picker with time selector

We change the maven layout of the projects from single projects into a multi-modul project structure

This plugin can be used to measure the time different phases of a workflow take.
The plugin can be configured by the activity result :

Example:

<item name='measurepoint' type='start'>durchlauf</item> 

efines a start point named 'durchlauf'

 <item name='measurepoint' type='stop'>durchlauf</item> 

definens a end point named 'durchlauf'

The result will be stored into the txtWOrkflowActivityLog (comments)
and also the Plugin will create the following fields:

datMeasurePointStart_durchlauf : contains the start time points (list latest entry on top!)
datMeasurePointEnd_durchlauf : contains the end time points (list)
numMeasurePoint_durchlauf: contains the total time in milis.

With this logic we can measure any time aspact of a process instance

If items contain xml content the reportPlugin should not encode the content. This makes it inpossible to use special xsl transformation during report processing.

E.g. if a field contains html content (htmlDiscription) it could be necessary to handle this content by special xsl template. We see this in fo processing (pdf output)

RulePlugin should provide a way to modify attributes of the activity entity.
This makes it able to develop more complex rules and avoids the posibillity to manupulate the workitem directryl from script.
So a custom plugin can alos evaluate custom activity atributes computed by a business rule.

This will extend the flexiblity of the rule plugin in a rich way

If a new workitem is added into the lucen index the property $crated is not added because the workitem is not yet saved.

We need to redesign the WorkflowScheduler.
Goals: better ways for subclassing

Main problem: workitem selection can not be influenced from a controller

See: imixs/imixs-marty#22

The lookup of the mail resouce should be done if a mail is send.
currently at the beginning of the plugin the plugin fetches the mail resource.

we should move the JSONParser from imixs-workflow-jax-rs to imixs-workflow-core

And we should add a simpleParser method to parse simple json strings into ItemCollections

the ItemCollection method copyAllItems did copy per reference.
This leads into problems (e.g. in the version plugin) if a value need to be changed later.

http://stackoverflow.com/questions/869033/how-do-i-copy-an-object-in-java

It should be easy to simulate this unexpected behavior in a junit test

ItemCollection itemColNewVersion = new ItemCollection();

itemColNewVersion.replaceAllItems(sourceItemCollection.getAllItems());

change a value in newVersion ...
=> value also changed in sourceCollection

See new method: saveByNewTransaction

redesign workitem view concept.
To get a better way to subclass we need to change method visibility.

We should focus on a way to subclass only query builders and not the viewAdapter

The post xml ItemCOlleciton method should return the processed xml item

We need to find a way to embed fonts int fop documetns.

There for it is necessary to config the fopFactory with a custom user configuration file
See: http://xmlgraphics.apache.org/fop/0.95/embedding.html

We can use the following line to add a config file:

 String configFile=propertyService.getProperties().getProperty("imixs.fop.userconfiguration");
 FopFactory fopFactory = FopFactory.newInstance();
 fopFactory.setUserConfig(configFile);

Than it should be possible to configure embedded fonts

http://xmlgraphics.apache.org/fop/trunk/fonts.html

We need public gettern methods to compute 'form' and 'replyTo'
otherwise there is no way to customize these values later throug a plugin!

We need to check the encoding and test for UTF-8 when parsing a JSON Element....

trying to upload this model fails.
need to be checked