Giter Club home page Giter Club logo

marklogicworkflow's Introduction

A way to import and execute BPMN2 process models using MarkLogic's CPF feature.

What is MarkLogic Workflow?

This project aims to provide a way to use the dominant BPMN2 process modelling standard in order to provide a way to build workflows that can be executed within MarkLogic's Content Processing Framework (CPF) feature.

Many customers of MarkLogic desire simple content and human centric workflows rather than complex ESB/JMS system-to-system integration. A full-fledged separate BPM suite therefore forces them to install a great deal more infrastructure and has a higher learning curve than is necessary to support these workflows.

MarkLogic Workflow uses the MarkLogic Enterprise NoSQL Database and it's document state processing functionality - CPF - in order to provide a single integrated platform for MarkLogic-document and human user centric workflow modelling and processing.

This provides customers with an easy entry point in to content-centric BPM for long-lived processes that use MarkLogic stored content. It does this at minumum cost, with no extra working parts (other than a modelling tool), and uses open standards and open source software to minimise vendor lock in.

Example processes that can be implemented:-

  • Content review/approval workflows
  • Content change request workflows
  • Content creation request workflows
  • Case management workflows using content about a person/event/customer/place as context for a human user's decision making
  • Long term content review and disposition (policy driven deletion) workflows

Why use MarkLogic Workflow?

Good question. A few quick reasons:-

  • No additional cost for existing MarkLogic customers
  • Extends CPF to allow parallel execution of sub-processes and inclusion of multiple documents, or none, in a single process
  • Uses MarkLogic Alerting to precisely identify which documents affect which processes, at a more granular level that CPF Domains
  • Introduces the concept of human steps, and work queues, which CPF does not support
  • Implemented using CPF - which means long running processes have ZERO PROCESSING COST when not active - this is UNIQUE in the BPM space
  • Uses the dominant BPMN2 standard and provides a MarkLogic view in the Eclipse BPMN2 modeler, a leading open source BPM modelling application, used by jBPM also, avoiding vendor lock-in

MarkLogic workflow does not aim to be a full fledge BPM Suite providing round trip business process re-engineering. It is also not aimed at system to system integration, although will be able to invoke and be invoked by web services in a future release. MarkLogic workflow can, therefore, act as part of an overall SOA architecture that also includes system to system BPM environments like IBM WebSphere Process Server or TIBCO ActiveMatrix BPM.

Information on MarkLogic Workflow

marklogicworkflow's People

Contributors

adamfowleruk avatar chunyuwilson avatar eouthwaite avatar kentune avatar luisjmart avatar peetkes avatar steliann avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

marklogicworkflow's Issues

Use Alerting to kick off a new process

Use alerts to start a process (via creating a process instance XML document)

Must support multiple alerts creating a single type of process, and passing on the initiating attachment.

alert-action-process.xqy - create a process doc once an alert is fired
lib-alerts.xqy - basic alert creation
lib-search-subscribe.xqy - saving a search and creating/removing alerts
/v1/resources/processsubscription.xqy - create, update and delete an alert configuration
workflow-import.xqy - Support for Alert configuration within a diagram (multiple alerts, single process)
Eclipse Workflow extension - Alert custom message type potentially required - cts:query support, external XSD import (cts:query et al)

MBO REST POST process

Update or complete a process step. E.g. complete a human step
Complete with JSON and XML and TXT (for testing scripts - process id)
See documentation/RESTAPI.md

MBO REST PUT processsubscription

Create a process subscription (alert to fire a new process instance. Also creates a CPF domain as required)
Complete with JSON and XML.
See documentation/MODELLING.md

MBO REST GET process

Fetch the current status and data for a process instance
Complete with JSON and XML.
See documentation/RESTAPI.md

MBO REST GET processsubscription

Fetch the named processsubscription (or all of the current user's subscriptions?)
Complete with JSON and XML.
See documentation/MODELLING.md

Support Alerting configuration within BPMN modeler

Must support multiple alert configurations linked to the same Workflow Domain (start event)

Implement as:-

  • Actual Process (with start event with domain message configuration) as a sub model
  • Other models within file show start event (with alert message configuration) pass signal to sub process (the actual process model)

Just in idea. Actual implementation may need to vary depending upon BPMN2 standard conventions/rules.

Support business rules invocation

As per BPMN2 spec

Implement via semantic inferencing?

From BPMN2 spec PDF page 163:-
"A Business Rule Task provides a mechanism for the Process to provide input to a Business Rules Engine and to get the output of calculations that the Business Rules Engine might provide. The InputOutputSpecification of the Task (see page 211) will allow the Process to send data to and receive data from the Business Rules Engine."

MBO REST GET processroleinbox

Get the process tasks waiting for a user in a named role (and visible to this user, who also has that role)
Complete with JSON and XML.
See documentation/RESTAPI.md

BPMN2 escalation timer step

Support multiple parallel timers
Used timed task server task, with escalation query. E.g. if still have a timer element with specified name, re-route to different CPF step
Start Timer task (duration or specified date time, specify escalation step)
Cancel timer task
task action

MBO Support BPMN2 send and receive

Once event handling implemented, and sub processes, should be able to easily implement sending a message (fire and forget) and receiving a message (async messaging).

Support for basic BPMN2 elements

User Task (Basic HTML step processor)
Task (subclasses only for MarkLogic actions, not generic support)
Send task (SOAP)
Send task (REST)
Loop characteristic (on individual activities)
Exclusive gateway
Error event (route to specific CPF task as per BPMN2 spec)

Support basic BPMN2 events

Escalation event (non timer escalation)
Error event (custom or generic)
Custom event (any event)

Treat events as being CPF states that immediately start 1 or more parallel sub processes. (multiple event listeners)

Message and Signal events both out of scope

MBO REST GET processsearch

Fetch all processes, or all instances of the named process or process-version. (visible to the current user only?)
Complete with JSON and XML.
See documentation/RESTAPI.md

Support BPMN2 call activity

process invokes another separate process (as opposed to sub process contained within same process model document)

MBO MarkLogic Start CPF pipeline task

Specify pipeline or domain to invoke
Take single attachment as parameter - this will be the document the pipeline/domain is invoked on
Set the document's properties such that CPF kicks off the domain

MBO Docs Sprint 002 example process models

  • BPMN2 specification test process models modified and tested to MarkLogic executable standard
    • Advanced
    • Collapsed sub process
    • Correlation example seller
    • email voting 2
    • expanded subprocess
    • laneset
    • pool
    • Process
    • Procurement Processes with Error Handling - Stencil Trosotech 2 pages
    • Travel Booking
    • triso - Hardware Retailer v2
    • Triso - Order process for pizza v4
    • Extended
    • Call activity
    • Nobel prize process

Don't forget to test them all!

MBO MarkLogic Detach task

Detach a named attachment from a process. Could be a single attachment, entire array, or specific instance within an array.

Support BPMN2 Service task

Complex. May be sync, async, or fire and forget
As per BPMN2 standards
service task -> invoke service and process response

MarkLogic CPF to BPMN2 exporter

Export CPF pipelines (domain) to BPMN2 model and test reintegration of any alterations.

Needs to be 100% round trippable.

MBO MarkLogic Search Task

Perform a search in MarkLogic and store result references in an Attachment array in the process document

MBO REST PUT process

Instantiate a process.
Complete with JSON and XML and TXT (for testing scripts - process id)

MBO REST GET processqueue

Fetch the named queue's workload (those items visible to this user - whether locked or not)
Complete with JSON and XML.
See documentation/RESTAPI.md

Support fork and rendezvous (Parallel Gateway)

Implement as a 'sub process document' under /workflow/processes/sub
Create requisite pipeline, add to overarching domain (same as root process)
Fork Task
(hidden - CPF only, not BPMN2) Check rendezvous task - update subprocess completion state element in parent process, thus kicking rendezvous task on to next step (condition set to if all elements are true, proceed)
Rendezvous (awaiting) Task
Implement just 'all routes' to start with
Parent process flow uses subprocesses element with specific UUID (in case in future we allow parallel of parallel of subprocesses), and sub element for each fork created.

Support BPMN2 sub-processes

Allow a process flow to initiate and return from a sub process flow.
Be aware of escalation timers, and parallel execution use cases too alongside this feature

MBO Docs Create example executable process models

  • DEFERRED BPMN2 specification test process models modified and tested to MarkLogic executable standard
    • Basic
    • Incident Management Level 1
    • Incident Management Account Manager Only
    • Incident Management Process Engine only

Don't forget to test them!

Security audit and features

  • Enterprise features
    • Set security as relevant to the process document at each step in the process - irrelevant, sorted by amps and internal role
      • Especially around assignment - N/A
    • Allow a set security permissions on documents feature - postponed until MarkLogic specific steps in later sprints
    • Ensure installation creates relevant roles and permission sets
    • Creation of roles - (Requires mlnodetools enhancement)
    • creation of privileges - (Requires mlnodetools enhancement)
    • creation of amps - (Requires mlnodetools enhancement)
    • settings of modules db document security - DONE (mlnodetools enhancement)
    • setting of rest extension security - RFE logged
    • Ensure rest services honour the 'locked by' guarantee - not relevant, handled by internal privileges

Eclipse CPF modelling support fully working

Made an initial start, but needs significant debugging and testing. Also need to support all CPF state, status, condition, and action configuration within BPMN2 model.

Requires Java expertise. Knowledge of Eclipse SWT and EMF also useful.

MBO MarkLogic set process variable

Activity: Set process variable activity with multiple from and to (simple XPath evaluation)

  • See if there is an equivalent BPMN2 method - may just be a variable assignment on each step instead
  • Create import step for this BPMN2 method
  • Create CPF action to represent this

MBO REST GET processinbox

Fetch the current user's (or specified user's?) process inbox
Complete with JSON and XML.
See documentation/RESTAPI.md

MBO Test evaluation support in XQuery

Specifically: Support for (/some/path/one,/some/path/two)[1] style evaluation within conditions (exclusive gateway)

See wfu:evaluate() and data/examples/bpmn2/011_gateways_test.bpmn

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.