de-jcup / asp Goto Github PK

Per default external process launcher ( ExternalProcessAsciidoctorJServerLauncher) shall
not show password but xxxxxxxxxxxxxxxxxxx.

Shall be configurable - so if somebody needs this inside logs it can be enabled.

Currently a call to ASPClient e.g. for "isAlive" cannot be canceled
Every client action should be canceable.

So an interface must be introduced to handle those operations:
ProgressMonitor. Method isCanceled()

Situation

Newer IDE variants do separate between src/main/java and src/test/java and do not accept application starts from there.
For example: we cannot start e.g. AspClientTestMain.java from eclipse IDE.

Wanted

Inside IDE the test applications shall be startable again.

Solution

Move the "*TestMain" applications to integration test src/main/java

we need a "SERVER_LOGS" entry in json for server info.

Release issue.

After this release has been done, ASP parts will become available at

• https://s01.oss.sonatype.org/content/groups/public/de/jcup/asp/
• and also at maven central by automatic sync

Why another release when no code has changed?

Reason: The jars will be now signed and the upload and build process will change. So technically they have changed at least by signature.... Providing here also a 1.3.0 would lead to inconsequent change management. So a "hotfix" version was necessary.

currently diagrams are not rendered

see https://github.com/de-jcup/asp/projects/2

Jars must be available by a wellknown mavenrepository. So easy to integrate into builds using
this as dependency - e.g. https://github.com/de-jcup/eclipse-asciidoctor-editor/

see https://github.com/de-jcup/asp/projects/1

Asciidoctor has much newer versions which should be integrated in next ASP server.

Will change dependencies to:

compile group: 'org.asciidoctor', name: 'asciidoctorj', version: '2.2.0'
compile group: 'org.asciidoctor', name: 'asciidoctorj-pdf', version: '1.5.2'
compile group: 'org.asciidoctor', name: 'asciidoctorj-diagram', version: '2.0.1'

asciidoctorj-pdf 1.5.3 has been released yesterday
https://mvnrepository.com/artifact/org.asciidoctor/asciidoctorj-pdf/1.5.3
containing
https://github.com/asciidoctor/asciidoctor-pdf/releases/tag/v1.5.3

so use new version for 1.1.1

see https://github.com/de-jcup/asp/projects/1

Situation

We currently use Options.asMap() - but this is deprecated

Wanted

Remove the deprecated usage - because newer releases could remove the method or change the (internal) behaviour

Solution

• introduce additional attributes parameters for communication protocol
• provide own builder in client api which can be used (e.g. inside eclipse asciidoctor editor)

drop travis files and create github action

create release 1.2.0

This is the ASP issue for add support for supporting de-jcup/eclipse-asciidoctor-editor#322

ASP must use custom graphviz location

Complete client-server communication shall be encrypted.

Server will return on startup a secret-key inside console log. This key must be used by client(s) wanting to communicate with server. All communication will be encrypted.

This will protect data, but also prevent #10, because if a decryption fails on client side this can only happen because of communicate with an unknown ASP server.

see https://github.com/de-jcup/asp/projects/2

Situation

Because of the bintray sundown the asp libraries are no longer available as a gradle or maven dependency.

Wanted

ASP shall be available as a gradle or maven dependency.

Solution

Provide a deployment to maven cenral

Provide a github action "release" which automatically releases new versions to OSSRH and so to maven central.
dist shall be available as well

see https://github.com/de-jcup/asp/projects/1 , but without bintray deployment.

The libraries are currently only available at local maven repository - bintray is no longer available and the change to maven central is still ongoing (unfortunately).

Situation

currently the deploy.sh script must be used for deployment.
It also uses the old gradle "maven" plugin instead of "maven-publish".

Wanted

Release to maven central by github action

Convert *.adoc files by using ASP

PlantUML code:

component Client as client
component "ASP server" as aspServer
aspServer <- Alice  :start (1)
client <- Alice
aspServer <- client : connect, define path to asciidoctor file to render (2)
aspServer -> client :delivers path to output (e.g.PDF/HTML..) (3)

use 1.6.0 see https://mvnrepository.com/artifact/org.asciidoctor/asciidoctorj-pdf

see https://github.com/de-jcup/asp/projects/1

see https://github.com/de-jcup/asp/projects/2

currently ./gradlew fullintegrationtest does fail

de.jcup.asp.server.asciidoctorj.launcher.ExternalProcessAsciidoctorJServerLauncherIntTest > server_launch_by_jar FAILED
    de.jcup.asp.core.LaunchException at ExternalProcessAsciidoctorJServerLauncherIntTest.java:58

de.jcup.asp.server.asciidoctorj.launcher.ExternalProcessAsciidoctorJServerLauncherIntTest > long_running_action_like_convert_file_to_pdf_can_be_canceled FAILED
    de.jcup.asp.core.LaunchException at ExternalProcessAsciidoctorJServerLauncherIntTest.java:65

It shall be possible to enable output for client/server communication json values. Interesting for debugging etc.

see de-jcup/eclipse-asciidoctor-editor#336 (comment)

To provide de-jcup/eclipse-asciidoctor-editor#213 ASP we must provide an interface to get all attributes into a map - necessary for asciidoctor editor.

see AsciiDoctorAttributesProvider#resolveAttributes

This is the corresponding issue to
de-jcup/eclipse-asciidoctor-editor#282

ExternalProcessAsciidoctorJServerLauncher does not work under windows:

javaCommand = pathToJava + "/java";
File test = new File(javaCommand);
if (!test.exists()) {
...

Either change path to executable or integrate logic for windows

Server shall be started from outside by java with process builder.

If there is port already in use like
java.net.BindException: Address already in use (Bind failed)
we give back 10.

For any other error just a 1
Normal exit: 0

Situation

When rendering source code with ASP 1.4.0 the asciidoctor output does no longer contain syntax highlighting

Wanted

Shall work again

Analyze

The problem is, that with the #45 the sent json content differs now between options and attributes :

{
  "version" : "0.0.0",
  "command" : "convert_file",
  "source_filepath" : "/somewhere/example1.adoc",
  "options" : {
    "backend" : "pdf"
  },
  "attributes" : {
    "source-highlighter" : "coderay",
    "coderay-css" : "style"
  }
}

formerly the options element did contain the attribute element. The ASP server 1.4.0 version does still use the old structure ando so no attributes are used.

Solution

open

Currently build fails because oracle jdk 8 is set and not found by travis...

We will update dependencies:

   // https://mvnrepository.com/artifact/org.asciidoctor/asciidoctorj
    asciidoctorj_version = "2.5.3"
    
    // https://mvnrepository.com/artifact/org.asciidoctor/asciidoctorj-pdf
    asciidoctorj_pdf_version = "1.6.2"
    
    // https://mvnrepository.com/artifact/org.asciidoctor/asciidoctorj-pdf
    asciidoctorj_diagram_version = "2.2.1"

Problem

The key value approach is okay for single values, but when it comes up to lists or maps the approach does not really work (or to cumbersome).

So JSON would be the simplest and proven way to contain communication data.

But... dependencies shall be small and we need a fast library, so ...

TODO

• Find a library being small, fast and is ready for production
• Integrate it and replace key value approach

At de-jcup/eclipse-asciidoctor-editor#248 there were problems with external launcher: The console at windows does use \r\n instead \n so key was having always a \r inside, so encryption failed...

This is fixed in de-jcup/eclipse-asciidoctor-editor#248 by just trimming the key - this must be done inside ASP launcher.

Risk estimation

Current Situation

• there is always a client TCP/IP connection to an ASP server and always trusted.
• the communication is not encrypted, but plain json
• But Server IP is always only loopback address, ports can differ
• Client accepts only ASP connections from/to loopback address, so localhost...

What could a Hacker do with ASP?

• Kill the process of the real ASP server instance
• Create a fake server ASP, or an adapted ASP server, but create a control server
• The control server could create and deploy an exploited PDF or HTML

When could a Hacker do this with ASP?

• Hacker must have already access to local machine
• Must have the right to kill the origin process
• Must have the possiblity to create a server process for necessary port on loop back address

If the hacker is able to do the steps described above, he/she is already having access to system.

What would be the "benefit" for an hacker to have access to ASP

Exploit PDF and deliver by ASP:

One potential target: Attacker has normal user/developer rights available but no admin rights. A malware PDF could be a delivered by fake ASP server to client caller and have

• either rights of the caller (not really a "hack benefit")
• maybe gain more user rights (priviledge escalation) by using an HTML or PDF exploit delivered...

Update README.doc and describe gradle and maven integration via maven central

Because of bintray sundown - see de-jcup/eclipse-asciidoctor-editor#333 for details - the library must be hosted somewhere else.

Will include

• new infrastructure
• change way of deployment - remove old bintray auto deploy script
• use github packages (or maybe better: maven central...)

Provide a first version which will include

• api client (java)
• server,
  • shall provide possibility to convert a local asciidoc file into PDF or HTML by a client call
    Result shall be either contain file path to render result or error message
  • shall have a warmup phase, so when first call to server will not be slow

Kind of communciation between client and server

• synchron
• sequential, no parallel work: just one client call per time, others will wait

Protocol:

• just simple tcp communication by lines (ending with \n) and key=value data transfer
• this will be full encapsulated in client, so caller does not bother with this

Currently the attributes are fetched only one time, than cached.

But they cannot be reset by client in any way...

So either remove the method at all (client +server) or provide a reset method as well
see also de-jcup/eclipse-asciidoctor-editor#229

It seems to be necessery, not to only provide the possiblity to start ASP with special environment values, but also to treat some of them in a special way.

"graphvizdot@" is used in attribute builder by intellij plugin:
https://github.com/asciidoctor/asciidoctor-intellij-plugin/blob/57ad18b78bf1d414f342e87eb706310a491b92fd/src/main/java/org/asciidoc/intellij/AsciiDoc.java#L1034-L1037

We should do same here.

It leads to errors when using an eclipse workspace at a location containing such characters (eg: é)

The issue comes from the request/response encryption that only support ISO-8859-1 charset. To avoid this issue I've made a PR suggesting to escape non ISO-8859-1 characters with their unicodes using Apache commons-text StringEscapeUtils escapeJava and unescapeJava method.

Currently server output is always shown. Should have a toggle flag, per default disabled

Situation

The client does always send "0.0.0" as version.

Example:

{
  "version" : "0.0.0",
  "command" : "convert_file",
  "source_filepath" : "/somewhere/example1.adoc",
  "options" : {
    "backend" : "pdf"
  },
  "attributes" : {
    "source-highlighter" : "coderay",
    "coderay-css" : "style"
  }
}

Wanted

Version of client shall be send.

Solution

The version is fetched from MANIFEST.MF - but the version is currently only injected by gradle for dist jars.
Must be injected in all jars by gradle build.

Situation

Inside V1.4.0 the new Option builder API was introduced and the origin "just map" was removed inside ASPClient
Inside de-jcup/eclipse-asciidoctor-editor#371 the for asciidoctor editor plugin build is being changed from gradle (+manual parts) to full automated maven tycho.

The tycho build does currently work with asp 1.3.1 but ... the asciidoctorj dependencies are missing.
The changes done for 1.4.0 shall not be applied inside the maven-tycho branch but inside master branch.

So... as long as maven tycho branch is still under development, we rely on the API in 1.3.1 way.

Wanted

Having a version based on 1.3.1 but with compile dependencies to asciidoctorj api parts.

Solution

Provide Version 1.3.2 based on 1.3.1 without the changes done in 1.4.0 and publish it on maven central

Currently always version "1.0" is used in communication, no matter which version client or server has.

This should be changed to real version numbering, so communication sends client and server versions.

Upgrade

• 'asciidoctorj', version: '2.5.1'
• 'asciidoctorj-pdf', version: '1.5.4'
• 'asciidoctorj-diagram', version: '2.1.2'

Situation

The method AspClient#resolveAttributes is marked as deprecated and the implementation on server side does use deprecated parts from asciidoctorj

Wanted

No longer provide or use deprecated parts

Solution

Remove the attributes