Giter Club home page Giter Club logo

apache / openwhisk-runtime-java Goto Github PK

View Code? Open in Web Editor NEW
45.0 30.0 63.0 3.12 MB

Apache OpenWhisk Runtime Java supports Apache OpenWhisk functions written in Java and other JVM-hosted languages

Home Page: https://openwhisk.apache.org/

License: Apache License 2.0

Shell 3.29% Java 36.95% Scala 34.88% Dockerfile 6.93% Makefile 3.26% Python 14.68%
openwhisk apache serverless faas functions-as-a-service cloud serverless-architectures serverless-functions docker functions openwhisk-runtime java jvm

openwhisk-runtime-java's Introduction

Apache OpenWhisk runtimes for java

License Continuous Integration

Changelogs

Quick Java Action

A Java action is a Java program with a method called main that has the exact signature as follows:

public static com.google.gson.JsonObject main(com.google.gson.JsonObject);

For example, create a Java file called Hello.java with the following content:

import com.google.gson.JsonObject;

public class Hello {
    public static JsonObject main(JsonObject args) {
        String name = "stranger";
        if (args.has("name"))
            name = args.getAsJsonPrimitive("name").getAsString();
        JsonObject response = new JsonObject();
        response.addProperty("greeting", "Hello " + name + "!");
        return response;
    }
}

In order to compile, test and archive Java files, you must have a JDK 8 installed locally.

Then, compile Hello.java into a JAR file hello.jar as follows:

javac Hello.java
jar cvf hello.jar Hello.class

Note: google-gson must exist in your Java CLASSPATH when compiling the Java file.

You need to specify the name of the main class using --main. An eligible main class is one that implements a static main method as described above. If the class is not in the default package, use the Java fully-qualified class name, e.g., --main com.example.MyMain.

If needed you can also customize the method name of your Java action. This can be done by specifying the Java fully-qualified method name of your action, e.q., --main com.example.MyMain#methodName

Not only support return JsonObject but also support return JsonArray, the main function would be:

import com.google.gson.JsonArray;
import com.google.gson.JsonObject;

public class HelloArray {
    public static JsonArray main(JsonObject args) {
        JsonArray jsonArray = new JsonArray();
        jsonArray.add("a");
        jsonArray.add("b");
        return jsonArray;
    }
}

And support array result for sequence action as well, the first action's array result can be used as next action's input parameter.

So the function would be:

import com.google.gson.JsonArray;

public class Sort {
    public static JsonArray main(JsonArray args) {
        return args;
    }
}

Create the Java Action

To use as a docker action:

wsk action update helloJava hello.jar --main Hello --docker openwhisk/java8action

This works on any deployment of Apache OpenWhisk

To use on a deployment of OpenWhisk that contains the runtime as a kind:

wsk action update helloJava hello.jar --main Hello --kind java:8

Invoke the Java Action

Action invocation is the same for Java actions as it is for Swift and JavaScript actions:

wsk action invoke --result helloJava --param name World
  {
      "greeting": "Hello World!"
  }

Local development

Pre-requisites

  • Gradle
  • Docker Desktop (local builds)

Build and Push image to a local Docker registry

  1. Start Docker Desktop (i.e., Docker daemon)

  2. Build the Docker runtime image locally using Gradle:

./gradlew core:java8:distDocker

This will produce the image whisk/java8action and push it to the local Docker Desktop registry with the latest tag.

  1. Verify the image was registered:
$ docker images whisk/*
REPOSITORY           TAG     IMAGE ID            CREATED             SIZE
whisk/java8action    latest  35f90453905a        7 minutes ago       521MB

Build and Push image to a remote Docker registry

Build the Docker runtime image locally using Gradle supplying the image Prefix and Registry domain (default port):

docker login
./gradlew core:java8:distDocker -PdockerImagePrefix=$prefix-user -PdockerRegistry=docker.io

Deploying the Java runtime image to OpenWhisk

Deploy OpenWhisk using ansible environment that contains the kind java:8 Assuming you have OpenWhisk already deployed locally and OPENWHISK_HOME pointing to root directory of OpenWhisk core repository.

Set ROOTDIR to the root directory of this repository.

Redeploy OpenWhisk

cd $OPENWHISK_HOME/ansible
ANSIBLE_CMD="ansible-playbook -i ${ROOTDIR}/ansible/environments/local"
$ANSIBLE_CMD setup.yml
$ANSIBLE_CMD couchdb.yml
$ANSIBLE_CMD initdb.yml
$ANSIBLE_CMD wipe.yml
$ANSIBLE_CMD openwhisk.yml

Or you can use wskdev and create a soft link to the target ansible environment, for example:

ln -s ${ROOTDIR}/ansible/environments/local ${OPENWHISK_HOME}/ansible/environments/local-java
wskdev fresh -t local-java

Testing

Install dependencies from the root directory on $OPENWHISK_HOME repository

pushd $OPENWHISK_HOME
./gradlew install
popd $OPENWHISK_HOME

Using gradle to run all tests

./gradlew :tests:test

Using gradle to run some tests

./gradlew :tests:test --tests *ActionContainerTests*

Using IntelliJ:

  • Import project as gradle project.
  • Make sure working directory is root of the project/repo

Using container image to test

To use as docker action push to your own dockerhub account

docker tag whisk/java8action $user_prefix/java8action
docker push $user_prefix/java8action

Then create the action using your the image from dockerhub

wsk action update helloJava hello.jar --main Hello --docker $user_prefix/java8action

The $user_prefix is usually your dockerhub user id.


Troubleshooting

Gradle build fails with "Too many files open"

This may occur on MacOS as the default maximum # of file handles per session is 256. The gradle build requires many more and is unable to open more files (e.g., java.io.FileNotFoundException). For example, you may see something like:

> java.io.FileNotFoundException: /Users/XXX/.gradle/caches/4.6/scripts-remapped/build_4mpzm2wl8gipqoxzlms7n6ctq/7gdodk7z6t5iivcgfvflmhqsm/cp_projdf5583fde4f7f1f2f3f5ea117e2cdff1/cache.properties (Too many open files)

You can see this limit by issuing:

$ ulimit -a
open files                      (-n) 256

In order to increase the limit, open a new terminal session and issue the command (and verify):

$ ulimit -n 10000

$ ulimit -a
open files                      (-n) 10000

Gradle Task fails on :core:java8:tagImage

Docker daemon is not started and the Task is not able to push the image to your local registry.

License

Apache 2.0

openwhisk-runtime-java's People

Contributors

cbickel avatar chetanmeh avatar csantanapr avatar dgrove-oss avatar dubee avatar falkzoll avatar fxulusoy avatar iainduncani avatar ioana-blue avatar jasonpet avatar jeremiaswerner avatar joachimvaldez avatar luke-roy-ibm avatar markusthoemmes avatar mdeuser avatar mrutkows avatar ningyougang avatar nwspeete-ibm avatar paulcastro avatar perryibm avatar psuter avatar rabbah avatar rsulzmann avatar sciabarracom avatar sjfink avatar skywalkeretw avatar starpit avatar tysonnorris avatar vinodmut avatar vvraskin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

openwhisk-runtime-java's Issues

Travis build failure openwhisk-admin-tools dependency not found

Environment details:

  • Travic-ci build

Steps to reproduce the issue:

  1. Trigger Travis-ci build for the runtime

Provide the expected results and outputs:

Build should pass successfully.

Provide the actual results and outputs:

FAILURE: Build failed with an exception.
* What went wrong:
Could not resolve all files for configuration ':tests:testCompileClasspath'.
> Could not find org.apache.openwhisk:openwhisk-admin-tools:1.0.0-SNAPSHOT.
  Searched in the following locations:
      https://repo.maven.apache.org/maven2/org/apache/openwhisk/openwhisk-admin-tools/1.0.0-SNAPSHOT/maven-metadata.xml
      https://repo.maven.apache.org/maven2/org/apache/openwhisk/openwhisk-admin-tools/1.0.0-SNAPSHOT/openwhisk-admin-tools-1.0.0-SNAPSHOT.pom
      https://repo.maven.apache.org/maven2/org/apache/openwhisk/openwhisk-admin-tools/1.0.0-SNAPSHOT/openwhisk-admin-tools-1.0.0-SNAPSHOT.jar
      file:/home/travis/.m2/repository/org/apache/openwhisk/openwhisk-admin-tools/1.0.0-SNAPSHOT/maven-metadata.xml
      file:/home/travis/.m2/repository/org/apache/openwhisk/openwhisk-admin-tools/1.0.0-SNAPSHOT/openwhisk-admin-tools-1.0.0-SNAPSHOT.pom
      file:/home/travis/.m2/repository/org/apache/openwhisk/openwhisk-admin-tools/1.0.0-SNAPSHOT/openwhisk-admin-tools-1.0.0-SNAPSHOT.jar
  Required by:
      project :tests > org.apache.openwhisk:openwhisk-tests:1.0.0-SNAPSHOT

Additional information you deem important:

  • Issue is not occurred due to a direct commit to the repository rather a change occurred with apache/openwhisk#3722 where the openwhisk-admin-tools might not be building during the travis build.

JVM heap size is not set correctly for Java actions

@rabbah commented on Thu Jun 15 2017

Java actions run with no heap settings, regardless of the container's memory limit: https://github.com/apache/incubator-openwhisk/blob/master/core/javaAction/Dockerfile#L38

This will cause a reused Java action to eventually be killed because the GC may not get a chance to run.

>  docker run -m 64M -it javaaction bash
# java -XX:+PrintFlagsFinal -version 2>&1 | grep MaxHeapSize
    uintx MaxHeapSize                              := 1035993088                          {product}

That's ~1GB even though memory limit for container is 64M.


@jsanda commented on Mon Oct 23 2017

Is something like this what you have in mind? run-java.sh is the default command for the image. Among other things it sets the JVM heap size based on container limits.

Support .java source actions in some fashion

by @gorkem from #12 (comment)

We should add direct .java invocation to this epic too. I have it already implemented on the devtools repo. [1]. The only reason I did not PR it to here was the gson dependency. Since we are taking care of it with this one, I see no reason not to implement it.
[1] https://github.com/apache/incubator-openwhisk-devtools/tree/master/java-local

by @csantanapr #12 (comment)

@gorkem I See no issue on adding .java also
@rabbah do you recall why this was not done originally for Java actions and only .jar?
Maybe the extra latency for compile step on /init too bad or takes longer than 1minute those would be the only two I see, put that’s a problem we have today with swift with single source file it’s convenient for dev/demo but for production is better to have a precompiled to the action
Maybe it’s time again to bring discussion of init actions that will help with compiling and saving the compiled output to the action stamping with runtime image and maybe source on the action this way it only compiled when the source changes or the runtime image changes.

by @csantanapr #12 (comment)

Hum 🤔
What about simple one liner functions between sequences ?
Meaning like to transform output parameters into input parameters in sequences
Maybe there is a OW provider/operator that included libraries and frameworks into a custom runtime
IBM including Watson sdk, COS SDK, Cloudant sdk, http client library
RedHat including spring cloud framework and adapters
Regular user extending a Java runtime with own libraries
Someone offering a UI web based editor useful for demo to edit a java file in web editor assuming that runtime has some useful libraries installed like Jax-rs related ones to handle web actions input.

by @rabbah #12 (comment)

It’s nice to have - quick start, demo all good.

by @kameshsampath #12 (comment)

This has been my thinking last couple of days in these lines
I am not sure where are storing the jars today when we say wsk action create foo foo.jar --main foo - is it as a blob in couchdb ??
As you mentioned that biggest stuff we need to resolve dependency resolution for Java - wondering we can deploy a Maven Repo Manager as part of the OW deployment, this could solve the above point as well,. Right now the biggest hurdle of Java Adoption is this as functions need to be quick

Behaviors of timeout is different for NodeJS actions and Java actions.

@daisy-ycguo commented on Sat Nov 04 2017

Environment details:

I'm running a local openwhisk and testing nodejs actions and java actions which run more than 1 minutes. The timeout of these actions are set to 60000.

Steps to reproduce the issue:

  1. Create nodejs action by wsk action create run2 action_fixedlen.js -i --param timeout 2. I find no errors after wsk action invoke run2 -i. The log shows the action is halted when it's actually not finished.
  2. Create java action by wsk action create javarun1 action_fixedlen.jar --main FixedTime -i. I find an error java.io.IOException: Broken pipe in log while wsk action invoke javarun1 -i.

The source code of action_fixedlen.js and action_fixedlen.jar can refer here.

Provide the expected results and outputs:

I expected clear output if the action is halted because of timeout, not java.io.IOException: Broken pipe, not "no errors" in logs.

Provide the actual results and outputs:

java.io.IOException: Broken pipe because of java action is timeout

......
    "2017-11-04T15:50:23.267170396Z stdout: I’m busy",
    "2017-11-04T15:50:24.267475348Z stdout: I’m busy",
    "2017-11-04T15:50:25.19915025Z  stdout: time out!",
    "2017-11-04T15:50:25.279663546Z stderr: java.io.IOException: Broken pipe",
    "2017-11-04T15:50:25.279751044Z stderr: at sun.nio.ch.FileDispatcherImpl.write0(Native Method)",
    "2017-11-04T15:50:25.279884854Z stderr: at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)",
    "2017-11-04T15:50:25.279953363Z stderr: at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)",
    "2017-11-04T15:50:25.279989627Z stderr: at sun.nio.ch.IOUtil.write(IOUtil.java:65)",
    "2017-11-04T15:50:25.279997402Z stderr: at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)",
    "2017-11-04T15:50:25.280038692Z stderr: at sun.net.httpserver.Request$WriteStream.write(Request.java:391)",
    "2017-11-04T15:50:25.280082249Z stderr: at sun.net.httpserver.FixedLengthOutputStream.write(FixedLengthOutputStream.java:78)",
    "2017-11-04T15:50:25.280131273Z stderr: at java.io.FilterOutputStream.write(FilterOutputStream.java:97)",
    "2017-11-04T15:50:25.280174979Z stderr: at sun.net.httpserver.PlaceholderOutputStream.write(ExchangeImpl.java:439)",
    "2017-11-04T15:50:25.280218671Z stderr: at openwhisk.java.action.Proxy.writeResponse(Proxy.java:60)",
    "2017-11-04T15:50:25.280226498Z stderr: at openwhisk.java.action.Proxy.access$400(Proxy.java:39)",
    "2017-11-04T15:50:25.280265324Z stderr: at openwhisk.java.action.Proxy$RunHandler.handle(Proxy.java:146)",
    "2017-11-04T15:50:25.280309312Z stderr: at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:79)",
    "2017-11-04T15:50:25.280354495Z stderr: at sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:83)",
    "2017-11-04T15:50:25.280396143Z stderr: at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:82)",
    "2017-11-04T15:50:25.281136929Z stderr: at sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:675)",
    "2017-11-04T15:50:25.281153061Z stderr: at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:79)",
    "2017-11-04T15:50:25.281157559Z stderr: at sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:647)",
    "2017-11-04T15:50:25.281161357Z stderr: at sun.net.httpserver.ServerImpl$DefaultExecutor.execute(ServerImpl.java:158)",
    "2017-11-04T15:50:25.281165196Z stderr: at sun.net.httpserver.ServerImpl$Dispatcher.handle(ServerImpl.java:431)",
    "2017-11-04T15:50:25.281189194Z stderr: at sun.net.httpserver.ServerImpl$Dispatcher.run(ServerImpl.java:396)",
    "2017-11-04T15:50:25.281195336Z stderr: at java.lang.Thread.run(Thread.java:748)"

No errors in log if a nodejs action is timeout.

Activation: run2 (08b49032e2144781b49032e214278111)
[
    "2017-11-04T16:18:59.38856493Z  stdout: timeout=2",
    "2017-11-04T16:18:59.388652855Z stdout: interval=10",
    "2017-11-04T16:19:02.402712303Z stdout: I'm busy.",
    "2017-11-04T16:19:03.408028403Z stdout: I'm busy.",
    .......
    "2017-11-04T16:19:26.438918288Z stdout: I'm busy.",
    "2017-11-04T16:19:27.443382697Z stdout: I'm busy.",
    "2017-11-04T16:19:28.444408797Z stdout: I'm busy.",
    ......
    "2017-11-04T16:20:02.521283865Z stdout: I'm busy.",
    "2017-11-04T16:20:03.523249329Z stdout: I'm busy."
]

@kevingrozav commented on Mon Nov 06 2017

I received a broken pipe error as well when trying to run the CLI installer (Mac amd64).
screen shot 2017-11-06 at 4 43 57 pm


@daisy-ycguo commented on Tue Nov 07 2017

@kevingrozav Your error looks like nothing related with an action execution. Which command are you trying to run?


@kevingrozav commented on Tue Nov 07 2017

@daisy-ycguo I followed the instructions on the CLI readme for building the binary from src with Go, and it is working now. Sorry for the confusion.


@rabbah commented on Fri Nov 24 2017

I'm not able to reproduce this with the information provided in the description.

Retroactively add openJ9 JDK version bump to changelog

PR https://github.com/apache/incubator-openwhisk-runtime-java/pull/77/files

bumped OpenJ9 base image to use release "jdk8u181-b13" but failed to note that in the 1.13 CHANGELOG.txt file

i.e.,:
FROM adoptopenjdk/openjdk8-openj9:x86_64-ubuntu-jdk8u181-b13_openj9-0.9.0

ironically, the PR https://github.com/apache/incubator-openwhisk-runtime-java/pull/69/files which also tried to bump the version (and was never merged) did attempt to update the changelog.

[Thought] Using Volumes for java actions

currently Java action jars are stored in couch DB and the streamed to the JarLoader while invocation. @gorkem is working to refactor that approach to save load and invocation time.

One of the thought is define a m2 kind of lay out and Java Action pick the jar from it instead of streaming. It could be modelled around GlusterFS and PV.

Some source files miss Apache license headers

Following Apache license header guideline, all human-readable Apache-developed files that are included within a distribution must include the header text with few exceptions. You can find few exceptions here: which files do not require a license header.

I used Apache Rat to check this repository after excluding a few files, and I got this report. We need to add Apache licensing header to those files.

Unapproved licenses:

  openwhisk-runtime-java/core/javaAction/Dockerfile
  openwhisk-runtime-java/CHANGELOG.md
  openwhisk-runtime-java/tests/src/test/resources/application.conf
  openwhisk-runtime-java/ansible/environments/local/group_vars/all
  openwhisk-runtime-java/ansible/environments/local/hosts

The excluded files are:

**/*.json
**/**.gradle
**/gradlew
**/gradle/**
**/.**
**/templates/**
**/*.j2.*
**/.github/**
**/auth.whisk.system
**/auth.guest
**/i18n_resources.go

java Actionruntime of jar confilcts

@xingyunhui commented on Sun May 13 2018

hi,
I use java8 actionRuntime, and have two jars are conflicted.
for example, there is a jar -- base-0.0.1.jar in java8 actionRuntime, my function use a jar -- base-0.0.2.jar, when I run my function in java8 actionRuntime, I find the actionRuntime always use base-0.0.1.jar , and I wanna use base-0.0.2.jar for my funtion


@csantanapr commented on Mon May 14 2018

Hi @xingyunhui

I don't understand what's the problem.

The java runtime doesn't have any file named base-0.0.1.jar

$ docker run --rm -it openwhisk/java8action find / -name base-0.0.1.jar

Could you open a new issue in the runtime-java repo, that's the repo that deals with java actions.

And provide more details on step by step on how to reproduce your error

Look to update adoptopenjdk base image

The base image we use to build our runtime is from 9+ months ago according to
https://hub.docker.com/r/adoptopenjdk/openjdk8-openj9/tags

Our current image uses specifically:

FROM adoptopenjdk/openjdk8-openj9:x86_64-ubuntu-jdk8u181-b13_openj9-0.9.0

Newer official released images include (released as of this issue creation):

  • x86_64-ubuntu-jdk8u192-b12_openj9-0.11.0r0 (9 months ago)
  • x86_64-ubuntu-jdk8u202-b08_openj9-0.12.1 (3 months ago)
  • x86_64-ubuntu-jre8u212-b03_openj9-0.14.0 (2 months ago)
  • x86_64-ubuntu-jre8u212-b04_openj9-0.14.2 (12 days ago)
  • x86_64-ubuntu-jre8u222-b10_openj9-0.15.1 (1 day ago)

Which should we bump to? Should we attempt to follow releases quarterly perhaps?

Note: these are not nightly builds but are selected builds promoted by the adopt community...

Making Java Function return Future

Currently there is no way to return Future from Java Action like how they are done in JavaScript via Promises. Making Java Action return future will help in running Asynchronous code inside Java Action

Use Class Data Sharing for Java actions

@alexkli commented on Thu Oct 05 2017

As described in this article, Java has a feature called class data sharing (CDS) that allows to share common class data for different JVMs.

This seems to be available since Java 5 (for JVM & JDK classes), and since Java 8 (u40) support application code as well (AppCDS).

The current OW java action runtime doesn't do anything like that yet.

Benefits:

  • faster JVM startup time: from 120ms to 90ms for a simple hello world program (in that article)
  • reduces memory footprint when running multiple JVMs, apparently works in a containerized environment, should be great within OpenWhisk

There are two steps for the basic CDS AFAICS:

  • prepare the shared class files once via java -Xshare:dump: could be done during the docker image build (?)
  • call the JVM with the flag -Xshare:on to enable CDS (there is some talk about -Xshare:auto becoming the default which I assume would enable it automatically if the prepared files are present)

Leveraging AppCDS in OpenWhisk would mean to include the action Java code, and that might be tricky, as that can only be shared between containers with the same action code IIUC.

Same for Ahead-Of-Time compilation (AOT), new in Java 9, mentioned in the article as well, which has a costly pre-compilation step (that should include the application aka action code), not sure how that could be done within OW. It would be beneficial if an action needs to scale up, i.e. you are starting many new Java containers with a particular action and you want to get the fastest startup time. But maybe pre-warming makes this less useful. One would have to be able to run that action specific pre-compilation up front, and that might require to embed it in the docker image IIUC, which in turn would require building it dynamically from OW, which sounds complex.


@dgrove-oss commented on Thu Oct 05 2017

There should be benefits just from enabling shared classes for the core libraries + framework code (not specific to any user action).


@alexkli commented on Thu Oct 05 2017

Yes, that's what I meant with "basic CDS" (as opposed to AppCDS). It could include the java action framework code.


@rabbah commented on Thu Oct 05 2017

The Java runtime is moving to a new repository - when that's complete, we'll move this issue there.

FYI @csantanapr.

[Epic] Improvements to Java Runtime avoid com.sun.*, support MANIFEST main and other runtime image optimizations

The following are list of tasks that we need to work to improve the OW Java Action runtime:

  1. avoid usage of com.sun.* packages - this is used in https://github.com/apache/incubator-openwhisk-runtime-java/blob/master/core/javaAction/proxy/src/main/java/openwhisk/java/action/Proxy.java, instead we can use readily available pre-built httpservers e.g. using vert.x is one of the options for high performance

  2. Right now we rely on the --main parameter passed to run Java Aciton, instead we can make the Loader to infer it via MANIFEST.mf Main-class or add custom attribute for OW apps

  3. It would be great to have making OW Java Action use standard Java main method signature with just String[] args and we define custom Launcher that convert the args to JSON and delegate to OW Java Function Launcher, this way it will be easy for applications to be built and tested outside before testing in OW
    Edited by @csantanapr 3 is now a new issue: #15

@csantanapr @ddragosd @gorkem - thoughts comments and feedback ??

fix .travis it contains deploy section twice

I messed up the .travis file with duplicate lines deploy

deploy:
  - provider: script
    script: "./tools/travis/publish.sh openwhisk ${TRAVIS_TAG%@*} ${TRAVIS_TAG##*@}"
    on:
      tags: true
      all_branches: true
deploy:
  - provider: script
    script: "./tools/travis/publish.sh openwhisk 8 master"
    on:
      branch: master

Multiple Distros for Java Runtimes

It will be nice to have OpenWhisk JDK runtimes available on multiple distributions

  • centos
  • debian
  • alpine

The initial work of this can follow the pattern like https://github.com/kameshsampath/adoptopenjdk .

Moving this discussion from #24, #18 to discuss and freeze on this.

@Param-S lets continue the discussion here

@csantanapr - we are moving this discussion into its own thread .

@bbrowning - this is something we started to discuss the other day slack. Please add your thoughts comments here.

about custom jre

Hi, sir, I wonder if there is a way to use my customed JRE on openwhisk? That's to say, I build the openjdk locally, how can I use this openjdk on openwhisk?

Thx a lot!!!

Support Scala Actions

One person on Slack, and now a customer using IBM Cloud Functions want to use Scala to write their OpenWhisk Actions.

It looks like the java runtime can be use to run Scala actions.

What's need it to support it a bit better than today is

  1. Add automation test cases with some scala examples, including process to generate the jar
  2. Docs explaining how an user can get started with a simple hello world example code and steps to build jar.
  3. (bonus) The runtime image can be use to compile the scala and produce the jar

Here is an example:

Create Directory structure

$ tree
.
├── Hello.scala
├── build.sbt
└── project
    └── plugins.sbt

Create Action Hello.scala

$ cat Hello.scala
import com.google.gson.JsonObject
object Hello {
  def main(args: JsonObject): JsonObject = {
    println("Log something")
    val name = if (args.has("name")) args.getAsJsonPrimitive("name").getAsString() else "stranger"
    val response = new JsonObject()
    response.addProperty("greeting", "Hello " + name + "!")
    response
  }
}

Setup Project

$ cat build.sbt

scalaVersion := "2.11.8"
libraryDependencies ++= Seq (
  "com.google.code.gson" % "gson" % "latest.integration"
)
$ cat project/plugins.sbt
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.6")

Build Jar

$ sbt assembly
[info] Loading global plugins from /Users/csantanapr/.sbt/0.13/plugins
[info] Loading project definition from /Users/csantanapr/Documents/dev/whisk/demos/scala_action/project
[info] Set current project to scala_action (in build file:/Users/csantana23/Documents/dev/whisk/demos/scala_action/)
[info] Including from cache: gson-2.8.2.jar
[info] Including from cache: scala-library-2.11.8.jar
[info] Checking every *.class/*.jar file's SHA-1.
[info] Merging files...
[warn] Merging 'META-INF/MANIFEST.MF' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.google.code.gson/gson/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.google.code.gson/gson/pom.xml' with strategy 'discard'
[warn] Strategy 'discard' was applied to 3 files
[info] Assembly up to date: /Users/csantanapr/Documents/dev/whisk/demos/scala_action/target/scala-2.11/scala_action-assembly-0.1-SNAPSHOT.jar
[success] Total time: 1 s, completed Apr 23, 2018 8:53:44 PM

Deploy Action

$ wsk action update scala target/scala-2.11/scala_action-assembly-0.1-SNAPSHOT.jar --kind java --main Hello
ok: updated action scala

Run Action

$ wsk action invoke scala -p name Carlos -r
{
    "greeting": "Hello Carlos!"
}

Java action that depends on packages jar file fails

I have a set of Java actions packaged in a jar file that I deploy on Bluemix. All Java code is packaged in a jar file with a lib directory that contains dependant jar files, in particular the Cloudant client. Simple actions that do not use any dependant libraries work fine. Actions that use the Cloudant library fail with a NoClassDefFoundError.

Environment details:

  • Bluemix
  • Build on Ubuntu 16.04

Steps to reproduce the issue:

  1. Build the code with gradle
  2. Deploy on bluemix (I use wskdeploy)
  3. Invoke the example/greeting action
  4. Invoke the example/cloudant-test action

Provide the expected results and outputs:

Result invoking example/greeting:

$ bx wsk action invoke example/greeting  -b
ok: invoked /_/example/greeting with id 48d37cc2dfff42c2937cc2dfff42c2cb
{
    "activationId": "48d37cc2dfff42c2937cc2dfff42c2cb",
    "annotations": [
        {
            "key": "path",
            "value": "consideratehoteliers.com_dev/example/greeting"
        },
        {
            "key": "waitTime",
            "value": 493
        },
        {
            "key": "kind",
            "value": "java"
        },
        {
            "key": "limits",
            "value": {
                "logs": 10,
                "memory": 256,
                "timeout": 60000
            }
        },
        {
            "key": "initTime",
            "value": 434
        }
    ],
    "duration": 460,
    "end": 1522921174650,
    "logs": [],
    "name": "greeting",
    "namespace": "consideratehoteliers.com_dev",
    "publish": false,
    "response": {
        "result": {
            "greeting": "Hello !"
        },
        "status": "success",
        "success": true
    },
    "start": 1522921174190,
    "subject": "[email protected]",
    "version": "0.0.1"
}

Provide the actual results and outputs:

$ bx wsk action invoke example/cloudant_test -b
ok: invoked /_/example/cloudant_test with id b7b7df5cbc754685b7df5cbc752685c5
{
    "activationId": "b7b7df5cbc754685b7df5cbc752685c5",
    "annotations": [
        {
            "key": "path",
            "value": "consideratehoteliers.com_dev/example/cloudant_test"
        },
        {
            "key": "waitTime",
            "value": 520
        },
        {
            "key": "kind",
            "value": "java"
        },
        {
            "key": "limits",
            "value": {
                "logs": 10,
                "memory": 256,
                "timeout": 60000
            }
        },
        {
            "key": "initTime",
            "value": 420
        }
    ],
    "duration": 447,
    "end": 1522851950713,
    "logs": [],
    "name": "cloudant_test",
    "namespace": "consideratehoteliers.com_dev",
    "publish": false,
    "response": {
        "result": {
            "error": "An error has occured while invoking the action (see logs for details): java.lang.NoClassDefFoundError: com/cloudant/client/api/ClientBuilder"
        },
        "status": "action developer error",
        "success": false
    },
    "start": 1522851950266,
    "subject": "[email protected]",
    "version": "0.0.1"
}

Additional information you deem important:

See attached files that include:

  • the wskdeploy manifest
  • the gradle build file
  • two Java classes that implement the actions mentioned above, Hello.java that works and CloudantTest.java that doesn't

code.zip

HTTP requests/responses not correctly parsed in expected format in webactions

Given a WebAction call of:

https://localhost:31001/api/v1/web/guest/foo/testFunction/foo?aQuery=bob&aQuery2=bob

... the parsed JSON object extracts and passes the query parameters as fields on the root object and not in the __ow_query field (which was expected and would be consistent with other runtimes). So for the above request, the JSON object looks like:

{
   "__ow_method":"get",
   "__ow_headers":{
      "accept-encoding":"gzip",
      "host":"owdev-controller.default.svc.cluster.local:8080",
      "user-agent":"okhttp/4.7.2"
   },
   "__ow_path":"/foo",
   "value2":"bob",
   "value":"bill"
}

Is this expected behaviour? Additionally, even with the above, neither header or queries would cope with multiple parameters with the same name - for this it seems like the format of those blocks should be along the lines of:

{
    "__ow_method": "get",
    "__ow_headers": {
        "accept-encoding": "gzip",
        "host": "owdev-controller.default.svc.cluster.local:8080",
        "user-agent": "okhttp/4.7.2"
    },
    "__ow_path": "/foo",
    "__ow_query": {
        "value2": [
            "bob"
        ],
        "value": [
            "bill"
        ]
    }
}

Additionally, the response code of an HTTP action is hardcoded to 200 in the Proxy.java. I'm not sure if this is required by OpenWhisk or is something that could be fixed in the case that an HTTP response object was returned by the underlying action.

Current master build failing in ansible command

Current master build is failing with following error

fatal: [controller0]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: {u'replicationFactor': u\"{{ kafka_replicationFactor | default((groups['kafkas']|length)|int) }}\", u'version': u'0.11.0.1', u'port': 9092, u'heap': u\"{{ kafka_heap | default('1g') }}\", u'ras': {u'port': 8093}}: 'dict object' has no attribute 'kafkas'\n\nThe error appears to have been in '/home/travis/build/apache/openwhisk/ansible/roles/controller/tasks/deploy.yml': line 77, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: (re)start controller\n  ^ here\n"}

add additional Volume to java run time.

I need to access config file from host when calling action(Java action) by openwhisk .
But am unable to read from host(showing file not found exception).
So for that i need to add volume when docker java8action image/container bootup.
How i will add that volume info..
Please suggest..

More flexibility to parsing and passing JSON input/out in Java runtime

originally proposed by @kameshsampath

It would be great to have making OW Java Action use standard Java main method signature with just String[] args and we define custom Launcher that convert the args to JSON and delegate to OW Java Function Launcher, this way it will be easy for applications to be built and tested outside before testing in OW

@rabbah suggested the use of FDK (Function Developer Kit) to hide complexity to deal with a string from end users

On 3, did you consider an openwhisk java FDK which provides a standard wrapper instead for local development?

created out of #12

JarLoader breaks strong encapsulation of JDK classes

JarLoader#augmentEnv performs illegal access on core JDK classes by breaking accessibility rules. This breaks the strong encapsulation rules outlined in JEP 261. Starting with Java 9 this will print a warning by default on the command line (and possibly have a performance and runtime impact). At some yet to be defined point in the future this will throw an exception by default. At some later yet to be defined point in the future this will always throw an exception.

I don't see an alternative to entirely moving away from modifying environment variables. They are readonly in Java. Modifying them is a hack that makes testing hard and multi threading impossible.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.