Current Travis tests only run our unit tests; they don't run any end to end tests. We need to trigger an end to end test on a Kubernetes cluster on each PR.

Hi,

It's on the roadmap to add PHP in the environments ? I think it's easy, if it's not in the roadmap i will try to add it.

Thanks

I'm not sure this is intended behaviour:

If I add two functions to the same URL they are both accepted by fission and fission will execute the first one, however when listing functions the order is reversed and its not clear which one is active

fission route create --method GET --url /hello --function hello
trigger 'd09e4d12-5e5b-4ebd-bf66-c4cd67e846ae' created
fission route create --method GET --url /hello --function hello2
trigger '59add1e2-630d-42dc-ab7e-e993a7de4e77' created
fission route list
NAME                                 METHOD URL    FUNCTION_NAME FUNCTION_UID
59add1e2-630d-42dc-ab7e-e993a7de4e77 GET    /hello hello2
d09e4d12-5e5b-4ebd-bf66-c4cd67e846ae GET    /hello hello

in this setup hello is executed and not hello2 , only after removing the route to hello , hello2 takes over

The controller handles the case that /filestore doesn't exist, but not the case where stat failed for any other reason. And fission-bundle doesn't handle controller.MakeFilestore errors at all.

Hi,

The fission.io offers clients for Linux and Mac

Since minikube is available for Windows as well it might be convenient to provide a Windows client as well so people would not need to build one themselves.

I cloned the repo on my Windows machine, installed Go, build the client and it worked like a charm.

If no windows server is available to build one in a CI process then cross compiling might be an option.

Cheers,
Hans

Hi,

is there a roadmap for implementing Java env in fission?

Cheers,
Marcin

Both can probably work on Alpine instead of Ubuntu.

fission appears to use a buffered context for both receiving and sending data.

This prevents the ability to create streaming transformations, or process any data coming into the function as a stream. Do you have any plans to add streaming support? Seems like a pretty big thing to not have

microcule has streaming support ( as well as a plugin system and support for 14+ languages ) https://github.com/stackvana/microcule

We have test that, if we are using k8s service in a large scale cluster and massive service and pod scenario, it will face the serious latency issues.

In a 1000 node, 4000 pod, 600 service condition, the 1000 req/sec request's average latency will be 8 times of directly forwarding to the pod's port. Because the k8s' default mechanism will add many iptable's item in one node as the service number increase, that cause a dramatic performance decreasing( we using another network implementation to avoid this).

In a large scale serverless scenario, there are many functions and function runtime instance, so it's not a small problem in time critical task.

The kubewatcher service is pretty much independent of Fission. Only the API is dependent on the fission.Watch type, which may as well be a K8s resource; then watchSync.go can be changed to a the usual K8s controller loop instead of watching the Fission API.

This issue will track the work of removing the fission-specific bits and moving to a K8s resource and controller.

Issue for tracking Ruby support. We will work on this based on interest; of course, contributions are always welcome. Please 👍 this issue if you're interested in Ruby support.

If anyone's interested in helping out with this, the Python environment is a good place to learn how to add language support to Fission.

This issue is for fission logs itself, not function logs; we've been undisciplined about using logrus properly. We need to go over most log statements and change them to use the WithFields logging method.

This is an issue that's tracking the work around redesigning the environment interface. This covers #24 but also various other issues, such as dependency resolution, better handling of compile-time errors, etc.

Current status: writing up requirements and user stories. The doc is in Documentation/wip/environments-v2.md.

Today fission uses a persistent volume + etcd to store functions. Each is wrapped in a simple interface (FileStore and ResourceStore) so we can have different storage backends for functions. We could support S3/GCS/other cloud storage backends for functions. We could also support Kubernetes ThirdPartyResources for both file and resource stores (despite etcd size limits, there's some operability advantages, like not having to provision persistent volumes and manage another etcd instance).

To support Go, Rust, and other compiled languages, we'll need a separate build step. This should be triggered at function create/update.

Environments will need a link to an image that will have the compiler and other tools (such as dependency managers).

We will need to figure out a way to specify dependencies for functions. Maybe a new fission resource type for a dependency spec, and a way to associate that resource with functions.

Environments should include a file extension field, and then functions could be automatically mapped to the right environment. (Except when there are multiple envs with the same file extension; in that case the user would need to resolve the ambiguity.)

Fission API needs some sort of authentication, either with client certs or some sort of user/password file.

pod name hits limit of 63 chars if env name is greater than 17 chars
may want to drop adding entire uid

2016/12/09 06:42:23 eager-create pool failed: Deployment.extensions "fission-alpha9-mod-b20e9823-72a8-4cbd-8ecc-d6fbfef8dadb-hxyrtehs" is invalid: [spec.selector.matchLabels: Invalid value: "fission-alpha9-mod-b20e9823-72a8-4cbd-8ecc-d6fbfef8dadb-hxyrtehs": must be no more than 63 characters, spec.selector: Invalid value: {"matchLabels":{"pool":"fission-alpha9-mod-b20e9823-72a8-4cbd-8ecc-d6fbfef8dadb-hxyrtehs"}}: invalid label selector.]

Functions need a way of requesting access to Kubernetes secrets and configmaps, either as volumes or env vars. They should also be able to request access to persistent volumes.

Instead of deploying your own etcd use the etcd operator to take advantage of scaling, upgrades, etc.

cc @xiang90 @hongchaodeng

We need to add a connection timeout (and perhaps retries) to the fission client so it doesn't just hang when the server is unavailable; we should also output some useful error in the CLI when that happens.

The current Python support is more like a proof of concept than something usable. It needs a lot of improvements:

1. Request parameters in the URL or in the body need to be parsed and provided to the function in the context object
2. Parse bodies based on Content-Type
3. Support all HTTP methods (today only GET is supported)

Investigate using flask or another framework to do all this, instead of implementing it from scratch.

Kubewatcher today only has a webhook publisher; investigate adding a NATS-Streaming-based publisher for reliable message delivery.

Hi,

I noticed in https://github.com/fission/fission/blob/master/Documentation/Architecture.md that a swagger spec is mentioned.
However api/swagger.json does not exist (anymore ?)

In my view the project would benefit from formally specified API's between the components.
Say one would like to build a fission dashboard then having to go through the Go code to find the URL's is rather cumbersome. Its much easier to generate a javascript client from a swagger spec.

And if you would like to take it even further then its possible to generate a the server stubs, the client stubs and even the command-line client (e.g. using http://swagger.io/tools/ or https://goa.design/ )

Last but not least: formally specifying the API's increase security as it stops invalid inputs early in the process :-)

Cheers,
Hans

One should be able to limit resource usage (cpu, ram, etc) for a given function. This could be possibly specify through the CLI

Set up travis CI, or something like it, for testing.

Hi,

Great idea and I'm liking the ease of implementation. The company I'm working with at the moment has deployed Openshift 3 rather than pure kubernetes, so I was hoping to have native support for Openshift 3 in Fission.

Openshift is mostly a superset of Kubernetes so it's not not too difficult to get working, I managed to get Fission up and running today with broad strokes, but was hoping to have it done at the application level so there are fewer security concessions.

Openshift has stricter security rules than Kubernetes with a few specifically affecting Fission. That being said the security rules are pretty good, so might be something worth adding to Fission on Kubernetes as well. The main things are:

• Containers run as a random numeric user (so either need a volume or world writable directory to make file system changes)
• Containers run with fewer capabilities
• There is a concept of projects in Openshift. They are directly mapped to namespaces, but you need to explicitly allow permissions for API calls between projects

Summary
In short the following needs doing to allow Fission to work on Openshift 3:

1. The fission etcd deployment can't create it's config store directory due to permission. The fix for Openshift would be to specify ETCD_DATA_DIR to either use a volume or create the etcd config dir in something like /tmp, which is world writable
2. The fission controller can't create it's /filestore directory. In Openshift it needs to create the /filestore directory somewhere that is world writable in the container or on a volume. The controller also seem to fail to function correctly unless the filestore directory is created but doesn't panic/error on that failure, so I think it's worth changing the behaviour to panic/exit on /filestore creation failure so that it's easier to debug and more clearly shows as an error (pod restarts)
3. The pool manager doesn't have permission to call cross namespace in Openshift 3, so calls to the fission-function namespace are rejected. The best fix in Openshift would be to create a custom role for the pool manager in the deployment template (with only the API calls to the other project the pool manager needs to make) and assigned it to the correct service account in the fission-function project. An easier but less security fix is to assign the cluster-administrator role in the fission-function project to the default fission service account (system:serviceaccount:fission:default). Due to those missing rules this means that at least the following API calls fail from the pool manager in the current deployment:
   • list extensions.deployments in the fission-function namespace
   • create extensions.deployments in the fission-function namespace
4. It might just be the file permissions above, but until I granted the capability to run as root for the controller (the same set of capabilities as Docker assigns by default) I would get the exception right at the bottom of this ticket. The only way I could fix the issue was to grant those capabilities (as I couldn't change the directory without changing source code like for etcd) so I'm not sure if there's more that needs doing.

Developing on Openshift
Openshift is super simple to get up and running, the client tool sets up a docker component based cluster to work with. The CLI tool is also just a superset of Kubectl so there's not much new to learn other than the additional concepts, to get going:

• Get the latest client (cli) tool version from https://github.com/openshift/origin/releases/tag/v1.4.1
• Run oc cluster up --version 'v1.4.0'. This will pull down the Openshift Origin containers and run them, making your cluster
• NOTE: If using docker toolbox you might need to switch docker-machine context context to the newly created docker machine if it fails half way through and run the same command again, e.g.
  • eval $(docker-machine env --shell bash openshift)
  • oc cluster up --version 'v1.4.0'
• oc adm policy add-cluster-role-to-user cluster-admin system --as=system:admin
• oc login -u system:admin
• Use the oc CLI tool (like kubectl) and/or login to the console with the same credentials on port 8443 on the host that is running the docker containers (so maybe your vbox host docker-machine ip openshift)

Getting Fission Deployed on Openshift
Below are the steps I took to get Fission up and running on Openshift 3, including a script with the hacks necessary to bypass the security features.

First of all you need to patch http://fission.io/fission.yaml with the following to change the etcd configstore dir:

142a143,144
>         - name: ETCD_DATA_DIR
>           value: /tmp/default.etcd

Then running the following script will do the job (in a shell that is logged in with oc):

#!/bin/bash

 oc new-project fission # The existing fission templates already create the project correctly by using the namespace attribute, but I still do it in order to make the changes below before creation of the resources
 oc adm policy add-scc-to-user anyuid -z default # Hack 1
 oc new-project fission-function
 oc policy add-role-to-user cluster-admin system:serviceaccount:fission:default # Hack 2
 oc create -f fission.yaml # Hack 3
 oc create -f fission-nodeport.yaml

1. Hack 1: This fixes item number 2 from the summary section, but in a broad way which gives the container more capabilities than it needs
2. Hack 2: This fixes item number 3 from the summary section, but in a way that is overly permissive for the pool manager.
3. Hack 3: Deploying the etcd pod so it creates the configstore in /tmp, not necessarily a hack.

With the above hacks I was able to successfully deploy the hello world function, so it's definitely possible to get it working!

Exception
The exception referenced in the last point of the summary section

2017/02/06 00:45:07 http: panic serving 172.17.0.1:59594: runtime error: invalid memory address or nil pointer dereference
goroutine 4036 [running]:
net/http.(*conn).serve.func1(0x19f62e00)
	/usr/local/go/src/net/http/server.go:1491 +0xa2
panic(0x8fe7940, 0x19c1e008)
	/usr/local/go/src/runtime/panic.go:458 +0x40b
github.com/fission/fission/controller.(*ResourceStore).writeFile(0x19e2bea0, 0x1a018a80, 0x5, 0x1a073800, 0x53, 0x60, 0x0, 0x0, 0x0, 0x0, ...)
	/Users/soam/gopath/src/github.com/fission/fission/controller/resourceStore.go:156 +0x18d
github.com/fission/fission/controller.(*FunctionStore).Create(0x19e2bea0, 0x19f3c9c0, 0x0, 0x0, 0x0, 0x0)
	/Users/soam/gopath/src/github.com/fission/fission/controller/functionStore.go:31 +0xec
github.com/fission/fission/controller.(*API).FunctionApiCreate(0x19e2bea0, 0xf76a6988, 0x19f13fc0, 0x1a032380)
	/Users/soam/gopath/src/github.com/fission/fission/controller/functionApi.go:67 +0x232
github.com/fission/fission/controller.(*API).FunctionApiCreate-fm(0xf76a6988, 0x19f13fc0, 0x1a032380)
	/Users/soam/gopath/src/github.com/fission/fission/controller/api.go:73 +0x38
net/http.HandlerFunc.ServeHTTP(0x19fdf738, 0xf76a6988, 0x19f13fc0, 0x1a032380)
	/usr/local/go/src/net/http/server.go:1726 +0x34
github.com/fission/fission/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0x19e4c270, 0xf76a6988, 0x19f13fc0, 0x1a032380)
	/Users/soam/gopath/src/github.com/fission/fission/vendor/github.com/gorilla/mux/mux.go:103 +0x210
github.com/fission/fission/vendor/github.com/gorilla/handlers.loggingHandler.ServeHTTP(0x9887b70, 0x19c1e0d8, 0x9885430, 0x19e4c270, 0x988dd40, 0x19d041b0, 0x1a032380)
	/Users/soam/gopath/src/github.com/fission/fission/vendor/github.com/gorilla/handlers/handlers.go:69 +0xf0
github.com/fission/fission/vendor/github.com/gorilla/handlers.(*loggingHandler).ServeHTTP(0x1a031240, 0x988dd40, 0x19d041b0, 0x1a032380)
	<autogenerated>:8 +0x9b
net/http.serverHandler.ServeHTTP(0x19f88410, 0x988dd40, 0x19d041b0, 0x1a032380)
	/usr/local/go/src/net/http/server.go:2202 +0x136
net/http.(*conn).serve(0x19f62e00, 0x988e940, 0x19f13e40)
	/usr/local/go/src/net/http/server.go:1579 +0xe86
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2293 +0x489

[WARN]	Lock file may be out of date. Hash check of YAML failed. You may need to run 'update'
[INFO]	Downloading dependencies. Please wait...
[INFO]	--> Fetching github.com/coreos/go-oidc.
[INFO]	--> Fetching github.com/coreos/pkg.
[INFO]	--> Fetching github.com/dchest/uniuri.
[INFO]	--> Fetching github.com/coreos/etcd.
[INFO]	--> Fetching github.com/davecgh/go-spew.
[INFO]	--> Fetching github.com/blang/semver.
[INFO]	--> Fetching github.com/docker/distribution.
[INFO]	--> Fetching github.com/docopt/docopt-go.
[INFO]	--> Fetching github.com/emicklei/go-restful.
[INFO]	--> Fetching github.com/ghodss/yaml.
[INFO]	--> Fetching github.com/go-openapi/jsonpointer.
[INFO]	--> Fetching github.com/go-openapi/jsonreference.
[INFO]	--> Fetching github.com/go-openapi/spec.
[INFO]	--> Fetching github.com/go-openapi/swag.
[INFO]	--> Fetching github.com/gogo/protobuf.
[INFO]	--> Fetching github.com/golang/glog.
[INFO]	--> Fetching github.com/golang/protobuf.
[INFO]	--> Fetching github.com/google/gofuzz.
[INFO]	--> Fetching github.com/gorilla/context.
[INFO]	--> Fetching github.com/gorilla/handlers.
[INFO]	--> Fetching github.com/gorilla/mux.
[INFO]	--> Fetching github.com/jonboulle/clockwork.
[INFO]	--> Fetching github.com/juju/ratelimit.
[INFO]	--> Fetching github.com/mailru/easyjson.
[INFO]	--> Fetching github.com/pborman/uuid.
[INFO]	--> Fetching github.com/PuerkitoBio/purell.
[INFO]	--> Fetching github.com/PuerkitoBio/urlesc.
[INFO]	--> Fetching github.com/satori/go.uuid.
[INFO]	--> Fetching github.com/Sirupsen/logrus.
[INFO]	--> Fetching github.com/spf13/pflag.
[INFO]	--> Fetching github.com/ugorji/go.
[INFO]	--> Fetching github.com/urfave/cli.
[INFO]	--> Fetching golang.org/x/net.
[INFO]	--> Fetching golang.org/x/oauth2.
[INFO]	--> Fetching golang.org/x/sys.
[INFO]	--> Fetching golang.org/x/text.
[INFO]	--> Fetching google.golang.org/appengine.
[WARN]	Unable to checkout golang.org/x/net
[ERROR]	Update failed for golang.org/x/net: Cannot detect VCS
[INFO]	--> Fetching google.golang.org/cloud.
[WARN]	Unable to checkout golang.org/x/oauth2
[ERROR]	Update failed for golang.org/x/oauth2: Cannot detect VCS
[INFO]	--> Fetching gopkg.in/inf.v0.
[INFO]	--> Fetching gopkg.in/yaml.v2.
[INFO]	--> Fetching k8s.io/client-go.
[WARN]	Unable to checkout golang.org/x/sys
[ERROR]	Update failed for golang.org/x/sys: Cannot detect VCS
[WARN]	Unable to checkout golang.org/x/text
[ERROR]	Update failed for golang.org/x/text: Cannot detect VCS
[WARN]	Unable to checkout google.golang.org/appengine
[ERROR]	Update failed for google.golang.org/appengine: Cannot detect VCS
[WARN]	Unable to checkout google.golang.org/cloud
[ERROR]	Update failed for google.golang.org/cloud: Cannot detect VCS
[ERROR]	Failed to install: Cannot detect VCS
Cannot detect VCS
Cannot detect VCS
Cannot detect VCS
Cannot detect VCS
Cannot detect VCS

How about using godep?

Thanks,

The kubernetes client can close watches after a certain period; we need to restart them when this happens.

Pods may become ready before having an IP; today poolmgr just fails if pod.Status.PodIP is empty. It should wait and retry a few times up to some time limit before failing.

Current implementation seems only start one pod for one function, is there a road map for scale out on request like AWS Lambda? It's important feature for serverless.

The yaml snippit for the controller deoployment ends in crash loop backoff with the following logs:

$ kcn fission logs controller-2839000588-pt9wb
2016/11/28 22:35:52 Error: path /filestore does not exist

We need a way to autoscale specialized function pods up (and down) based on load. There are a few possible metrics to scale based upon (CPU usage, latency, queue depth) and there's some design work needed around how these metrics should be tracked, what component should make the autoscaling decision, and so on.

Today a user can access function logs by looking for the pod and using kubectl logs:

kubectl --namespace fission-function get pods -l functionName=foo

kubectl --namespace fission-function logs podname

This has two problems:
(a) It doesn't work after the pod is deleted
(b) It's cumbersome, the user has to learn how to do this

We need to enable something like

fission function logs --name <function>

And have this solve both problems (a) and (b) above. Of course (a) will require persisting logs somewhere. We need to investigate if it's easy to integrate with one of the existing solutions, or whether we need to roll our own solution for that.

I got this error "Internal server error (fission)".

I ran these command as follows.

docker-machine start default
minikube start
export FISSION_URL=http://$(minikube ip):31313
export FISSION_ROUTER=$(minikube ip):31314
fission env create --name nodejs --image fission/node-env
echo 'module.exports = function(context, callback) { callback(200, "Hello, world!\n"); }' > hello.js
fission function create --name hello --env nodejs --code hello.js
fission route create --method GET --url /hello --function hello
curl http://$FISSION_ROUTER/hello
Internal server error (fission)

Any idea?

When a pool manager start, it will clear the resources that not belong to it's instanceId, that means one namespace only allow one pool manager.

Seems that we only can deploy multi pool manager by using different namespace. But two pool manager will start two seperate function service. When function call coming, router will forward the request to one service randomly. Is this right?

Using multi service should be hard to deal with the autoscaling limit.

I've just installed the Client CLI by following the instructions in the README.

Running fission -v confirms I am running v0.5.0

If I try to follow along and create the environment using fission env create I get following listing:

Usage: fission [options] COMMAND [arguments]
    -h, --help                       Displays this message
    -v, --version                    Output the version of fission

Commands:
clone                Clone a VM
delete               Delete a VM
info                 Show information for a VM
snapshot create      Create a snapshot of a VM
snapshot delete      Delete a snapshot of a VM
snapshot list        List the snapshots of a VM
snapshot revert      Revert a VM to a snapshot
start                Start a VM
status               Show the status of all VMs
stop                 Stop a VM
suspend              Suspend a VM

None of the commands seem to match any that are described in the documentation or the codebase here in Github?

Am I missing something completely?

Thanks,
Ville

The poolmgr cache needs to be invalidated when the latest function version is requested and the function has been updated.

Poolmgr doesn't keep track of whether there's an ongoing request, so the easiest thing to do would be to kill the old version after the function's timeout, and send new requests to a new pod containing the latest version.

The router's cache also needs invalidated on function update for a route that points to the latest version.

Golint reports a bunch of warnings, mostly about missing comments. We should clean up and remove unused exports [any tools for that?], and add comments where it makes sense.

https://goreportcard.com/report/github.com/platform9/fission#golint

When I'm going to create a nodejs environment with tutorial, I get stuck.

fission env create --name nodejs --image fission/node-env

There is no response with my terminal. I'm using fission on Mac.

Kubernetes has a 3rd party resource type that allows extending the kubernetes api by 3rd parties.

It would be very interesting to add support for this in fission. The user could then upload an environment description as a 3rd party object, and the script file itself as a 3rd party object.

This even would allow the user to deploy those objects in helm charts along with other k8s objects.

fission env create --image nonexistent-image leads to poolmgr creating a new deployment every few minutes, with no limit ( 😱 )

Root cause: GenericPoolManager.eagerPoolCreator calls MakeGenericPool to create a pool for any environment that doesn't have one. Since this call never succeeds, and returns failure without cleaning up, we just end up re-creating deployments over and over. (Those duplicated deployments don't create new pods though, since they're all labelled the same way.)

We should (a) clean up properly on MakeGenericPool failure, (b) back off in eagerPoolCreator on retries, and (c) have a test for this case.

Hello,

I've followed the tutorial and I was able to call fission route create --method GET --url /hello --function hello multiple times:

fission route list
NAME                                 METHOD URL    FUNCTION_NAME FUNCTION_UID
e6ae6ce9-4a4d-4975-acce-c1e76fee8578 GET    /hello hello
5c008b2d-9248-41a8-a43b-a8ca87b70a19 GET    /hello hello
55931415-aea4-4edd-976f-779eb7e34a7d GET    /hello hello
7e61f913-52d6-4d66-8dbb-70cfff3b1a23 GET    /hello hello
0f8550b1-5efd-429f-b1d6-f1922d555105 GET    /hello hello

I don't think it should be possible to specify the same tuple (method,route) multiple times, or am I missing something?

There are use cases where it's safe to reuse pods for multiple functions. In fact it's probably more common than cases where it's unsafe to do so.

This issue is for (a) figuring out the API/UX change, if any, and (b) the implementation for loading and routing to multiple functions per pod.

Poolmgr leaks deployments in the following cases:

1. An environment is deleted.
2. Poolmgr is restarted, either due to failure or due to upgrade

Poolmgr also leaks pods if it is restarted while there are unmanaged pods.

Since we run etcd already, we should use that to keep track of this state and clean up properly.

The python environment should set app.logger, so that the user's function can simply call app.log() to log. The logs should be set up to go to stdout, so that "kubectl logs" etc. can work.

See #123, #101.

Fission only support code pieces now, I think it's not good enough, because user may need to include the 3rd party libs needed to run the function, so ...

If you create a function:

fission fn create --name foo --env nodejs --code foo.js

then try to use the same name again:

fission fn create --name foo --env nodejs --code foo2.js

You get:

ERRO[0000] Failed to create function                     err="(Error 0) HTTP error 500" name=hello
Failed to create function: (Error 0) HTTP error 500

I think the controller API could be improved a bit to give a better status code and message in this case (and probably others). I'm happy to take a look at doing that if there is interest.