ncarlier / webhookd Goto Github PK

Using the command sudo curl https://raw.githubusercontent.com/ncarlier/webhookd/master/install.sh | bash gives an error eventually:

Downloading https://github.com/ncarlier/webhookd/releases/download/v1.6.1/webhookd-linux-armv6l.tgz to /tmp/webhookd-linux-armv6l.tgz ...

curl: (22) The requested URL returned error: 404 Not Found
error: Unable to download binary for your architecture.

The page isn't there. Is this just something on my end?

Hi, thanks for this excellent tool!

It would be great if the "data : " prefix sent in response to GET requests could be toggleable or removed.

With this, this tool could be used to output Prometheus Metrics :)

Current behavior:

$ curl -XGET -s 'localhost:8080/metrics.sh'
data: some_custom_metric_generated_in_a_shell_script{hook_name="metrics"} 1

Behavior suggested by this issue

$ curl -XGET -s 'localhost:8080/metrics.sh'
some_custom_metric_generated_in_a_shell_script{hook_name="metrics"} 1

Code producing the current behavior:

I getting cors error when calling the API from remote FE client.

Is there any way to add support for the cross-origin request?

Would be cool to log X-Forwarded-For header in your logs instead of reverse proxy ip address. Thank you.

to make this tool better , that per my test for the docker container.

latest: Pulled from ncarlier/webhookd

1, put the "long.sh" which collected in issue#36 into scripts directory
"
#!/bin/bash

echo "Starting long script..."
for i in {1..20}; do
sleep 1
echo "running ${i} ..."
done
echo "Long script end"
exit 0
"

2,run the docker command,

docker run -d --name=webhookd
-v ${PWD}/scripts:/var/opt/webhookd/scripts
-p 63008:8080
ncarlier/webhookd
webhookd --scripts=/var/opt/webhookd/scripts

3, curl it at docker host
webhookd$ curl localhost:63008/long
data: Starting long script...
data: running 1 ...
data: running 2 ...
data: running 3 ...
data: running 4 ...
data: running 5 ...
data: running 6 ...
data: running 7 ...
data: running 8 ...
data: running 9 ...
data: error: signal: killed <<<< this should expect that killed by timeout default 10s.
4,login into container at docker host.
webhookd$ docker exec -it webhookd bash
bash-5.1# ps
PID USER TIME COMMAND
1 root 0:00 webhookd --scripts=/var/opt/webhookd/scripts
22 root 0:00 [sleep] <<<< this is zombie process.
23 root 0:00 bash
27 root 0:00 ps

I followed the instructions to add a .htpasswd file.
I created one with: htpasswd -cb .htpasswd hefty 1234
Testing locally with htpasswd -v proves that the file works fine.

When I restart the webhookd process and test from a browser, then the browser prompts for a user and password. But the correct password is not accepted.
Same behavior when I try it from the command line by curl -u hefty:1234 <url>: 401 unauthorized
I also tried curl -u hefty:1234 -XPOST <url>, but then it is waiting for a reply forever.

Did I misunderstand something?

Would be cool to override mailto:[email protected] parameter in WHD_NOTIFICATION_URI, when a script prefix also contains a mail address, for example:

echo "notify:[email protected]: Hello World"

or:

echo "notify:[email protected],[email protected],[email protected]: Hello World"

If there is no mail address after the prefix, default address will be used.

Same should be possible with HTTP post hook notifications:

echo "notify:http://requestb.in/another987: Hello World"

Thank you.

Would be cool to add a prefix to htpasswd file to use different users for different paths. For example:

/services1:api1:abcdef654321 # will be used for all scripts in /services1, eg. /services1/foo
/services2:api2:abcdef123456 # will be used for all scripts in /services2, eg. /services2/bar
/:api:default000000 # will be used for all other paths or scripts in the root path, eg. /other/foo or /bar

Thank you.

Hello,

Thank you for this awesome project !

I use my custom Dockerfile and I need a specific version of nodejs to run the nodejs scripts

I tried these methods in the Dockerfile without success:

Method 1 refuses to install nodejs (nodejs-current-17.9.0-r0: breaks: world[nodejs=14.19.0])

FROM ncarlier/webhookd:1.15.0
RUN apk add --update nodejs=14.19.0

Method 2 installs nvm but node is not available to run the scripts (env: can't execute 'node': No such file or directory)

FROM ncarlier/webhookd:1.15.0
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
RUN source $HOME/.nvm/nvm.sh && nvm install 14

I would love to use webhookd in silex v3 💯

Have a nice day

Hi there,

Forgive me if I am doing something wrong here. I have a simple script that looks like this:

#!/bin/bash
echo "$action" | tee /tmp/webhook.log

When I start webhookd and trigger the script with the queries in the URL look so curl http://domain:port/script?action=test it works perfectly, however when I try and pass those variables from a JSON structure like so curl -d '{"action":"test"}' -H "Content-Type: application/json" 'http://domain:port/script' it does not work at all, it appears that "action" is not being received as a variable by webhookd. Could I get some help on this please?

GET https://hook.PRIVATEDOMAIN.at/
{"name":"webhookd","version":""}

GET https://hook.PRIVATEDOMAIN.at/echo
hook not found

Same from within the container:

bash-5.0# curl localhost:8080
{"name":"webhookd","version":""}bash-5.0#
bash-5.0# curl localhost:8080/echo
hook not found
bash-5.0#

Check everything is in place:

bash-5.0# ls
echo.sh     examples    execute.sh
bash-5.0# cat echo.sh
#!/bin/bash

# Usage: http POST :8080/echo msg==hello foo=bar

echo "Echo script:"

echo "Command result: hostname=`hostname`"

echo "Header variable: User-Agent=$user_agent"

echo "Query parameter: msg=$msg"

echo "Body payload: $1"
bash-5.0# pwd
/var/opt/webhookd/scripts
bash-5.0#

LGTM

Kubernetes YAML:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/acme-challenge-type: dns01
    cert-manager.io/acme-dns01-provider: cloudflare
    cert-manager.io/cluster-issuer: letsencrypt-prod
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: 'true'
  name: webhook
  namespace: webhook
spec:
  rules:
  - host: hook.PRIVATEDOMAIN.at
    http:
      paths:
      - backend:
          serviceName: webhook
          servicePort: 8080
  tls:
  - hosts:
    - hook.PRIVATEDOMAIN.at
    secretName: tls-webhook
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: webhook
  name: webhook
  namespace: webhook
spec:
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: webhook
  sessionAffinity: None
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: webhook
  namespace: webhook
  labels:
    app: webhook
spec:
  selector:
    matchLabels:
      app: webhook
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: webhook
    spec:
      containers:
      - name: webhook
        image: berndinox/webook
        imagePullPolicy: Always

Dockerfile:

FROM ncarlier/webhookd
RUN apk update && apk upgrade && \
    apk add --no-cache git wget
COPY execute.sh /var/opt/webhookd/scripts

... not very complex.

Container-Logs:

INF 2020/07/25 11:37:14.085990 9f8085841545355c0a603799554e9eea GET /echo 10.0.0.210:43724 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36 63.099µs
ERR 2020/07/25 11:37:14.085942 Script not found: scripts/echo.sh

.. am i missing something?

EDIT
YEP.. fixed it:

        env:
        - name: WHD_SCRIPTS
          value: "/var/opt/webhookd/scripts"

Hi @ncarlier
Thank you for this gorgues utility.
In my case, can scripts called by webhookd print result strings in docker logs contexts?
e.g.
#!/bin/bash
echo 111
then I can see 111 in "docker logs webhookd"

Just curious, why POST request is used instead of GET request?

I'm asking this because currently javascript SSE implementation only support GET request.
Besides, some opinion about SSE is GET request only

I know I can still use a reverse proxy to convert the request, but just wish to know the intention using POST request.

Seems desc should be "Private key file" in /tooling/httpsig/main.go:
KeyFile string flag:"key-file" desc:"Public key file (PEM format)" default:"./key.pem"?

where to pass smtp user credentials for sending email notifications?

After many days running (14days last time) the service stop without error.
I can see the service get down, without error :

Mar 26 17:55:21 packages systemd[1]: Stopping WEBHOOKD...
Mar 26 17:55:22 packages systemd[1]: webhookd.service: Succeeded.
Mar 26 17:55:22 packages systemd[1]: Stopped WEBHOOKD.

Strange thing, the service should restart https://www.freedesktop.org/software/systemd/man/systemd.service.html#Restart= with Restart=always

Hi,

I created an ansible recipe installing the webhookd by downloading the tgz.
Having the "release" folder inside the tgz make the installation more complex (3 steps instead of 1).

Ok it is not very important but since it doesn't seems to add any feature I guess, it may be interesting to remove it?

This new version is so much better.

Thanks a lot, you roxxxx man

I'm trying to make this work through a reverse proxy (I'm using Docker), but I keep getting this error:

INF 2020/06/01 22:01:42.976802 server is ready to handle requests at :443

ERR 2020/06/01 22:01:42.977083 could not listen on :443 : open server.pem: no such file or directory

Anyone got an idea what to do?

Hello,

x_hub_signature got github has changed, and add format "sha1=%s"
The gihut sample script returns an error message

error: bad hook signature: expecting sha1=b5522ca84a491c0e7e457829356b6d89a836622b and got b5522ca84a491c0e7e457829356b6d89a836622b
error: exit status 1

Please change line in file scripts/examples/github.sh
signature=$(echo -n "$1" | openssl sha1 -hmac "$GITHUB_SECRET" | sed -e 's/^.* //')

• [ "$signature" != "$x_hub_signature" ] && die "bad hook signature: expecting $x_hub_signature and got $signature"

• [ "sha1=$signature" != "$x_hub_signature" ] && die "bad hook signature: expecting $x_hub_signature and got $signature"

According the documentation, we should use WHD_HOOK_TIMEOUT but in the usages of webhookd, it says: WHD_TIMEOUT.

And finally, it works only with WHD_TIMEOUT.

Btw, great project !!

Hi,
I try to use this project with github. It's really easy, simple and useful.
I want to add authentication but it's look like github don't use authentication but pass the secret into json message.
I suppose you don't manage it ?
I think I need a script to manage the secret ? (with jq for example)
Any suggestion ?

Best regards
Azlux

First of all, thanks for your great work, i really like it.

I'm using this script with hetrixtools to trigger on downtime, I'm planning to take some security measure to prevent anyone from sending unsolicited request. I found this on their wiki → https://docs.hetrixtools.com/webhook-authentication-token/. I wondering if there any shell variable for token in Authorization: Bearer <token>.

Thanks !

I got a setup where a webhook script executes docker-compose and uses a custom compose file inside the same directory. So my script looks like:

#!/bin/bash

DIR=$(dirname $0)
cd $DIR

docker-compose -f ./docker-compose-staging.yml ps
docker-compose -f ./docker-compose-staging.yml pull
docker-compose -f ./docker-compose-staging.yml up -d --force-recreate

With version 1.8.0 of webhookd, the script executes just fine. With any later version (e.g. v1.9.0), docker-compose always reports that it is unable to find the specified compose file. This is even the case when I declare the whole path to the compose file inside the container, by e.g. docker-compose -f /scripts/tmp/docker-compose-staging.yml ps.

Hi

This is strange

bash-4.4# webhookd --version
webhookd ()
Copyright (C) 2018 Nunux, Org.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Nicolas Carlier.

I think it would be much more convenient if we can add authentication like HTTP Authentication, query parameter,IP restriction etc.

Having all the output (including the webhook scripts) in the console/log would make debugging easier.

When calling HTTP endpoint with a specific script hierarchy, error is thrown in the data output:

❯ curl -XPOST http://localhost:8080/wfs/deploy                                                                                                                                                                          
data: error: open /tmp/wfs/deploy_1_20181105_0959.txt: no such file or directory 
❯ cat scripts/wfs/deploy.sh                                                                                                                                                                                          
#!/bin/bash
echo "Deploying..." 

Hi

Looking at the build/dockerfile - I have a number of Lua scripts that I’d like to execute from the bash/shell/command line.

Here’s an example lua_test.lua to run.

#!/usr/bin/lua

print ("Hello World!")
print ("*******************")
local saydatetime = os.date('%H, %M. On. %A %B %d :')
print(saydatetime)
print ("*******************")

What would I need to do to enable that capability ?

Bonjour,
J'apprécie beaucoup ce projet. Simple et efficace.

Je dispose de mon propre repo apt : https://packages.azlux.fr/
Il est possible pour moi d’intégrer tout type de projet.
Je vais ajouter celui-ci à ma liste, car je pense fortement qu'il en vaut le coup.
Un fois de .deb fait, tout est automatisé de mon coté. Ce qui permet de donner aux utilisateurs un moyen de d'install et de mise à jour automatisé avec APT.

Qu'en pensez-vous ?
Azlux

then I had a new problem , I use github to post a request , then i got “ Request
We couldn’t deliver this payload: Service Timeout” , I try to set the WHD_HOOK_TIMEOUT ，then it don't work , i still got the msg ... is the shell script execution time too long ? how can i fix it .....

First, thank you for this wonderful package, it's exactly what I need.

However, followed your readme file, setup webhookd as docker container.
It reply me Script not found: scripts/test.sh.

command used:
docker run --rm --name=webhookd -v ~/webhookd:/var/opt/webhookd/scripts -p 8080:8080 ncarlier/webhookd

This is the directory structure inside the container:

bash-4.4# pwd
/var/opt/webhookd/script
bash-4.4# ls -al
total 12
drwxr-xr-x    2 1000     1000          4096 Dec 31 07:52 .
drwxr-xr-x    5 root     root          4096 Dec 30 21:45 ..
-rwxrwxrwx    1 1000     1000            19 Dec 31 08:02 test.sh
bash-4.4#

Below is the content of my test.sh:

#!/bin/bash
ps aux

This is the result of the post request:

ERROR: 2018/12/31 08:10:21 index.go:61: Script not found: scripts/test.sh
INFO: 2018/12/31 08:10:21 logger.go:25: 1546243821109422472 POST /test 192.168.33.1:57640 PostmanRuntime/7.4.0 54.439µs

This tool is awesome!

Is there any "cicd" using case with this?
It would be perfect with web!

Not an issue, thanks for the great code!
But, would that be possible to pass a header from the executed script?
Something like:
echo header: "X-Auth-Token $token"

Hi
When I execute:
curl -v -XPOST http://localhost:8080/foo/bar
I receive this error:
ERR 2021/08/12 13:49:06.514247 Script not found: scripts/foo/bar

I'm using the example script from here, anyone got a solution to it? I think it's related to jq though.

parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 2, column 1
error: bad body format: expecting JSON
error: exit status 1

Firstly, thanks you for creating this handy container. It help a lot when I just want a webhook daemon.

However when I translate your example docker run script to docker-compose, I encounter the error message:
"/docker-entrypoint.sh: exec: line 20: webhookd --scripts=/var/opt/webhookd/scripts: not found"

My test docker compose file:

version: '3'
services:
    webhookd:
        container_name: webhookd
        image: ncarlier/webhookd
        volumes:
            - ${PWD}/scripts:/scripts
        command:
            - webhookd --scripts=/var/opt/webhookd/scripts

        ports:
            - '37564:8080'

I tried following command but is doesn't workd.
command: /usr/local/bin/webhookd --scripts=/var/opt/webhookd/scripts
ErrMsg is "/docker-entrypoint.sh: exec: line 20: /usr/local/bin/webhookd --scripts=/var/opt/webhookd/scripts: not found"

However, if I remove command and add a new entry point, it works.
entrypoint: /docker-entrypoint.sh webhookd --scripts=/var/opt/webhookd/scripts

I don't get the difference between webhookd's preset entrypoint /docker-entrypoint.sh with command and my workaround..
Need your insight for this question.

Thanks,
Louis

Environment ( if it matters )
Kubuntu 20.04

Webhookd --version: ( it seems version number is not displayed.. )
webhookd | Version:
webhookd | Git commit: 6b3623f
webhookd | Built: Sun Jul 11 11:17:07 UTC 2021

My script is:

#!/bin/bash

echo "Starting background job"
echo "Hook information: name=$hook_name, id=$hook_id, method=$hook_method"
echo "Query parameter: foo=$foo"
echo "Header parameter: user-agent=$user_agent"
echo "Script parameters: ${1}"


nohup ./scripts/long.sh >/tmp/long.log 2>&1  &

echo "Background job started."

I run the test as is outlined in the README.md

curl -d @test.json http://***.compute-1.amazonaws.com:8080/shoebox-test -u aaron:***** 

where test.json is

% cat test.json 
{"message": "this is a test"}

and my response shows the payload does not get passed through.

% curl -d @test.json http:/***.compute-1.amazonaws.com:8080/shoebox-test -u aaron:****** 
Starting background job
Hook information: name=shoebox-test, id=12, method=POST
Query parameter: foo=
Header parameter: user-agent=curl/7.79.1
Script parameters: 
Background job started.

Can you suggest how I can debug?

I just wanted to create my own image with additional apk packages from :1.15.1, but the problem is that you set in 1.15.1 ARG USER=webhookd and ARG UID=1000. Now apk install ... doesn't work.

1.15.0 works because there are no ARG USER and UID variables.

I was wondering if it would be possible for you to add a config variable to change the /static path to something different. For example, I'd like to change it to /docs as a more intuitive way to provide documentation on the endpoints I've made using webhookd.

Thanks for making such an awesome and easy to use tool!

Hi,

I tried the webhookd with a small script.

#! /usr/bin/env bash

echo "Script parameters: $1"
echo "FOo parameters: $foo"
echo "FOo parameters: $service"
echo "FOo parameters: $version"

This give me a random result 😢

Sometimes:

curl --data @test.json "http://localhost:8080/reload?service=saphir-traefik&version=1"
data: Script parameters: {"tata":"toto"}

data: FOo parameters: 

data: FOo parameters: saphir-traefik

data: FOo parameters: 1

data: done

But sometimes:

curl --data @test.json "http://localhost:8080/reload?service=saphir-traefik&version=1"
data: Script parameters: {"tata":"toto"}

data: FOo parameters: 

data: done

And I've got some error in the logs:

NFO: 2018/12/13 12:53:01 work_runner.go:36: Work reload#15 started...
ERROR: 2018/12/13 12:53:01 work_runner.go:82: Work reload#15 is over. Unable to write more data into the channel: FOo parameters: 1
ERROR: 2018/12/13 12:53:01 work_runner.go:95: Error while flushing the log file: /tmp/reload_15_20181213_1253.txt write /tmp/reload_15_20181213_1253.txt: file already closed
INFO: 2018/12/13 12:53:01 work_runner.go:110: Work reload#15 done [SUCCESS]

Any idea?

Hi,

Short style option seems broken (and incoherent with "usage" display):

This is failing:

/usr/local/bin/webhookd -l :8080 --scripts /etc/webhook -t 120 

while this OK

/usr/local/bin/webhookd --listen :8080 --scripts /etc/webhook --timeout 120 

Usage display:

Dec 19 17:56:07 jungle webhookd[4698]: Password file (encoded with htpasswd) (default ".htpasswd")
Dec 19 17:56:07 jungle webhookd[4698]: -scripts string
Dec 19 17:56:07 jungle webhookd[4698]: Scripts directory (default "scripts")
Dec 19 17:56:07 jungle webhookd[4698]: -timeout int
Dec 19 17:56:07 jungle webhookd[4698]: Hook maximum delay before timeout (in second) (default 10)
Dec 19 17:56:07 jungle webhookd[4698]: -version
Dec 19 17:56:07 jungle webhookd[4698]: Print version

No shorten options displayed, one "-" or two?

Hey, I'm using image from hub.docker.com and I encountered a problem with "Script not found" message when I try to do your example foo/bar.
Regardless of specified value of APP_SCRIPTS_DIR in docker-compose file still i get message "Script not found".

Hi,

When triggering webhook using URL, I'm getting this error:

λ curl localhost:8080/path/to/directory/test
data: error: fork/exec /scripts/path/to/directory/test.sh: no such file or directory

Webhookd is launched using Docker:

webhook:
    image: ncarlier/webhookd:v1.6.0
    command: webhookd -d
    ports:
      - 8080:8080
    environment:
      - "APP_SCRIPTS_DIR=/scripts"
    volumes:
      - ./webhooks/scripts:/scripts

The script is present in the /scripts/path/to/directory/:

λ docker exec -it af5c9a0fb3ec ls -la /scripts/path/to/directory/
total 1
drwxrwxrwx    2 root     root             0 Jan 28 16:01 .
drwxrwxrwx    2 root     root             0 Jan 28 16:01 ..
-rwxr-xr-x    1 root     root            89 Jan 28 15:29 test.sh

λ docker exec -it af5c9a0fb3ec cat /scripts/path/to/directory/test.sh
#!/bin/bash
echo "test"

Container logs below:

DEBUG: 2020/01/28 16:07:44 dispatcher.go:33: Dispatching work request: path/to/directory/test#7
DEBUG: 2020/01/28 16:07:44 worker.go:46: Worker #1 received work request: path/to/directory/test#7
INFO: 2020/01/28 16:07:44 work_runner.go:28: Work path/to/directory/test#7 started...
DEBUG: 2020/01/28 16:07:44 work_runner.go:29: Work path/to/directory/test#7 script: /scripts/path/to/directory/test.sh
DEBUG: 2020/01/28 16:07:44 work_runner.go:30: Work path/to/directory/test#7 parameter: [user_agent=curl/7.55.1 accept=*/*]
DEBUG: 2020/01/28 16:07:44 work_runner.go:50: Work path/to/directory/test#7 output to file: /tmp/path_to_directory_test_7_20200128_1607.txt
INFO: 2020/01/28 16:07:44 work_request.go:73: Work path/to/directory/test#7 done [ERROR]
INFO: 2020/01/28 16:07:44 logger.go:25: 1580227664298761800 GET /path/to/directory/test 172.26.0.1:37280 curl/7.55.1 19.0572ms

Any ideas?
Thanks.

I can't find the scripts folder .... could you tell me where is it ?
i install webhooks use APT.

Many thanks for the great image and software!

It would be extremely handy for me to not only use docker but also docker-compose in scripts executed by the service. I've spotted an appropriate package for apk - https://pkgs.alpinelinux.org/package/edge/testing/x86/docker-compose - providing docker-compose. Would be great if docker-compose could be added to this image!

EDIT: alternatively use the instructions from the alpine wiki to install pip and docker-compose - https://wiki.alpinelinux.org/wiki/Docker

Hi,

Absolutely love this project for it's simplicity.

It would be useful to expose the Work ID (used for identifying which log to access) back out to the worker?
This would be useful for providing the URL for accessing the logs as part of the post-run notification for debugging etc.

data: Hook work request "test" queued...

data: Run sample test

curl: (18) transfer closed with outstanding read data remaining

script_runner.go:98: scripts/test.sh executed wit SUCCESS

I have set APP_HOOK_TIMEOUT="300" and it is getting executed successfully, but curl response is failing.

Hi @ncarlier,

thanks for this nice piece of software.

I want to execute a bash script as a background job, so that the HTTP request exits immediatly. I wasn't able to do this with

nohup ./do-stuff.sh >/dev/null 2>&1 &

Is there a way to solve this?

I ended up testing and reverse engineering the Makefile to find the correct settings. A simple run command, or an example docker-compose.yml file would help newcomers get started faster.