What is the simplest webhook-wrangling server-daemon-thing.

go-webhookd

What is the simplest webhook-wrangling server-daemon-thing.

In many ways this is nothing more than a fancy bucket-brigade. By design.

Receivers handle the actual webhook side of things, doing auth and basic sanity checking and validation. Assuming everything is as it should be receivers return a bag of bytes (the actual webhook message that may or may not be massaged depending the receiver). That bag is then handed to one or more dispatchers which do something with those bytes. Those details, including security considerations are left as an exercise to the reader.

In between (receivers and dispatchers) are an optional chain of transformations which accept bytes as their input, do something with those bytes, and then return bytes.

Install

You will need to have both Go and the make programs installed on your computer. Assuming you do just type:

make bin

All of this package's dependencies are bundled with the code in the vendor directory.

Usage

webhookd

./bin/webhookd -hUsage of ./bin/webhookd: -config string Path to a valid webhookd config file

webhookd is an HTTP daemon for handling webhook requests. Individual webhook endpoints (and how they are processed) are defined in a config file that is read at start-up time.

Caveats

TLS

TLS is not supported yet so you should not run webhookd on the public internets without first putting a TLS-enabled proxy in front of it.

Dynamic endpoints

At some point there might be dynamic (or runtime) webhook endpoints but today there are not.

In the meantime you can gracefully restart webhookd by sending its PID a USR2 signal which will cause the config file (and all the endpoints it defines) to be re-read. It's not elegant but it works. For example:

```$> ./bin/webhookd -config config.json2016/10/16 00:19:47 Serving 127.0.0.1:8080 with pid 2723

$> kill -USR2 27232016/10/16 00:19:59 Graceful handoff of 127.0.0.1:8080 with new pid 2724 and old pid 27232016/10/16 00:19:59 Exiting pid 2723.```

Setting up a webhookd server

While you can set up a webhookd server by hand it's probably easier to all that work with a config file and let code take care of all the details, including registering all the webhooks. Config files are discussed in detail below.

All error handling in the examples below have been removed for the sake of brevity.

Setting up a webhookd server with a handy config file

```import ( "github.com/whosonfirst/go-webhookd/config" "github.com/whosonfirst/go-webhookd/daemon")

wh_config, _ := config.NewConfigFromFile("config.json")

whdaemon, _ := daemon.NewWebhookDaemonFromConfig(whconfig)wh_daemon.Start()```

You can also just grab the HTTP handler func with wh_daemon.HandlerFunc() if you need or want to start a webhookd daemon in your own way.

Setting up a webhookd server "by hand"

```import ( "github.com/whosonfirst/go-webhookd"
"github.com/whosonfirst/go-webhookd/daemon" "github.com/whosonfirst/go-webhookd/dispatchers" "github.com/whosonfirst/go-webhookd/receivers" "github.com/whosonfirst/go-webhookd/transformations" "github.com/whosonfirst/go-webhookd/webhook"
)

wh_receiver, _ := receivers.NewInsecureReceiver()

null, _ := transfromations.NewNullTransformation()wh_transformations := []webhookd.WebhookTransformation{ null }

pubsub, _ := dispatchers.NewPubSubDispatcher("localhost", 6379, "websocketd")wh_dispatchers, _ := []webhookd.WebhookDispatcher{ pubsub }

wh, _ := webhook.NewWebhook("/foo", whreceiver, whtransformations, wh_dispatchers)

whdaemon, _ := daemon.NewWebhookDaemon("localhost", 8080)whdaemon.AddWebhook(wh)wh_daemon.Start()```

See the way we're using an Insecure receiver and a PubSub dispatcher with a Null transformation? All are these are discussed in detail below.

Sending stuff to webhookd

```curl -v -X POST http://localhost:8080/foo -data-binary @README.md

• upload completely sent off: 703 out of 703 bytes< HTTP/1.1 200 OK< Date: Fri, 20 May 2016 01:31:05 GMT< Content-Length: 0< Content-Type: text/plain; charset=utf-8<
• Connection #0 to host localhost left intact```

Where did it go...

./bin/subscribe webhookd{'pattern': None, 'type': 'subscribe', 'channel': 'webhookd', 'data': 1L}{'pattern': None, 'type': 'message', 'channel': 'webhookd', 'data': '# go-webhookd## ImportantYou should not try to use this, yet. No. No, really.## UsageIt _should_ work something like this. If you\'re reading this sentence that means it _doesn\'t_.import (\t"github.com/whosonfirst/go-webhookd"\t"github.com/whosonfirst/go-webhookd/dispatchers"\t"github.com/whosonfirst/go-webhookd/receivers")dispatcher := dispatchers.NewPubSubDispatcher("localhost", 6379, "pubsub-channel")receiver := receivers.NewGitHubReceiver("github-webhook-s33kret")endpoint := "/wubwubwub"webhook := webhookd.NewWebhook(endpoint, receiver, dispatcher)daemon := webhookd.NewWebHookDaemon(webhook)daemon.AddWebhook(webhook)daemon.Start()## See also'}

In this case, it went to Redis PubSub land! Where things go depend on how you've configured your dispatchers.

Config files

Config files for webhookd are JSON files consisting of five top-level sections. An example config file is included with this repository. The five top-level sections are:

daemon

"daemon": { "host": "localhost", "port": 8080 }

The daemon section is a dictionary defining configuration details for the webhookd daemon itself.

receivers

"receivers": { "insecure": { "name": "Insecure" }, "github": { "name": "GitHub", "secret": "s33kret" } }

The receivers section is a dictionary of "named" receiver configuations. This allows the actual webhook configurations (described below) to signal their respective receivers using the dictionary "name" as a simple short-hand.

transformations

"transformations": { "chicken": { "name": "Chicken", "language": "zxx", "clucking": false } }

The transformations section is a dictionary of "named" tranformation configuations. This allows the actual webhook configurations (described below) to signal their respective transformations using the dictionary "name" as a simple short-hand.

dispatchers

"dispatchers": { "pubsub": { "name": "PubSub", "host": "localhost", "port": 6379, "channel": "webhookd" } }

The dispatchers section is a dictionary of "named" dispatcher configuations. This allows the actual webhook configurations (described below) to signal their respective dispatchers using the dictionary "name" as a simple short-hand.

webhooks

"webhooks": [ { "endpoint": "/github-test", "receiver": "github", "dispatchers": [ "pubsub" ] }, { "endpoint": "/insecure-test", "receiver": "insecure", "dispatchers": [ "pubsub" ] }, { "endpoint": "/slack-test", "receiver": "slack", "transformations": [ "slack", "chicken" ], "dispatchers": [ "slack", "log"] } ]

The webhooks section is a list of dictionaries. These are the actual webhook endpoints that clients (out there on the internet) will access.

• endpoint This is the path that a client will access. It is the webhook URI that clients will send requests to.
• receiver The named receiver (defined in the receivers section) that the webhook will use to process requests.
• transformations An optional list of named transformations (defined in the transformations section) that the webhook process the message body with.
• dispatchers The list of named dispatchers (defined in the dispatchers section) that the webhook will relay a successful request to.

Receivers

GitHub

{ "name": "GitHub", "secret": "s33kret" }

This receiver handles Webhooks sent from GitHub. It validates that the message sent is actually from GitHub (by way of the X-Hub-Signature header) but performs no other processing.

Properties

• name string This is always GitHub.
• secret string The secret used to generate the HMAC hex digest of the message payload.
• ref string An optional Git ref to filter by. If present and a WebHook is sent with a different ref then the daemon will return a 666 error response.

Insecure

{ "name": "Insecure" }

As the name suggests this receiver is completely insecure. It will happily accept anything you send to it and relay it on to the dispatcher defined for that webhook.

This receiver exists primarily for debugging purposes and you should not deploy it in production.

Properties

• name string This is always Insecure.

Slack

{ "name": "Slack" }

This receiver handles Webhooks sent from Slack. It does not process the message at all. This receiver has not been fully tested yet so proceed with caution.

Properties

• name string This is always Slack.

Transformations

Chicken

{ "name": "Chicken", "language": "zxx", "clucking": false }

The Chicken transformation will convert every word in your message to 🐔 using the go-chicken package. If this seems silly that's because it is. It's also more fun that yet-another boring "make all the words upper-cased" example.

Properties

• name string This is always Chicken.
• language string A three-letter language code specifying which language go-chicken should use.
• clucking bool A boolean flag indicating whether or not to cluck when generating results.

GitHubCommits

{ "name": "GitHubCommits", "exclude_additions":false, "exclude_modification":false, "exclude_deletions":true, }

The GitHubCommits transformation will extract all the commits (added, modified, removed) from a push event and return a JSON encoded list of paths.

Properties

• name string This is always GitHubCommits.
• excludeadditions _bool A flag to indicate that new additions in a commit should be ignored. Optional; default false.
• excludemodification _bool A flag to indicate that modifications in a commit should be ignored. Optional; default false.
• excludedeletions _bool A flag to indicate that deletions in a commit should be ignored. Optional; default false.

Null

{ "name": "Null" }

The Null transformation will not do anything. It's not clear why you would ever use this outside of debugging but that's your business.

Properties

• name string This is always Null.

SlackText

{ "name": "SlackText" }

The SlackText transformation will extract and return the text property from a Webhook sent by Slack.

Properties

• name string This is always SlackText.

Dispatchers

Log

{ "name": "Log" }

The Log dispatcher will send messages to Go's logging facility. As of this writing that means everything is logged to STDOUT but eventually it will be more sophisticated.

Properties

• name string This is always Null.

Null

{ "name": "Null" }

The Null dispatcher will send messages in to the vortex, never to be seen again. This can be useful for debugging.

Properties

• name string This is always Null.

PubSub

{ "name": "PubSub", "host": "localhost", "port": 6379, "channel": "webhookd" }

The PubSub dispatcher will send messages to a Redis PubSub channel.

Properties

• name string This is always PubSub.
• host string The address of the Redis host you want to connect to.
• port int The port number of the Redis host you want to connect to.
• channel string The name of the Redis PubSub channel you want to send messages to.

Slack

{ "name": "Slack", "config": "/path/to/.slackcat.conf" }

The Slack dispatcher will send messages to a Slack channel using the slackcat package.

Properties

• name string This is always Slack.
• config string The path to a valid slackcat config file. Eventually you will be able to specify a plain-vanilla Slack Webhook URL but not today.

To do

• Add a general purpose "shared-secret/signed-message" receiver
• Add support for TLS
• Restrict access to receivers by host/IP
• Better logging

See also

• https://en.wikipedia.org/wiki/Webhook

Related

• https://github.com/whosonfirst/go-pubssed

To restore the repository download the bundle

wget https://archive.org/download/github.com-whosonfirst-go-webhookd_-_2017-12-28_19-02-19/whosonfirst-go-webhookd_-_2017-12-28_19-02-19.bundle

and run:

 git clone whosonfirst-go-webhookd_-_2017-12-28_19-02-19.bundle 

Source: https://github.com/whosonfirst/go-webhookd
Uploader: whosonfirst
Upload date: 2017-12-28

plus-circle Add Review

comment

Reviews