🔄 gosmee - A webhook forwarder/relayer and replayer

✨ Gosmee is a powerful webhook relayer that runs anywhere with ease! 📡 It also serves as a GitHub Hooks replayer using the GitHub API.

📝 Description

🌉 Gosmee enables you to relay webhooks from itself (as a server) or from https://smee.io to your local laptop or infrastructure hidden from the public internet.

🔌 It makes exposing services on your local network (like localhost) or behind a VPN super simple! This allows public services, such as GitHub, to push webhooks directly to your local environment.

Here's how it works:

1. 🎯 Configure your Webhook to send events to a https://smee.io/ URL or to your publicly accessible Gosmee server.
2. 🚀 Run the Gosmee client on your local machine to fetch these events and forward them to your local service.

This creates a seamless bridge between GitHub webhooks and your local development environment!

🔄 Alternatively, if you prefer not to use a relay server, you can use the GitHub API to replay webhook deliveries directly.

📊 Diagram

For those who prefer a visual explanation of how gosmee works:

Simple

Detailed

sequenceDiagram
    participant SP as Service Provider (e.g., GitHub)
    participant GS as Gosmee Server (Public URL / smee.io)
    participant GC as Gosmee Client (Local / Private Network)
    participant LS as Local Service (e.g., localhost:3000)

    Note over GC, LS: Runs in private network/local machine
    Note over SP, GS: Accessible on the public internet

    GC->>+GS: 1. Connect & Listen via SSE
    SP->>+GS: 2. Event triggers -> Sends Webhook Payload (HTTP POST)
    GS->>-GC: 3. Relays Webhook Payload (via SSE connection)
    GC->>+LS: 4. Forwards Webhook Payload (HTTP POST)
    LS-->>-GC: 5. (Optional) HTTP Response
    GS-->>-SP: 6. (Optional) HTTP Response (e.g., 200 OK)

Loading

📰 Blog Post

✍️ Learn more about the background and features of this project in this blog post: https://blog.chmouel.com/posts/gosmee-webhook-forwarder-relayer

🖼️ Screenshot

📥 Install

📦 Release

🔽 Go to the release page and choose the appropriate archive or package for your platform.

🍺 Homebrew

brew tap chmouel/gosmee https://github.com/chmouel/gosmee
brew install gosmee

🏹 Arch

yay -S gosmee-bin

🐳 Docker

Gosmee client with Docker

docker run ghcr.io/chmouel/gosmee:latest

Gosmee server with Docker

docker run -d -p 3026:3026 --restart always --name example.org ghcr.io/chmouel/gosmee:latest server --port 3026 --address 0.0.0.0 --public-url https://example.org

🔧 GO

go install -v github.com/chmouel/gosmee@latest

📂 Git

Clone the repository and use:

-$ make build
-$ ./bin/gosmee --help

❄️ Nix/NixOS

Gosmee is available from nixpkgs.

nix-env -iA gosmee
nix run nixpkgs#gosmee -- --help # your args are here

⚙️ System Services

System Service example files for macOS and Linux are available in the misc directory.

☸️ Kubernetes

You can expose an internal kubernetes deployment or service with gosmee by using this file.

Adjust the SMEE_URL to your endpoint and the http://deployment.name.namespace.name:PORT_OF_SERVICE URL is the Kubernetes internal URL of your deployment running on your cluster, for example:

http://service.namespace:8080

🔤 Shell completion

Shell completions are available for gosmee:

# BASH
source <(gosmee completion bash)

# ZSH
source <(gosmee completion zsh)

🚀 Usage

💻 Client

If you plan to use https://smee.io, you can generate your own smee URL by visiting https://smee.io/new.

Once you have it, the basic usage is:

gosmee client https://smee.io/aBcDeF https://localhost:8080

This command will relay all payloads received at the smee URL to a service running on http://localhost:8080.

✨ You can also save all relays as shell scripts for easy replay:

gosmee client --saveDir /tmp/savedreplay https://smee.io/aBcDeF https://localhost:8080

This command saves the JSON data of new payloads to /tmp/savedreplay/timestamp.json and creates shell scripts with cURL options at /tmp/savedreplay/timestamp.sh. Replay webhooks easily by running these scripts!

🙈 You can ignore certain events (identified by GitLab/GitHub/Bitbucket) with one or more --ignore-event flags.

If you only want to save payloads without replaying them, use --noReplay.

🎨 By default, you'll get colorful emoji output unless you specify --nocolor.

📊 Output logs as JSON with --output json (which implies --nocolor).

🖥️ Server

With gosmee server you can run your own relay server instead of using https://smee.io.

By default, gosmee server binds to localhost on port 3333. For practical use, you'll want to expose it to your public IP or behind a proxy using the --address and --port flags.

🔒 For security, you can use Let's Encrypt certificates with the --tls-cert and --tls-key flags.

There are many customization options available - check them with gosmee server --help.

To use your server, access it with a URL format like:

https://myserverurl/RANDOM_ID

The random ID must be 12 characters long with characters from a-zA-Z0-9_-.

🆕 Generate a random ID easily with the /new endpoint:

% curl http://localhost:3333/new
http://localhost:3333/NqybHcEi

🟢 Caddy

Caddy is the ideal way to run gosmee server:

https://webhook.mydomain {
    reverse_proxy http://127.0.0.1:3333
}

It automatically configures Let's Encrypt certificates for you!

🔵 Nginx

Running gosmee server behind nginx requires some configuration:

    location / {
        proxy_pass         http://127.0.0.1:3333;
        proxy_set_header Connection '';
        proxy_http_version 1.1;
        chunked_transfer_encoding off;
        proxy_read_timeout 372h;
    }

⚠️ Long-running connections may occasionally cause errors with nginx. Contributions to debug this are welcome!

🔁 Replay webhook deliveries via the GitHub API (beta)

🔄 If you prefer not to use a relay server with GitHub, you can replay webhook deliveries directly via the GitHub API.

This method is more reliable as you don't depend on relay server availability. You'll need a GitHub token with appropriate scopes:

• For repository webhooks: read:repo_hook or repo scope
• For organization webhooks: admin:org_hook scope

Currently supports replaying webhooks from Repositories and Organizations (GitHub Apps webhooks not supported).

First, find the Hook ID:

gosmee replay --github-token=$GITHUB_TOKEN --list-hooks org/repo

List hooks for an organization:

gosmee replay --github-token=$GITHUB_TOKEN --list-hooks org

Start listening and replaying events on a local server:

gosmee replay --github-token=$GITHUB_TOKEN org/repo HOOK_ID http://localhost:8080

This will listen to all new events and replay them to http://localhost:8080.

⏱️ Replay all events received since a specific time (UTC format 2023-12-19T12:31:12):

gosmee replay --time-since=2023-12-19T09:00:00 --github-token=$GITHUB_TOKEN org/repo HOOK_ID http://localhost:8080

To find the right date, list all deliveries:

gosmee replay --github-token=$GITHUB_TOKEN --list-deliveries org/repo HOOK_ID

Note

gosmee replay doesn't support paging yet and lists only the last 100 deliveries. Specifying a date older than the last 100 deliveries won't work.

When rate limited, gosmee will fail without recovery mechanisms.

🔄 Beyond Webhook

Gosmee is webhook-specific. For other tunneling solutions, check https://github.com/anderspitman/awesome-tunneling. Recommended alternatives include go-http-tunnel or tailscale.

⚠️ Caveats ⚠️

⚠️ This tool is intended for local development and testing environments only! It hasn't undergone thorough security and performance reviews and should not be deployed in production systems.

🙏 Thanks

• Most of the work is powered by the go-sse library.
• Previously used pysmee but its underlying SSE library had issues with chunked transfers.

📜 Copyright

Apache-2.0

👥 Authors

👨‍💻 Chmouel Boudjnah

• 🐘 Fediverse - <@[email protected]>
• 🐦 Twitter - <@chmouel>
• 📝 Blog - <https://blog.chmouel.com>