# devops

Apps and scripts built as part of the DevOps pipeline for the monoaf.com
server.

The main piece is the **deploy pipeline**: when a pull request is merged into
`main` in Gogs, the exact merge commit is built, deployed, tagged
`deploy/<target>/<UTC time>` and logged.

```
 PR merged in Gogs
   │  webhook (HMAC-signed), 127.0.0.1:9000/deploy
   ▼
 deployd  ── checks signature, merged into main ──▶  release <repo> <sha>
                                                     │
              clean checkout of that commit ◀────────┤
              go build  /  rsync static files ◀──────┤
              rsync content/ (if configured) ◀───────┤
              sudo systemctl restart goapp@<name> ◀──┤
              tag deploy/<target>/<time>, push ◀─────┤
              line in /var/log/deploys.log ◀─────────┘
```

## Layout

| Path | Installed as | What it is |
|---|---|---|
| `deployd/` | `/usr/local/bin/deployd` | Webhook receiver (Go, standard library only) |
| `release/release` | `/usr/local/bin/release` | Builds, deploys, tags and logs one commit |
| `release/test.sh` | not installed | Runs `release` end to end in a sandbox |
| `sites.json` | `/etc/deployd/sites.json` | Which repo deploys to which site |
| `systemd/deployd.service` | `/etc/systemd/system/` | Runs `deployd` as the `deploy` user |
| `sudoers/60-deploy` | `/etc/sudoers.d/60-deploy` | Lets `deploy` restart `goapp@<name>` only |
| `certs/issue-cert` | `/usr/local/sbin/issue-cert` | Certificate wrapper (Phase 10 of the server plan) |

Everything is installed by the admin, owned by root. The `claude` user can
propose changes here through pull requests but can't install anything.

## sites.json

```json
{
  "bajdev/posts": { "kind": "go", "name": "posts", "content": "content" }
}
```

| Key | Kinds | Meaning |
|---|---|---|
| `kind` | all | `go` (a `goapp@<name>` app) or `static` (files served by nginx) |
| `name` | go | App name: binary at `/opt/apps/<name>/<name>`, unit `goapp@<name>` |
| `content` | go | Optional folder in the repo synced to `/opt/apps/<name>/content/` |
| `host` | static | Site under `/srv/www/vhosts/<host>/` |
| `dir` | static | Folder in the repo to publish (default `public`) |

Repos not listed are refused.

## Install (once, as the admin)

Steps 1–4 match Phase 7 of the server plan.

**1. The `deploy` user**, its log, and its sudo rule:

```bash
sudo useradd -r -m -d /var/lib/deploy -s /bin/bash -G webops deploy
sudo install -m 664 -o deploy -g webops /dev/null /var/log/deploys.log
sudo -iu deploy sh -c 'mkdir -p -m 700 ~/.ssh && ssh-keygen -t ed25519 -N "" -f ~/.ssh/id_ed25519 -C deploy@vps'
sudo -iu deploy git config --global user.name Deploy
sudo -iu deploy git config --global user.email deploy@monoaf.com
sudo cat /var/lib/deploy/.ssh/id_ed25519.pub        # for the Gogs deploy account
```

In Gogs, create a `deploy` user, add that key to it, and add it as a **Write**
collaborator on each repo it releases. It needs Write to push tags; branch
protection still stops it pushing to `main`. Then trust Gogs' host key:

```bash
sudo -iu deploy ssh -o StrictHostKeyChecking=accept-new git@git.monoaf.com true
```

**2. Build and install** from a checkout of this repo:

```bash
git clone https://git.monoaf.com/bajdev/devops.git ~/devops && cd ~/devops
(cd deployd && go test ./... && CGO_ENABLED=0 go build -trimpath -o deployd .)
sudo install -o root -g root -m 755 deployd/deployd /usr/local/bin/deployd
sudo install -o root -g root -m 755 release/release /usr/local/bin/release
sudo install -d -o root -g root -m 755 /etc/deployd
sudo install -o root -g root -m 644 sites.json /etc/deployd/sites.json
sudo install -o root -g root -m 644 systemd/deployd.service /etc/systemd/system/deployd.service
# sudo rule: install under a name sudo ignores, check it, then move it into place
sudo install -o root -g root -m 440 sudoers/60-deploy /etc/sudoers.d/60-deploy.new
sudo visudo -cf /etc/sudoers.d/60-deploy.new && sudo mv /etc/sudoers.d/60-deploy.new /etc/sudoers.d/60-deploy
```

**3. Secret and service:**

```bash
printf 'WEBHOOK_SECRET=%s\n' "$(openssl rand -hex 32)" | sudo tee /etc/deployd/secret.env >/dev/null
sudo chmod 600 /etc/deployd/secret.env
sudo systemctl daemon-reload && sudo systemctl enable --now deployd
sudo cat /etc/deployd/secret.env        # the secret for the Gogs webhook
```

**4. Let Gogs call it.** Since 0.14.3 Gogs refuses webhooks to local
addresses unless allowed. In `/home/git/gogs/custom/conf/app.ini`, then
`sudo systemctl restart gogs`:

```ini
[security]
LOCAL_NETWORK_ALLOWLIST = 127.0.0.1
```

## Moving a site onto the pipeline

1. Add it to `sites.json` (through a PR here) and reinstall that file.
2. Hand its deployed files to `deploy`, so `release` can update them. For a
   Go app:

   ```bash
   sudo chown -R deploy /opt/apps/<name>/<name> /opt/apps/<name>/content
   ```

   Leave `data/` alone. After this, deploy the site only through the
   pipeline; `deploy-go` is for a site's first deploy.
3. In the repo's Gogs settings: **Webhooks → Add Webhook → Gogs**. Payload URL
   `http://127.0.0.1:9000/deploy`, content type `application/json`, secret
   from `/etc/deployd/secret.env`, events "Let me choose what I need" → only
   **Pull Request**.
4. Merge a test PR and check the result (below).

## Checking it works

- A merged PR produces a `deploy/<target>/...` tag in Gogs and a line in
  `/var/log/deploys.log`.
- A PR closed without merging deploys nothing.
- `journalctl -u deployd` shows each release's output.
- In Gogs, the webhook's **Recent Deliveries** shows each request and
  deployd's reply: `202` started a release, `204` was ignored, `401` was a
  bad signature.

## Rollback

Redeploy any earlier release by its tag, as the admin:

```bash
sudo -iu deploy env DEPLOY_BY="rollback by bajdev" release bajdev/posts deploy/posts/20261026T120000Z
```

That checks out the tag, redeploys it, and records the rollback as a new tag
and log line.

## Development

```bash
(cd deployd && go vet ./... && go test -race ./...)
release/test.sh ~/src/posts      # sandboxed end-to-end run of release
```

`release/test.sh` uses a local bare git remote and stand-in `sudo` and
`systemctl`, so nothing on the server is touched. It covers a Go release
with content sync, a rollback, a static site, refused input, and a service
that fails to start (no tag, no log line).