ReKoPub/pull-releases/003-web-convert

Convert articles on the website (v1.2.0)

What changed

• Converter form on rekopub.monoexp.com, under the summary: paste an article address, choose EPUB (the default), KEPUB or Both, and get download links. "Links work for 24 hours." is shown above them.
• rekopub-web (cmd/rekopub-web, internal/web): a small server on 127.0.0.1:8004, run as goapp@rekopub-web behind nginx. For each conversion it runs the rekopub command as a separate process, saves the books in a new folder with a random name and serves them from /books/<folder>/<file>. Books are deleted after 24 hours.
• Limits, all settable in /etc/goapps/rekopub-web.env:
  • 50 conversions in 15 minutes for everyone together. Over that, the form is disabled and says "Conversions are temporarily paused. Please try again in N minutes."
  • 10 per visitor in 15 minutes, so one visitor can't use up everyone's.
  • 2 conversions running at once, up to 10 waiting (a minute at most).
  • 2 GB of kept books; over that, conversions pause until old books are deleted.
  • nginx adds a per-address rate limit in front (6 conversion requests a minute, burst 5).
• Local addresses now need -LOCAL (CLI change): local and private addresses are refused by default. A local -URL without the flag stops with "-URL … is on this computer or a private network; add -LOCAL to convert it" (exit code 2), before anything is downloaded. Before this, rekopub -URL http://localtest.me:3000/ turned the server's own Gogs page into a book. The website never passes -LOCAL, and also checks the address itself before running anything. People who convert pages on their own network need to add -LOCAL from this version on.
• "Download / Use ReKoPub" section title above Download and Usage, which become its subsections. The example command moved from the summary into Usage, and the usage table lists -LOCAL.
• deploy/: the nginx site file and an example settings file.
• VERSION 1.2.0.

Notes

• Needs bajdev/devops PR "cli kind: also deploy a server" merged and installed first: it builds ./cmd/rekopub-web and a server rekopub into /opt/apps/rekopub-web/ and restarts goapp@rekopub-web before publishing the site.
• Server setup before the merge (see the PR description for who does what): /opt/apps/rekopub-web/data/, /etc/goapps/rekopub-web.env with PORT=8004, the new nginx site file.
• The form works without JavaScript too: the server then answers with a page of its own.
• Cross-site form posts are refused, so other sites can't use the converter through their visitors' browsers.

Testing

• go vet ./..., go test ./... and go test -race ./internal/web.
• New tests: -LOCAL (with the real pipeline: a localhost page is refused without it, before any request, and converted with it), and the server: form checks, local addresses, both limits, busy, storage limit, failed conversions, unexpected output paths, link expiry and path tricks on /books/, deleting old books, HTML answers (escaped), cross-site posts, visitor keys for IPv4/IPv6, and the real process runner with a stand-in script.
• Ran the whole stack locally: the built rekopub and rekopub-web behind nginx with this PR's server block (without TLS). Converted real pages to EPUB, KEPUB and both, downloaded them, and checked: local addresses refused (localtest.me, 127.0.0.1, [::1]), the per-visitor limit, nginx's 429, five conversions sent at once (two at a time, each in its own folder), and the page in its normal, finished and paused states.