Hardened VPS, Set Up By Your AI Agent

A copy-paste guide to building a secure Ubuntu 24.04 server where an AI agent (Hermes Agent) does ~80% of the work — driven from a desktop app you install on your own machine first.

The idea

Most VPS guides make you hand-type dozens of commands. This one flips it: you install the agent in minute five, hand it a single detailed build prompt, and it executes and verifies each phase. Your manual part is provisioning, one login URL, and approving the sensitive steps.

Time budget

Step	Who does it	Time
Provision VPS	You	10 min
Install desktop app locally	You	5 min
Install Hermes Agent on the VPS	You	5 min
Hardening, firewall, Tailscale, Docker, Caddy	Agent	15 min (unattended)
Tailscale login click-through	You	2 min
Remote desktop (optional)	Agent	5 min
Messaging gateway (optional)	You	10 min

Phase 1 — Provision the server You

Create a Hetzner Cloud account: console.hetzner.cloud
New server, Ubuntu 24.04 LTS. CPX11 (2 vCPU / 4 GB) is plenty to start; go CPX21 if you plan to run containers with real load.
Add your SSH public key at creation time — Hetzner SSH key docs. Password-based SSH gets brute-forced within minutes of a box going live, so keys-only from the start.
Note the IPv4 address. Log in: ssh root@YOUR_SERVER_IP

Any provider works

The rest of this guide is provider-agnostic. Hetzner is used because it's cheap and well-supported, but the same steps apply to DigitalOcean, Linode, Vultr, AWS Lightsail, etc. Ubuntu 24.04 LTS is the assumed OS throughout.

Phase 2 — Install the Hermes desktop app You

Do this on your own computer, before touching the VPS further. It's the friendliest way to use Hermes — especially if this is your first time with an AI agent — and it becomes your control panel for the entire rest of the build.

hermes desktop    # after installing Hermes locally (see install.sh below)

If you haven't installed Hermes on your local machine yet, run the same installer you'd use on the server:

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup      # pick your LLM provider + paste its API key
hermes desktop    # launches the native app (alias: hermes gui)

Native app for macOS, Linux, and Windows. What you get:

Streaming chat with the agent, drag-and-drop files, native notifications
Session browser — review past sessions, resume where you left off
⌘K command palette for quick actions
Extendable UI — custom panes/widgets via plugins, themable skins

Why this order

Your first Hermes experience shouldn't be inside a tmux session over SSH. The desktop app is a client front-end for one or more Hermes agents — including agents running on remote machines. Install it locally first, get comfortable, then register your VPS as a connection and drive the whole server build from it.

One app, many agents

The desktop app manages multiple agent connections side by side — a local one (this machine) and any number of remote ones (your VPS, LAN boxes, etc.), switchable from the Sessions sidebar. They don't share memory, sessions, or skills — that's intentional isolation, so the agent doing work on your laptop never touches your server's state and vice versa. For this guide you'll be talking to the VPS agent from Phase 3 onward.

Phase 3 — Install the agent on the VPS You

This is the last server-side manual step. SSH in and install Hermes Agent — an open-source terminal AI agent that executes shell commands, edits files, and verifies its own work:

ssh root@YOUR_SERVER_IP
apt update && apt full-upgrade -y
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup     # interactive wizard: pick an LLM provider, paste its API key
hermes doctor    # health check — everything should be green

Docs: hermes-agent.nousresearch.com/docs

Run it inside tmux

Long agent runs must survive SSH disconnects: tmux new -s build, run hermes, detach with Ctrl-B D, reattach later with tmux attach -t build.

Connect your desktop app to it

Once the VPS agent is running, register it in the desktop app you installed in Phase 2: Settings → Gateways → Add connection. The SSH kind is easiest — enter user@host:22 and the app opens the tunnel and starts the remote dashboard for you (no extra server setup). Alternatively, Remote gateway connects over HTTP(S) via LAN, Tailscale, or the internet with a session token. Click Test and wait for "Reachable". From here on, every phase below is driven from your desktop app — you never need to SSH back in except for emergencies. Docs: desktop connections guide.

Phase 4 — Hand the agent the build sheet Agent

Paste this entire prompt into Hermes and let it work. It is deliberately explicit about ordering and safety gates — keep those in if you edit it.

Build prompt — paste into your agent

Harden and set up this fresh Ubuntu 24.04 VPS. Work step by step, verifying each
phase with real command output before moving to the next. Ask before any reboot.

1. Create a sudo user (I'll tell you the username) and copy root's SSH keys
   from /root/.ssh/authorized_keys to the new user.
2. Harden SSH in /etc/ssh/sshd_config.d/99-hardening.conf:
   PermitRootLogin no, PasswordAuthentication no, PubkeyAuthentication yes.
   Validate with `sshd -t`, then open a SECOND verified login before restarting
   ssh. Never restart sshd without that check.
3. Install and enable ufw: default deny incoming, allow outgoing, allow OpenSSH.
   Install and enable fail2ban.
4. Install Tailscale (https://tailscale.com/install.sh), run `tailscale up`,
   and give me the login URL.
5. Install Docker from the official repo
   (https://docs.docker.com/engine/install/ubuntu/), add my user to the docker
   group. CRITICAL RULE: every container port mapping must bind to loopback
   (127.0.0.1:PORT:PORT), never 0.0.0.0. Anything public goes through Caddy.
6. Install Caddy from the official repo (https://caddyserver.com/docs/install).
   Confirm the service is running.
7. Final verification — show me the output of each:
   - `ssh root@localhost` on port 22 should be refused / pubkey-only
   - `ufw status verbose` shows a minimal ruleset
   - `docker ps` format shows no 0.0.0.0 bindings (fine if none running)
   - `tailscale status` connected
   - `systemctl is-active caddy` active
8. Write an AGENTS.md in my home directory documenting the conventions:
   apps live in ~/apps/<appname>/ with docker-compose.yml, public sites get a
   block in /etc/caddy/Caddyfile followed by `systemctl reload caddy`, all
   Docker ports bound to 127.0.0.1, never install nginx, never edit ufw or sshd
   config without asking first.

The one dangerous step

SSH hardening (item 2) is the only step that can lock you out if mis-ordered. That's why the prompt demands sshd -t plus a second working login before restarting sshd — and why you should keep your current session open until the agent confirms both. If it ever skips that sequence, stop it and make it comply. Recovery if it all goes wrong: Hetzner console → your server → Rescue tab gives you an out-of-band root shell.

Why the agent verifies better than you do

A tired human types ufw enable and moves on. The agent pastes back the actual ufw status verbose output, which you can eyeball in ten seconds. Ask it to show verification output at every phase, not to claim "done" — the build prompt above already does this.

Phase 5 — Tailscale: your admin path Agent

The build prompt installs Tailscale. Your part: open the login URL it prints, and sign in (free account covers up to 100 devices: tailscale.com).

Install Tailscale on your laptop/phone too: tailscale.com/download. After that, prefer SSH-ing to the server's tailnet name or tailscale ip -4 address instead of the public IP. The public SSH port stays open as a fallback, but daily admin traffic never touches the open internet.

Phase 6 — Docker & Caddy conventions Agent

Both are installed by the build prompt. The important part is the layout, which the agent writes into AGENTS.md so every future session follows it:

Apps live in ~/apps/<appname>/, each with a docker-compose.yml. No raw docker run.
Loopback binding rule: every port map is 127.0.0.1:PORT:PORT. Docker writes its own iptables rules and bypasses UFW — an unbound public port is exposed even behind a firewall. Audit with docker ps --format '{{.Ports}}' | grep 0.0.0.0 (should print nothing).
Public access goes through Caddy: add a block to /etc/caddy/Caddyfile, then sudo systemctl reload caddy. Caddy provisions HTTPS certificates automatically — no certbot.

Phase 7 — Remote desktop on the VPS (optional) Agent

A full desktop costs ~1 GB RAM. If you don't genuinely need a GUI, skip this. If you do, tell the agent:

Prompt

Install XFCE and xrdp for remote desktop. Enable xrdp. Do NOT open port 3389
in ufw — RDP must only be reachable over the Tailscale network. Verify xrdp is
listening and confirm ufw rules unchanged.

Then connect from your machine's RDP client to the server's Tailscale IP (tailscale ip -4), never the public IP. RDP exposed to the open internet is one of the most aggressively scanned services on the web.

Phase 8 — Chat with your server from your phone (optional) You

Hermes connects to Discord, Telegram, Slack, and a dozen other platforms, giving the same agent your phone's chat apps:

hermes gateway install

After this you can message the VPS ("check disk usage", "restart the myapp container") from anywhere — which, combined with Tailscale, makes the remote-desktop step unnecessary for most admin work.

Final checklist You

The server isn't "done" when things are running — it's done when these all pass:

ssh root@YOUR_SERVER_IP → refused
Password SSH → refused (key only)
sudo ufw status verbose → minimal ruleset (22/80/443 + Tailscale)
docker ps --format '{{.Ports}}' | grep 0.0.0.0 → nothing
tailscale status → server listed, laptop can reach it
systemctl is-active caddy → active
hermes doctor → green
xrdp (if installed) → reachable on tailnet IP only

Pitfalls & recovery

Locked out during SSH hardening → Hetzner console → server → Rescue tab → out-of-band root shell. Fix the config, reboot normally.
Docker port exposed despite UFW → Docker bypasses UFW entirely via iptables. Always use 127.0.0.1: bindings; route public traffic through Caddy.
Agent breaks prompt caching / mid-run drift → don't edit its past context; steer with new messages. If a run goes sideways, fix the specific step manually and tell it what changed.
Forgotten fail2ban → check your own bans: sudo fail2ban-client status sshd.
Certificate not issuing → Caddy fetches Let's Encrypt certs on first request; allow ~10 seconds, check journalctl -u caddy for certificate obtained successfully.