tai/runbooks/ssh.md

service, symptoms, tags

service	symptoms	tags
ssh	connection refused, authentication failed, host key mismatch, permission denied, timeout connecting, no route to host	ssh, sshd, openssh, authentication, network, connectivity

Symptoms

• ssh: connect to host <hostname> port 22: Connection refused
• Permission denied (publickey) — key not accepted or wrong user
• WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! — host key mismatch
• Connection timed out — firewall blocking or host unreachable
• No route to host — routing issue or host is down

Diagnostics

Is sshd running?

systemctl status sshd
systemctl status ssh

A stopped or failed sshd is the most common cause of "connection refused".

Check sshd configuration

sshd -t
cat /etc/ssh/sshd_config

Look for: PasswordAuthentication, PubkeyAuthentication yes, AuthorizedKeysFile.

Check authorised keys

ls -la ~/.ssh/
cat ~/.ssh/authorized_keys

Permissions must be: ~/.ssh → 700, authorized_keys → 600. Wrong permissions cause silent auth failure even with the correct key.

Check sshd logs

journalctl -u sshd -n 100
journalctl -u ssh -n 100
grep sshd /var/log/auth.log | tail -50

Look for: Invalid user, Failed publickey, Connection reset by peer, Too many authentication failures.

Check listening port

ss -tlnp | grep sshd
netstat -tlnp | grep :22

If sshd is running but not listening on the expected port, check Port in /etc/ssh/sshd_config.

Firewall rules

iptables -L INPUT -n -v
nft list ruleset
ufw status verbose

A DROP rule on port 22 causes silent timeouts, not "connection refused".

Remediation

sshd not running:

systemctl enable --now sshd

Wrong permissions on authorized_keys:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chown -R $USER:$USER ~/.ssh

sshd config error: Fix the error reported by sshd -t, then:

systemctl restart sshd

Host key mismatch (expected after reinstall/reprovisioning): Remove the old key from the client:

ssh-keygen -R <hostname>

Only do this if you are certain the host was intentionally reprovisioned. If the key change is unexpected, treat as a potential MITM and investigate before connecting.