Troubleshooting guides
Start with the symptom. Follow a path you can verify.
Every runbook follows the same flow: scope impact, verify a hypothesis, recover safely, then add a prevention action. Adapt commands to your distribution, permissions, and change process.
Nginx 502 Bad Gateway: trace the upstream before changing timeouts
A 502 is an upstream connectivity or response problem. Establish whether the backend is reachable, alive, and responding within its budget before changing proxy settings.
Open runbookMySQL connection pool exhaustion: find the leak or the wait
Too many connections is an outcome of saturation, leaks, or slow transactions. Identify which clients hold connections before increasing capacity.
Open runbookExpired or failed TLS certificate replacement: restore HTTPS safely
Inspect the certificate actually served on the user path, then validate renewal, deployment, and CDN propagation instead of only checking a PEM file on disk.
Open runbookKubernetes CrashLoopBackOff: use the exit reason to find the startup failure
CrashLoopBackOff is a restart state, not the root cause. Previous logs, termination reasons, events, and dependency health reveal what is breaking startup.
Open runbookDocker disk full: measure before you prune
When a Docker node runs out of space, identify images, containers, volumes, and logs first. A blind prune can remove data you still need.
Open runbookRedis memory pressure and eviction: distinguish policy, hot keys, and TTL debt
Increasing maxmemory is not a root-cause fix. Inspect eviction policy, key lifetimes, large values, and workload growth before changing capacity.
Open runbookLinux DNS resolution failure: verify resolv.conf through container DNS
Separate missing network reachability, an unavailable resolver, wrong records, and stale application caching before changing DNS settings.
Open runbookLinux disk full with no obvious large file: find deleted files and inode exhaustion
When df and du disagree, a deleted file may still be held open by a process. The other common cause is inode exhaustion from many small files.
Open runbookCron job not running: environment, permissions, timezone, and logging checklist
Cron runs with a smaller environment than an interactive shell. Use absolute paths, explicit logs, and explicit timezones to remove ambiguity.
Open runbookSSH connection failure: distinguish network, port, authentication, and host key issues
The SSH error text points to different layers. Read it first, rather than changing authentication before the route and listener are known-good.
Open runbookPostgreSQL deadlock detected: use the lock graph to repair concurrency order
A deadlock is a cycle of mutually waiting transactions. PostgreSQL aborts one to protect correctness; the lasting fix is usually in application access order.
Open runbookHigh Linux load average: tell CPU saturation from I/O wait and D-state tasks
Load average is not CPU usage alone. It includes runnable work and uninterruptible I/O waits, so pair it with queue, iowait, and process-state evidence.
Open runbookCloudflare 522 Connection Timed Out: trace reachability from edge to origin
A 522 means the edge could not establish or complete a timely connection to origin. Focus on origin reachability, filtering, and capacity rather than repeatedly purging cache.
Open runbookHow to use a runbook
Begin with the scope and prerequisites. Compare your evidence with the interpretation table, adapt example commands to your environment, and define recovery checks before making a change. If a step is unclear or version-specific, send Kevin a correction.