MONITORING
July 23, 2026

How to Set Up Alerts Before Your Server Runs Out of Disk

5 min read
Author
CloudStick Team
Backend Developer
Share this article
How to Set Up Alerts Before Your Server Runs Out of Disk
CloudStick
Disk Space Alerts

Why a full disk takes everything down with it

A disk that reaches 100% doesn't just run out of storage — it takes the rest of the server down with it. MySQL and MariaDB refuse writes once their data directory has no free space, so INSERT and UPDATE queries start failing even though the service is still running and still responds to a health check. PHP-FPM can't write session files, so logged-in users get bounced without warning. On Ubuntu and Debian systems, systemd-journald and even sshd can fail to allocate space for new log entries or PAM session state, which is why a server that looks "up" in a ping check can be completely unusable for new SSH logins. None of this happens as a single dramatic event. It's almost always a slow leak — a log file growing a few megabytes a day, a backup directory that never gets pruned, an application writing debug output to disk — until one day the graph crosses zero and every service depending on that filesystem fails at once, usually outside business hours when nobody is watching.

Checking usage right now

Before you write anything that runs unattended, check what df is telling you right now. df -h reports space usage per mounted filesystem in human-readable units, but it only tells half the story — a filesystem can sit at 40% space usage and still be completely full if it has run out of inodes, which happens on servers with millions of small files: session caches, mail queues, or a deploy pipeline that leaves old node_modules trees lying around. Check inode usage separately with df -i; the percentage column behaves the same way, and a filesystem at 100% inode usage refuses new files exactly like one at 100% space usage, even while df -h still shows gigabytes free. Once you know a filesystem is genuinely tight, find out why before assuming it's normal growth: du -sh /var/log /home/*/logs 2>/dev/null | sort -rh | head walks the usual suspects and sorts the biggest directories to the top, which is almost always faster than guessing.

Writing a disk-alert script

The most reliable disk-space alert isn't a SaaS monitoring agent — it's a short bash script that reads df and gets out of the way if there's nothing to report. The script below loops over every mounted filesystem using df -P, POSIX output format, which is more predictable to parse than the default df -h because column widths and delimiters don't vary between distributions. It strips the percent sign off the usage figure and posts to a webhook whenever a filesystem crosses the threshold.

#!/bin/bash
# /usr/local/bin/disk-alert.sh — warns when any mounted filesystem crosses a threshold
THRESHOLD=85
WEBHOOK_URL="https://hooks.slack.com/services/XXX/YYY/ZZZ"
df -P | awk 'NR>1 {print $5, $6}' | while read -r pcent mount; do
usage=${pcent%\%}
if [ "$usage" -ge "$THRESHOLD" ]; then
curl -s -X POST -H 'Content-type: application/json' \
--data "{\"text\":\"Disk alert: ${mount} is at ${usage}% on $(hostname)\"}" \
"$WEBHOOK_URL"
fi
done

Save the script as /usr/local/bin/disk-alert.sh and make it executable with chmod +x /usr/local/bin/disk-alert.sh. Swap the curl call for a mail command if you'd rather receive plain email, or leave both in and let the script fan out to more than one channel.

Scheduling it with cron

Cron is the right tool for running this check, not a long-running daemon — the check takes a fraction of a second and doesn't need a persistent process sitting in memory. Add a line like the one below with crontab -e, or drop a file into /etc/cron.d/ if you want the schedule version-controlled alongside your provisioning scripts, to run the check every fifteen minutes.

# crontab -e
*/15 * * * * /usr/local/bin/disk-alert.sh >> /var/log/disk-alert.log 2>&1

Cron jobs run with a minimal PATH — typically just /usr/bin:/bin — so always call the script by its absolute path, and make sure the script itself doesn't depend on tools resolved through a login shell's PATH. df and curl live in /usr/bin or /bin by default on Ubuntu and Debian, so this rarely causes problems, but it does bite people who install tools via nvm, rbenv, or a custom PATH set in .bashrc, none of which cron ever loads.

Choosing a threshold and alert channel

A single threshold isn't enough, because a disk at 80% full and a disk at 98% full are different emergencies that deserve different responses. Use two tiers: a warning around 80%, which gives you time to investigate calmly during working hours, and a critical alert at 90%, which means act now. Adjust the exact numbers to the workload — a database server with large binary logs might want to warn earlier than a static file server — but the two-tier pattern itself is the important part, not the specific percentages you pick.

TIP

Email is the easiest channel to wire up first, but it's also the slowest one to actually reach a person, especially outside working hours. For the warning tier, email is fine — nobody needs to be woken up over a slow, predictable trend. For the critical tier, route the same script's output to Slack or a webhook that triggers SMS. A disk that's growing fast enough to hit 90% can go from warning to full within an hour on a busy logging server, and by the time someone checks their inbox the outage has already started.

Automatic cleanup as a backstop

A cron script tells you a disk is already close to full; CloudStick's Server Stats panel — available on every plan, with no extra monitoring agent to install — shows the trend line before that happens. Because Server Stats graphs disk usage over time alongside CPU and RAM, a slow leak, a log directory growing by a couple hundred megabytes a day, for example, is visible as a steadily rising line days before it becomes a middle-of-the-night page. Pairing that visibility with the cron alert script above is a reasonable belt-and-suspenders setup: CloudStick shows you the trend so the root cause gets fixed on your own schedule, and the script catches you if something spikes faster than expected between check-ins. Set the threshold today, before the disk fills on its own schedule instead of yours.

Leave a comment
Full Name
Email Address
Message
Contents