How to Audit Who Changed What on Your Servers

Why This Is Hard to Answer

A site goes down, a vhost stops responding, or a cron job that used to work silently starts failing. The first question is never "what broke" — that part is usually obvious from an error log. The first question is "who touched this, and when." On most small teams, that question takes an hour of grepping through scattered logs, and sometimes it never gets a real answer, because the trail was never being kept in the first place.

The reason it's hard is structural, not a tooling gap. A typical small team shares one root SSH key or one set of sudo credentials across three or four people. Everyone edits the same nginx.conf, restarts the same services, and every one of those actions lands in the logs as "root did this" — not "Priya did this." The system was never told who the human was, so it can't tell you later.

Start With SSH Login History

On Ubuntu 22.04 and 24.04, every SSH authentication attempt — successful or not — is recorded in /var/log/auth.log by the system logger. This is the first place to check when you need to establish who was even on the box during a given window. It won't tell you which file someone edited, but it narrows the timeline to who had a shell open.

This is genuinely useful for identifying an unfamiliar login IP or an access attempt outside business hours. But if every teammate connects with the same shared key labeled "deploy," auth.log will faithfully tell you "the deploy key logged in at 3:14am" without telling you which person was holding it. The log is only as attributable as the credentials feeding it.

Track File Changes with auditd

auth.log tells you who was logged in; it does not tell you which files they touched once they were in. For that, Ubuntu ships the Linux Audit framework, installed as the auditd package. Once running, you can watch specific paths — an nginx config directory, a wp-config.php, a crontab — and get a kernel-level record of every read, write, or attribute change against them, tagged with the user, process, and timestamp.

Rules added with auditctl directly don't survive a reboot, so put the ones you want permanently active in a file under /etc/audit/rules.d/ and reload with sudo augenrules --load. Even a handful of watches on config directories and credential files is usually enough to answer "did anyone touch this" with certainty instead of a guess.

Shell History Is Not an Audit Trail

Per-user .bash_history is the first place most people look, and it's genuinely handy for reconstructing a sequence of commands right after an incident. But it was never designed as a security control, and it fails as one in three specific ways.

First, it's trivially cleared: history -c wipes the in-memory list, and unsetting HISTFILE before running a command stops it from ever being written at all. Second, it's local to the account and the machine — if someone works as root directly rather than through sudo, there is no per-user attribution to recover, only a single shared file. Third, and most importantly, anyone with write access to the account can edit the history file itself after the fact, so even an intact-looking history is not proof of what actually ran.

Treat shell history as a convenience for your own recall, never as evidence for an audit. If a command needs to be provable after the fact, it needs to be logged by something the person running it can't edit — which is exactly what auditd and centralized logging are for.

Get Logs Off the Server

Every audit trail described above has the same weak point: it lives on the box that got compromised. Anyone with root can run rm /var/log/auth.log or truncate the audit log and remove the evidence along with the intrusion. A trail that can be deleted by the same access level that created the problem isn't reliable — it needs to leave the server as it's written.

rsyslog, already running on Ubuntu by default, can forward logs to a remote collector with a few lines in /etc/rsyslog.d/:

A dedicated log shipper — Filebeat, Vector, or Promtail feeding into whatever you already use for aggregation — does the same job with more structure, shipping auth.log and ausearch output as they're written rather than waiting for someone to pull them manually. Once logs are off-box in near real time, deleting the local copy after an incident doesn't erase the record; it just means someone was clumsy enough to leave evidence of trying.

This is also where shared credentials keep causing pain, independent of where the logs end up. If three people SSH in as the same deploy user, no amount of forwarding turns that into per-person attribution — the shipper can only forward what the source system recorded. Because every CloudStick teammate manages servers, websites, and services through their own dashboard login rather than a shared root password, actions like restarting a service, editing a vhost, or rolling back a deployment are already tied to a named person in CloudStick's own logs the moment they happen. You're not reconstructing who did it from raw auth.log after the fact — the platform already knew, because role-based access meant that action could only have come from one account.

Putting It Together

A working audit trail is layered, not a single tool. Check auth.log to establish who was on the server and when. Use auditd with watches on the paths that actually matter — web server configs, credential files, cron directories — to see what changed once someone was in. Stop relying on shell history for anything you might need to prove later; it's a convenience, not a record. And ship everything off the box with rsyslog forwarding or a log shipper so the trail survives even if the server itself doesn't.

None of this replaces the simplest fix: stop sharing root. The moment each teammate has their own account with a defined role, most of the "who did this" investigation disappears before it starts, because the system was recording the right name from the beginning.