LVM & Linux Storage Tools
The LFCS blueprint's Storage domain is worth a full fifth of the exam, and its published competency list reads almost like this page's table of contents: LVM, the virtual filesystem, filesystem creation and troubleshooting, remote filesystems and network block devices, swap, filesystem automounters, storage performance monitoring. LVM — the Logical Volume Manager — sits between a raw block device and the filesystem on top of it, and its entire argument for existing is one word: resize, without unmounting, without rebuilding, without downtime. Below: the physical-volume → volume-group → logical-volume stack and why pooling capacity this way matters, growing a live filesystem with the exact one-line command the blueprint itself highlights, swap treated as a first-class resizable volume instead of a fixed partition carved out at install time, static fstab mounts held up against autofs for shares that shouldn't hang a login shell when a server vanishes, and the handful of commands — iostat, du, lsblk, findmnt — that answer "why is this node out of disk, or slow" before you guess.
Picture a warehouse floor built from modular tiles instead of a wall built from bricks. A brick wall is one size forever — making it bigger means demolition, and so does making it smaller. The modular floor works differently: raw floor tiles (physical volumes) bolt together into one shared pool of open floor space (a volume group), and then you rope off however much of that pool becomes one usable storage room (a logical volume) — any size, as many rooms as you like. Need one room bigger? Walk the rope out further and restack the shelves inside it — nobody in the next room over even notices, and nothing had to be knocked down. LVM is that modular floor. A plain partition is the brick wall.
vgs and lvs output twice before typing a single resize command, and who already hosts the LFCS blueprint and the Linux fundamentals deep-dive this page sits beside.Architecture: physical volumes, volume groups, logical volumes
☺ Like you're 10: One raw tile becomes floor space, tiles pool into one shared floor, and rope marks off however much of that floor becomes one usable room.
pvcreate initializes a raw block device — a whole disk or a partition on one — as a Physical Volume (PV): it writes an LVM label plus a small metadata area, and carves the rest into fixed-size Physical Extents (PEs), 4 MiB each by default. vgcreate pools one or more PVs into a Volume Group (VG) — the VG's total capacity is simply the sum of every member PV's extents, and it doesn't care that they might be different physical disks. lvcreate then carves a Logical Volume (LV) out of the VG's free extents — an ordinary-looking block device at /dev/<vg>/<lv> (a symlink to /dev/mapper/<vg>-<lv>) that a filesystem gets built on exactly like it would on a plain partition. The whole point of that extra layer of indirection: capacity and physical layout become two separate questions. Run out of room in the VG, and vgextend adds another PV to the pool — no reformat, no data movement, no downtime — and every LV in that VG can immediately grow into the new space.
# PV — initialize raw devices for LVM's own bookkeeping pvcreate /dev/sdb /dev/sdc # VG — pool them into one shared capacity vgcreate vg_data /dev/sdb /dev/sdc # LV — carve out usable volumes from the pool lvcreate -n lv_var -L 20G vg_data # a fixed 20G volume lvcreate -n lv_data -l 100%FREE vg_data # everything that's left in the VG # a filesystem goes ON TOP of the LV — LVM itself has no opinion about ext4 vs xfs mkfs.ext4 /dev/vg_data/lv_var mkfs.xfs /dev/vg_data/lv_data
LVM decouples "how much storage I have" from "how it's physically partitioned." A plain partition ties a filesystem's ceiling to one device's boundary drawn at install time. A logical volume ties it to whatever the volume group currently pools — which you can grow, live, by adding another disk, without ever touching the filesystem's own on-disk layout until the very last step.
"I assumed LVM was operations trivia until our staging database's disk filled up mid-demo. Someone typed four commands — pvcreate on a disk the cloud console had already attached, vgextend, lvextend -r, done — and the database kept running the entire time, no downtime window, no maintenance ticket. I'd have reached for a full backup-and-migrate. That gap is the whole reason this exists."
Growing a filesystem live — the one command pair that's the whole point
☺ Like you're 10: Walk the rope out further to make the room bigger, then restack the shelves inside it to actually use the new space — both in one breath, no one has to move out.
Growing is the operation LVM exists for, and it's genuinely live: no unmount, no maintenance window, for both major Linux filesystems. The sequence is always the same shape — check the VG has room (or give it more with vgextend), grow the LV, then grow the filesystem to fill it. lvextend -r does the last two steps in one command, dispatching to the correct filesystem-resize tool automatically. It's worth memorizing as one line, because it's exactly the one the LFCS blueprint itself calls out for this domain:
# is there room in the VG already? (VFree column) vgs vg_data # out of room in the pool — attach a new disk and extend the VG onto it pvcreate /dev/sdd vgextend vg_data /dev/sdd # the one-liner: grow the LV by 10G AND resize its filesystem, live, in one step lvextend -r -L +10G /dev/vg_data/lv_var # the same thing, spelled out as two commands — useful when you need to pick the resize tool yourself lvextend -L +10G /dev/vg_data/lv_var resize2fs /dev/vg_data/lv_var # ext4 — takes the DEVICE xfs_growfs /var # xfs — takes the MOUNT POINT, never the device
-L +10G grows the LV by that much on top of its current size; drop the + and -L 30G sets an absolute target instead — an easy way to grow ten gigabytes when you meant thirty. -l (lowercase) works in extents or a percentage instead of bytes, which is why -l 100%FREE appeared above: "give this LV every extent the VG has left," a common shape for a volume that should simply consume whatever room remains.
No tool exists to shrink an XFS filesystem, live or otherwise — xfs_growfs has no opposite. The only path to a smaller XFS volume is: create a new, smaller one, copy the data across, and swap it in. Plan XFS volumes generously up front, or use ext4 where you genuinely expect to shrink later. Shrinking ext4 is possible but only offline, and — the reverse order from growing — the filesystem must shrink before the logical volume does: umount, e2fsck -f, resize2fs to the smaller size, then lvreduce -L <size>. Shrink the LV first and you've truncated live filesystem data out from under itself — silent, and often unrecoverable, corruption.
Swap: a resizable logical volume, not a fixed partition
☺ Like you're 10: Swap is spare floor space the computer borrows from disk when it runs out of desk space in memory — and on LVM it's just another room you can resize, not a wall you poured in concrete at install time.
Swap is disk space the kernel uses as overflow when physical RAM is under pressure, and treating it as an LV instead of a dedicated partition means it inherits everything else on this page — lvextend, vgs, living in the same pool as your filesystems. The one operation swap can't do live is resize: unlike a mounted filesystem, an active swap device has to be switched off entirely before it changes shape, because there's no equivalent of resize2fs for a raw swap area — you simply reformat it with mkswap after the LV itself is bigger.
# set up swap on a dedicated LV lvcreate -n lv_swap -L 4G vg_data mkswap /dev/vg_data/lv_swap swapon /dev/vg_data/lv_swap echo '/dev/vg_data/lv_swap none swap sw 0 0' >> /etc/fstab # persist across reboots # quick alternative when there's no spare LV — a swap FILE instead fallocate -l 2G /swapfile && chmod 600 /swapfile mkswap /swapfile && swapon /swapfile # resizing swap — the one volume type on this page that needs a full stop first swapoff /dev/vg_data/lv_swap lvextend -L +2G /dev/vg_data/lv_swap mkswap /dev/vg_data/lv_swap # re-format — swap has no live-resize equivalent to resize2fs swapon /dev/vg_data/lv_swap # how aggressively does the kernel reach for swap? 0-100, default 60 sysctl vm.swappiness sysctl -w vm.swappiness=10 # takes effect now, gone at reboot echo 'vm.swappiness=10' >> /etc/sysctl.d/99-swap.conf # persists
Low vm.swappiness tells the kernel to prefer reclaiming page cache over swapping out application memory — the right default on most servers, where a slow, thrashing swap-in is a worse outage than briefly evicting a cache page. Watch for actual swap pressure with vmstat 1: nonzero, sustained numbers in the si/so (swap in/swap out) columns mean the box is genuinely thrashing, not just holding some swap allocated and idle — free -h alone can't tell you which of those two is happening.
Mounting it: fstab, UUIDs, and automounters for shares that come and go
☺ Like you're 10: A static mount is furniture bolted to the floor before anyone moves in. An automounter is furniture that only appears when you walk toward it, and quietly leaves again once nobody's used it for a while.
/etc/fstab is the classic answer: one line per mount, applied at boot. Identify devices by UUID, never a device name like /dev/sdb1 — device letters can and do shift across reboots as disks are added, removed, or enumerated in a different order, while a filesystem's UUID is stable for its lifetime. blkid and lsblk -f both list it; mount -a replays every fstab entry immediately, so you can catch a typo before it costs you a reboot, and findmnt shows what's genuinely mounted right now, options included — the ground truth, not what a file merely claims should be true.
blkid /dev/sdb1 # find the UUID before you write the fstab line lsblk -f # device tree, filesystem type, UUID, and mountpoint, all at once mount -a # replay fstab NOW — catches a typo before the next reboot does findmnt /var # confirm what's ACTUALLY mounted, and with what options
# /etc/fstab — one line per persistent mount, always applied at boot # device (UUID preferred) mountpoint fstype options dump pass UUID=1a2b3c4d-5e6f-7890-abcd-ef1234567890 /var ext4 defaults 0 2 UUID=9f8e7d6c-5b4a-3210-fedc-ba0987654321 /data xfs defaults 0 2 10.0.4.20:/export/shared /mnt/shared nfs defaults,nofail 0 0
A static mount has a real failure mode: nothing gates the boot on the target actually being reachable unless you say so. Always add nofail to a network filesystem line — without it, an unreachable NFS server at boot time can stall the whole machine waiting on a mount that will never succeed. That's the exact problem autofs solves structurally rather than by remembering a flag: nothing mounts until something actually touches the path, and it unmounts itself again after an idle timeout — a dead server only matters the moment someone tries to use it, not at every boot whether anyone needs it or not.
# /etc/auto.master — maps a mount-point prefix to a map file, plus an idle timeout /mnt/auto /etc/auto.shared --timeout=600 # /etc/auto.shared — the map file: key, NFS options, server:/export/path shared -rw,soft,intr 10.0.4.20:/export/shared
systemctl enable --now autofs ls /mnt/auto/shared # the FIRST access to this path triggers the mount, not before
NFS mounts default to hard: if the server stops responding, the client retries forever rather than returning an error — any process touching that mount blocks in uninterruptible sleep (D state), immune to Ctrl-C and to kill -9 alike, until the server comes back or the mount is force-unmounted. That's the correct choice when silent data loss is worse than a hang — a database volume, say. For anything where availability matters more than that guarantee, mount soft with a sane timeo/retrans (as in the map file above) so a stuck call eventually returns an error instead of hanging a shell, or the whole node, forever.
Monitoring storage performance and space
☺ Like you're 10: Four different questions get four different commands: how full is it, who ate the space, how busy is the disk right now, and what does the kernel actually think is mounted where.
"Storage performance monitoring" is its own named competency in the LFCS blueprint, and in practice it's a short, memorizable ladder rather than one tool. Start with fullness, then find what's eating it, then ask whether the disk itself is the bottleneck, and only then look at the underlying hardware's own health.
# FULLNESS — filesystem level df -hT # human sizes, WITH filesystem type lsblk -f # the block-device tree: type, UUID, mountpoint findmnt # every mount, canonically, including binds # WHO ATE IT — walk the tree, largest first du -xhd1 /var | sort -h | tail # -x stays on one filesystem; a classic culprit is old logs/images journalctl --disk-usage # journald specifically is a frequent /var space eater # IS THE DISK ITSELF BUSY — sysstat's iostat, per device iostat -xz 1 # %util near 100 = saturated; watch await (ms) climb under load iotop -oa # top(1), but per PROCESS, for disk I/O specifically # IS THE HOST SWAPPING — vmstat's si/so columns vmstat 1 # sustained nonzero si/so = genuinely thrashing, not just idle swap # IS THE UNDERLYING DISK ITSELF HEALTHY smartctl -a /dev/sda # SMART attributes — catch a dying disk before it becomes an outage
Two columns in iostat -x matter more than the rest: %util tells you whether the device is saturated at all, and await tells you how long, in milliseconds, requests are actually waiting — a device pinned at 100% %util with a low, flat await can still be healthy if it's simply well-provisioned for the load; a climbing await is the real early warning sign, whatever the utilization number reads.
On a throwaway VM with a spare virtual disk: pvcreate, vgcreate, and lvcreate a 2G volume, format it xfs, and mount it. Fill it deliberately with fallocate -l 1900M /mnt/test/pad until df -hT reads uncomfortably full. Now grow it live: vgextend another virtual disk in, then run the exact lvextend -r -L +2G one-liner from earlier and watch df -hT update with zero downtime and no unmount. Set up a swap LV, swapoff it, grow it, and bring it back. Finally build the auto.master/auto.shared pair above pointing at any NFS export you have access to, and confirm with mount | grep auto that nothing is mounted until you cd into it — then watch it vanish from mount's output again after the idle timeout passes.
Gotchas, and LVM against the alternatives
☺ Like you're 10: Most surprises come from growing the room but forgetting to restack the shelves, moving the rope before you finish emptying the room, or promising more floor space than the warehouse actually has.
Forgetting -r on lvextend is the single most common self-inflicted gap: the block device genuinely grows, but the filesystem sitting on it doesn't know that yet, so df -hT keeps reporting the old, smaller size until you separately run resize2fs or xfs_growfs — the fix isn't wrong, it's just one command short. Pulling a PV that still has live extents on it — with vgreduce or by physically removing the disk — either fails safely if LVM notices in time, or destroys data if forced; move the extents off first with pvmove. Thin-provisioned LVs (lvcreate -T) can overcommit deliberately, which means vgs's free-space column can look healthy while the underlying thin pool itself is nearly full — watch the pool's own data_percent with lvs -o+data_percent specifically, because when a thin pool fills, every thin LV drawing from it can go read-only at once, not just the one that happened to write last.
| Option | Flexibility | Operational cost | Choose it when… |
|---|---|---|---|
| LVM (this page) | Resize, snapshot, and span disks live, at any time | Low — a handful of well-understood commands, native to nearly every distro | The default answer on any bare-metal or VM node where you don't yet know exactly how much room you'll need |
| Plain partitions | None — fixed at creation, resizing means a rescue boot and real risk | Lowest — but only because there's nothing to operate | A throwaway VM, or storage you are certain will never need to change shape |
| Btrfs / ZFS | Copy-on-write snapshots, checksums and volume management built into the filesystem itself — no separate LVM layer needed | Medium — a genuinely different mental model and its own failure modes to learn | You want snapshots and checksumming as a property of the filesystem, not a separate tool bolted on top |
| Cloud block storage + CSI | Resized (and often snapshotted) through a cloud API call, no SSH required | Low from the operator's seat — the provider runs the layer underneath | Kubernetes workloads via a PersistentVolumeClaim; many CSI drivers, TopoLVM among them, implement the volume itself with exactly the pvcreate/vgcreate/lvcreate stack above, one layer down |
That last row is worth sitting with: a PersistentVolumeClaim resizing live in a cluster is frequently this exact command sequence, just issued by a CSI controller instead of a human at a terminal — the same pvcreate/vgextend/lvextend primitives, one layer further from view. Platform Engineering's Storage & State picks up exactly where this page's "cloud block storage + CSI" row leaves off, and if the node underneath any of this ever goes NotReady, the same disk-space and I/O commands above are usually where a CKA-style investigation actually starts. systemd & journald, next in this section, covers the other half of "why is this node unhappy" — the process supervision half rather than the block-device half.
Foxy: The staging volume's nearly full. I ran lvextend -L +20G /dev/vg_data/lv_var and df still says it's almost out of room. Is lvextend just broken?
Sol the Sloth: Slow down and read what you actually typed, Foxy. No -r. The block device grew. The filesystem sitting on it has no idea yet — it's still measuring the room by its old walls.
Gizmo: Or — hear me out — just lvreduce it back down and reformat from scratch. Fresh start! Nobody will miss whatever was on there. 🎉
Timmy the Turtle: Gizmo, that's a production staging volume with a live database on it. lvreduce before the filesystem shrinks first is exactly the order that corrupts data — and here nobody even wants to shrink anything.
Sol the Sloth: resize2fs /dev/vg_data/lv_var. One command. The extents are already there — we just never told the filesystem to go claim them.
Benny the Beaver: And has anyone actually checked whether the VG had 20G free to give in the first place, or did lvextend quietly do less than we asked?
Sol the Sloth: vgs vg_data — check first, always. On a good day it tells you exactly what's left before you type a single resize command.
1. Name the three layers between a raw disk and a mounted filesystem under LVM, and the command that creates each. 2. What does vgextend let you do that a plain partition scheme cannot? 3. What single flag turns lvextend from "grow the block device only" into "grow the block device and its filesystem," and what does forgetting it look like in df's output? 4. Which of the two major Linux filesystems on this page can never shrink, under any circumstance? 5. Why does resizing swap require swapoff first, when growing an ordinary mounted filesystem doesn't? 6. What's the practical difference in behavior between a plain /etc/fstab entry and an autofs map for the same NFS export? 7. Which two iostat -x columns tell you whether a disk is actually the bottleneck, and what does each one mean?
Check your answers
- Physical Volume (
pvcreate) → Volume Group (vgcreate) → Logical Volume (lvcreate); a filesystem then goes on top of the LV withmkfs. - Add another physical disk to the pool with
vgextendand every logical volume in that group can immediately grow into the new capacity, live — a plain partition's ceiling is fixed at the physical device boundary drawn when it was created. -r— without it,lvextendresizes only the underlying block device; the filesystem on top keeps reporting its old size indfuntil you separately runresize2fs(ext4) orxfs_growfs(xfs, using the mount point, not the device).- XFS — it can only grow. Ext4 can shrink, but only offline, and only in the order filesystem-first, then
lvreduce— shrinking the LV before the filesystem truncates live data out from under it. - Because there's no live-resize equivalent of
resize2fsfor a raw swap area — an active swap device has to be switched off, extended withlvextend, and reformatted withmkswapbefore it's switched back on; a mounted filesystem, by contrast, can be resized while still mounted and in active use. - A plain
fstabentry mounts at boot unconditionally and stays mounted whether or not anyone uses it — a dead server withoutnofailcan hang the whole boot.autofsmounts only on first access to the path and unmounts again after an idle timeout, so a dead server only matters the moment someone actually reaches for that path. %util— how saturated the device is, as a percentage of time it was busy servicing requests — andawait— how long, in milliseconds, requests actually waited. A high but flat%utilcan still be healthy; a climbingawaitis the more reliable early sign of a real bottleneck.