The ferrosys command line
The ferrosys-cli crate ships one binary, ferrosys. It writes ext2, ext3, and ext4
filesystems, FAT12, FAT16, and FAT32 volumes, exFAT volumes, btrfs filesystems, and APFS
containers. It reports on them, reads their contents back out, and changes what an ext
filesystem is known by. It is the library’s surface for anyone not writing Rust.
$ ferrosys format --size SIZE --uuid HEX --time SECS [options] OUT.img
$ ferrosys inspect [options] IMAGE
$ ferrosys extract [options] IMAGE (--to-tar F|- | --to-dir DIR | --cat PATH | --stat PATH | --list)
$ ferrosys detect [options] IMAGE
$ ferrosys identity [options] IMAGE
This binary carries every family, APFS included, so detect and inspect identify an image
whatever it turns out to hold. The library’s default build leaves APFS out, and a program
that wants one family turns the library’s defaults off. format -t is where you say which
one to write.
Install it from the workspace:
$ cargo install --path crates/ferrosys-cli
Everything the image depends on is an input
Everything an image’s bytes depend on is an input you supply, so two runs given the same inputs write the same image, byte for byte. Reproducibility is the only mode the tool has, and it is always on.
That has one consequence worth stating plainly: an identity is required, and --time is
required for every family whose format records an instant. Each family names its own
identity. That is --uuid for ext, --volume-id for a FAT, --volume-serial for exFAT,
--fsid for btrfs, and --container-uuid for APFS, beside each volume’s own in --volume.
The tool mints neither an identity nor a time. Pipe an identity in
from a tool that does, and pass the time explicitly or set SOURCE_DATE_EPOCH:
$ ferrosys format --size 512M --uuid "$(uuidgen)" --time 1700000000 rootfs.img
$ SOURCE_DATE_EPOCH=1700000000 ferrosys format --size 512M --uuid "$(uuidgen)" rootfs.img
The directory-hash seed defaults to the UUID’s bytes, so it too is an identity you
supplied. --hash-seed overrides it.
Streams and exit codes
The standard output carries exactly one artifact per run: a report, a listing, a tar
stream, or one file’s bytes. Everything else — a format’s summary, warnings, errors —
goes to the standard error. So ferrosys extract img --to-tar - | tar -t never has a
summary line spliced into its input, and ferrosys inspect --json img > report.json
never has a warning in it.
The exit codes mirror e2fsck’s. The line between 4 and 8 is whether an opinion about a
filesystem could be formed at all:
| Code | Meaning |
|---|---|
0 | The command did what it was asked, and any filesystem it read is sound. |
4 | A filesystem was read, and it is bad. |
8 | The command could not be carried out. The host got in the way, the bytes are no filesystem, a cap stopped a read, or the request is beyond the build or family. |
16 | The command line could not be understood. |
format
$ ferrosys format --size 512M --uuid f0e17055-0000-4000-8000-000000000000 \
--time 1700000000 --from-tar rootfs.tar rootfs.img
The image streams out through the library’s streaming writer, which touches only the blocks the filesystem uses. The file therefore stays sparse, and the tool writes a filesystem far larger than memory.
Where the contents come from
--from-tar FILE and --from-dir DIR are the two sources. Without either, an ext
filesystem holds only /lost+found, and every other family an empty root. Giving both is a
usage error, since nothing here decides the rules for a merge.
--from-tar FILE reads an uncompressed tar archive. A named file is left on disk, and
each member is read as its file is placed, a mebibyte at a time. Peak memory therefore does
not grow with the archive. --from-tar - reads the standard input, which cannot be sought
back over and so is held whole. That is the one case where a large archive needs the
memory to match, and the one that carries a size cap. A stream past four gibibytes is
refused, and naming the archive as a file is both the way past it and the cheaper route.
A compressed archive is named as such rather than reported as malformed tar:
$ gunzip -c rootfs.tar.gz | ferrosys format ... --from-tar - rootfs.img
--from-dir DIR walks a directory tree on this machine. DIR itself becomes the
filesystem root. Modes, ownership, all three times, symlinks, hard links, device, FIFO and
socket nodes, and extended attributes with their POSIX ACLs all come across. Each file is
read as it is placed, a mebibyte at a time, so a file costs one mebibyte of memory however
large it is.
The walk records Linux inode metadata and Linux extended attributes, so it runs on Linux
alone. So do --volume-from-dir and extract --to-dir. A binary built elsewhere refuses
each by name and exits 8, having opened nothing. Everything else in the tool is the same on
every platform: an empty filesystem, --from-tar, inspect, the rest of extract,
detect, and every geometry option.
The walk records the uid and gid the host files carry, which for a build that does not run
as root is that user’s own. Use --owner UID:GID to replace them, and a rootless
build almost always wants --owner 0:0:
$ ferrosys format --size 512M --uuid "$(uuidgen)" --time 1700000000 \
--from-dir staging/rootfs --owner 0:0 rootfs.img
Sizing the image to its contents
--size auto works the size out from what goes in the filesystem, instead of being told:
$ ferrosys format --size auto --from-dir staging \
--uuid "$(uuidgen)" --time 1700000000 rootfs.img
It finds the smallest filesystem that holds the contents by planning candidate sizes and
placing the contents into each. The size it settles on is therefore one that formats, and
one allocation unit less does not. Nothing is written while it searches, so a size that
cannot be found leaves the destination untouched like any other planning failure. Pair it
with --dry-run to learn the number without writing anything at all.
It works for every family, and each measures itself in its own unit. An ext filesystem and an APFS container are searched a block at a time. A FAT volume, an exFAT volume and a btrfs are searched a sector at a time. A cluster or a chunk is derived from the size there, rather than given.
The smallest filesystem holding something is one with nothing left in it. That is right
for an image that will only be read, and useless for one that will be written to.
--slack says how much must stay free:
$ ferrosys format --size auto --slack 20% --from-dir staging \
--uuid "$(uuidgen)" --time 1700000000 rootfs.img
$ ferrosys format --size auto --slack 64M --from-dir staging \
--uuid "$(uuidgen)" --time 1700000000 rootfs.img
The share is of the finished filesystem, measured in the room each family leaves free. That
is blocks for ext, and clusters for FAT and exFAT. On btrfs it is room for file data, as
df reports it, and on APFS the blocks macOS lets any user write. So --slack 20% is what
df reports as 80% used. On ext the super-user reservation (--reserved-percent) is
separate accounting over the same blocks, so an unprivileged writer sees 15% of it at the 5%
default. --slack belongs to --size auto and is a usage error over a size that was named
outright, which has no room to find.
On btrfs the share is room for file data, as df reports it. That is the space left in its
data chunks, and the device space no chunk claims shared out between the data profile’s
copies.
On FAT a share can leave a volume unchanged where a byte figure does not. FAT32 needs 65525
clusters whatever it holds, so a small tree already lands on a volume that is almost
entirely free. Only --slack 200M and the like ask for more than that floor already gives.
On a small image the floor is often the journal rather than the contents. A log costs
about 4 MiB whatever goes in the filesystem, so --size auto -t ext2 is what fits a
boot partition to what it holds.
The destination must be a regular file
A format writes only the blocks the filesystem uses, and extends the file to its full size with a single byte at the end. Every byte it does not write must already read as zero, which creating or truncating a regular file guarantees and a block device does not. Formatting a device would leave whatever it held interleaved with the new filesystem, so the tool refuses.
A failing run leaves the destination alone
The source is parsed, the geometry planned, and the inode model built and checked before the destination is opened. A run that cannot succeed therefore never truncates the image already at that path.
--dry-run goes one step further. It reports the geometry the command would realize,
without opening the destination at all.
--atomic covers the other end. It writes to a sibling temporary file, and renames it over
the destination once the image is whole. A failure part-way through the write is therefore
not visible either. Note what --atomic costs: the destination becomes a new file. Its
mode comes from this process’s umask, and any ownership, ACLs, or extra hard links the old
file carried do not survive the rename. Without --atomic the image is written in place.
The two do not combine. --dry-run opens no destination, so there is nothing left for
--atomic to decide, and a flag that decides nothing reads as one that worked.
A destination is never a file the run reads
An archive named by path is read a member at a time, as each file is placed. So is a walked tree. Both reads happen after the destination is opened. A destination written in place that is one of those files is truncated before it is read. The image would then carry zeros, or its own bytes, where the source’s belong.
So the tool refuses such a destination and leaves it as it was. It judges by the file rather
than by the name. The archive itself is refused, and so is a hard or symbolic link to it, or
a file already inside the walked tree. --atomic writes a new file and renames it into place
once everything has been read, so it carries out the same command.
Geometry the command line names, and geometry it does not
A family’s layout can be decided by more than its size. Where it is, the choice is spelled
out for the family that records it in its own superblock and reads it back as identity.
btrfs takes --sector-size, --node-size, and the two replication profiles. ext takes
--block-size and --inode-size for the same reason, and APFS takes --block-size.
FAT and exFAT derive their cluster size, table count, reserved sectors, and root capacity from the volume’s size, by the rules their reference formatters use. The command line does not offer to override them. The library does. A caller can pin a cluster size, a sector size, an OEM name, a boot sector, or a FAT volume’s disk geometry. That caller builds a plan request directly and formats through the crate. What comes out of either route is the same filesystem.
Choosing a filesystem
-t (or --type) selects which filesystem to write: ext2, ext3, ext4 (the default),
fat12, fat16, fat32, exfat, btrfs, or apfs. It names the family as well as the
variant.
Each family takes its own identity and its own options, and an option belonging to another
one is refused by name rather than passed over. A line that named two families has said two
things that cannot both be honored, and is told so.
exfat is one word where the others are three, because the format has one revision and
every volume records it. The family is the finest answer there is, so there is no variant
to choose between. btrfs is one word because what varies between two btrfs filesystems is a
feature word and a geometry, and both are options. apfs is one word for the same kind of
reason. What varies between two containers is a block size, a release profile and their
volumes, and each is an option.
Within the ext family it seeds the whole feature set from that profile’s baseline. The -O
list and the geometry options layer on top, exactly as they do for mke2fs -t:
$ ferrosys format --size 256M --uuid "$(uuidgen)" --time 1700000000 -t ext2 rootfs.img
$ ferrosys format --size 256M --uuid "$(uuidgen)" --time 1700000000 -t ext3 rootfs.img
An image is judged by the features it carries, not the profile it started from, so the two
compose freely. -t ext2 -O has_journal writes exactly what -t ext3 does, which is a
journal over the ext2 baseline, and ferrosys inspect labels either one ext3. The order
of -t and -O on the line does not matter. The profile is always the base, and -O
always layers on top.
Feature words
Features are named on disk, and -O turns them on and off left to right over the selected
profile (ext4 unless -t says otherwise):
$ ferrosys format --size 64M --uuid "$(uuidgen)" --time 1 \
-O ^has_journal,^orphan_file,^metadata_csum_seed,^metadata_csum small.img
That clears the journal and the checksums, and leaves the rest of the profile intact.
extent, 64bit, flex_bg, huge_file, dir_nlink, and extra_isize are all still
set. The image is therefore an ext4 one without a journal, and ferrosys inspect labels
it ext4.
-t ext2 is not a shorthand for a list of ^ words. It selects a different base, and
reaching that base by clearing words means clearing every ext4-layer word, extent
included. Name the profile you want, and layer -O on top of it.
A combination that must never reach disk is refused by name. Dropping has_journal while
orphan_file remains is one, since the orphan file’s entries are journaled. That is why
the line above clears both.
The same rule covers what the source holds, because a feature word is a promise about the
structures the filesystem carries. Two examples are refused by name and by path. Dropping
ext_attr from a source whose entries have extended attributes is one. Dropping
large_file from one holding a file of 2 GiB or more is the other. The attribute is
neither dropped nor written into a filesystem whose words deny it.
^large_file at the 4096-byte block size is refused on its own, before any source is read.
The resize inode a growable filesystem carries is itself a file that large.
-O reads btrfs’s feature words too, in btrfs’s own vocabulary — the words its tooling
takes and ferrosys inspect prints back, which are not ext’s:
$ ferrosys format --size 1G --fsid "$(uuidgen)" --time 1700000000 -t btrfs \
-O ^block-group-tree old-kernel.img
That is the one most worth naming. A filesystem carrying block-group-tree needs a kernel
of 6.1 or newer. Clearing it is how you write an image for an older one.
The grammar is the same in both families. A bare word sets a feature, ^ clears it, and
none starts from nothing. The list applies left to right. The vocabularies are disjoint,
so a word belonging to the other family is refused rather than quietly ignored. A feature
this tool does not write is refused by name, and so is one whose prerequisites were not
asked for. block-group-tree rests on free-space-tree and no-holes, so clearing either
of those means clearing it in the same list.
ext geometry and defaults
--grow
--grow sizes the reserved descriptor blocks that let the filesystem grow online without
relocating its descriptor table. It defaults to max, which reserves the most the format
allows. --grow 4G reserves exactly enough to reach a known target, and --grow none
reserves nothing.
--block-size and --inode-size
--block-size and --inode-size set the two sizes every other ext number is derived from.
A block is 1024, 2048, or 4096 bytes, and 4096 by default. An inode is a power of two from
128 up to the block size, and 256 bytes by default. A 128-byte inode holds a whole second
from 1901 to 2038 and no creation time. A time outside that, or a fraction of a second, is
refused. The inode size is therefore the choice that decides which times an image can carry.
A host’s times carry fractions, so a --from-dir tree takes --fixed-time to fit a 128-byte
inode.
--journal
--journal N sizes the journal in filesystem blocks, and --journal auto, the default,
sizes it from the filesystem. The journal is a real file and costs its size in free space,
which is four mebibytes of a sixteen-mebibyte filesystem. -t ext2 writes none, and is the
way to spend that space on contents instead.
--errors
--errors sets what a kernel does when it detects an error in the filesystem. continue
notes it and carries on, remount-ro remounts read-only, and panic stops the machine. It
defaults to continue, which is the kernel’s own default.
--fixed-time
--fixed-time forces every inode’s times to one instant, whatever the source recorded. It
is for the image whose contents came from a tree whose timestamps are not reproducible.
--time alone stamps the filesystem, and this stamps everything in it.
--hash and --hash-signedness
--hash and --hash-signedness decide how a directory’s names are hashed for its index.
The hash is half_md4 by default, and tea and legacy are the alternatives. The
signedness decides whether a name’s bytes are read as signed or unsigned, and it is
unsigned by default. That is what makes the hash independent of the host. A char is
signed on one architecture and unsigned on another. A directory hashed one way is not
readable by a kernel hashing the other.
--label
--label names the filesystem, up to sixteen bytes. A longer label is refused rather than
truncated, so the name a filesystem carries is the one that was asked for:
$ ferrosys format --size 64M --uuid "$(uuidgen)" --time 1 --label rootfs fs.img
--inodes and --bytes-per-inode
--inodes and --bytes-per-inode set how many inodes the filesystem has, overriding the
count the size alone would choose. --inodes 20000 names the count directly.
--bytes-per-inode 16384 names the density it is derived from, which is one inode for
every so many bytes. The two share one setting, and the last given wins. A density that
would need more inodes in a group than its one-block bitmap indexes is refused rather than
quietly reduced.
A named count is a target, not a floor. It is spread across the groups. Each group’s share
is rounded up to fill whole inode-table blocks, and then down to a multiple of eight. The
group’s inodes therefore end on a byte boundary in the inode bitmap. That is the same
s_inodes_per_group mke2fs derives for the request.
The rounding meets or exceeds the request wherever an inode-table block holds a multiple of
eight inodes. Where a block holds fewer than eight, the multiple-of-eight step can leave
the realized total a few inodes short. That is the 1024-byte block at the default 256-byte
inode, and the 2048-byte block once --inode-size 512 halves how many fit. ferrosys inspect reports the count the filesystem actually carries.
--reserved-percent
--reserved-percent sets the share of blocks held back for the super-user, from 0 to 50,
with up to two decimal places. --reserved-percent 1.5 reserves 1.5%. It defaults to 5.
The count is exact, floor(blocks × percent) in integer arithmetic, so a filesystem’s
reservation is reproducible to the block.
--json
--json prints the geometry the format realized on the standard output:
$ ferrosys format --size 64M --uuid "$(uuidgen)" --time 1 --json fs.img
{"schema":3,"family":"ext","variant":"ext4","uuid":"f0e17055-...","volume_name":"",
"created":1,"features":[...],...}
Every JSON document this tool writes opens with the same "schema" field. The shape is a
contract of its own that no command-line signature describes, so it names its own version.
The receipt also carries free_blocks and free_inodes, read back from the filesystem
that was just written. A format’s overhead is otherwise invisible, and on a small image it
is most of it. On a --dry-run, where no filesystem was written to have them, both are
null and "written" is false.
Writing a FAT volume
An EFI System Partition is FAT and has no alternative, so it is the case this family exists for:
$ ferrosys format -t fat32 --size 512M --volume-id "$(od -An -N4 -tx1 /dev/urandom | tr -d ' ')" \
--time 1700000000 --label ESP --owner 0:0 \
--accept-loss change-time,time-precision --from-dir esp-staging esp.img
--volume-id is this family’s identity. It is a 32-bit serial number, taken as eight hex
digits in either the bare form or the dashed one every tool prints (1A2B-3C4D). It is a
separate option from --uuid rather than four bytes cut from one. A value silently
narrowed is a value you did not choose.
The type is what the geometry must derive to, not what gets written down. Nothing in a
FAT volume records which of the three it is, and every driver counts the clusters and
compares against two thresholds. So -t fat32 on a size that cannot reach a FAT32 cluster
count is refused rather than written as a FAT16 wearing a FAT32 label.
The command line names a type outright. The library additionally offers the type the geometry reaches on its own, and a FAT32 volume below the cluster count the specification sets for one. That is a shape every mainstream driver reads and no specification blesses. Asking for it is therefore explicit, and the command line does not ask.
--label goes through this family’s own rules: eleven bytes, upper-cased, in the OEM
character set. A label the field cannot hold is refused rather than truncated, as on the ext
side. So is --time. A FAT directory entry represents 1980-01-01 through 2107-12-31 at a
two-second granularity. An instant outside that is refused rather than written as a
plausible-looking one inside it. --label "" writes a volume with no label, and a label
beginning with a space is refused, as mkfs.fat refuses one.
What a FAT volume cannot carry, and why the build stops
A FAT directory entry holds a name, one attribute byte, three coarse timestamps, a first
cluster, and a length. There is no field for an owner, a group, or permission bits. There
is none for a symbolic link, a second name for a file, a device number, or an extended
attribute. So a build that would drop any of those fails and names the entry and the
property. --accept-loss is how you name the ones that can go:
$ ferrosys format -t fat32 --size 512M --volume-id 1A2B3C4D --time 1700000000 \
--from-dir staging esp.img
ferrosys: /EFI: a FAT volume cannot carry the ownership of this entry
Refusing is the default because the alternative is worse than inconvenient. A root filesystem written to FAT with silent mode loss is a security bug wearing a convenience feature’s clothes. No report after the fact makes that acceptable.
A property counts as lost when the value does not survive, rather than when the format
has no field for it. That distinction is what keeps the acknowledgement meaningful. A read
of a FAT image fills an owner and a mode in from --assume-owner and --assume-modes,
which default to root and 0644/0755. A root-owned tree of conventionally moded files
therefore goes in, comes back unchanged, and loses nothing by them.
Two losses a tree walked off a host always takes:
change-time, because the format records no change time on anything.time-precision, because it stores a write time to two seconds and an access time to the day.
So the line above is the realistic minimum for --from-dir. Properties are named one by
one on purpose. A caller who accepted losing permission bits must not thereby have accepted
every symbolic link in the tree disappearing. That is the surprise the option exists to
prevent. --accept-loss all is the deliberate exception, and it covers a property a later
version names as well.
Set --assume-owner and --assume-modes to whatever the extraction will use, so the two
ends of a round trip agree about what survived. What the build did cost is the last thing
the summary prints, in the same words --accept-loss reads:
DIRECTION PROPERTY ENTRIES
dropped time-precision 4
dropped change-time 4
A build that lost nothing says nothing dropped or synthesized rather than printing an
empty table, so silence never has to be interpreted.
Writing an exFAT volume
An SDXC card specifies exFAT, and every current desktop operating system reads one as it ships. This is the family for a card or a stick that has to be read somewhere else:
$ ferrosys format -t exfat --size 8G \
--volume-serial "$(od -An -N4 -tx1 /dev/urandom | tr -d ' ')" \
--label MEDIA --owner 0:0 \
--accept-loss change-time,time-precision --from-dir card-staging card.img
--volume-serial is this family’s identity. It is the VolumeSerialNumber both boot
regions record, taken as eight hex digits in either the bare form or the dashed one every
tool prints. It is an option of its own, as FAT’s --volume-id is, because each is a field
of its own format. The two have one width and hold separate values.
There is no --time here, and the absence is the format’s. An exFAT volume records no
instant of its own anywhere. Every time on it belongs to an entry, and comes from the
source that named it. The flag is refused for this family, the way every option of a family
that was not named is.
--label goes through this family’s own rules again. It takes up to eleven UTF-16 code
units, which is eleven characters rather than eleven bytes for anything outside ASCII.
It takes text rather than bytes. The format stores a label as code units, so there is
no encoding to guess for bytes that are not text. As everywhere else, a label the field
cannot hold is refused rather than truncated.
This family does two things its neighbor cannot. A name is stored whole, up to 255 UTF-16 code units, in the case it was given, with no second shortened name derived beside it. What goes in is therefore what a listing shows. A lookup also folds through the volume’s own up-case table. A path in any case therefore reaches the entry a driver reading the same volume would reach:
$ ferrosys extract card.img --cat /DCIM/IMG_0001.JPG
$ ferrosys extract card.img --cat /dcim/img_0001.jpg # the same file
--size auto finds the smallest volume that holds the contents. Its cluster is the
conventional one for the size it settles on, which grows at 7 MiB, past 256 MiB and past
32 GiB. A tree of many small files that fits below one of those steps is fitted there. Past
each of the first two steps a small file takes eight times the room, and past 32 GiB four
times.
What an exFAT volume cannot carry
The same properties a FAT volume cannot, for the same reason. An entry set records a name, five attribute bits, three times, and two lengths. It has no field for an owner, a group, or permission bits. It has none for a symbolic link, a second name for a file, a device number, or an extended attribute.
--accept-loss, --assume-owner and --assume-modes mean exactly what they mean for FAT.
A build that would lose something you have not named fails, and says which entry and which
property.
What differs is how much time survives, and it is worth knowing before choosing between
the two families. exFAT records a creation and a modification time to ten milliseconds, and
each of its three times with a UTC offset. FAT records a write time to two seconds, an
access time to the day, and no zone at all. A volume this tool writes says its times are UTC
rather than leaving a reader to guess a locality. A host tree still loses time-precision,
because exFAT’s access time is two-second granular like FAT’s. It loses far less of it.
Writing a btrfs filesystem
Fedora and openSUSE default to btrfs, and both lay out subvolumes rather than one flat tree. This is therefore the family for a distribution root image, and the layout is the point rather than a refinement:
$ ferrosys format -t btrfs --size 8G --fsid "$(uuidgen)" --time 1700000000 \
--label fedora --owner 0:0 \
--subvol "$(uuidgen):/@" --subvol "$(uuidgen):/@home" --default-subvol /@ \
--from-dir root-staging root.img
--fsid is this family’s identity. It is the id blkid reports, and the id a UUID= line
in /etc/fstab names. It is taken as 32 hex digits in either the bare form or the dashed
one. It is an option of its own, as ext’s --uuid is, because each is a field of its own
format.
A btrfs records five identifiers, and the other four have options of their own.
--metadata-uuid is the id every tree block is stamped with, where it is to differ from the
visible one. That is what lets the visible id change later without rewriting every block,
and what a filesystem records as a feature bit in that state.
--chunk-tree-uuid, --device-uuid, and --subvolume-uuid are the chunk tree’s, the
device’s, and the top-level subvolume’s. Each defaults to zero, which is a legitimate value
and an obviously unset one. None of them is derived from another. A filesystem whose bytes
you can reproduce is one that states all five. A value this tool invented would be a value
you could not state.
--subvol [ro:]UUID:PATH makes the source directory at PATH the root of a subvolume of
its own. It is repeatable, and each needs its own identifier. The filesystem’s UUID tree
is keyed by them, so two subvolumes sharing one would produce a tree with a repeated key.
The command line says so by name, rather than letting the writer discover it. ro: in front
makes one read-only. The identifier leads and the path is everything after it, so a path can
contain a colon.
--default-subvol PATH says which subvolume a mount that was told none lands on. Without it
a mount lands on the top-level tree every btrfs starts with, which subvolid=5 names. That
is rarely what a root image wants:
# mount -o subvol=@ root.img /mnt # what --default-subvol /@ makes the default
A subvolume root is still a directory. The source declares it as one and --subvol says how
to lay it out, which is why the same staging tree feeds every family unchanged. A hard link
cannot span two subvolumes, and no btrfs holds one. A source naming one is refused rather
than silently written as two files.
--label is up to 255 bytes, taken as they come. The superblock’s label field records no
encoding, so what you supply is what every reader of the image sees. A label the field
cannot hold is refused rather than truncated, as everywhere else.
The geometry is four options. --sector-size is the smallest addressable unit of file data,
and --node-size is the size of a tree block. Each is a power of two from 4K to 64K, or
auto, the default: 4096 for the sector size and 16K for the node. The node size decides
how much a leaf holds, and so how large a file can be before it stops fitting inside the
metadata.
--metadata-profile and --data-profile say how each kind of block group is replicated.
dup writes two copies on the one device, which is what protects the trees against a bad
sector, and single writes one. Metadata defaults to dup and data to single, which is
what the format’s own tooling does on one device.
Unlike the other three families, nothing is lost on the way in. btrfs has a field for every property a source states:
- An owner and a group.
- A full mode.
- A link count.
- A device number.
- Four timestamps, each to the nanosecond.
- As many names per object as you give it.
- An extended attribute, as a record of its own.
--accept-loss is therefore not an option this family takes, and a build here never refuses
a tree for what it cannot hold. The format summary says so in a line rather than by silence.
--size auto finds the smallest filesystem that holds the contents, never below the minimum
of its profiles.
The smallest a btrfs can be is 109 MiB at the default profile pairing, and 45 MiB at
--metadata-profile single. A volume below whichever applies is refused with the number it
would have taken.
Two btrfs images from one directory can differ, and it is the directory that changed
Everything above holds the general rule: the same inputs write the same bytes. What is worth knowing for this family in particular is what counts as the same inputs when the input is a host directory tree.
Walking a tree reads its files, and on a filesystem that records access times, reading a
file updates that file’s access time. btrfs stores an access time to the nanosecond, as
ext4 and APFS do, so an image built from the tree shows the change. Two --from-dir builds of one
tree, run one after the other, can differ by a millisecond in every entry’s atime. The
tool read no clock. The tree it was pointed at was not the same tree twice.
If images have to be byte-identical across runs, give the build a source that reading does
not change. --from-tar is one, and its member times are in the archive. Mounting the
staging tree noatime is another, and so is stamping the tree’s times before each build.
Every other input is already fixed. --time and the five identifiers are values you state,
and nothing else in the tool reads anything but the source.
Writing an APFS container
APFS is the filesystem of every Mac, and of the RAM disk an Apple kernel roots on. A container is a pool of blocks shared by its volumes, each a filesystem with a name and an identifier of its own:
$ ferrosys format -t apfs --size auto --slack 8M --time 1700000000 \
--container-uuid "$(uuidgen)" --volume "case-sensitive:$(uuidgen):ramdisk" \
--owner 0:0 --from-dir root-staging ramdisk.img
--container-uuid is this family’s identity: the identifier the container superblock
records. It is an option of its own, as each family’s identity is, because it is a field of
this format. Every volume in the container records an identifier of its own beside it.
A volume is one directive, --volume [KIND:]UUID:NAME, carrying all three things a
volume has: how it compares names, its identifier and its name. It is repeatable, and each
volume takes the next slot. The first holds the contents, and every other one holds what
--volume-from-dir gives it, or only its root directory. Each needs its own identifier,
apart from the container’s, and a line where two share one is refused. The identifier leads
and the name is everything after it, so a name can contain a colon: UTF-8, up to 255 bytes.
KIND is case-insensitive, the default and what macOS formats, or case-sensitive. Those
are the words inspect prints for a volume. Either kind compares a name by its canonical
decomposition, so two spellings of one accented letter are one name. A source holding two
names its volume compares as one is refused, naming both. A Linux tree holding Makefile and
makefile is written to a case-sensitive volume.
A container has one volume slot for every 512 MiB of it or part of 512 MiB, fixed when it is
formatted. Two volumes therefore need a container larger than 512 MiB, and --size auto
finds one. The smallest container is 1,024 blocks, which is 4 MiB at the default block size.
The geometry is three options. --block-size is a power of two from 4096, the default,
to 65536. --profile names the macOS release whose shape the container is written in:
macos14, macos15, macos26 (the default) or macos27. It decides which Unicode version a
name is held to, refusing a code point that version leaves unassigned. It also decides two
words of each volume’s superblock, and macOS 26’s kernel mounts either shape.
--formatted-by is the program every volume records as having formatted it, up to 31 bytes.
It defaults to this tool’s name and version.
--root-snapshot UUID:NAME gives the first volume a snapshot of itself, which the volume
records as the one it roots from. The snapshot keeps the whole tree as it is written. Each
object it adds is laid out as macOS lays out one it takes. Apple’s tools list the snapshot,
mount it, and keep it as the volume changes. Its identifier is apart from the container’s and every
volume’s, and its name is UTF-8, up to 255 bytes. inspect reports the transaction a volume
roots from, and lists the snapshot beside the volume’s others.
--volume-from-dir SLOT:DIR fills a later volume from a directory tree, as --from-dir
fills the first. SLOT counts from 0 in --volume order and is 1 or more, once per volume,
and --owner applies to each tree.
--volume-group SYSTEM:DATA joins two volumes as a volume group, the System and Data
volumes macOS boots from. Each records its role and the group’s identifier, which is the Data
volume’s own. The System volume numbers its files from a range of its own. inspect reports
each volume’s group beside its role.
--firmlinks FILE marks the System volume’s firmlinks, the directories macOS joins to a
path in the Data volume. Each line of FILE is a directory’s absolute path, a tab, and the
target’s path relative to the Data volume, ending in a newline. A target is at most 1,023
bytes, the longest path macOS resolves. Each listed directory the System volume holds gets the
flag 0x800000 and the attribute com.apple.fs.firmlink holding its target. The summary counts
the directories marked against the lines listed.
$ ferrosys format -t apfs --size 1G --container-uuid "$(uuidgen)" \
--volume "$(uuidgen):System" --volume "$(uuidgen):Data" \
--root-snapshot "$(uuidgen):root" --volume-group 0:1 --firmlinks firmlinks.txt \
--owner 0:0 --from-dir system-tree --volume-from-dir 1:data-tree boot.img
Nothing is lost on the way in, as on btrfs. A volume has a field for every property a
source states. That is owners, modes, four times to the nanosecond, hard links, device
nodes, pipes, sockets and extended attributes. --accept-loss is therefore not an option
this family takes, and the format summary says so in a line.
--size auto searches a block at a time. A container fitted with no slack has no free block
at all: macOS mounts it and takes no new file. That is right for a volume that will only be
read, as a root RAM disk is.
--slack is room macOS lets you write. Before a change, macOS asks for the blocks it needs
and keeps a margin free beside them. It refuses the change where fewer are free. The margin is
counted in blocks, whatever their size. It is 100 in a small container and a hundredth of the
blocks from 10,000 on, at most 4,000. A process that is not root is kept a quarter more in a
small container, and about twice as much in a large one.
A file that grows is also given room ahead of it. Once the file is large, that is as much as
4,095 blocks in one change. The write that draws it fails where they are not free. --slack
counts all of this, so the room it leaves is room no write misses. What it keeps beside the room
is at least 4,535 blocks, and a mebibyte’s worth of blocks more. That is 4,791 blocks, 19 MiB,
at 4 KiB, and 4,551 blocks, 284 MiB, at 64 KiB.
With --slack 64M, any user can write 64 MiB into the container on macOS 15, 26 and 27, a
mebibyte or less a write. Each file’s records take room beside its data, about a block for
every sixteen files. The format summary gives the container’s Free blocks and its Writable blocks. The second is the first less what macOS keeps.
A small partition
A format’s defaults are sized for a general-purpose filesystem, and on a small one they
cost more than they are worth. Two of them dominate. The growth headroom (--grow) is one.
The journal is the other, and it is a real file costing its size in free space, which is
4 MiB of a 16 MiB filesystem. On a partition that will not grow and does not need a
journal, say so:
$ ferrosys format --size 16M --uuid "$(uuidgen)" --time 1700000000 \
-t ext2 --grow none --reserved-percent 0 boot.img
That leaves 3830 of the 16 MiB image’s 4096 blocks free, which is 93.5% of it usable,
against 2710 at the defaults. If the partition will be mounted read-write and power loss is
a real risk, keep the journal (-t ext3). Drop it where the partition is written once and
read afterwards, which is what a boot partition usually is. The format summary reports what
was reserved and what is left free, so the cost of any combination is one command away.
inspect
$ ferrosys inspect rootfs.img
Filesystem family: ext
Filesystem variant: ext4
Filesystem size: 67108864
Allocation unit: 4096
Filesystem identifier: f0e17055-0000-4000-8000-000000000000
Filesystem volume name: <none>
Filesystem UUID: f0e17055-0000-4000-8000-000000000000
Filesystem magic number: 0xEF53
Filesystem features: has_journal ext_attr resize_inode dir_index orphan_file ...
Filesystem state: clean
...
Inode count: 16384
Block count: 16384
...
no findings
Every report opens with the same five lines, whatever the image holds. Those are which family, which variant of it, how large it is, what it allocates in, and what identifies it. Everything after them is that family’s own:
- An ext image describes itself in superblock fields.
- A FAT volume describes itself in its parameter block.
- An exFAT volume describes itself in its boot region.
- A btrfs describes itself in its superblock, its chunk map, and its trees.
- An APFS container describes itself in its superblock and each volume’s.
An ext filesystem left needing recovery is reported as a mount would present it. Its journal
is replayed in memory first, and a Journal recovery: line gives the transactions and blocks
applied, or says the journal was not replayed. The JSON report carries the same under
journal_recovery. Every command that reads an image reads it that way, except identity,
which refuses one needing recovery.
A filesystem a driver had mounted and did not unmount cleanly is reported the same way in
every family. A scan gives one cosmetic finding, and its words are the same for all five:
the filesystem was not cleanly unmounted, so its metadata may not describe its contents.
The record is the format working rather than failing. A strict read of a card pulled out of
a reader therefore succeeds, and --fail-on conformance passes it.
Each body also says how the filesystem was left. FAT and exFAT print a Volume state line,
btrfs a Filesystem state line, and APFS a Container state line. All four use the same
words: clean, not cleanly unmounted, and a recorded medium failure where a FAT or exFAT
driver met one. ext prints Filesystem state in dumpe2fs’s words, clean or not clean,
with with errors where a driver recorded errors. A driver records an unclean shutdown of
an ext filesystem with a journal as needs_recovery in its features, and leaves its state
clean. On a filesystem with an orphan file it also leaves orphan_present, which the
features list.
So a tool that only wants to know what an image is, and whether it is sound, reads the head and stops:
$ ferrosys inspect esp.img
Filesystem family: fat
Filesystem variant: fat32
Filesystem size: 67108864
Allocation unit: 512
Filesystem identifier: 1A2B-3C4D
Volume label: ESP
Volume state: clean
Volume serial number: 1A2B-3C4D
Type string: FAT32
OEM name: ferrosys
Bytes per cluster: 512
Allocation tables: 2
Sectors per table: 1009
Clusters: 129022
...
no findings
The FAT body reports the volume’s own claim about its type beside the count the head’s variant was derived from. No driver reads that string, because the type follows from the cluster count and from nothing else. A volume whose string disagrees with its geometry is read by its geometry, and this is the one place the disagreement is visible.
An exFAT report is the same envelope over a third body, and four of its fields are ones no boot sector holds:
$ ferrosys inspect card.img
Filesystem family: exfat
Filesystem variant: exfat
Filesystem size: 8589934592
Allocation unit: 32768
Filesystem identifier: 5E71-A10C
Volume label: MEDIA
Volume state: clean
Percent in use: 0%
Partition offset: 0
Bytes per sector: 512
Sectors per cluster: 64
Bytes per cluster: 32768
Allocation table at sector: 2048
Sectors per table: 2048
Cluster heap at sector: 4096
Total sectors: 16777216
Clusters: 262080
Allocation bitmap at cluster: 2
Allocation bitmap bytes: 32760
Up-case table at cluster: 3
Up-case table bytes: 5836
Root directory at cluster: 4
no findings
Where the allocation bitmap and the up-case table are, and how long each is, are recorded nowhere but in the root directory. The format stores them as ordinary directory entries. Those lines are therefore what a reading of that directory recovered, rather than a field read off the boot sector.
Volume state is the two flags a mounted driver writes, which sit outside the boot checksum
precisely so it can.
A btrfs report is the same envelope again, over the one body in this tool that describes two layers, because the format has two. Above is a logical address space with the trees on it. Below is a chunk map that translates every address in it onto the device. A filesystem whose trees all verify and whose map is missing a range is a different report from one where both are sound. Both are therefore there:
$ ferrosys inspect fedora-root.img
Filesystem family: btrfs
Filesystem variant: btrfs
Filesystem size: 8589934592
Allocation unit: 4096
Filesystem identifier: 1b4e28ba-2fa1-11d2-883f-0016d3cca427
Label: fedora
Filesystem state: clean
Metadata identifier: 1b4e28ba-2fa1-11d2-883f-0016d3cca427
Generation: 42
Bytes used: 1735983104
Sector size: 4096
Tree block size: 16384
Filesystem features: free-space-tree free-space-tree-valid block-group-tree ...
Superblock copies: present, present, outside the device
Mapped chunks: 5
Mapped bytes: 2214592512
ROOT_TREE at: 30523392 (level 0)
CHUNK_TREE at: 22036480 (level 0)
EXTENT_TREE at: 30507008 (level 1)
...
Subvolume <top-level>: id 5
Subvolume root: id 256, default
Subvolume home: id 257
no findings
The last section is the one nothing else in this tool has. A subvolume is a filesystem tree
inside the filesystem, with inode numbers of its own. Someone points this command at a
btrfs to ask which ones are on an image. They ask, too, which is the default a mount
reaches with no subvol=. The top-level tree is a subvolume too, and it is the one no
directory entry names.
Filesystem features is one list across the three words the superblock carries, in the words
format -O takes. A feature read off a report can therefore be typed straight back into a
format. Nothing has to be translated from the all-capitals spelling the format’s own header
uses.
A bit no feature of this crate’s own table covers is reported on its own line, as the word it belongs to and its value. The table prints that line when such a bit is set, and the JSON report always carries the unknown bits, zero or not. An image carrying something this tool does not understand can therefore never read as one it does.
Superblock copies is listed one entry at a time rather than counted, because which copy is
damaged is what a person acts on. The format writes a copy at each of three fixed locations
the device is long enough to hold all of. A filesystem smaller than 256 GiB therefore has
two, and two is the whole count for it.
An APFS report describes a container rather than a filesystem, because that is what an APFS
partition holds. It gives the container’s own fields, then one block per volume it lists,
then the snapshots of the volume the scan read. That is the volume --volume names, by name,
by the identifier a report prints, or by slot. --snapshot scans one of its snapshots instead:
$ ferrosys inspect disk.img --volume Data
Filesystem family: apfs
Filesystem variant: apfs
Filesystem size: 2147483648
Allocation unit: 4096
Filesystem identifier: c5941820-3bf0-46f0-a201-6d4d3d9349c1
Block count: 524288
Block size: 4096
Transaction: 10
Container state: clean
Block zero: block zero holds the live superblock
Compatible features: 0x0
Read-only compatible: 0x0
Incompatible features: 0x2
Volumes: 2 of 4
Volume 0: Macintosh HD
Role: system
...
Read here: no: unsupported volume features: a sealed volume (0x20)
Volume 1: Data
Role: data
Identifier: 6428f854-0626-447e-99a9-1230723961ac
Names: case-insensitive
Features: 0x2, read-only compatible 0x0, incompatible 0x1
Holds: 2583 files, 39 directories, 7 symbolic links, 6 other
Snapshots: 1
Blocks allocated: 1636
Formatted by: newfs_apfs (2332.140.13)
Read here: yes, and reported on
Snapshot before-upgrade: transaction 8, taken 2026-10-09T20:25:20Z
no findings
Without --volume the report reads the first volume this tool reads. On a Mac’s own disk the
first slot holds a sealed system volume, which is passed over, and the standard error says so.
extract chooses the same way.
A volume whose superblock cannot be read is listed as <unreadable>, with the reason. The
other volumes of the container still read, and only a choice of that one is refused. A read
naming no volume stops there rather than passing damage over.
--volume and --snapshot belong to this family, and are refused for an image of any
other. None of the others holds more than one filesystem.
An option belonging to one family is refused for another rather than passed over.
--groups reports the block-group descriptors of an ext filesystem. A FAT or exFAT volume
has one flat cluster heap, and a btrfs is divided by a chunk tree instead. Asking for them
there is a question with no answer, and is told so. A report that quietly omitted the
section would read as a volume with no groups in it, which is a different claim.
The whole image is scanned by default: every group descriptor, bitmap, inode, extent
tree, and directory block, with each metadata checksum recomputed. That is what makes a
bad image bad (exit 4) rather than merely described. --quick reports the superblock
alone and reaches no verdict. It cannot be combined with --fail-on, which is a verdict on
the scan --quick skips. The two together would be a CI gate that looks armed and exits 0
on a filesystem whose bytes are destroyed. They are refused rather than accepted with one of
them inert.
--fail-on moves the line at which the scan’s findings make the filesystem bad. It defaults
to integrity. A filesystem is bad when its own bytes contradict each other, which is a
checksum that does not match what it covers. It is bad, too, when a structure the reader
must follow cannot be followed.
The threshold below it, conformance, means something else, and is worth knowing about
before you reach for it. It faults a filesystem that is valid for its format but not the
form this tool writes. A filesystem mke2fs or mkfs.fat made is exactly that, and it is
not thereby broken. So conformance is a check on this tool’s own output, an opt-in
self-check, and not what inspect does by default.
The four severities mean the same thing for every family. The categories a finding falls
into do not, and are each family’s own. A superblock and a boot sector are not one subsystem
under two names. --fail-on structural faults only an image whose structures cannot be
followed at all. --fail-on never reports every finding and exits 0 regardless, which is
what to use when you want the report and not the judgment.
A read the image file fails during the scan is the host’s failure, not the filesystem’s.
A disk returning an error is one, and a network share going away is another. Each is
reported as a source failure, after the findings in the table and as "source_failures" in
the JSON report.
None counts towards the verdict. The scan did not look at what the failed read held, so the
run exits 8 whatever --fail-on says. extract exits 8 on the same failure. A file that
ends early is different: that is the image cut short, and a finding.
--groups is bounded the same way and for the same reason. A superblock’s group count is
its own claim, and a crafted one reaches four billion. The listing stops at a million
groups and says how many of the claimed total it showed. It does not grow to a count no
image behind it could hold. The JSON report carries the same fact as "groups_complete".
A scan reads an image it has no reason to trust, so what it collects is bounded. It stops
at ten thousand findings and says so, in the table, in the "truncated" field of the JSON
report, and as a SARIF notification. A truncated report is a floor. The image holds at
least these findings, and the rest of it went unread, so the verdict it reaches reads
“at least n findings”. Ten thousand findings is far past what a filesystem needs to be
called bad.
--groups adds every block group’s descriptor. --json reports the same data as a JSON
document, shaped as the same envelope the table is. That is a head which means the same
thing whatever the image holds, then a body named for the family:
$ ferrosys inspect --json rootfs.img
{"schema":3,"family":"ext","variant":"ext4","size":67108864,"allocation_unit":4096,
"identifier":"f0e17055-...","offset":0,"findings":{"schema":3,"clean":true,"count":0,
"truncated":false,"findings":[],"source_failures":[]},"ext":{"superblock":{...},
"features":{...},"journal_recovery":{...}}}
The head is six fields and the findings. The body under "ext" carries the superblock, with
its clean and errors state. It also carries the feature names split by word, the unknown
feature bits, the journal recovery, and with --groups every group descriptor. The unknown
bits are reported whether or not there are any. An image carrying a feature the tool does
not know can therefore never read as one it understood.
A consumer that reads only the head never learns what a block group is. One that wants
ext4’s geometry reads the body. Each family’s body sits under its own key below the same head:
"ext", "fat", "exfat", "btrfs", or "apfs".
The two dialects differ in one way on purpose. The table omits a row it has nothing to say in, because a person reading a report wants what is there. An image with no journal grows no journal rows.
The JSON body’s shape is fixed instead. The same keys are present for every image of a family, carrying the value the filesystem holds. A consumer therefore indexes a key without first asking whether this image has one. That is why the document carries a field or two the table does not print.
Every finding carries its severity, the family that found it, that family’s own word for the
subsystem, and the byte offset when there is one. It carries that family’s own coordinates
as well. Those are {"group":3,"inode":12,"block":40} for an ext image, and whatever
addresses a finding in some other format.
--sarif reports the scan’s findings as a SARIF 2.1.0
log, so a static-analysis or forensic pipeline can consume them as it would any other
tool’s. It reports those findings alone,
and not the superblock description.
Each finding becomes one result. Its severity maps to the SARIF level, where structural
and integrity are error, conformance is warning, and cosmetic is note. The exact
severity travels in the result’s properties, together with the family, the subsystem, the
byte offset, and the coordinates it sits at. A rule is named family/subsystem, so two
formats that both call something a directory do not merge into one rule.
--sarif reports scan findings, so it runs the scan and cannot be combined with --quick.
It selects a different output format from --json:
$ ferrosys inspect --sarif rootfs.img > findings.sarif
--offset reads a filesystem that begins partway into a larger file — a partition inside
a whole-disk image:
$ ferrosys inspect --offset 1M disk.img
The JSON document carries that coordinate back as offset, the same field detect --json
carries. A caller that scanned a disk and then described what it found lines the two
documents up by it.
--length says where the partition ends. Nothing past that end is read, so a filesystem
longer than its partition is reported as truncated. Without it, such a filesystem is read on
into the next partition, and what is read there is taken as its own:
$ ferrosys inspect --offset 1M --length 16M disk.img
detect, extract and identity take the same pair.
extract
Exactly one of five things comes out.
Whatever the family, what comes out is the filesystem rather than one of its parts. On a btrfs that is worth saying plainly, because the format has parts a path could stop at and does not. A subvolume is a filesystem tree of its own, with inode numbers that start again from the same value the top-level tree’s do. Every mode here crosses that boundary the way it crosses a directory.
/home/user/notes reaches the file whether or not /home is a subvolume, and a listing of
the image names everything in every subvolume once. Symbolic links are resolved through as
well. A path continuing through /bin on a tree where that is a link into /usr therefore
reaches what it names.
A btrfs also stores a file’s bytes compressed where a mount was told to. Every distribution
that defaults to this filesystem tells it to for at least some of the tree.
This binary carries the three decoders the format defines, which are DEFLATE, LZO1X, and
Zstandard. A file stored that way comes out as the file rather than as a refusal. The
checksums are checked either way. They cover the bytes the filesystem stored, so inspect
verifies a compressed extent without expanding it.
An APFS container is read one volume at a time, since each volume is a filesystem of its own
and none is inside another. --volume names it by name, by the identifier a report prints,
or by slot. Without it, the first volume this tool reads is read, as inspect chooses.
--snapshot reads that volume as one of its snapshots keeps it:
$ ferrosys extract disk.img --volume Data --to-tar - | tar -tv
$ ferrosys extract disk.img --volume Data --snapshot before-upgrade --cat /notes.txt
A file macOS compressed comes out as its content. This binary carries the zlib, LZVN, LZFSE and LZBITMAP decoders its chunks are stored with. It also reads the chunks macOS keeps as their content behind a marker. A compression type it does not decode is refused by its number, naming the file.
A tar archive
Ownership, modes, symlinks, hard links, and device and FIFO nodes travel in the header. The paths, the ids, the times (to the nanosecond, and negative for a file older than the epoch), the extended attributes, and the POSIX ACLs travel in PAX records, because the header cannot hold them. GNU tar and bsdtar both read it.
$ ferrosys extract rootfs.img --to-tar rootfs.tar
$ ferrosys extract rootfs.img --to-tar - | tar -tv
The archive opens with a ./ member describing the root directory and omits an ext
filesystem’s /lost+found, which the format makes for itself. On every other family a
directory of that name is the volume’s own and is written like any other. What comes out is what
format --from-tar reads back in, so a filesystem survives a round trip through the
archive unchanged.
A socket is the one kind of node tar cannot express, because it has no entry type for one. An image holding a socket is a typed error rather than an archive quietly missing a file.
A tar header has no field for the BSD flags a Darwin filesystem records, UF_HIDDEN and the
rest. The archive is written without them, and the standard error counts the entries that had
any, as Not kept: bsd-flags.
A device number goes into the header exactly. Seven octal digits hold numbers to 2097151. Past that the field is filled with eight, as bsdtar writes it and GNU tar reads it. Eight digits hold a Mac’s twenty-four-bit minor numbers. A number past 16777215 fits neither form. An image holding one is a typed error rather than an archive naming another device.
That refusal comes part-way through the walk, as an inode that does not read does, and as an
ACL that does not decode does. A named destination is created and truncated before the walk
starts. --atomic covers that, as it does for format. The archive is written to a sibling
temporary file, and renamed over the destination once the walk is complete. A walk that
fails therefore leaves whatever was at that path untouched.
--atomic applies to --to-tar FILE, the only mode with a destination to rename into.
Asking for it anywhere else is refused rather than accepted and ignored.
The image is read while the archive is written. So an archive written in place over the
image is refused, by any name, as format refuses a destination it reads.
$ ferrosys extract rootfs.img --to-tar rootfs.tar --atomic
A directory tree
This is format --from-dir in reverse:
$ ferrosys extract rootfs.img --to-dir unpacked
Names written: 1284
The destination is made if it is not there, and it must be empty. An extraction states what
the filesystem holds. A name already present would be an entry that could not be created,
found part-way through with the tree half written. The destination takes the filesystem
root’s own mode, ownership, times, and extended attributes. Everything the filesystem holds
appears beneath it at the path it holds inside the image. An ext filesystem’s /lost+found
is not written, so the tree is one format --from-dir reads straight back.
Everything the archive carries is carried here too, set on the files themselves. That is modes, ownership, symlinks, hard links, device and FIFO nodes, extended attributes, and POSIX ACLs, and sockets besides, which an archive refuses. Two things no host lets a caller set, so the tree carries the time it was written for those two alone. They are an inode’s change time and its creation time. Access and modification times are set to the nanosecond, and held to what the destination keeps.
Four parts of a tree need privileges:
- A device node needs
CAP_MKNOD. - Setting a recorded owner needs
CAP_CHOWN, and in a user namespace an id the namespace maps. - An extended attribute in the
securityortrustednamespace is the host’s to write. - A destination filesystem with no notion of a second name for a node refuses a hard link.
An unprivileged run therefore stops at the first of them and names it. That is the right
answer for a tree meant to be faithful. A rootfs quietly missing /dev/null is a rootfs
that boots differently, and one whose ping lost its security.capability is one that no
longer runs unprivileged.
--skip-privileged is the opt-in for a run that wants what it can have. What it left out is
named on the standard error rather than assumed:
$ ferrosys extract rootfs.img --to-dir unpacked --skip-privileged
Names written: 1282
Ownership: not applied — this process may not set another owner
Attribute not set: security.capability
Not kept: special-bits (12 entries)
Skipped: /dev/console
Skipped: /dev/null
A tree written without its owners is owned by the process that wrote it. A set-user-ID or set-group-ID bit there would make a program run as that process. So a file whose recorded owner was not set is written without those two bits. A directory keeps its set-group-ID, which runs nothing.
The reserved attributes are what an unprivileged extraction of a real root filesystem meets
first. A Debian tree carries security.capability on the binaries that hold one, and a tree
built under SELinux carries security.selinux on nearly every inode. Each attribute is named
once, however many entries carry it.
Some of a tree has no place on the destination, whatever privilege the process holds. A name can be longer than the destination takes. A format storing up to 255 UTF-16 units holds names of up to 765 bytes of UTF-8, and a Linux filesystem takes 255 bytes.
An extended attribute can be one the destination will not hold. Another operating system names
its attributes in no namespace this host knows, such as com.apple.quarantine. The kernel
holds a user attribute only on a regular file or a directory. It takes at most 64 KiB for
one value. A destination also keeps a room of its own for a node’s attributes, and a large
value can be past it.
A device node can record a number wider than this host’s, as another operating system’s can. The node is read back once made. One the host made as another device is taken out again rather than left naming it.
A destination also keeps a time to a range and a fraction of its own. So every node’s times are read back too, and one held as another time is refused like the rest. Each limit is the destination’s own. By default the first refusal stops the run and names the entry.
No destination holds the BSD flags a Darwin filesystem records, so an entry carrying any is refused like the rest.
--skip-unsupported is the opt-in for a run that wants what the destination will hold. An
entry whose name is refused is left out, and a directory takes everything under it along. Each
is named as Skipped:. An attribute the destination refuses is not set, and is named once as
Attribute not set:. A destination that holds more leaves out less. --to-tar keeps every
name and attribute either opt-in leaves out.
Under the same opt-in, BSD flags are not set. The entries that had any are counted as
Not kept: bsd-flags.
Under the same opt-in, a time the destination holds as another is kept as it holds it. The
entry is named as Time not held:.
A file’s contents are written as the filesystem reports them, and a hole reads as zeros. A sparse file therefore lands in the destination fully allocated. A tree holding one occupies more space on the host than it does in the image.
A directory’s mode is applied once its contents are in place, so one the image records as read-only still receives them. One recorded without owner-search permission waits until the whole run is done. A second name for a hard-linked file is written by traversing to the first, and a directory that cannot be searched cannot be traversed. Applying such a mode early is what would make an ordinary user’s extraction of an ordinary image fail part-way.
The image is untrusted input, and a name in it is not a path to resolve. Every directory is
created and then opened, and everything beneath it is created through that open handle by
its single-component name. A name holding a separator, a .., or a NUL is therefore refused
rather than followed, and nothing lands outside the destination.
The reader refuses such a name too, where it resolves it. Both --to-dir and --to-tar are
therefore safe against one, and neither depends on the other’s check. No path through the
destination tree is walked a second time. Nothing swapped into it while a run is in flight
can redirect a write. Symbolic links are written exactly as the image records them,
absolute targets and all, which is safe precisely because nothing here ever follows one.
There is no --atomic for a tree: no rename publishes a whole tree at once, and inventing
one would promise something the run cannot do. The empty destination is what stands in its
place. A failure part-way leaves a partial tree in a directory that held nothing, rather
than mixed into one that did.
How strictly the image is held to its format
Extraction is the one command whose output is the image’s contents. What it makes of a filesystem it does not fully understand therefore matters more here than anywhere else. An image carrying a feature this reader does not follow can be interpreted best-effort, and the result looks complete without being it.
So the strict read is tried first, always. --strict makes its refusal the answer. Without
it, the read falls back to a lenient one, which is what makes a damaged or unfamiliar image
recoverable at all. That read names on the standard error the deviation it decided to
interpret through:
$ ferrosys extract odd.img --to-dir unpacked
ferrosys: reading odd.img leniently: ext image refused: unsupported incompatible feature bits 0x00001000: the reader cannot be certain it interprets the on-disk format correctly
ferrosys: what it holds is interpreted best-effort; --strict refuses instead
A run that says nothing read an image it could hold to its format entirely.
What a filesystem does not record
--assume-owner U:G and --assume-modes F:D say what to record where the filesystem being
read has no field for it. They change nothing about an ext image. ext records an owner, a
mode, and three times on every entry, so what comes out is what was stored. They change
nothing about a btrfs either, which records those and a creation time besides.
They exist because a FAT volume does not. A format with no notion of an owner still has to
become host files with some owner, and something has to decide which. Every driver that
mounts such a format has uid=, gid=, fmask=, and dmask= mount options for exactly
this reason. Here the same decision is a flag, and its default is the conservative one. That
default is root ownership, 0644 for a file and 0755 for a directory, and never anything
more permissive. A tree that landed world-writable because nothing was named would be a
security bug made out of a format limitation.
Whatever was assumed is reported on the standard error beside what the host refused. An extraction therefore says which parts of the tree came from the image, and which were policy:
Assumed: ownership (1282 entries)
Assumed: permissions (1282 entries)
Assumed: change-time (1282 entries)
The properties are named in the same words format --accept-loss reads. A property an
extraction reports can therefore be typed straight back into the build that would preserve
it.
One file’s bytes
Nothing else goes to the standard output:
$ ferrosys extract rootfs.img --cat /etc/hostname
ferrosys
The path is a path inside the image, taken as the bytes you typed, because an ext4 name
need not be text. Symbolic links are followed against the image’s own root, so /lib/modules
resolves on a merged-/usr tree where /lib is a link into /usr. A .. component
ascends, whether you wrote it or a link in the image stores it. /usr/lib64 -> ../lib is
the shape a multiarch root filesystem has.
At the root there is nothing to ascend to, so a run of them stays there. Every path names something inside the image, and nothing here reaches the machine reading it.
A listing
$ ferrosys extract rootfs.img --list
drwxr-xr-x 2 0 0 4096 2023-11-14T22:13:20Z /etc
-rw-r--r-- 1 1000 1000 9 2023-11-14T22:13:20Z /etc/hostname
lrwxrwxrwx 1 0 0 17 2023-11-14T22:13:20Z /etc/mtab -> /proc/self/mounts
--list --json produces the same listing as a JSON document. It adds each entry’s inode
number, its extended attributes, and any POSIX ACL decoded into readable entries. The inode
number is what tells one file with two names from two files with the same contents.
A field the family has no notion of is absent rather than null or zero. A FAT entry
carries no inode and no links, because the format has neither inode numbers nor a
second name for a node. A one there would be this tool answering a question the format never
asked. What it does carry is a synthesized list naming the properties the report filled in
rather than read. That list is always present, empty or not, so “the image recorded
everything” and “this document did not say” stay distinguishable. In the table the
link-count column reads - for such a family, which keeps the columns where a reader
expects them.
One path’s metadata
This is the answer to “what is /usr/bin/ping, exactly” without listing a hundred thousand
other lines:
$ ferrosys extract rootfs.img --stat /usr/bin/ping
Path: /usr/bin/ping
Inode: 15
Type: file
Mode: 0755 (-rwxr-xr-x)
Owner: 0:0
Links: 1
Size: 76672
Blocks: 152
Accessed: 2023-11-14T22:13:20Z (0 ns)
Modified: 2023-11-14T22:13:20Z (0 ns)
Changed: 2023-11-14T22:13:20Z (0 ns)
Created: 2023-11-14T22:13:20Z (0 ns)
Xattr security.capability: \x01\x00\x00\x02\x00 \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
It reports the type, the mode both ways, ownership, size, and times. Where the family
records them, it also reports the inode number, the link count, a device node’s numbers, a
symlink’s target, and every extended attribute. A stored POSIX ACL is decoded rather than
shown as bytes. A path naming a symlink describes the link itself, not its target. --json
reports the same as a document.
On a family that records fewer of them the report is shorter, and it says what it assumed:
$ ferrosys extract esp.img --stat /EFI/BOOT/BOOTX64.EFI
Path: /EFI/BOOT/BOOTX64.EFI
Type: file
Mode: 0644 (-rw-r--r--)
Owner: 0:0
Size: 76672
Accessed: 2023-11-14T00:00:00Z (0 ns)
Modified: 2023-11-14T22:13:20Z (0 ns)
Changed: 2023-11-14T22:13:20Z (0 ns)
Created: 2023-11-14T22:13:20Z (640000000 ns)
Assumed: ownership, permissions, change-time
Three things to read out of that. There is no Inode: and no Links: line, because the
format has neither. The access time is midnight, because FAT records a date and no time of
day for it. And the change time equals the modification time, because the format records no
change time at all and the closest thing it has stands in. That is what the Assumed:
line is there to say.
An attribute’s value is bytes, and it is rendered the way a name is. Printable characters go
out as themselves, and everything else as a \xNN escape, with the backslash escaping
itself so the rendering names exactly one value.
A bidirectional formatting character is escaped too, as \u{202e}. It is not a control
character, and left alone it reorders the rest of the line, so photo\u{202e}gnp.exe would
display as photo exe.png. To read the bytes rather than look at them, take --json. A
value that is not text carries a value_hex field beside it holding the bytes exactly. The
field’s presence is itself the signal that the value is not text.
A JSON document escapes both classes as well, as the \uXXXX escape the grammar defines. A
parser therefore reads back the character the name held, and nothing between the document
and the parser acts on it. The same rule covers every name a document carries: a path, a
symlink target, an attribute name, a volume label.
In a JSON document the mode field is the permission bits as a decimal number, since
JSON has no octal literal, so 509 is 0o775. mode_octal beside it carries the usual
spelling.
--stat reports more of an entry than --list does. It adds a creation time and a device
node’s numbers. Where the family records them, it adds how many blocks the entry occupies and
its BSD flags. A listing leaves those out on every family, including the families that record
them. Reading them costs a read per entry, and a listing is the shape a caller uses to walk a
whole tree. Extended attributes are in --stat and --list --json, and the table listing
omits them.
So a field’s absence from a listing entry means “a listing does not carry this”. A field’s
absence from --stat means “this filesystem does not record it”. When the question is what
one path holds, --stat is the answer that leaves nothing out.
Reading an image you do not trust
A file’s size is the image’s own claim about it, and a sparse file legitimately dwarfs the filesystem holding it. Nothing structural therefore bounds what a read of one would write. An inode claiming sixteen tebibytes and mapping nothing costs an extraction sixteen tebibytes of zeros, from an image of a hundred kilobytes. A hole reads back as zeros and occupies nothing.
So extract caps it whether or not you ask. The default is sixteen times the length of
the filesystem being read, which no ordinary file approaches and no crafted one survives.
--max-file-bytes N names a different cap, for an image that holds a file legitimately
sparser than that:
$ ferrosys extract suspect.img --cat /etc/passwd --max-file-bytes 64M
$ ferrosys extract sparse.img --to-dir out --max-file-bytes 512G
There is no spelling for “no cap”, and none is needed. The cap is a size, so a run that means to read a file of some size names that size.
Over the cap the read is an error, not a short file. A truncated file that looked whole would be the worse outcome, because a pipeline would carry it forward and never know. Where the cap that stopped a read was the default rather than one you named, the command says so on the standard error. It also says what raises it.
The cap is the same cap in every mode and on every family. --cat streams a file into the
standard output and holds nothing, and it answers to the cap all the same. What a stream
writes follows the length the image declares, so a mode that streamed without the cap
would be the one way past it. A flag that meant something different depending on which
family answered would be the worst of both.
A walk holds one more bound, and this one has no flag: the longest path it builds, 4,096
bytes, which is Linux’s PATH_MAX. A tree nested deeper is well-formed in every format, and
inspect reads it whole. It names what it finds below the bound by the path’s last
components. --list, --to-tar and --to-dir hand out whole paths, so each stops at the
first path past the bound. The message names the limit, and the run exits 8.
detect
$ ferrosys detect rootfs.img
ext4
$ ferrosys detect --offset 1M disk.img
ext2
$ ferrosys detect esp.img
fat32
One word goes to the standard output, so it reads well in a shell test. That word is
ext2, ext3, ext4, fat12, fat16, fat32, exfat, btrfs, apfs, or
unrecognized. --json produces a document instead.
For every family it is the same word format -t takes. What detect says about an image is
therefore what you would type to write another like it. --offset points it at a partition
inside a whole-disk image, or at a region a carver located.
One further answer, unknown, is what a tool prints when the library classifies a family
that build has no name for. Something recognized the image, so unrecognized would be the
wrong word for it.
The families are tried in order of how distinctive their magic is. ext, and any family with
a multi-byte magic at a fixed offset, are classified first. FAT is classified last and never
by its boot signature. 0xAA55 at the end of sector 0 appears on every bootable sector
ever written. That includes the master boot record of a disk whose partition holds an
ext4 filesystem.
A FAT volume is claimed only when the whole parameter block is internally consistent. That is what keeps a healthy filesystem of another family from being misidentified as this one.
This asks what an image is, not whether it is sound. An image with a quirk inspect
would refuse still classifies here. An unrecognized image exits 8, since there is no
filesystem to have an opinion about.
identity
$ ferrosys identity --uuid "$(uuidgen)" --label rootfs rootfs.img
3 superblock copies written, journal superblock updated
Changes what an existing ext filesystem is known by: its UUID, its volume label, or the seed its metadata checksums derive from. It is the one command that writes to an image it did not create, and the only one whose destination already holds something worth keeping.
Every superblock copy is rewritten, the primary and each group’s backup alike. So is the journal’s own record of the UUID. No copy is left claiming the old identity. Each copy is patched where it lies, and keeps every field this change does not name, including the ones this tool has no opinion about.
Nothing is written until every copy has been read and every check has passed, so a refusal
leaves the image exactly as it was. There is no --atomic. An image is rewritten in place,
and a sibling temporary file would mean copying every byte of it to change sixteen.
A run cut short partway through the writing is finished by running the same command again. That holds whichever copies reached the disk, which after a power loss is any subset of them. The second run leaves the image byte for byte as one uninterrupted run would have. Success is reported only once the writes are on the disk.
The journal keeps its own copy of the UUID, and on most images a checksum over it. Linux
sets csum_v3 on the log of any metadata_csum filesystem the first time it mounts one. An
image that has ever been used therefore carries a crc32c covering the whole journal
superblock. A rescue image being cloned is one such image, and so is a rootfs that was
booted once and is now being re-stamped per device. That word is recomputed with the UUID,
so the log stays one the kernel will load.
A journal whose stored checksum does not match its contents is refused, as a damaged filesystem superblock is. Writing a correct checksum over wrong bytes would replace a fault a checker finds with one it does not.
A filesystem needing journal recovery is refused too. Recovery at its next mount would write
the journal’s copy of the superblock back over the new identity. Mount it, or run e2fsck,
first.
At least one of --uuid, --label, and --set-checksum-seed is required. A run that would
write nothing is a command line that meant to say something.
--offset and --length point it at a filesystem inside a whole-disk image, as every
image-reading verb takes them. Nothing past the length is read or written. This command
rewrites ext identity fields. Pointed at a sound volume of another family, it refuses with
the word detect prints for what the image holds. The volume is fine and the request does
not apply to it. That is a different verdict from a damaged filesystem, and it carries a
different exit code.
When the UUID is the checksum seed
$ ferrosys identity --uuid "$(uuidgen)" legacy.img
ferrosys: legacy.img: changing the UUID would invalidate every metadata checksum: this filesystem
has metadata_csum without metadata_csum_seed, so its checksums are seeded from the UUID
itself — set the checksum seed to keep them valid
$ ferrosys identity --uuid "$(uuidgen)" --set-checksum-seed legacy.img
1 superblock copies written, journal superblock updated, metadata_csum_seed set
Under metadata_csum, every metadata object in the filesystem carries a checksum seeded
from the filesystem’s seed. Those objects are each group descriptor, inode, bitmap,
directory block, and extent node. Where metadata_csum_seed is set, that seed is a
superblock field and the UUID is free to move. Where it is not, the seed is the UUID, so
changing the UUID invalidates every checksum in the image at once.
That is refused rather than half-performed. --set-checksum-seed is the way through. It
records the seed the current UUID implies and turns metadata_csum_seed on, after which
the UUID moves and every existing checksum stays valid. It is asked for rather than assumed
because it sets an incompatible feature. A kernel that does not know metadata_csum_seed
will not mount the result.
Images ferrosys format writes carry metadata_csum_seed already, so this is a question
about filesystems made elsewhere.
A round trip, end to end
$ tar -cf rootfs.tar -C rootfs .
$ ferrosys format --size 512M --uuid "$(uuidgen)" --time 1700000000 \
--from-tar rootfs.tar rootfs.img
$ ferrosys inspect rootfs.img
$ ferrosys extract rootfs.img --to-tar - | tar -tv
Or with a tree at both ends, and the size taken from the tree rather than named:
$ ferrosys format --size auto --slack 20% --uuid "$(uuidgen)" --time 1700000000 \
--from-dir rootfs --owner 0:0 rootfs.img
$ ferrosys extract rootfs.img --to-dir unpacked --skip-privileged
Every time is UTC, printed as YYYY-MM-DDTHH:MM:SSZ. The tool has no time zone and no
locale.