Kenneth at 23:02, 8 February 2016

2016-02-08T23:02:47Z

New page

== File System Support ==

"Unixy" filesystems : ext2, ext3, ufs, xfs, reiserfs, jfs,... all support Sparse files

FAT16 and FAT32 do not

NTFS does support Sparse files

References:
* Sparse files - MLDonkey - http://mldonkey.sourceforge.net/Sparse_files#FAT16.2C_FAT32
* File System Functionality Comparison (Windows) - https://msdn.microsoft.com/en-us/library/windows/desktop/ee681827%28v=vs.85%29.aspx

== Sparse File ==

[http://en.wikipedia.org/wiki/Sparse_file Wiki: Sparse Files]

In computer science, a sparse file is a type of computer file that attempts to use file system space more efficiently. When space has been allocated to a file but not actually filled with data it is not written to the file system. Instead, meta-information about these "empty" regions is stored until they are filled with data.

The obvious advantage of sparse files is that storage is only allocated when actually needed. Large files can be created even if there isn't enough free space yet. A disadvantage is that sparse files can become very fragmented. Also, filling up partitions to the maximum can have unpleasant effects.

Will create a file of one megabyte in size, but with only one byte actually stored on disk:
dd if=/dev/zero of=sparse-file bs=1 count=1 seek=1M

== Sparse Tools ==

View sparse size (in blocks)
# both will report the full file size, but the block size will show sparse size
ls -ls [file]
stat [file]

du -s -B1 --apparent-size sparse-file
du -s -B1 sparse-file

tar
-S, --sparse # handle sparse files efficiently

cp - # control creation of sparse files
# cp tries to detect sparse files automatically. This control can be forced
cp --sparse=WHEN [...]
# --sparse=auto
# --sparse=always
# --sparse=never

man cp:
By default, sparse SOURCE files are detected by a crude heuristic and
the corresponding DEST file is made sparse as well. That is the behav-
ior selected by --sparse=auto. Specify --sparse=always to create a
sparse DEST file whenever the SOURCE file contains a long enough
sequence of zero bytes. Use --sparse=never to inhibit creation of
sparse files.

This is especially useful if a sparse-file has somehow become non-sparse (i.e. the empty blocks have been written out to disk in full). Disk space can be recovered by doing:
cp --sparse=always formerly-sparse-file recovered-sparse-file

rsync
-S, --sparse # handle sparse files efficiently
rsync -aP --sparse local-file remote-host:remote-file

cpio
--sparse

Not supported tools:
# Using such a utility would make a sparse file balloon in size.
pax, scp, sftp, and ftp

== keywords ==

sparse file

[[Category:Linux]]

Sparse File - Revision history

Kenneth at 23:02, 8 February 2016