SQUASHFS 4.3 - A squashed read-only filesystem for Linux
Copyright 2002-2014 Phillip Lougher <[email protected]>
Released under the GPL licence (version 2 or later).
Welcome to Squashfs 4.3. This is the first release in over 3 years, and
there are substantial improvements to stability, new compression options
and compressors, speed optimisations, and new options for Mksquashfs/Unsquashfs.
This is a tools only release, support for Squashfs filesystems is
in mainline (2.6.29 and later).
Changes in Squashfs-tools 4.3
-----------------------------
1. Stability improvements. Better checking of user input for out of
range/invalid values. Better handling of corrupted Squashfs filesystems
(Mksquashfs append mode, and Unsquashfs). Better handling of buffer
overflow/underflow.
2. GZIP compressor now supports compression options, allowing different
compression levels to be used.
3. Rewritten LZO compressor with compression options, allowing different
LZO algorithms and different compression levels to be used.
4. New LZ4 compressor (note not yet in mainline kernel)
5. Better default memory usage for Mksquashfs. Mksquashfs by default now
uses 25% of physical memory.
6. Duplicate checking in Mksquashfs further optimised. With certain
"problem filesystems" greater than 2x performance improvement.
Filesystems with a lot of duplicates should see at least 10-20% speed
improvement.
7. The -stat option in Unsquashfs now displays the compression options
used to generate the original filesystem. Previously -stat only displayed
the compression algorithm used.
8. The file being compressed/uncompressed in Mksquashfs/Unsquashfs is now
displayed if CTRL-\ (SIGQUIT from keyboard) typed.
9. The status of the internal queues/caches in Mksquashfs/Unsquashfs is
now displayed if CTRL-\ (SIGQUIT from keyboard) is typed twice within
one second. Normally only useful for "power users", but it can be
used to discover if there‘s any bottlenecks affecting performance
(the bottleneck will normally be the compressors/fragment compressors).
10. Miscellaneous new options for Mksquashfs/Unsquashfs to fine tune behaviour.
11. Fixes for CVE-2012-4024 and CVE-2012-4025.
Compatiblity
------------
Mksquashfs 4.3 generates 4.0 filesystems. These filesystems are fully
compatible/interchangable with filesystems generated by Mksquashfs 4.0 and are
mountable on 2.6.29 and later kernels.
Compressors
-----------
New compression options and compressors are now supported.
The new options and compressors are:
1. gzip
-Xcompression-level <compression-level>
<compression-level> should be 1 .. 9 (default 9)
-Xwindow-size <window-size>
<window-size> should be 8 .. 15 (default 15)
-Xstrategy strategy1,strategy2,...,strategyN
Compress using strategy1,strategy2,...,strategyN in turn
and choose the best compression.
Available strategies: default, filtered, huffman_only,
run_length_encoded and fixed
2. lzo
-Xalgorithm <algorithm>
Where <algorithm> is one of:
lzo1x_1
lzo1x_1_11
lzo1x_1_12
lzo1x_1_15
lzo1x_999 (default)
-Xcompression-level <compression-level>
<compression-level> should be 1 .. 9 (default 8)
Only applies to lzo1x_999 algorithm
3. lz4
-Xhc
Compress using LZ4 High Compression
The compression specific options are, obviously, specific to the compressor
in question, and you should read the compressor documentation and check
their web sites to understand their behaviour.
In general the defaults used by Mksquashfs for each compressor are optimised
to give the best performance for each compressor, where what constitutes
best depends on the compressor. For gzip/xz best means highest compression
(trying multiple filters/strategies can improve compression, but this is
extremely expensive computationally, and hence, not suitable for the defaults),
for LZO/LZ4 best means a tradeoff between compression and (de)-compression
overhead (LZO/LZ4 by definition are intended for weaker processors).
New Mksquashfs options
----------------------
1. -mem <size>
Set the amount of memory used by Mksquashfs to <size> bytes. G/M and K
post-fixes are supported.
By default Mksquashfs uses 25% of the physical memory. Increasing
this with the -mem option can increase performance (note it does not have
any effect on compression). Reducing it can prevent thrashing if the
system is busy and there is not 25% of physical memory free (again, note
it does not have any effect on compression).
2. -exit-on-error
By default Mksquashfs treats certain errors as benign, if these
errors occur Mksquashfs prints the error on the console but continues.
These errors are typically failure to read a file from the source filesystem.
This is deliberate, in many cases users prefer Mksquashfs to flag
the error but continue rather than abort what may be hours of compression.
But there are times where failure to read any file is considered critical,
and users (especially in the case of automated scripts where the
errors output to the console may be missed) prefer Mksquashfs to exit.
The new -exit-on-error option can be used in this scenario. This option
makes Mksquashfs treat all benign errors as fatal.
3. -progress
By default if -info is specified, the progress bar is disabled as it gets
in the way. Occasionally you might want the progress bar enabled whilst
-info is enabled. This option forces Mksquashfs to output the progress
bar when -info is specified.
4. -Xhelp
Display the usage text for the currently selected compressor.
New Unsquashfs options
----------------------
1. -u[ser-xattrs]
Only write user xattrs. This forces Unsquashfs to ignore system xattrs.
This is useful when Unsquashing a filesystem as a non-root user, and the
filesystem contains system xattrs which are only writable by root.
Major bugs fixed
----------------
1. If Mksquashfs ran out of space in the destination filesystem, this
would not cause Mksquashfs to immediately abort, and Mksquashfs would
continue to process the source filesystem. Mksquashfs now immediately
aborts on out of space in the destination filesystem.
2. Unsquashfs ignored the maximum number of open files limit, and if that
was lower than the default limit for Linux, it would run out of file
descriptors. Unsquashfs now limits the number of open files to the
limit currently in force (e.g. specified by setrlimit).
3. If huge numbers of dynamic pseudo files were specified, Mksquashfs
could exceed the maximum number of open files limit. This was because
Mksquashfs created all the dynamic file processes up front before
commencing source filesystem reading and compression. Mksquashfs
now creates the dynamic file processes on demand whilst reading
and compressing the source filesystem, thus limiting the number of
dynamic pseudo file processes in existence at any one time.
4. When outputting Unsquashfs used to set the permissions of directories
as it recursively descended. This in hindsight had an obvious oversight,
if a directory had only read permission (or was otherwise restricted), then
Unsquashfs would fail to write its contents when descending into it. Fixed
by setting directory permissions as Unsquashfs recursively unwinds.