Showing posts with label tar. Show all posts
Showing posts with label tar. Show all posts

Wednesday, November 8, 2017

Parallel compression and decompression command

There is option for tar program:
option c for compression, x for extraction.
-I, --use-compress-program PROG
      filter through PROG (must accept -d)
You can use multithread version of archiver or compressor utility.
Most popular multithread archivers are pigz (instead of gzip) and pbzip2 (instead of bzip2). For instance:
$ tar -I pbzip2 -cf OUTPUT_FILE.tar.bz2 paths_to_archive
$ tar --use-compress-program=pigz -cf OUTPUT_FILE.tar.gz paths_to_archive
Archiver must accept -d. If your replacement utility hasn't this parameter and/or you need specify additional parameters, then use pipes (add parameters if necessary):
$ tar cf - paths_to_archive | pbzip2 > OUTPUT_FILE.tar.gz
$ tar cf - paths_to_archive | pigz > OUTPUT_FILE.tar.gz
Use lbzip2

$ time tar cf tmp.tar.bz2 tmp --use-compress-program=lbzip2
$ time tar xf tmp.tar.bz2 --use-compress-program=lbzip2