Friday, November 17, 2017

Mount a hard disk in ubuntu + reboot (may have different filesystem type)

1. $ sudo fdisk -l
display device
Disk /dev/sdc: 2 TiB, 2197949513728 bytes, 4292870144 sectors

2. $ mkdir /media/DiskC
Better use /media instead of /mnt, because /mnt is usually the automatic mount point, may be replaced when restarting.

3 $ mount /dev/sdc /media/DiskC
Then error occured
mount: wrong fs type, bad option, bad superblock on /dev/sdc,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

(for several filesystems (e.g. nfs, cifs) you might need a /sbin/mount. helper program)
This is relevant given that you are trying to mount NFS. The /sbin/mount.nfs helper program is provided by nfs-common. You can install it with:
sudo apt install nfs-common
On the other hand, if you are trying to mount CIFS, the helper program is provided by cifs-utils. You can install it with:
sudo apt install cifs-utils
4. convert file type
$ sudo mkfs.ext4 /dev/sdc

goto step 3.
Done!


Auto mount on start up.

  1. [IMPORTANT] sudo cp /etc/fstab /etc/fstab.old - Create a backup of the fstab file just in case something unwanted happens.
  2. sudo blkid - Note the UUID of the partition you want to automount. 
    ls -al /dev/disk/by-uuid/
    lrwxrwxrwx  1 root root  10 Sep  2 17:45 842c5c25-xxxx-xxxx-xxxx-6da95691d994 -> ../../sdb1
    lrwxrwxrwx  1 root root  10 Sep  2 17:45 b1c5d9e4-xxxx-xxxx-xxxx-5d4a0e0ffd5a -> ../../sdc1
    lrwxrwxrwx  1 root root   9 Sep  2 17:45 ceedf9aa-xxxx-xxxx-xxxx-863293374348 -> ../../sdd
    lrwxrwxrwx  1 root root  10 Sep  2 17:46 fc9a41df-xxxx-xxxx-xxxx-e5999bd67182 -> ../../sda1
  3. sudo nano /etc/fstab - Copy the following line to the end of the file, save it and reboot afterwards to check if it worked.

Examples

A common setup is:
UUID=<uuid> <pathtomount> <filesystem> defaults 0 0
Use lsblk -o NAME,FSTYPE,UUID to find out the UUIDs and filesystems of the partition you want to mount. For example:
$ lsblk -o NAME,FSTYPE,UUID
NAME   FSTYPE UUID
sda
├─sda2
├─sda5 swap   498d24e5-7755-422f-be45-1b78d50b44e8
└─sda1 ext4   d4873b63-0956-42a7-9dcf-bd64e495a9ff

NTFS

UUID=<uuid> <pathtomount> ntfs uid=<userid>,gid=<groupid>,umask=0022,sync,auto,rw 0 0
Examples for the <> variables:
  • <uuid>=3087106951D2FA7E
  • <pathtomount>=/home/data/
  • <userid>=1000
  • <groupid>=1000
Use id -u <username> to get the userid and id -g <username> to get the groupid.

# Mount all disks
sudo mount -a

Wednesday, November 8, 2017

Parallel compression and decompression command

There is option for tar program:
option c for compression, x for extraction.
-I, --use-compress-program PROG
      filter through PROG (must accept -d)
You can use multithread version of archiver or compressor utility.
Most popular multithread archivers are pigz (instead of gzip) and pbzip2 (instead of bzip2). For instance:
$ tar -I pbzip2 -cf OUTPUT_FILE.tar.bz2 paths_to_archive
$ tar --use-compress-program=pigz -cf OUTPUT_FILE.tar.gz paths_to_archive
Archiver must accept -d. If your replacement utility hasn't this parameter and/or you need specify additional parameters, then use pipes (add parameters if necessary):
$ tar cf - paths_to_archive | pbzip2 > OUTPUT_FILE.tar.gz
$ tar cf - paths_to_archive | pigz > OUTPUT_FILE.tar.gz
Use lbzip2

$ time tar cf tmp.tar.bz2 tmp --use-compress-program=lbzip2
$ time tar xf tmp.tar.bz2 --use-compress-program=lbzip2

Python locale error: unsupported locale setting

When using pip install, the local error problem occur.
export LC_ALL="en_US.UTF-8"
export LC_CTYPE="en_US.UTF-8"
sudo dpkg-reconfigure locales
Then select en_US.UTF-8 to install

Occurred when install virtualenv.