Postgres on Btrfs

Postgres performs random writes on big database files, which can cause huge performance impact on files systems featured with COW(Copy On Write) like btrfs and zfs.

A good practice is add "nodatacow" option to database volume in /etc/fstab

Run

mount <mount-point> -o remount

to apply the edited mount options.

Find the original user of a process invoked by sudo

Let's say you want to find out who kicked off the most memory consuming process using root.

1) open top.
2) sort by mem by pressing >
3) write down the pid
4) sudo vim /proc/<pid>/environ , you will find the the user in: SUDO_USER='\<user\>'

There are some other ways: http://unix.stackexchange.com/questions/7334/using-top-to-see-processes-run-by-a-user-on-behalf-of-sudo

Join odd and even numbered lines

sed:

sed '$!N;s/\n/,/'


awk:

awk 'NR%2==0 {print p","$0;} NR%2 {p=$0;}'


paste:

paste - -


--------------------------
Seems  I like paste most. To join 3 consecutive lines:

paste - - -

IO Redirection - Swapping stdout and stderr (Advanced)

% (sh myscript.sh 3>&2 2>&1 1>&3) 2>/dev/null
I'm stderr
% (sh myscript.sh 3>&2 2>&1 1>&3) >/dev/null 
I'm stdout

brackets, parentheses, curly braces in BASH

In Bash, test and [ are builtins.

The double bracket enables additional functionality. For example, you can use && and || instead of-a and -o and there's a regular expression matching operator =~.

The braces, in addition to delimiting a variable name are used for parameter expansion so you can do things like:

• Truncate the contents of a variable
  $ var="abcde"; echo ${var%d*}
abc
• Make substitutions similar to sed
  $ var="abcde"; echo ${var/de/12}
abc12
• Use a default value
  $ default="hello"; unset var; echo ${var:-$default}
hello
• and several more

List recently modified files under a directory in linux shell

In order to list files that have been modified recently, we could use the find command to retrieve the file information and sort them by modified date:

find . -type f -exec stat --format '%Y :%y %n' {} \; | sort -nr | cut -d: -f2- | head

However, this will miss some folders if the folders are symoblic links. So in this case we could specify find to follow symbolic links.

find -L . -type f -exec stat --format '%Y :%y %n' {} \; | sort -nr | cut -d: -f2- | head

Fourther more, if you just want to get files modified in last a few days, it's build in in find:

find . -mtime n

list files that modified n*24 hours ago.

The difference between soft link and hard link in linux

From: http://linuxgazette.net/105/pitcher.html

Unix files consist of two parts: the data part and the filename part.

The data part is associated with something called an 'inode'. The inode carries the map of where the data is, the file permissions, etc. for the data.

                               .---------------> ! data ! ! data ! etc
                              /                  +------+ !------+
        ! permbits, etc ! data addresses !
        +------------inode---------------+

The filename part carries a name and an associated inode number.

                         .--------------> ! permbits, etc ! addresses !
                        /                 +---------inode-------------+
        ! filename ! inode # !
        +--------------------+

More than one filename can reference the same inode number; these files are said to be 'hard linked' together.

        ! filename ! inode # !
        +--------------------+
                        \
                         >--------------> ! permbits, etc ! addresses !
                        /                 +---------inode-------------+
        ! othername ! inode # !
        +---------------------+

On the other hand, there's a special file type whose data part carries a path to another file. Since it is a special file, the OS recognizes the data as a path, and redirects opens, reads, and writes so that, instead of accessing the data within the special file, they access the data in the file named by the data in the special file. This special file is called a 'soft link' or a 'symbolic link' (aka a 'symlink').

command sort on last column

As you know, the linux sort command is agile, which can sort on column using -k.

However, if we gonna sort on the last column and each row has different number of columns, we can still achieve that with the help of sed.

sed 's/^\(.\+\s\+\)\([0-9]\+ *\)$/\2 \1/'

zsh cd tab completion slow

It is quite annoing that zsh seems stuck when completion using tab in a huge directory.

To speed up the zsh completion, append the following lines to ~/.zshrc

# Speed up the zsh completion

zstyle ':completion:*' accept-exact '*(N)'
zstyle ':completion:*' use-cache on
zstyle ':completion:*' cache-path ~/.zsh/cache


Sometimes, the directory auto completion works slow under git and svn directories.The solution is to disable it from getting git info and svn info:

For GIT, add following to .zshrc:


__git_files () {
    _wanted files expl 'local files' _files  
}


For SVN, append this line to .zshrc:

compdef -d svn

SSH without manually input password

On client side generate the ssh key:


client$ cd ~/.ssh
client$ ssh-keygen -q -f id_rsa -t rsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:

Copy the public key to the server:


# Step 1:
client$ scp ~/.ssh/id_rsa.pub user@your.ssh.server:~/.ssh/tmpfile

# Step 2:
server$ chmod 700 ~/.ssh
server$ cat ~/.ssh/tmpfile >> ~/.ssh/authorized_keys
server$ chmod 600 ~/.ssh/authorized_keys
server$ rm ~/.ssh/tmpfile

GNOME Shell Extension development document

I don't know why it is so hard to search for the right development document for GNOME Shell Extension. The Gjs seems wraps all the functionalities to call the shell, but hard to find out how.
After searching and coding. I find out this may be a good one:

http://sander.github.com/tmp/gsdoc/documentation.html

Recursively chmod directories only

find . -type d -exec chmod 755 {} \;

This will recursively search your directory tree (starting at dir ‘dot’) and chmod 755 all directories only.

Similarly, the following will chmod all files only (and ignore the directories):

find . -type f -exec chmod 644 {} \;

Read binary in Hex

$ hexdump dog

or

$ cat dog | xxd

od -x can be used in mac, but the hight byte and low byte position is different

Popular Command Lines for Linux

The original article are in three parts:

• Part I: Top Ten One-Liners from CommandLineFu Explained
• Part II: The Next Ten One-Liners from CommandLineFu Explained
• Part III: Another Ten One-Liners from CommandLineFu Explained
• Part IV: Yet Another Ten One-Liners from CommandLineFu Explained

In my post, part_1, part_2, part_3, part_4

I love working in the shell. Mastery of shell lets you get things done in seconds, rather than minutes or hours, if you chose to write a program instead.
In this article I’d like to explain the top one-liners from the commandlinefu.com. It’s a user-driven website where people get to choose the best and most useful shell one-liners.

Use screen run background program in linux server

$screen
$run batch command
Press Ctrl-a,d

$screen -ls

There are screens on:

15914.pts-1.centos (Detached)

15887.pts-0.centos (Detached)

2 Sockets in /var/run/screen/S-root.

$screen -r 15914

(& , nohup)

opencv

g++ hello-world.cpp -o hello-world \
    -I /usr/local/include/opencv -L /usr/local/lib  \
    -lm -lcv -lhighgui -lcvaux

c++ find_obj.cpp -o o -I /usr/local/include -L /usr/local/lib -lopencv_core -lopencv_imgproc -lopencv_highgui -lopencv_ml -lopencv_video -lopencv_features2d -lopencv_calib3d -lopencv_objdetect -lopencv_contrib -lopencv_legacy -lopencv_flann

Color grep in terminal

alias grep='GREP_COLOR="1;37;41" LANG=C grep --color=auto'

!$ in command

$vim testfile.txt

then $cat !$ will give u the testfile.txt

Check the size of the directory -- du -sh *

$du -sh *

vimrc

set nocompatible
set mouse=a
behave mswin