mirror of
https://github.com/onceupon/Bash-Oneliner.git
synced 2024-10-23 00:46:08 +00:00
6 lines
24 KiB
JSON
6 lines
24 KiB
JSON
{
|
|
"name": "Bash-oneliner",
|
|
"tagline": "Bash Oneliner learning station. This blog will focus on bash commands for parsing biological data, which are tsv files(tab-separated values); some of the commands are for Ubuntu system maintaining. I apologize that there won't be any citation of the code, but they are probably from dear Google and Stackoverflow. Not all the code here are oneliner (if the ';' counts..). English and bash are not my first language, so... correct me anytime, tks!!",
|
|
"body": "##Handy Bash oneliner commands for tsv file editing\r\n\r\n- [Grep](#grep)\r\n- [Sed](#sed)\r\n- [Awk](#awk)\r\n- [Xargs](#xargs)\r\n- [Find](#find)\r\n- [Loops](#loops)\r\n- [Download](#download)\r\n- [Random](#random)\r\n- [Others](#others)\r\n- [System](#system)\r\n\r\n##Grep\r\n#####extract text bewteen words (e.g. w1,w2)\r\n \r\n```bash\r\ngrep -o -P '(?<=w1).*(?=w2)'\r\n```\r\n\r\n#####grep lines without word (e.g. bbo)\r\n \r\n```bash\r\ngrep -v bbo\r\n```\r\n\r\n#####grep and count (e.g. bbo)\r\n \r\n```bash\r\ngrep -c bbo filename\r\n```\r\n\r\n#####insensitive grep (e.g. bbo/BBO/Bbo)\r\n \r\n```bash\r\ngrep -i \"bbo\" filename \r\n```\r\n\r\n#####count occurrence (e.g. three times a line count three times)\r\n \r\n```bash\r\ngrep -o bbo filename \r\n```\r\n\r\n#####COLOR the match (e.g. bbo)!\r\n \r\n```bash\r\ngrep --color bbo filename \r\n```\r\n\r\n#####grep search all files in a directory(e.g. bbo)\r\n \r\n```bash\r\ngrep -R bbo /path/to/directory \r\n```\r\n\r\nor\r\n \r\n```bash\r\ngrep -r bbo /path/to/directory \r\n```\r\n#####search all files in directory, only output file names with matches(e.g. bbo)\r\n \r\n```bash\r\ngrep -Rh bbo /path/to/directory \r\n```\r\nor\r\n```bash \r\ngrep -rh bbo /path/to/directory \r\n```\r\n\r\n#####grep OR (e.g. A or B or C or D)\r\n \r\n```\r\ngrep 'A\\|B\\|C\\|D' \r\n```\r\n\r\n#####grep AND (e.g. A and B)\r\n \r\n```bash\r\ngrep 'A.*B' \r\n```\r\n\r\n#####grep all content of a fileA from fileB\r\n \r\n```bash\r\ngrep -f fileA fileB \r\n```\r\n\r\n#####grep a tab\r\n \r\n```bash\r\ngrep $'\\t' \r\n```\r\n\r\n##Sed\r\n[[back to top](#handy-bash-oneliner-commands-for-tsv-file-editing)]\r\n\r\n#####remove lines with word (e.g. bbo)\r\n \r\n```bash\r\nsed \"/bbo/d\" filename\r\n```\r\n\r\n#####edit infile (edit and save)\r\n \r\n```bash\r\nsed -i \"/bbo/d\" filename\r\n```\r\n#####when using variable (e.g. $i), use double quotes \" \"\r\ne.g. add >$i to the first line (to make a FASTA file)\r\n \r\n```bash\r\nsed \"1i >$i\" \r\n```\r\n//notice the double quotes! in other examples, you can use a single quote, but here, no way! \r\n//'1i' means insert to first line\r\n\r\n\r\n#####delete empty lines\r\n \r\n```bash\r\nsed '/^\\s*$/d' \r\n``` \r\nor\r\n \r\n```bash\r\nsed 's/^$/d' \r\n```\r\n#####delete last line\r\n \r\n```bash\r\nsed '$d' \r\n```\r\n\r\n#####add \\n every nth character (e.g. every 4th character)\r\n \r\n```bash\r\nsed 's/.\\{4\\}/&\\n/g' \r\n```\r\n\r\n#####substitution (e.g. replace A by B)\r\n \r\n```bash\r\nsed 's/A/B/g' filename \r\n```\r\n#####select lines start with string (e.g. bbo)\r\n \r\n```bash\r\nsed -n '/^@S/p' \r\n```\r\n#####delete lines with string (e.g. bbo)\r\n \r\n```bash\r\nsed '/bbo/d' filename \r\n```\r\n#####print every nth lines\r\n \r\n```bash\r\nsed -n '0~3p' filename\r\n```\r\n//catch 0: start; 3: step\r\n\r\n\r\n#####print every odd # lines\r\n \r\n```bash\r\nsed -n '1~2p' \r\n```\r\n#####print every third line including the first line\r\n \r\n```bash\r\nsed -n '1p;0~3p' \r\n```\r\n#####remove leading whitespace and tabs\r\n \r\n```bash\r\nsed -e 's/^[ \\t]*//'\r\n```\r\n//notice a whitespace before '\\t'!!\r\n\r\n\r\n#####remove only leading whitespace\r\n \r\n```bash\r\nsed 's/ *//'\r\n```\r\n//notice a whitespace before '*'!!\r\n\r\n\r\n#####remove ending commas\r\n \r\n```bash\r\nsed 's/,$//g' \r\n```\r\n#####add a column to the end\r\n \r\n```bash\r\nsed \"s/$/\\t$i/\"\r\n```\r\n//$i is the valuable you want to add\r\ne.g. add the filename to every last column of the file\r\n \r\n```bash\r\nfor i in $(ls);do sed -i \"s/$/\\t$i/\" $i;done\r\n```\r\n\r\n#####remove newline\\ nextline\r\n \r\n```bash\r\nsed ':a;N;$!ba;s/\\n//g'\r\n```\r\n\r\n#####print a number of lines (e.g. line 10th to line 33 rd)\r\n \r\n```bash\r\nsed -n '10,33p' <filename\r\n```\r\n\r\n\r\n#Awk\r\n[[back to top](#handy-bash-oneliner-commands-for-tsv-file-editing)]\r\n\r\n#####set tab as field separator\r\n \r\n```bash\r\nawk -F $'\\t' \r\n```\r\n\r\n#####output as tab separated (also as field separator)\r\n \r\n```bash\r\nawk -v OFS='\\t' \r\n```\r\n\r\n#####pass variable\r\n\r\n```bash\r\na=bbo;b=obb;\r\nawk -v a=\"$a\" -v b=\"$b\" \"$1==a && $10=b' filename \r\n```\r\n\r\n#####print number of characters on each line\r\n \r\n```bash\r\nawk '{print length ($0);}' filename \r\n```\r\n\r\n#####find number of columns\r\n \r\n```bash\r\nawk '{print NF}' \r\n```\r\n\r\n#####reverse column order\r\n \r\n```bash\r\nawk '{print $2, $1}' \r\n```\r\n\r\n#####check if there is a comma in a column (e.g. column $1)\r\n \r\n```bash\r\nawk '$1~/,/ {print}' \r\n```\r\n\r\n#####split and do for loop\r\n \r\n```bash\r\nawk '{split($2, a,\",\");for (i in a) print $1\"\\t\"a[i]} filename \r\n```\r\n\r\n#####print all lines before nth occurence of a string (e.g stop print lines when bbo appears 7 times)\r\n \r\n```bash\r\nawk -v N=7 '{print}/bbo/&& --N<=0 {exit}'\r\n```\r\n\r\n#####add string to the beginning of a column (e.g add \"chr\" to column $3)\r\n \r\n```bash\r\nawk 'BEGIN{OFS=\"\\t\"}$3=\"chr\"$3' \r\n```\r\n\r\n#####remove lines with string (e.g. bbo)\r\n \r\n```bash\r\nawk '!/bbo/' file \r\n```\r\n\r\n#####column subtraction\r\n \r\n```bash\r\ncat file| awk -F '\\t' 'BEGIN {SUM=0}{SUM+=$3-$2}END{print SUM}'\r\n```\r\n\r\n#####usage and meaning of NR and FNR\r\ne.g.\r\nfileA:\r\na\r\nb\r\nc\r\nfileB:\r\nd\r\ne\r\n \r\n```bash\r\nawk 'print FILENAME, NR,FNR,$0}' fileA fileB \r\n```\r\n\r\nfileA 1 1 a\r\nfileA 2 2 b\r\nfileA 3 3 c\r\nfileB 4 1 d\r\nfileB 5 2 e\r\n\r\n#####and gate\r\n\r\ne.g.\r\nfileA:\r\n1 0\r\n\r\n2 1\r\n\r\n3 1\r\n\r\n4 0\r\n\r\nfileB:\r\n\r\n1 0\r\n\r\n2 1\r\n\r\n3 0\r\n\r\n4 1\r\n \r\n```bash\r\nawk -v OFS='\\t' 'NR=FNR{a[$1]=$2;next} NF {print $1,((a[$1]=$2)? $2:\"0\")}' fileA fileB \r\n```\r\n\r\n1 0\r\n\r\n2 1\r\n\r\n3 0\r\n\r\n4 0\r\n\r\n#####round all numbers of file (e.g. 2 significant figure)\r\n \r\n```bash\r\nawk '{while (match($0, /[0-9]+\\[0-9]+/)){\r\n \\printf \"%s%.2f\", substr($0,0,RSTART-1),substr($0,RSTART,RLENGTH)\r\n \\$0=substr($0, RSTART+RLENGTH)\r\n \\}\r\n \\print\r\n \\}'\r\n```\r\n\r\n#####give number/index to every row\r\n \r\n```bash\r\nawk '{printf(\"%s\\t%s\\n\",NR,$0)}'\r\n```\r\n\r\n##Xargs\r\n[[back to top](#handy-bash-oneliner-commands-for-tsv-file-editing)]\r\n\r\n#####set tab as delimiter (default:space)\r\n\r\n```bash\r\nxargs -d\\t\r\n```\r\n\r\n#####display 3 items per line\r\n\r\n```bash\r\necho 1 2 3 4 5 6| xargs -n 3\r\n```\r\n\r\n//1 2 3\r\n 4 5 6\r\n\r\n\r\n#####prompt before execution\r\n\r\n```bash\r\necho a b c |xargs -p -n 3\r\n```\r\n\r\n#####print command along with output\r\n\r\n```bash\r\nxargs -t abcd\r\n```\r\n\r\n///bin/echo abcd\r\n//abcd\r\n\r\n\r\n#####with find and rm\r\n\r\n```bash\r\nfind . -name \"*.html\"|xargs rm -rf\r\n```\r\n\r\ndelete fiels with whitespace in filename (e.g. \"hello 2001\")\r\n\r\n```bash\r\nfind . -name \"*.c\" -print0|xargs -0 rm -rf\r\n```\r\n\r\n#####show limits\r\n\r\n```bash\r\nxargs --show-limits\r\n```\r\n\r\n#####move files to folder\r\n\r\n```bash\r\nfind . -name \"*.bak\" -print 0|xargs -0 -I {} mv {} ~/old\r\n```\r\n\r\nor\r\n\r\n```bash\r\nfind . -name \"*.bak\" -print 0|xargs -0 -I file mv file ~/old\r\n```\r\n\r\n#####move first 100th files to a directory (e.g. d1)\r\n\r\n```bash\r\nls |head -100|xargs -I {} mv {} d1\r\n```\r\n\r\n#####parallel\r\n\r\n```bash\r\ntime echo {1..5} |xargs -n 1 -P 5 sleep\r\n```\r\na lot faster than\r\n```bash\r\ntime echo {1..5} |xargs -n1 sleep\r\n```\r\n\r\n#####copy all files from A to B\r\n\r\n```bash\r\nfind /dir/to/A -type f -name \"*.py\" -print 0| xargs -0 -r -I file cp -v -p file --target-directory=/path/to/B\r\n```\r\n\r\n//v: verbose|\r\n//p: keep detail (e.g. owner)\r\n\r\n\r\n#####with sed\r\n\r\n```bash\r\nls |xargs -n1 -I file sed -i '/^Pos/d' filename\r\n```\r\n\r\n#####add the file name to the first line of file\r\n\r\n```bash\r\nls |sed 's/.txt//g'|xargs -n1 -I file sed -i -e '1 i\\>file\\' file.txt\r\n```\r\n\r\n#####count all files\r\n\r\n```bash\r\nls |xargs -n1 wc -l\r\n```\r\n\r\n#####to filter txt to a single line\r\n\r\n```bash\r\nls -l| xargs\r\n```\r\n\r\n#####count files within directories\r\n\r\n```bash\r\necho mso{1..8}|xargs -n1 bash -c 'echo -n \"$1:\"; ls -la \"$1\"| grep -w 74 |wc -l' --\r\n```\r\n// \"--\" signals the end of options and display further option processing\r\n\r\n\r\n#####download dependencies files and install (e.g. requirements.txt)\r\n \r\n```bash\r\ncat requirements.txt| xargs -n1 sudo pip install\r\n```\r\n\r\n#####count lines in all file, also count total lines\r\n\r\n```bash\r\nls|xargs wc -l\r\n```\r\n\r\n\r\n\r\n##Find \r\n[[back to top](#handy-bash-oneliner-commands-for-tsv-file-editing)]\r\n#####list all sub directory/file in the current directory\r\n \r\n```bash\r\nfind .\r\n```\r\n\r\n#####list all files under the current directory\r\n \r\n```bash\r\nfind . -type f\r\n```\r\n\r\n#####list all directories under the current directory\r\n \r\n```bash\r\nfind . -type d\r\n```\r\n\r\n#####edit all files under current directory (e.g. replace 'www' with 'ww')\r\n \r\n```bash\r\nfind . name '*.php' -exec sed -i 's/www/w/g' {} \\;\r\n```\r\nif no subdirectory\r\n \r\n```bash\r\nreplace \"www\" \"w\" -- *\r\n```\r\n//a space before *\r\n\r\n\r\n#####find and output only filename (e.g. \"mso\")\r\n \r\n```bash\r\nfind mso*/ -name M* -printf \"%f\\n\"\r\n```\r\n\r\n#####find and delete file with size less than (e.g. 74 byte)\r\n \r\n```bash\r\nfind . -name \"*.mso\" -size -74c -delete\r\n```\r\n//M for MB, etc\r\n\r\n\r\n##Loops\r\n[[back to top](#handy-bash-oneliner-commands-for-tsv-file-editing)]\r\n#####while loop, column subtraction of a file (e.g. a 3 columns file)\r\n \r\n```bash\r\nwhile read a b c; do echo $(($c-$b));done < <(head filename)\r\n```\r\n//there is a space between the two '<'s\r\n\r\n#####while loop, sum up column subtraction\r\n \r\n```bash\r\ni=0; while read a b c; do ((i+=$c-$b)); echo $i; done < <(head filename)\r\n```\r\n\r\n#####if loop\r\n \r\n```bash\r\nif (($j==$u+2))\r\n```\r\n//(( )) use for arithmetic operation\r\n \r\n```bash\r\nif [[$age >21]]\r\n```\r\n//[[ ]] use for comparison\r\n\r\n#####for loop\r\n \r\n```bash\r\nfor i in $(ls); do echo file $i;done\r\n```\r\n\r\n\r\n##Download\r\n[[back to top](#handy-bash-oneliner-commands-for-tsv-file-editing)]\r\n#####download all from a page\r\n \r\n```bash\r\nwget -r -l1 -H -t1 -nd -N -np -A mp3 -e robots=off http://example.com\r\n```\r\n//-r: recursive and download all links on page\r\n\r\n//-l1: only one level link\r\n\r\n//-H: span host, visit other hosts\r\n\r\n//-t1: numbers of retries\r\n\r\n//-nd: don't make new directories, download to here\r\n\r\n//-N: turn on timestamp\r\n\r\n//-nd: no parent\r\n\r\n//-A: type (seperate by ,)\r\n\r\n//-e robots=off: ignore the robots.txt file which stop wget from crashing the site, sorry example.com\r\n\r\n\r\n\r\n##Random\r\n[[back to top](#handy-bash-oneliner-commands-for-tsv-file-editing)]\r\n#####random pick 100 lines from a file\r\n \r\n```bash\r\nshuf -n 100 filename\r\n```\r\n\r\n#####random order (lucky draw)\r\n \r\n```bash\r\nfor i in a b c d e; do echo $i; done| shuf\r\n```\r\n\r\n#####echo series of random numbers between a range (e.g. generate 15 random numbers from 0-10)\r\n \r\n```bash\r\nshuf -i 0-10 -n 15\r\n```\r\n\r\n#####echo a random number\r\n \r\n```bash\r\necho $RANDOM\r\n```\r\n\r\n#####random from 0-9\r\n \r\n```bash\r\necho $((RANDOM % 10))\r\n```\r\n\r\n#####random from 1-10\r\n \r\n```bash\r\necho $(((RANDOM %10)+1))\r\n```\r\n\r\n\r\n\r\n##Others\r\n[[back to top](#handy-bash-oneliner-commands-for-tsv-file-editing)]\r\n#####remove newline / nextline\r\n \r\n```bash\r\ntr --delete '\\n' <input.txt >output.txt\r\n```\r\n\r\n#####replace newline\r\n \r\n```bash\r\ntr '\\n' ' ' <filename\r\n```\r\n\r\n#####compare files (e.g. fileA, fileB)\r\n \r\n```bash\r\ndiff fileA fileB\r\n```\r\n//a: added; d:delete; c:changed\r\n\r\nor\r\n\r\n```bash\r\nsdiff fileA fileB\r\n```\r\n//side-to-side merge of file differences\r\n\r\n\r\n#####number a file (e.g. fileA)\r\n\r\n```bash\r\nnl fileA\r\n```\r\nor\r\n\r\n```bash\r\nnl -nrz fileA\r\n```\r\n//add leading zeros\r\n\r\n\r\n#####combine/ paste two files (e.g. fileA, fileB)\r\n \r\n```bash\r\npaste fileA fileB\r\n```\r\n//default tab seperated\r\n\r\n\r\n#####reverse string\r\n \r\n```bash\r\necho 12345| rev\r\n```\r\n\r\n#####read .gz file without extracting\r\n \r\n```bash\r\nzmore filename\r\n```\r\nor\r\n \r\n```bash\r\nzless filename\r\n```\r\n\r\n#####run in background, output error file\r\n \r\n```bash\r\n(command here) 2>log &\r\n```\r\n\r\nor\r\n\r\n```bash\r\n(command here) 2>&1| tee logfile\r\n```\r\n\r\nor\r\n\r\n```bash\r\n(command here) 2>&1 >>outfile\r\n```\r\n//0: standard input; 1: standard output; 2: standard error\r\n\r\n\r\n#####send mail\r\n \r\n```bash\r\necho 'heres the content'| mail -A 'file.txt' -s 'mail.subject' me@gmail.com\r\n```\r\n//use -a flag to set send from (-a \"From: some@mail.tld\")\r\n\r\n\r\n#####.xls to csv\r\n \r\n```bash\r\nxls2csv filename\r\n```\r\n#####append to file (e.g. hihi)\r\n \r\n```bash\r\necho 'hihi' >>filename\r\n```\r\n\r\n#####make BEEP sound\r\n \r\n```bash\r\nspeaker-test -t sine -f 1000 -l1\r\n```\r\n\r\n#####set beep duration\r\n \r\n```bash\r\n(speaker-test -t sine -f 1000) & pid=$!;sleep 0.1s;kill -9 $pid\r\n```\r\n\r\n#####history edit/ delete\r\n \r\n```bash\r\n~/.bash_history\r\n```\r\nor\r\n\r\n```bash\r\nhistory -d [line_number]\r\n```\r\n\r\n#####get last history/record filename\r\n \r\n```bash\r\nhead !$\r\n```\r\n\r\n#####clean screen\r\n \r\n```bash\r\nclear\r\n```\r\n\r\nor\r\n\r\n```bash\r\nCtrl+l\r\n```\r\n\r\n#####send data to last edited file\r\n \r\n```bash\r\ncat /directory/to/file\r\necho 100>!$\r\n```\r\n\r\n#####run history number (e.g. 53)\r\n \r\n```bash\r\n!53\r\n```\r\n\r\n#####run last command\r\n \r\n```bash\r\n!!\r\n```\r\n\r\n#####run last command that began with (e.g. cat filename)\r\n \r\n```bash\r\n!cat\r\n```\r\n\r\nor\r\n\r\n```bash\r\n!c\r\n```\r\n//run cat filename again\r\n\r\n\r\n#####extract .xf\r\n \r\n 1.unxz filename.tar.xz\r\n 2.tar -xf filename.tar\r\n\r\n#####install python package\r\n \r\n```bash\r\npip install packagename\r\n```\r\n\r\n\r\n#####Download file if necessary\r\n \r\n```bash\r\ndata=file.txt\r\nurl=http://www.example.com/$data\r\nif [! -s $data];then\r\n echo \"downloading test data...\"\r\n wget $url\r\nfi\r\n```\r\n\r\n#####wget to a filename (when a long name)\r\n \r\n```bash\r\nwget -O filename \"http://example.com\"\r\n```\r\n\r\n#####wget files to a folder\r\n\r\n```bash\r\nwget -P /path/to/directory \"http://example.com\"\r\n```\r\n\r\n#####delete current bash command\r\n \r\n```bash\r\nCtrl+U\r\n```\r\n\r\nor\r\n\r\n```bash\r\nCtrl+C\r\n```\r\n\r\nor\r\n\r\n```bash\r\nAlt+Shift+#\r\n```\r\n//to make it to history\r\n\r\n\r\n#####add things to history (e.g. \"addmetohistory\")\r\n \r\n```bash\r\n#addmetodistory\r\n```\r\n//just add a \"#\" before~~\r\n\r\n\r\n#####sleep awhile or wait for a moment or schedule a job\r\n \r\n```bash\r\nsleep 5;echo hi\r\n```\r\n\r\n#####count the time for executing a command\r\n \r\n```bash\r\ntime echo hi\r\n```\r\n\r\n#####backup with rsync\r\n \r\n```bash\r\nrsync -av filename filename.bak\r\nrsync -av directory directory.bak\r\nrsync -av --ignore_existing directory/ directory.bak\r\nrsync -av --update directory directory.bak\r\n```\r\n//skip files that are newer on receiver (i prefer this one!)\r\n\r\n\r\n#####make all directories at one time!\r\n \r\n```bash\r\nmkdir -p project/{lib/ext,bin,src,doc/{html,info,pdf},demo/stat}\r\n```\r\n//-p: make parent directory\r\n//this will create project/doc/html/; project/doc/info; project/lib/ext ,etc\r\n\r\n\r\n#####run command only if another command returns zero exit status (well done)\r\n \r\n```bash\r\ncd tmp/ && tar xvf ~/a.tar\r\n```\r\n\r\n#####run command only if another command returns non-zero exit status (not finish)\r\n \r\n```bash\r\ncd tmp/a/b/c ||mkdir -p tmp/a/b/c\r\n```\r\n\r\n#####extract to a path\r\n \r\n```bash\r\ntar xvf -C /path/to/directory filename.gz\r\n```\r\n\r\n#####use backslash \"\\\" to break long command\r\n \r\n```bash\r\ncd tmp/a/b/c \\\r\n> || \\\r\n>mkdir -p tmp/a/b/c\r\n```\r\n\r\n#####get pwd\r\n \r\n```bash\r\nVAR=$PWD; cd ~; tar xvf -C $VAR file.tar\r\n```\r\n//PWD need to be capital letter\r\n\r\n#####list file type of file (e.g. /tmp/)\r\n \r\n```bash\r\nfile /tmp/\r\n```\r\n//tmp/: directory\r\n\r\n\r\n#####bash script\r\n \r\n```bash\r\n#!/bin/bash\r\nfile=${1#*.}\r\n```\r\n//remove string before a \".\"\r\n\r\n```bash\r\nfile=${1%.*}\r\n```\r\n//remove string after a \".\"\r\n\r\n#####search from history\r\n \r\n```bash\r\nCtrl+r\r\n```\r\n\r\n#####python simple HTTP Server\r\n \r\n```bash\r\npython -m SimpleHTTPServer\r\n```\r\n\r\n#####variables\r\n \r\n```bash\r\n{i/a/,}\r\n```\r\ne.g. replace all\r\n```bash\r\n{i//a/,}\r\n```\r\n//for variable i, replace all 'a' with a comma\r\n\r\n#####read user input\r\n \r\n```bash\r\nread input\r\necho $input\r\n```\r\n\r\n#####generate sequence 1-10\r\n \r\n```bash\r\nseq 10\r\n```\r\n\r\n#####sum up input list (e.g. seq 10)\r\n \r\n```bash\r\nseq 10|paste -sd+|bc\r\n```\r\n\r\n#####find average of input list/file\r\n \r\n```bash\r\ni=`wc -l filename|cut -d ' ' -f1`; cat filename| echo \"scale=2;(`paste -sd+`)/\"$i|bc\r\n```\r\n\r\n#####generate all combination (e.g. 1,2)\r\n \r\n```bash\r\necho {1,2}{1,2}\r\n```\r\n//1 1, 1 2, 2 1, 2 2\r\n\r\n#####generate all combination (e.g. A,T,C,G)\r\n \r\n```bash\r\nset = {A,T,C,G}\r\ngroup= 5\r\nfor ((i=0; i<$group; i++));do\r\n repetition=$set$repetition;done\r\n bash -c \"echo \"$repetition\"\"\r\n```\r\n\r\n#####read file content to variable\r\n```bash\r\nfoo=$(<test1)\r\n```\r\n\r\n#####echo size of variable\r\n\r\n```bash\r\necho ${#foo}\r\n```\r\n\r\n#####array\r\n```bash\r\ndeclare -A array=()\r\n```\r\n\r\n#####send a directory\r\n\r\n```bash\r\nscp -r directoryname user@ip:/path/to/send\r\n```\r\n\r\n\r\n\r\n\r\n##System\r\n[[back to top](#handy-bash-oneliner-commands-for-tsv-file-editing)]\r\n\r\n#####snapshot of the current processes\r\n\r\n```bash\r\nps \r\n```\r\n\r\n#####check graphics card\r\n\r\n```bash\r\nlspci\r\n```\r\n\r\n#####show IP address\r\n \r\n```bash\r\n$ip add show\r\n```\r\nor\r\n \r\n```bash\r\nifconfig\r\n```\r\n\r\n#####check system version\r\n \r\n```bash\r\ncat /etc/*-release\r\n```\r\n\r\n#####Linux Programmer's Manuel: hier- description of the filesystem hierarchy\r\n \r\n```bash\r\nman hier\r\n```\r\n\r\n#####list job\r\n \r\n```bash\r\njobs -l\r\n```\r\n\r\n#####export PATH\r\n \r\n```bash\r\nexport PATH=$PATH:~/path/you/want\r\n```\r\n\r\n#####make file execuable\r\n \r\n```bash\r\nchmod +x filename\r\n```\r\n//you can now ./filename to execute it\r\n\r\n#####list screen\r\n \r\n```bash\r\nscreen -d -r\r\n```\r\n\r\n#####echo screen name\r\n \r\n```bash\r\nscreen -ls\r\n```\r\n\r\n#####check system (x86-64)\r\n \r\n```bash\r\nuname -i\r\n```\r\n\r\n#####surf the net\r\n\r\n```bash\r\nlinks www.google.com\r\n```\r\n\r\n#####add user, set passwd\r\n \r\n```bash\r\nuseradd username\r\npasswd username\r\n```\r\n\r\n#####edit variable for bash, (e.g. displaying the whole path)\r\n \r\n```bash\r\n1. joe ~/.bash_profile \r\n2. export PS1='\\u@\\h:\\w\\$' \r\n```\r\n//$PS1 is a variable that defines the makeup and style of the command prompt \r\n```bash\r\n3. source ~/.bash_profile\r\n```\r\n\r\n#####edit environment setting (e.g. alias)\r\n \r\n```bash\r\n1. joe ~/.bash_profile\r\n2. alias pd=\"pwd\" //no more need to type that 'w'!\r\n3. source ~/.bash_profile\r\n```\r\n\r\n#####list environment variables (e.g. PATH)\r\n \r\n```bash\r\n$echo $PATH\r\n```\r\n//list of directories separated by a colon\r\n\r\n#####list all environment variables for current user\r\n \r\n```bash\r\n$env\r\n```\r\n\r\n#####show partition format\r\n \r\n```bash\r\nlsblk\r\n```\r\n\r\n#####soft link program to bin\r\n \r\n```bash\r\nln -s /path/to/program /home/usr/bin\r\n```\r\n//must be the whole path to the program\r\n\r\n#####show hexadecimal view of data\r\n \r\n```bash\r\nhexdump -C filename.class\r\n```\r\n\r\n#####jump to different node\r\n \r\n```bash\r\nrsh node_name\r\n```\r\n\r\n#####check port (active internet connection)\r\n \r\n```bash\r\nnetstat -tulpn\r\n```\r\n\r\n#####find whick link to a file\r\n \r\n```bash\r\nreadlink filename\r\n```\r\n\r\n#####check where a command link to (e.g. python)\r\n \r\n```bash\r\nwhich python\r\n```\r\n\r\n#####list total size of a directory\r\n \r\n```bash\r\ndu -hs .\r\n```\r\nor\r\n \r\n```bash\r\ndu -sb\r\n```\r\n\r\n#####copy directory with permission setting\r\n \r\n```bash\r\ncp -rp /path/to/directory\r\n```\r\n\r\n#####store current directory\r\n \r\n```bash\r\npushd . $popd ;dirs -l \r\n```\r\n\r\n#####show disk usage\r\n \r\n```bash\r\ndf -h \r\n```\r\nor\r\n \r\n```bash\r\ndu -h \r\n```\r\nor\r\n \r\n```bash\r\ndu -sk /var/log/* |sort -rn |head -10\r\n```\r\n\r\n#####show current runlevel\r\n \r\n```bash\r\nrunlevel\r\n```\r\n\r\n#####switch runlevel\r\n \r\n```bash\r\ninit 3 \r\n```\r\nor\r\n```bash\r\ntelinit 3 \r\n```\r\n#####permanently modify runlevel\r\n \r\n```bash\r\n1. edit /etc/init/rc-sysinit.conf \r\n2. env DEFAULT_RUNLEVEL=2 \r\n```\r\n\r\n#####become root\r\n \r\n```bash\r\nsu\r\n```\r\n\r\n#####become somebody\r\n \r\n```bash\r\nsu somebody\r\n```\r\n\r\n#####report user quotes on device\r\n \r\n```bash\r\nrequota -auvs\r\n```\r\n\r\n#####get entries in a number of important databases\r\n \r\n```bash\r\ngetent database_name\r\n```\r\n(e.g. the 'passwd' database)\r\n \r\n```bash\r\ngetent passwd\r\n```\r\n//list all user account (all local and LDAP)\r\n(e.g. fetch list of grop accounts)\r\n \r\n```bash\r\ngetent group\r\n```\r\n//store in database 'group'\r\n\r\n#####little xwindow tools\r\n \r\n```bash\r\nxclock\r\nxeyes\r\n```\r\n\r\n#####change owner of file\r\n \r\n```bash\r\nchown user_name filename\r\nchown -R user_name /path/to/directory/\r\n```\r\n//chown user:group filename\r\n\r\n#####list current mount detail\r\n \r\n```bash\r\ndf\r\n```\r\n\r\n#####list current usernames and user-numbers\r\n \r\n```bash\r\ncat /etc/passwd\r\n```\r\n#####get all username\r\n \r\n```bash\r\ngetent passwd| awk '{FS=\"[:]\"; print $1}'\r\n```\r\n\r\n#####show all users\r\n \r\n```bash\r\ncompgen -u\r\n```\r\n\r\n#####show all groups\r\n \r\n```bash\r\ncompgen -g\r\n```\r\n\r\n#####show group of user\r\n \r\n```bash\r\ngroup username\r\n```\r\n\r\n#####show uid, gid, group of user\r\n \r\n```bash\r\nid username\r\n```\r\n\r\n#####check if it's root\r\n \r\n```bash\r\nif [$(id -u) -ne 0];then\r\n echo \"You are not root!\"\r\n exit;\r\nfi\r\n```\r\n//'id -u' output 0 if it's not root\r\n\r\n#####find out CPU information\r\n \r\n```bash\r\nmore /proc/cpuinfo\r\n```\r\nor\r\n \r\n```bash\r\nlscpu\r\n```\r\n\r\n#####set quota for user (e.g. disk soft limit: 120586240; hard limit: 125829120)\r\n \r\n```bash\r\nsetquota username 120586240 125829120 0 0 /home\r\n```\r\n\r\n#####show quota for user\r\n \r\n```bash\r\nquota -v username\r\n```\r\n\r\n#####fork bomb\r\n \r\n```bash\r\n:(){:|:&};:\r\n```\r\n//dont try this at home\r\n\r\n#####check user login\r\n \r\n```bash\r\nlastlog\r\n```\r\n\r\n#####edit path for all users\r\n \r\n```bash\r\njoe /etc/environment\r\n```\r\n//edit this file\r\n\r\n#####show running processes\r\n \r\n```bash\r\nps aux\r\n```\r\n\r\n#####find maximum number of processes\r\n \r\n```bash\r\ncat /proc/sys/kernal/pid_max\r\n```\r\n\r\n#####show and set user limit\r\n \r\n```bash\r\nulimit -u\r\n```\r\n\r\n\r\n=-=-=-=-=-A lot more coming!! =-=-=-=-=-=-=-=-=-=waitwait-=-=-=-=-=-=-=-=-=-",
|
|
"note": "Don't delete this file! It's used internally to help with page regeneration."
|
|
} |