Add terminal trick

Record a terminal session

README.md

@@ -33,22 +33,23 @@ Here's a more stylish version of [Bash-Oneliner](https://onceupon.github.io/Bash
 
 #####  Using Ctrl keys
 ```
+Ctrl + a : move to the beginning of line.
+Ctrl + d : if you've type something, Ctrl + d deletes the character under the cursor, else, it escapes the current shell.
+Ctrl + e : move to the end of line.
+Ctrl + k : delete all text from the cursor to the end of line.
+Ctrl + l : equivalent to clear.
 Ctrl + n : same as Down arrow.
 Ctrl + p : same as Up arrow.
+Ctrl + q : to resume output to terminal after Ctrl + s.
 Ctrl + r : begins a backward search through command history.(keep pressing Ctrl + r to move backward)
 Ctrl + s : to stop output to terminal.
-Ctrl + q : to resume output to terminal after Ctrl + s.
-Ctrl + a : move to the beginning of line.
-Ctrl + e : move to the end of line.
-Ctrl + d : if you've type something, Ctrl + d deletes the character under the cursor, else, it escapes the current shell.
-Ctrl + k : delete all text from the cursor to the end of line.
-Ctrl + x + backspace : delete all text from the beginning of line to the cursor.
 Ctrl + t : transpose the character before the cursor with the one under the cursor, press Esc + t to transposes the two words before the cursor.
-Ctrl + w : cut the word before the cursor; then Ctrl + y paste it
 Ctrl + u : cut the line before the cursor; then Ctrl + y paste it
-Ctrl + _ : undo typing.
+Ctrl + w : cut the word before the cursor; then Ctrl + y paste it
-Ctrl + l : equivalent to clear.
+Ctrl + x + backspace : delete all text from the beginning of line to the cursor.
 Ctrl + x + Ctrl + e : launch editor defined by $EDITOR to input your command. Useful for multi-line commands.
+Ctrl + z : stop current running process and keep it in background. You can use `fg` to continue the process in the foreground, or `bg` to continue the process in the background.
+Ctrl + _ : undo typing.
 ```
 ##### Change case
 ```bash
@@ -59,6 +60,7 @@ Esc + l
 Esc + c
 # converts letter under the cursor to uppercase, rest of the word to lowercase.
 ```
+
 ##### Run history number (e.g. 53)
 ```bash
 !53
@@ -73,15 +75,15 @@ sudo !!
 
 ##### Run last command and change some parameter using caret substitution (e.g. last command: echo 'aaa' -> rerun as: echo 'bbb')
 ```bash
-#last command: echo 'aaa'
+# last command: echo 'aaa'
 ^aaa^bbb
 
 #echo 'bbb'
 #bbb
 
-#Notice that only the first aaa will be replaced, if you want to replace all 'aaa', use ':&' to repeat it:
+# Notice that only the first aaa will be replaced, if you want to replace all 'aaa', use ':&' to repeat it:
 ^aaa^bbb^:&
-#or
+# or
 !!:gs/aaa/bbb/
 
 ```
@@ -118,6 +120,7 @@ $?   :most recent foreground pipeline exit status.
 $-   :current options set for the shell.
 $$   :pid of the current shell (not subshell).
 $!   :is the PID of the most recent background command.
+$_   :last argument of the previously executed command, or the path of the bash script.
 
 $DESKTOP_SESSION     current display manager
 $EDITOR   preferred text editor.
@@ -129,6 +132,31 @@ $USER   current username
 $HOSTNAME   current hostname
 ```
 
+##### Using vi-mode in your shell
+```bash
+set -o vi
+# change bash shell to vi mode
+# then hit the Esc key to change to vi edit mode (when `set -o vi` is set)
+k
+# in vi edit mode - previous command
+j
+# in vi edit mode - next command
+0
+# in vi edit mode - beginning of the command
+R
+# in vi edit mode - replace current characters of command
+2w
+# in vi edit mode - next to 2nd word
+b
+# in vi edit mode - previous word
+i
+# in vi edit mode - go to insert mode
+v
+# in vi edit mode - edit current command in vi
+man 3 readline
+# man page for complete readline mapping
+```
+
 ## Variable
 [[back to top](#handy-bash-one-liners)]
 ##### Variable substitution within quotes
@@ -144,7 +172,7 @@ echo '$foo'
 # single quotes within double quotes will not cancel expansion and will be part of the output
 echo "'$foo'"
 # 'bar'
-# doubled single quotes act as double quotes making variables expand
+# doubled single quotes act as if there are no quotes at all
 echo ''$foo''
 # bar
 ```
@@ -190,7 +218,7 @@ echo ${var[@]#0}
 
 ##### Grep lines with strings from a file (e.g. lines with 'stringA or 'stringB' or 'stringC')
 ```bash
-#with grep
+# with grep
 test="stringA stringB stringC"
 grep ${test// /\\\|} file.txt
 # turning the space into 'or' (\|) in grep
@@ -210,6 +238,12 @@ eval "$cmd"
 echo "$bar" # foo
 ```
 
+##### Record a terminal session
+```bash
+# https://github.com/asciinema/asciinema
+asciinema rec demo.cast
+```
+
 ## Math
 [[back to top](#handy-bash-one-liners)]
 ##### Arithmetic Expansion in Bash (Operators: +, -, *, /, %, etc)
@@ -278,8 +312,8 @@ echo "var=5;--var"| bc
 grep = grep -G # Basic Regular Expression (BRE)
 fgrep = grep -F # fixed text, ignoring meta-characters
 egrep = grep -E # Extended Regular Expression (ERE)
-pgrep = grep -P # Perl Compatible Regular Expressions (PCRE)
 rgrep = grep -r # recursive
+grep -P # Perl Compatible Regular Expressions (PCRE)
 ```
 
 #####  Grep and count number of empty lines
@@ -290,7 +324,7 @@ grep -c "^$"
 #####  Grep and return only integer
 ```bash
 grep -o '[0-9]*'
-#or
+# or
 grep -oP '\d*'
 ```
 #####  Grep integer with certain number of digits (e.g. 3)
@@ -313,7 +347,7 @@ grep -Po '\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}'
 ```bash
 grep -w 'target'
 
-#or using RE
+# or using RE
 grep '\btarget\b'
 ```
 #####  Grep returning lines before and after match (e.g. 'bbo')
@@ -351,7 +385,7 @@ grep -v '^#' file.txt
 ##### Grep variables with space within it (e.g. myvar="some strings")
 ```bash
 grep "$myvar" filename
-#remember to quote the variable!
+# remember to quote the variable!
 ```
 
 ##### Grep only one/first match (e.g. 'bbo')
@@ -430,8 +464,8 @@ grep $'\t'
 ```bash
 $echo "$long_str"|grep -q "$short_str"
 if [ $? -eq 0 ]; then echo 'found'; fi
-#grep -q will output 0 if match found
+# grep -q will output 0 if match found
-#remember to add space between []!
+# remember to add space between []!
 ```
 
 ##### Grep strings between a bracket()
@@ -517,7 +551,7 @@ sed -i '$ s/.$//' filename
 
 ##### Add string to beginning of file (e.g. "\[")
 ```bash
-sed -i '1s/^/[/' file
+sed -i '1s/^/[/' filename
 ```
 
 ##### Add string at certain line number (e.g. add 'something' to line 1 and line 3)
@@ -536,7 +570,7 @@ sed '$a\'
 
 ##### Add string to beginning of every line (e.g. 'bbo')
 ```bash
-sed -e 's/^/bbo/' file
+sed -e 's/^/bbo/' filename
 ```
 
 ##### Add string to end of each line (e.g. "}")
@@ -549,6 +583,13 @@ sed -e 's/$/\}\]/' filename
 sed 's/.\{4\}/&\n/g'
 ```
 
+##### Add a line after the line that matches the pattern (e.g. add a new line with "world" after the line with "hello")
+```bash
+sed '/hello*/a world' filename
+# hello
+# world
+```
+
 ##### Concatenate/combine/join files with a separator and next line (e.g separate by ",")
 ```bash
 sed -s '$a,' *.json > all.json
@@ -657,7 +698,7 @@ sed '$ s/.$//'
 
 ##### Insert character at specified position of file (e.g. AAAAAA --> AAA#AAA)
 ```bash
-sed -r -e 's/^.{3}/&#/' file
+sed -r -e 's/^.{3}/&#/' filename
 ```
 
 
@@ -712,7 +753,7 @@ awk -v N=7 '{print}/bbo/&& --N<=0 {exit}'
 
 ##### Print filename and last line of all files in directory
 ```bash
-ls|xargs -n1 -I file awk '{s=$0};END{print FILENAME,s}' file
+ls|xargs -n1 -I file awk '{s=$0};END{print FILENAME,s}' filename
 ```
 
 ##### Add string to the beginning of a column (e.g add "chr" to column $3)
@@ -722,12 +763,12 @@ awk 'BEGIN{OFS="\t"}$3="chr"$3'
 
 ##### Remove lines with string (e.g. 'bbo')
 ```bash
-awk '!/bbo/' file
+awk '!/bbo/' filename
 ```
 
 ##### Remove last column
 ```bash
-awk 'NF{NF-=1};1' file
+awk 'NF{NF-=1};1' filename
 ```
 
 ##### Usage and meaning of NR and FNR
@@ -792,7 +833,7 @@ awk '{printf("%s\t%s\n",NR,$0)}'
 # David    cat
 # David    dog
 
-awk '{split($2,a,",");for(i in a)print $1"\t"a[i]}' file
+awk '{split($2,a,",");for(i in a)print $1"\t"a[i]}' filename
 
 # Detail here:　http://stackoverflow.com/questions/33408762/bash-turning-single-comma-separated-column-into-multi-line-string
 ```
@@ -906,7 +947,7 @@ find /dir/to/A -type f -name "*.py" -print 0| xargs -0 -r -I file cp -v -p file
 
 ##### With sed
 ```bash
-ls |xargs -n1 -I file sed -i '/^Pos/d' file
+ls |xargs -n1 -I file sed -i '/^Pos/d' filename
 ```
 
 ##### Add the file name to the first line of file
@@ -1017,9 +1058,10 @@ else
   echo >&2 "Fatal error. This script requires mydir."
 fi
 
-# if variable is null
+# Check if a variable is null
-if [ ! -s "myvariable" ]; then echo -e "variable is null!" ; fi
+if [ -z "$var" ]; then echo "NULL"; else echo "Not NULL"; fi
-#True of the length if "STRING" is zero.
+# or 
+[ -z "$var" ] && echo "NULL"
 
 # Using test command (same as []), to test if the length of variable is nonzero
 test -n "$myvariable" && echo myvariable is "$myvariable" || echo myvariable is not set
@@ -1056,7 +1098,7 @@ if [[ $age -gt 21 ]]; then echo -e "forever 21!!" ; fi
 ```bash
 # Echo the file name under the current directory
 for i in $(ls); do echo file $i; done
-#or
+# or
 for i in *; do echo file $i; done
 
 # Make directories listed in a file (e.g. myfile)
@@ -1069,10 +1111,10 @@ for i in $(cat tpc_stats_0925.log |grep failed|grep -o '\query\w\{1,2\}'); do ca
 oifs="$IFS"; IFS=$'\n'; for line in $(cat myfile); do ...; done
 while read -r line; do ...; done <myfile
 
-#If only one word a line, simply
+# If only one word a line, simply
 for line in $(cat myfile); do echo $line; read -n1; done
 
-#Loop through an array
+# Loop through an array
 for i in "${arrayName[@]}"; do echo $i; done
 
 ```
@@ -1145,6 +1187,29 @@ date --date @1615852800
 
 ```
 
+##### Print current time point for N days ago or N days after
+```bash
+# print current date first (for the following example)
+date +"%F %H:%M:%S"
+# 2023-03-11 16:17:09
+
+# print the time that is 1 day ago
+date -d"1 day ago" +"%F %H:%M:%S"
+# 2023-03-10 16:17:09
+
+# print the time that is 7 days ago
+date -d"7 days ago" +"%F %H:%M:%S"
+# 2023-03-04 16:17:09
+
+# print the time that is a week ago
+date -d"1 week ago" +"%F %H:%M:%S"
+# 2023-03-04 16:17:09
+
+# add 1 day to date
+date -d"-1 day ago" +"%F %H:%M:%S"
+# 2023-03-12 16:17:09
+```
+
 ##### wait for random duration (e.g. sleep 1-5 second, like adding a jitter)
 ```bash
 sleep $[ ( $RANDOM % 5 ) + 1 ]
@@ -1158,7 +1223,7 @@ TMOUT=10
 
 ##### Set how long you want to run a command
 ```bash
-#This will run the command 'sleep 10' for only 1 second.
+# This will run the command 'sleep 10' for only 1 second.
 timeout 1 sleep 10
 ```
 
@@ -1493,9 +1558,17 @@ stat filename.txt
 ps aux
 ```
 
+##### List processes by top memory usage
+```bash
+ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head
+```
+
 ##### Display a tree of processes
 ```bash
 pstree
+
+# or
+ps aux --forest
 ```
 
 ##### Find maximum number of processes
@@ -1527,7 +1600,7 @@ who -r
 ##### Change SysV runlevel (e.g. 5)
 ```bash
 init 5
-#or
+# or
 telinit 5
 ```
 
@@ -1729,7 +1802,7 @@ pushd .
 # then pop
 popd
 
-#or use dirs to display the list of currently remembered directories.
+# or use dirs to display the list of currently remembered directories.
 dirs -l
 ```
 
@@ -1740,7 +1813,7 @@ df -h
 # or
 du -h
 
-#or
+# or
 du -sk /var/log/* |sort -rn |head -10
 ```
 
@@ -1770,7 +1843,7 @@ runlevel
 ```bash
 init 3
 
-#or
+# or
 telinit 3
 ```
 
@@ -1901,7 +1974,7 @@ ldconfig -p
 ldd /bin/ls
 ```
 
-##### Check user login
+##### Check the most recent login of all users
 ```bash
 lastlog
 ```
@@ -2051,11 +2124,6 @@ killall pulseaudio
 # then press Alt-F2 and type in pulseaudio
 ```
 
-##### When sound not working
-```bash
-killall pulseaudio
-```
-
 ##### List information about SCSI devices
 ```bash
 lsscsi
@@ -2092,7 +2160,7 @@ ps aux|grep python
 ```bash
 ps -p <PID>
 
-#or
+# or
 cat /proc/<PID>/status
 cat /proc/<PID>/stack
 cat /proc/<PID>/stat
@@ -2140,21 +2208,21 @@ sudo dpkg --purge <package_name>
 ```bash
 ssh -f -L 9000:targetservername:8088 root@192.168.14.72 -N
 #-f: run in background; -L: Listen; -N: do nothing
-#the 9000 of your computer is now connected to the 8088 port of the targetservername through 192.168.14.72
+# the 9000 of your computer is now connected to the 8088 port of the targetservername through 192.168.14.72
-#so that you can see the content of targetservername:8088 by entering localhost:9000 from your browser.
+# so that you can see the content of targetservername:8088 by entering localhost:9000 from your browser.
 ```
 ##### Get process ID of a process (e.g. sublime_text)
 ```bash
-#pidof
+# pidof
 pidof sublime_text
 
-#pgrep, you don't have to type the whole program name
+# pgrep, you don't have to type the whole program name
 pgrep sublim
 
-#pgrep, echo 1 if process found, echo 0 if no such process
+# pgrep, echo 1 if process found, echo 0 if no such process
 pgrep -q sublime_text && echo 1 || echo 0
 
-#top, takes longer time
+# top, takes longer time
 top|grep sublime_text
 ```
 
@@ -2219,7 +2287,12 @@ sar -f /var/log/sa/sa31|tail
 journalctl --file ./log/journal/a90c18f62af546ccba02fa3734f00a04/system.journal  --since "2020-02-11 00:00:00"
 ```
 
-##### Show a listing of last logged in users.
+##### Show a listing of last logged in users
+```bash
+last
+```
+
+##### Show a listing of unsuccessful (bad) login attempts 
 ```bash
 lastb
 ```
@@ -2397,6 +2470,11 @@ dig +short www.example.com
 host www.example.com
 ```
 
+##### Check public IP address
+```bash
+curl http://checkip.amazonaws.com
+```
+
 ##### Get DNS TXT record a of domain
 ```bash
 dig -t txt www.example.com
@@ -2437,7 +2515,7 @@ $ sudo nc -l 80
 
 ##### Check which ports are listening for TCP connections from the network
 ```bash
-#notice that some companies might not like you using nmap
+# note that some companies might not like you using nmap
 nmap -sT -O localhost
 
 # check port 0-65535
@@ -2445,7 +2523,7 @@ nmap  -p0-65535 localhost
 ```
 ##### Check if a host is up and scan for open ports, also skip host discovery.
 ```bash
-#skips checking if the host is alive which may sometimes cause a false positive and stop the scan.
+# skips checking if the host is alive. this may sometimes cause a false positive, stopping the scan.
 $ nmap google.com -Pn
 
 # Example output:
@@ -2467,7 +2545,7 @@ $ nmap -A -T4 scanme.nmap.org
 # -A to enable OS and version detection, script scanning, and traceroute; -T4 for faster execution
 ```
 
-##### Look up website information (e.g. name server), searches for an object in a RFC 3912 database.
+##### Look up website information (e.g. name server), searches for an object in a RFC 3912 database
 ```bash
 whois google.com
 ```
@@ -2477,7 +2555,7 @@ whois google.com
 openssl s_client -showcerts -connect www.example.com:443
 ```
 
-##### Display IP address
+##### Display network interfaces and their associated IP addresses
 ```bash
 ip a
 ```
@@ -2537,6 +2615,10 @@ curl -I http://example.com/
 # Accept-Ranges: bytes
 # Vary: Accept-Encoding
 ```
+##### Find out the time spent between request and response
+```
+curl -v -o /dev/null -s -w 'Total: %{time_total}s\n' google.com
+```
 
 ##### Find out the http status code of a URL
 ```bash
@@ -2678,11 +2760,11 @@ comm -23 fileA fileB
 ```bash
 nl fileA
 
-#or
+# or
 nl -nrz fileA
 # add leading zeros
 
-#or
+# or
 nl -w1 -s ' '
 # making it simple, blank separate
 ```
@@ -2817,7 +2899,7 @@ echo -e 'text here \c'
 
 ##### View first 50 characters of file
 ```bash
-head -c 50 file
+head -c 50 filename
 ```
 
 ##### Cut and get last column of a file
@@ -2837,12 +2919,33 @@ var=$((var+1))
 cat filename|rev|cut -f1|rev
 ```
 
-##### Cat to a file
+##### Create or replace a file with contents
 ```bash
 cat >myfile
 let me add sth here
-exit by control + c
+# exit with ctrl+d
-^C
+
+# or using tee
+tee myfile
+let me add sth else here
+# exit with ctrl+d
+```
+
+##### Append to a file with contents
+```bash
+cat >>myfile
+let me add sth here
+# exit with ctrl+d
+
+# or
+cat << EoF >> filename
+> add something here
+> EoF 
+
+# or using tee
+tee -a myfile
+let me add sth else here
+# exit with ctrl+d
 ```
 
 ##### Clear the contents of a file (e.g. filename)
@@ -2857,9 +2960,9 @@ echo 'hihi' >>filename
 
 ##### Working with json data
 ```bash
-#install the useful jq package
+# Install the useful jq package
-#sudo apt-get install jq
+# sudo apt-get install jq
-#e.g. to get all the values of the 'url' key, simply pipe the json to the following jq command(you can use .[]. to select inner json, i.e jq '.[].url')
+# e.g. to get all the values of the 'url' key, simply pipe the json to the following jq command(you can use .[]. to select inner json, i.e jq '.[].url')
 cat file.json | jq '.url'
 ```
 
@@ -2932,7 +3035,7 @@ while read a b; do yes $b |head -n $a ; done <test.txt
 ## Others
 [[back to top](#handy-bash-one-liners)]
 
-##### Describe the format and characteristics of image files.
+##### Describe the format and characteristics of image files
 ```bash
 identify myimage.png
 #myimage.png PNG 1049x747 1049x747+0+0 8-bit sRGB 1.006MB 0.000u 0:00.000
@@ -3055,7 +3158,7 @@ history -w
 vi ~/.bash_history
 history -r
 
-#or
+# or
 history -d [line_number]
 ```
 
@@ -3105,11 +3208,40 @@ rsync -av directory user@ip_address:/path/to/directory.bak
 # skip files that are newer on receiver (i prefer this one!)
 ```
 
+##### Create a temporary directory and `cd` into it
+```bash
+cd $(mktemp -d)
+# for example, this will create a temporary directory "/tmp/tmp.TivmPLUXFT"
+```
+
 ##### Make all directories at one time!
 ```bash
 mkdir -p project/{lib/ext,bin,src,doc/{html,info,pdf},demo/stat}
 # -p: make parent directory
-# this will create project/doc/html/; project/doc/info; project/lib/ext ,etc
+# this will create:
+# project/
+# project/bin/
+# project/demo/
+# project/demo/stat/
+# project/doc/
+# project/doc/html/
+# project/doc/info/
+# project/doc/pdf/
+# project/lib/
+# project/lib/ext/
+# project/src/
+#
+# project/
+# ├── bin
+# ├── demo
+# │   └── stat
+# ├── doc
+# │   ├── html
+# │   ├── info
+# │   └── pdf
+# ├── lib
+# │   └── ext
+# └── src
 ```
 
 ##### Run command only if another command returns zero exit status (well done)
@@ -3179,6 +3311,12 @@ scp -r directoryname user@ip:/path/to/send
 # :(){:|:&};:
 ```
 
+##### Trigger kernel crash
+```bash
+# Don't try this at home! 
+echo c > /proc/sysrq-trigger
+```
+
 ##### Use the last argument
 ```bash
 !$
@@ -3240,9 +3378,9 @@ fallocate -l 10G 10Gigfile
 
 ##### Create dummy file of certain size (e.g. 200mb)
 ```bash
-dd if=/dev/zero of=//dev/shm/200m bs=1024k count=200
+dd if=/dev/zero of=/dev/shm/200m bs=1024k count=200
 # or
-dd if=/dev/zero of=//dev/shm/200m bs=1M count=200
+dd if=/dev/zero of=/dev/shm/200m bs=1M count=200
 
 # Standard output:
 # 200+0 records in
@@ -3255,9 +3393,29 @@ dd if=/dev/zero of=//dev/shm/200m bs=1M count=200
 watch -n 1 wc -l filename
 ```
 
+##### Use Bash Strict Mode
+```bash
+# These options can make your code safer but, depending on how your pipeline is written, it might be too aggressive 
+# or it might not catch the errors that you are interested in
+
+# for reference see https://gist.github.com/mohanpedala/1e2ff5661761d3abd0385e8223e16425
+#               and https://mywiki.wooledge.org/BashPitfalls#set_-euo_pipefail
+
+set -o errexit      # exit immediately if a pipeline returns a non-zero status
+set -o errtrace     # trap ERR from shell functions, command substitutions, and commands from subshell
+set -o nounset      # treat unset variables as an error
+set -o pipefail     # pipe will exit with last non-zero status, if applicable
+set -Eue -o pipefail  # shorthand for above (pipefail has no short option)
+```
+
 ##### Print commands and their arguments when execute (e.g. echo `expr 10 + 20 `)
 ```bash
 set -x; echo `expr 10 + 20 `
+# or
+set -o xtrace; echo `expr 10 + 20 `
+
+# to turn it off..
+set +x
 ```
 
 ##### Print some meaningful sentences to you (install fortune first)