77 Commits

Author SHA1 Message Date
f9619440da Add terminal trick
Record a terminal session
2025-04-03 10:53:58 +08:00
3199d638ae fix command #last and add new commands for #System and #Others 2024-08-29 19:39:32 +00:00
80c5012f38 add commands for #Networking 2024-08-29 18:51:24 +00:00
6479b26656 fix command for #Check if a variable is null 2024-08-29 18:14:01 +00:00
7cfb4e286c Merge pull request #53 from fox-forks/hyperupcall-whitespace-consistent
Make formatting and names more consistent
2024-08-30 01:49:04 +08:00
88e095d5fc Make formatting and names more consistent
Signed-off-by: Edwin Kofler <edwin@kofler.dev>
2023-11-18 13:47:16 -08:00
212326d7a3 Add example for sed
Add a line after the line that matches the pattern (e.g. add a new line with "world" after the line with "hello")
2023-08-11 00:25:22 +08:00
6576e5a685 remove pgrep, fixing #40 2023-03-16 09:43:50 +00:00
a85131614c modify description for $_ 2023-03-16 07:05:25 +00:00
520f053c08 Merge pull request #38 from sarlej/add-$_
Add '$_'
2023-03-16 15:02:21 +08:00
3e6bbf7c20 fix issue #36 2023-03-15 10:02:31 +00:00
29902039d8 remove duplicated command 2023-03-15 09:38:46 +00:00
b6ff5a9fcc edit command order 2023-03-15 08:19:29 +00:00
1c5bd35608 update vi-mode #39 2023-03-15 08:18:16 +00:00
6054869173 Merge pull request #39 from mtatton/master
[ vi ] readline vi mode
2023-03-15 16:15:29 +08:00
a67ee7e167 update ctrl key order 2023-03-15 07:44:51 +00:00
58b858888a modify description for #41 2023-03-15 07:41:55 +00:00
9ae2d9403b Merge pull request #41 from 5c077m4n/add-ctrl-z-term-cmd
Add `Ctrl + z` command description
2023-03-15 15:40:24 +08:00
55fc47b685 modify output example for #43 2023-03-15 07:37:37 +00:00
ca89df83dc Merge pull request #43 from chapmanjacobd/patch-1
include full output
2023-03-15 15:34:45 +08:00
1b42d476e4 add example for mktemp 2023-03-15 07:29:08 +00:00
5d5b83b7b8 Merge pull request #44 from chapmanjacobd/patch-2
add mktemp
2023-03-15 15:17:21 +08:00
671c0eb16f update comment 2023-03-11 17:16:29 +00:00
51d112b5ce Merge pull request #45 from chapmanjacobd/patch-3
Use ctrl+d instead of ctrl+c
2023-03-12 01:14:06 +08:00
40c3b7d7b8 update title for bash strict mode, add command to turn it off 2023-03-11 17:08:42 +00:00
6ecdb7b85e Merge pull request #46 from chapmanjacobd/patch-4
add more info about BASH options
2023-03-12 00:58:05 +08:00
23a1efa249 add examples for 'current time point for N days ago or after N days' 2023-03-11 16:21:26 +00:00
a606d6be45 Merge pull request #48 from PoplarYang/master
fix typo '//dev/shm/200m' and  add 'current time point for N days ago or after N days'
2023-03-12 00:07:23 +08:00
16197f576e Merge pull request #29 from siansiansu/formatExampleCode
format bash commands
2023-03-11 16:40:42 +08:00
5f0c40efc5 Add Ctrl + z command desc
Signed-off-by: Roee Shapira <roee.shapira@tensorleap.ai>
Signed-off-by: Roee Shapira <ro33.sha@gmail.com>
2022-05-24 01:17:25 +03:00
232900349a fix typo
Co-authored-by: kang <1115610574@qq.com>
2022-05-17 18:07:22 +08:00
3cf8bdfe46 add 'current time point for N days ago or after' 2022-05-16 00:33:16 +08:00
794d4ba7b3 fix typo "//dev/shm" 2022-05-16 00:13:47 +08:00
3276c74d49 add more info about BASH options 2022-05-14 16:08:29 -05:00
f459755ade Use ctrl+d instead of ctrl+c 2022-05-14 13:10:26 -05:00
57085df9be add mktemp 2022-05-14 12:54:23 -05:00
7d1c903031 include full output 2022-05-14 12:18:16 -05:00
7d33353ca8 make some parts of example code more consistent
Signed-off-by: alexsu <alexsu@17.media>
2022-05-10 12:21:11 +08:00
a59f19e9cf [ vi ] readline vi mode
Signed-off-by: Unknown <dev@null>
2022-05-09 20:14:34 +02:00
c3a29f735a vi edit mode
Signed-off-by: Unknown <dev@null>
2022-05-09 20:14:34 +02:00
ebfb8f1be1 Create README.md
Add '$_'  it contains last of parameterers of recent command.
2022-05-09 13:32:42 +02:00
a59c8598b0 edfixed grep command "Regex with or without a certain character" 2022-05-09 00:18:05 +08:00
c70067d38f edRemove ‘ ’ style quotes from README.ME 2022-05-09 00:15:04 +08:00
bca6221ed3 edadd comm in "data wrangling" 2022-05-09 00:01:41 +08:00
0103101550 edit "Run last command" 2022-05-08 23:35:19 +08:00
bab726081b fixed example for Variable command 2022-05-08 23:33:28 +08:00
7d40456576 fixed content order 2022-05-08 23:07:47 +08:00
ab2ecc058d fixing example output for look command 2022-05-08 01:38:02 +08:00
9dbc77ba6e Merge pull request #28 from NateEag/patch-1
Fix typo in README.md (singer -> single)
2022-05-08 01:32:45 +08:00
8b2e16f0c1 Merge pull request #33 from JacksonBailey/patch-1
Variable substitution changes
2022-05-08 01:30:26 +08:00
18134f72c9 Merge pull request #30 from ember91/master
Fix spelling errors
2022-05-08 01:20:06 +08:00
b1137da792 Merge pull request #24 from linus/patch-2
Use correct quoting
2022-05-08 01:04:52 +08:00
b912cc99bf Merge pull request #23 from linus/patch-1
Use equivalent regex with PCRE
2022-05-08 01:02:27 +08:00
b07665cb43 Merge pull request #22 from herrcykel/patch-1
Simplify glob expression
2022-05-08 00:58:25 +08:00
7010074d16 Merge pull request #35 from chapeupreto/fixes
fix typo and markdown formatting issues
2022-05-08 00:54:56 +08:00
12023778bd fix typo and markdown formatting
Signed-off-by: Rod Elias <rodiney@gmail.com>
2022-05-06 10:50:27 -03:00
dc268515b7 Variable substitution changes
Clarify documentation and give a few more examples
2022-05-04 15:58:25 -04:00
d2549128c7 Fix spelling errors
Signed-off-by: Emil Berg <emil.berg@ericsson.com>
2022-05-04 07:50:27 +02:00
bd9a732dba Correct the description of Esc + c (Alt+c)
Signed-off-by: Страхиња Радић <contact@strahinja.org>
2022-05-04 07:47:31 +02:00
c136e0f36f Fix typo in README.md (singer -> single) 2022-05-03 21:45:55 -04:00
O
8ab7cf7a6d Simplify glob expression
Signed-off-by: O <herrcykel@users.noreply.github.com>
2022-05-03 22:15:57 +02:00
a97dcf2a3b Use equivalent regex with PCRE
Use the equivalent regex with PCRE.

`[0-9]*` (and `\d*`) will output each integer, e.g.:

```bash
echo 17 | grep -o '[0-9]*'
```

Whereas `\d` will split it into digits, e.g.:

```bash
echo 17 | grep -oP '\d'
```

Signed-off-by: Linus G Thiel <linus@yesbabyyes.se>
2022-05-03 22:02:09 +02:00
0f5fa26f06 Use correct quoting
Unquote the sed expression.

Signed-off-by: Linus G Thiel <linus@yesbabyyes.se>
2022-05-03 22:01:42 +02:00
c2b96736bd Merge pull request #21 from Strahinja/master
Correct the description of Esc + c (Alt+c, M-c)
2022-05-03 12:21:52 +08:00
8dcfa4904b Correct the description of Esc + c (Alt+c)
Signed-off-by: Страхиња Радић <contact@strahinja.org>
2022-01-26 18:17:03 +01:00
ce13d4af19 Merge pull request #20 from valerionew/patch-1
Swap * and ? in bash globbing sections to match with correct comments
2021-08-13 19:03:16 +08:00
81ec32bfb9 Swap * and ? in bash globbing sections to match with correct comments
Signed-off-by: valerionew <valerionappi995@gmail.com>
2021-07-01 15:05:55 +02:00
954f42c06a fixed typo in file name
xargs using With sed
2021-05-16 16:17:31 +08:00
c1c3f8bcb5 Merge pull request #19 from WardMaes/master
Add instructions to exit virtual environment
2021-05-06 17:30:17 +08:00
26db64f1b2 Add deactivate command
Signed-off-by: Ward <WardMaes@users.noreply.github.com>
2021-04-11 11:40:52 +02:00
5e32496e5d Merge branch 'master' of https://github.com/onceupon/Bash-Oneliner 2021-03-16 22:10:53 +08:00
f1e9f30e78 + find 1. Recursively count all the files in a directory
+ Condition and loop
1.  Using test command, to test if the length of variable is nonzero

+ Others
1. Create a screen and name it 'test'

+ Time
1. Get the seconds since epoch (Jan 1 1970) for a given date (e.g Mar 16 
2021)
2. Convert the number of seconds since epoch back to date

+ fix some markdown syntax issue
2021-03-16 22:10:49 +08:00
248ae4c50e Merge branch 'master' of https://github.com/onceupon/Bash-Oneliner 2021-03-16 22:08:23 +08:00
5740e44d62 + find 1. Recursively count all the files in a directory
+ Condition and loop
1.  Using test command, to test if the length of variable is nonzero

+ Others
1. Create a screen and name it 'test'

+ Time
1. Get the seconds since epoch (Jan 1 1970) for a given date (e.g Mar 16 
2021)
2. Convert the number of seconds since epoch back to date

+ fix some markdown syntax issue
2021-03-16 22:08:02 +08:00
19d402cc84 + find
1. Recursively count all the files in a directory

+ Condition and loop
1.  Using test command, to test if the length of variable is nonzero

+ Others
1. Create a screen and name it 'test'

+ Time
1. Get the seconds since epoch (Jan 1 1970) for a given date (e.g Mar 16 
2021)
2. Convert the number of seconds since epoch back to date

+ fix some markdown syntax issue
2021-03-16 22:03:57 +08:00
244a568bd7 fix typo mentioned in issue https://github.com/onceupon/Bash-Oneliner/issues/18 2021-03-16 20:13:56 +08:00
093edef55a + time
returns the current time with nanoseconds.
2021-03-16 19:43:41 +08:00

430
README.md
View File

@ -1,7 +1,7 @@
# Bash-Oneliner
I am glad that you are here! I was working on bioinformatics a few years ago and was amazed by those single-word bash commands which are much faster than my dull scripts, time saved through learning command-line shortcuts and scripting. Recent years I am working on cloud computing and I keep recording those useful commands here. Not all of them is oneliner, but i put effort on making them brief and swift. I am mainly using Ubuntu, Amazon Linux, RedHat, Linux Mint, Mac and CentOS, sorry if the commands don't work on your system.
This blog will focus on simple bash commands for parsing data and Linux system maintenance that i acquired from work and LPIC exam. I apologize that there are no detailed citation for all the commands, but they are probably from dear Google and Stackoverflow.
This blog will focus on simple bash commands for parsing data and Linux system maintenance that i acquired from work and LPIC exam. I apologize that there are no detailed citation for all the commands, but they are probably from dear Google and Stack Overflow.
English and bash are not my first language, please correct me anytime, thank you.
If you know other cool commands, please teach me!
@ -12,13 +12,13 @@ Here's a more stylish version of [Bash-Oneliner](https://onceupon.github.io/Bash
- [Terminal Tricks](#terminal-tricks)
- [Variable](#variable)
- [Math](#math)
- [Grep](#grep)
- [Sed](#sed)
- [Awk](#awk)
- [Xargs](#xargs)
- [Find](#find)
- [Condition and Loop](#condition-and-loop)
- [Math](#math)
- [Time](#time)
- [Download](#download)
- [Random](#random)
@ -33,22 +33,23 @@ Here's a more stylish version of [Bash-Oneliner](https://onceupon.github.io/Bash
##### Using Ctrl keys
```
Ctrl + a : move to the beginning of line.
Ctrl + d : if you've type something, Ctrl + d deletes the character under the cursor, else, it escapes the current shell.
Ctrl + e : move to the end of line.
Ctrl + k : delete all text from the cursor to the end of line.
Ctrl + l : equivalent to clear.
Ctrl + n : same as Down arrow.
Ctrl + p : same as Up arrow.
Ctrl + q : to resume output to terminal after Ctrl + s.
Ctrl + r : begins a backward search through command history.(keep pressing Ctrl + r to move backward)
Ctrl + s : to stop output to terminal.
Ctrl + q : to resume output to terminal after Ctrl + s.
Ctrl + a : move to the beginning of line.
Ctrl + e : move to the end of line.
Ctrl + d : if you've type something, Ctrl + d deletes the character under the cursor, else, it escapes the current shell.
Ctrl + k : delete all text from the cursor to the end of line.
Ctrl + x + backspace : delete all text from the beginning of line to the cursor.
Ctrl + t : transpose the character before the cursor with the one under the cursor, press Esc + t to transposes the two words before the cursor.
Ctrl + w : cut the word before the cursor; then Ctrl + y paste it
Ctrl + u : cut the line before the cursor; then Ctrl + y paste it
Ctrl + _ : undo typing.
Ctrl + l : equivalent to clear.
Ctrl + w : cut the word before the cursor; then Ctrl + y paste it
Ctrl + x + backspace : delete all text from the beginning of line to the cursor.
Ctrl + x + Ctrl + e : launch editor defined by $EDITOR to input your command. Useful for multi-line commands.
Ctrl + z : stop current running process and keep it in background. You can use `fg` to continue the process in the foreground, or `bg` to continue the process in the background.
Ctrl + _ : undo typing.
```
##### Change case
```bash
@ -57,8 +58,9 @@ Esc + u
Esc + l
# converts text from cursor to the end of the word to lowercase.
Esc + c
# converts letter under the cursor to uppercase.
# converts letter under the cursor to uppercase, rest of the word to lowercase.
```
##### Run history number (e.g. 53)
```bash
!53
@ -69,7 +71,6 @@ Esc + c
!!
# run the previous command using sudo
sudo !!
# of course you need to enter your password
```
##### Run last command and change some parameter using caret substitution (e.g. last command: echo 'aaa' -> rerun as: echo 'bbb')
@ -98,16 +99,16 @@ sudo !!
##### Bash globbing
```bash
# '*' serves as a "wild card" for filename expansion.
/b?n/?at #/bin/cat
# '?' serves as a single-character "wild card" for filename expansion.
/etc/pa*wd #/etc/passwd
# [] serves to match the character from a range.
# '?' serves as a single-character "wild card" for filename expansion.
/b?n/?at #/bin/cat
# '[]' serves to match the character from a range.
ls -l [a-z]* #list all files with alphabet in its filename.
# {} can be used to match filenames with more than one patterns
ls {*.sh,*.py} #list all .sh and .py files
# '{}' can be used to match filenames with more than one patterns
ls *.{sh,py} #list all .sh and .py files
```
##### Some handy environment variables
@ -119,6 +120,7 @@ $? :most recent foreground pipeline exit status.
$- :current options set for the shell.
$$ :pid of the current shell (not subshell).
$! :is the PID of the most recent background command.
$_ :last argument of the previously executed command, or the path of the bash script.
$DESKTOP_SESSION current display manager
$EDITOR preferred text editor.
@ -130,14 +132,49 @@ $USER current username
$HOSTNAME current hostname
```
##### Using vi-mode in your shell
```bash
set -o vi
# change bash shell to vi mode
# then hit the Esc key to change to vi edit mode (when `set -o vi` is set)
k
# in vi edit mode - previous command
j
# in vi edit mode - next command
0
# in vi edit mode - beginning of the command
R
# in vi edit mode - replace current characters of command
2w
# in vi edit mode - next to 2nd word
b
# in vi edit mode - previous word
i
# in vi edit mode - go to insert mode
v
# in vi edit mode - edit current command in vi
man 3 readline
# man page for complete readline mapping
```
## Variable
[[back to top](#handy-bash-one-liners)]
##### Variable substitution within quotes
```bash
# foo=bar
echo $foo
# bar
echo "$foo"
# bar
# single quotes cause variables to not be expanded
echo '$foo'
# $foo
# single quotes within double quotes will not cancel expansion and will be part of the output
echo "'$foo'"
# 'bar'
# double/single quotes around single quotes make the inner single quotes expand variables
# doubled single quotes act as if there are no quotes at all
echo ''$foo''
# bar
```
##### Get the length of variable
```bash
@ -178,9 +215,11 @@ echo ${var[@]#0}
```bash
{var//a/,}
```
##### Grep lines with strings from a file (e.g. lines with 'stringA or 'stringB' or 'stringC')
```bash
# with grep
test="god the father"
test="stringA stringB stringC"
grep ${test// /\\\|} file.txt
# turning the space into 'or' (\|) in grep
```
@ -199,15 +238,21 @@ eval "$cmd"
echo "$bar" # foo
```
##### Record a terminal session
```bash
# https://github.com/asciinema/asciinema
asciinema rec demo.cast
```
## Math
[[back to top](#handy-bash-one-liners)]
##### Arithmetic Expansion in Bash (Operators: +, -, *, /, %, etc)
```bash
echo $(( 10 + 5 )) #15
x=1
echo $(( x++ )) #1 , notice that it is still 1, since it's post-incremen
echo $(( x++ )) #1 , notice that it is still 1, since it's post-increment
echo $(( x++ )) #2
echo $(( ++x )) #4 , notice that it is not 3 since it's pre-incremen
echo $(( ++x )) #4 , notice that it is not 3 since it's pre-increment
echo $(( x-- )) #4
echo $(( x-- )) #3
echo $(( --x )) #1
@ -265,10 +310,10 @@ echo "var=5;--var"| bc
##### Type of grep
```bash
grep = grep -G # Basic Regular Expression (BRE)
fgrep = grep -F # fixed text, ignoring meta-charachetrs
fgrep = grep -F # fixed text, ignoring meta-characters
egrep = grep -E # Extended Regular Expression (ERE)
pgrep = grep -P # Perl Compatible Regular Expressions (PCRE)
rgrep = grep -r # recursive
grep -P # Perl Compatible Regular Expressions (PCRE)
```
##### Grep and count number of empty lines
@ -280,15 +325,15 @@ grep -c "^$"
```bash
grep -o '[0-9]*'
# or
grep -oP '\d'
grep -oP '\d*'
```
##### Grep integer with certain number of digits (e.g. 3)
```bash
grep [0-9]\{3\}
grep '[0-9]\{3\}'
# or
grep -E [0-9]{3}
grep -E '[0-9]{3}'
# or
grep -P \d{3}
grep -P '\d{3}'
```
##### Grep only IP address
@ -395,14 +440,14 @@ grep 'A\|B\|C\|D'
grep 'A.*B'
```
##### Regex any singer character (e.g. ACB or AEB)
##### Regex any single character (e.g. ACB or AEB)
```bash
grep 'A.B'
```
##### Regex with or without a certain character (e.g. color or colour)
```bash
grep colou?r
grep 'colou\?r'
```
##### Grep all content of a fileA from fileB
@ -456,9 +501,10 @@ sed 1,100d filename
##### Remove lines with string (e.g. 'bbo')
```bash
sed "/bbo/d" filename
- case insensitive:
# case insensitive:
sed "/bbo/Id" filename
```
##### Remove lines whose nth character not equal to a value (e.g. 5th character not equal to 2)
```bash
sed -E '/^.{5}[^2]/d'
@ -505,12 +551,12 @@ sed -i '$ s/.$//' filename
##### Add string to beginning of file (e.g. "\[")
```bash
sed -i '1s/^/[/' file
sed -i '1s/^/[/' filename
```
##### Add string at certain line number (e.g. add 'something' to line 1 and line 3)
```bash
sed -e '1isomething -e '3isomething'
sed -e '1isomething' -e '3isomething'
```
##### Add string to end of file (e.g. "]")
@ -524,7 +570,7 @@ sed '$a\'
##### Add string to beginning of every line (e.g. 'bbo')
```bash
sed -e 's/^/bbo/' file
sed -e 's/^/bbo/' filename
```
##### Add string to end of each line (e.g. "}")
@ -537,7 +583,14 @@ sed -e 's/$/\}\]/' filename
sed 's/.\{4\}/&\n/g'
```
##### Concatenate/combine/join files with a seperator and next line (e.g separate by ",")
##### Add a line after the line that matches the pattern (e.g. add a new line with "world" after the line with "hello")
```bash
sed '/hello*/a world' filename
# hello
# world
```
##### Concatenate/combine/join files with a separator and next line (e.g separate by ",")
```bash
sed -s '$a,' *.json > all.json
```
@ -645,7 +698,7 @@ sed '$ s/.$//'
##### Insert character at specified position of file (e.g. AAAAAA --> AAA#AAA)
```bash
sed -r -e 's/^.{3}/&#/' file
sed -r -e 's/^.{3}/&#/' filename
```
@ -700,7 +753,7 @@ awk -v N=7 '{print}/bbo/&& --N<=0 {exit}'
##### Print filename and last line of all files in directory
```bash
ls|xargs -n1 -I file awk '{s=$0};END{print FILENAME,s}' file
ls|xargs -n1 -I file awk '{s=$0};END{print FILENAME,s}' filename
```
##### Add string to the beginning of a column (e.g add "chr" to column $3)
@ -710,12 +763,12 @@ awk 'BEGIN{OFS="\t"}$3="chr"$3'
##### Remove lines with string (e.g. 'bbo')
```bash
awk '!/bbo/' file
awk '!/bbo/' filename
```
##### Remove last column
```bash
awk 'NF{NF-=1};1' file
awk 'NF{NF-=1};1' filename
```
##### Usage and meaning of NR and FNR
@ -774,13 +827,13 @@ awk '{printf("%s\t%s\n",NR,$0)}'
##### Break combine column data into rows
```bash
# For example, seperate the following content:
# For example, separate the following content:
# David cat,dog
# into
# David cat
# David dog
awk '{split($2,a,",");for(i in a)print $1"\t"a[i]}' file
awk '{split($2,a,",");for(i in a)print $1"\t"a[i]}' filename
# Detail here: http://stackoverflow.com/questions/33408762/bash-turning-single-comma-separated-column-into-multi-line-string
```
@ -982,6 +1035,11 @@ find . -type f -empty
find . -type f -empty -delete
```
##### Recursively count all the files in a directory
```bash
find . -type f | wc -l
```
## Condition and loop
[[back to top](#handy-bash-one-liners)]
@ -991,7 +1049,7 @@ find . -type f -empty -delete
if [[ "$c" == "read" ]]; then outputdir="seq"; else outputdir="write" ; fi
# Test if myfile contains the string 'test':
if grep -q hello myfile; then
if grep -q hello myfile; then echo -e "file contains the string!" ; fi
# Test if mydir is a directory, change to it and do other stuff:
if cd mydir; then
@ -1000,9 +1058,13 @@ else
echo >&2 "Fatal error. This script requires mydir."
fi
# if variable is null
if [ ! -s "myvariable" ]
#True of the length if "STRING" is zero.
# Check if a variable is null
if [ -z "$var" ]; then echo "NULL"; else echo "Not NULL"; fi
# or
[ -z "$var" ] && echo "NULL"
# Using test command (same as []), to test if the length of variable is nonzero
test -n "$myvariable" && echo myvariable is "$myvariable" || echo myvariable is not set
# Test if file exist
if [ -e 'filename' ]
@ -1017,16 +1079,17 @@ then
fi
# Test if the value of x is greater or equal than 5
if [ "$x" -ge 5 ]; then
if [ "$x" -ge 5 ]; then echo -e "greater or equal than 5!" ; fi
# Test if the value of x is greater or equal than 5, in bash/ksh/zsh:
if ((x >= 5)); then
if ((x >= 5)); then echo -e "greater or equal than 5!" ; fi
# Use (( )) for arithmetic operation
if ((j==u+2))
if ((j==u+2)); then echo -e "j==u+2!!" ; fi
# Use [[ ]] for comparison
if [[ $age -gt 21 ]]
if [[ $age -gt 21 ]]; then echo -e "forever 21!!" ; fi
```
[More if commands](http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_07_01.html)
@ -1106,6 +1169,45 @@ date +%F
# or
date +'%d-%b-%Y-%H:%M:%S'
# 10-Apr-2020-21:54:40
# Returns the current time with nanoseconds.
date +"%T.%N"
# 11:42:18.664217000
# Get the seconds since epoch (Jan 1 1970) for a given date (e.g Mar 16 2021)
date -d "Mar 16 2021" +%s
# 1615852800
# or
date -d "Tue Mar 16 00:00:00 UTC 2021" +%s
# 1615852800
# Convert the number of seconds since epoch back to date
date --date @1615852800
# Tue Mar 16 00:00:00 UTC 2021
```
##### Print current time point for N days ago or N days after
```bash
# print current date first (for the following example)
date +"%F %H:%M:%S"
# 2023-03-11 16:17:09
# print the time that is 1 day ago
date -d"1 day ago" +"%F %H:%M:%S"
# 2023-03-10 16:17:09
# print the time that is 7 days ago
date -d"7 days ago" +"%F %H:%M:%S"
# 2023-03-04 16:17:09
# print the time that is a week ago
date -d"1 week ago" +"%F %H:%M:%S"
# 2023-03-04 16:17:09
# add 1 day to date
date -d"-1 day ago" +"%F %H:%M:%S"
# 2023-03-12 16:17:09
```
##### wait for random duration (e.g. sleep 1-5 second, like adding a jitter)
@ -1456,9 +1558,17 @@ stat filename.txt
ps aux
```
##### List processes by top memory usage
```bash
ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head
```
##### Display a tree of processes
```bash
pstree
# or
ps aux --forest
```
##### Find maximum number of processes
@ -1707,6 +1817,18 @@ du -h
du -sk /var/log/* |sort -rn |head -10
```
##### check the Inode utilization
```
df -i
# Filesystem Inodes IUsed IFree IUse% Mounted on
# devtmpfs 492652 304 492348 1% /dev
# tmpfs 497233 2 497231 1% /dev/shm
# tmpfs 497233 439 496794 1% /run
# tmpfs 497233 16 497217 1% /sys/fs/cgroup
# /dev/nvme0n1p1 5037976 370882 4667094 8% /
# tmpfs 497233 1 497232 1% /run/user/1000
```
##### Show all file system type
```bash
df -TH
@ -1852,7 +1974,7 @@ ldconfig -p
ldd /bin/ls
```
##### Check user login
##### Check the most recent login of all users
```bash
lastlog
```
@ -2002,11 +2124,6 @@ killall pulseaudio
# then press Alt-F2 and type in pulseaudio
```
##### When sound not working
```bash
killall pulseaudio
```
##### List information about SCSI devices
```bash
lsscsi
@ -2163,13 +2280,19 @@ sar -n ALL
# reading SAR log file using -f
sar -f /var/log/sa/sa31|tail
```
##### Reading from journal file
```bash
journalctl --file ./log/journal/a90c18f62af546ccba02fa3734f00a04/system.journal --since "2020-02-11 00:00:00"
```
##### Show a listing of last logged in users.
##### Show a listing of last logged in users
```bash
last
```
##### Show a listing of unsuccessful (bad) login attempts
```bash
lastb
```
@ -2347,6 +2470,11 @@ dig +short www.example.com
host www.example.com
```
##### Check public IP address
```bash
curl http://checkip.amazonaws.com
```
##### Get DNS TXT record a of domain
```bash
dig -t txt www.example.com
@ -2380,14 +2508,14 @@ nc -vw5 google.com 22
# From server A:
$ sudo nc -l 80
# then you can connect to the 80 port from another server (e.g. server B):
# e.g. telent <server A IP address> 80
# e.g. telnet <server A IP address> 80
# then type something in server B
# and you will see the result in server A!
```
##### Check which ports are listening for TCP connections from the network
```bash
#notice that some companies might not like you using nmap
# note that some companies might not like you using nmap
nmap -sT -O localhost
# check port 0-65535
@ -2395,7 +2523,7 @@ nmap -p0-65535 localhost
```
##### Check if a host is up and scan for open ports, also skip host discovery.
```bash
#skips checking if the host is alive which may sometimes cause a false positive and stop the scan.
# skips checking if the host is alive. this may sometimes cause a false positive, stopping the scan.
$ nmap google.com -Pn
# Example output:
@ -2417,7 +2545,7 @@ $ nmap -A -T4 scanme.nmap.org
# -A to enable OS and version detection, script scanning, and traceroute; -T4 for faster execution
```
##### Look up website information (e.g. name server), searches for an object in a RFC 3912 database.
##### Look up website information (e.g. name server), searches for an object in a RFC 3912 database
```bash
whois google.com
```
@ -2427,7 +2555,7 @@ whois google.com
openssl s_client -showcerts -connect www.example.com:443
```
##### Display IP address
##### Display network interfaces and their associated IP addresses
```bash
ip a
```
@ -2487,6 +2615,10 @@ curl -I http://example.com/
# Accept-Ranges: bytes
# Vary: Accept-Encoding
```
##### Find out the time spent between request and response
```
curl -v -o /dev/null -s -w 'Total: %{time_total}s\n' google.com
```
##### Find out the http status code of a URL
```bash
@ -2522,16 +2654,16 @@ sudo iptables A INPUT s <IP> -p tcp dport 80 j DROP
```bash
# If file is not specified, the file /usr/share/dict/words is used.
look phy|head -n 10
# Phil
# Philadelphia
# Philadelphia's
# Philby
# Philby's
# Philip
# Philippe
# Philippe's
# Philippians
# Philippine
# phycic
# Phyciodes
# phycite
# Phycitidae
# phycitol
# phyco-
# phycochrom
# phycochromaceae
# phycochromaceous
# phycochrome
```
##### Repeat printing string n times (e.g. print 'hello world' five times)
@ -2589,6 +2721,40 @@ sdiff fileA fileB
diff fileA fileB --strip-trailing-cr
```
##### Find common/differing lines
```bash
# having two sorted and uniqed files (for example after running `$ sort -uo fileA fileA` and same for fileB):
# ------
# fileA:
# ------
# joey
# kitten
# piglet
# puppy
# ------
# fileB:
# ------
# calf
# chick
# joey
# puppy
#
# Find lines in both files
comm -12 fileA fileB
# joey
# puppy
#
# Find lines in fileB that are NOT in fileA
comm -13 fileA fileB
# calf
# chick
#
# Find lines in fileA that are NOT in fileB
comm -23 fileA fileB
# kitten
# piglet
```
##### Number a file (e.g. fileA)
```bash
@ -2733,7 +2899,7 @@ echo -e 'text here \c'
##### View first 50 characters of file
```bash
head -c 50 file
head -c 50 filename
```
##### Cut and get last column of a file
@ -2753,12 +2919,33 @@ var=$((var+1))
cat filename|rev|cut -f1|rev
```
##### Cat to a file
##### Create or replace a file with contents
```bash
cat >myfile
let me add sth here
exit by control + c
^C
# exit with ctrl+d
# or using tee
tee myfile
let me add sth else here
# exit with ctrl+d
```
##### Append to a file with contents
```bash
cat >>myfile
let me add sth here
# exit with ctrl+d
# or
cat << EoF >> filename
> add something here
> EoF
# or using tee
tee -a myfile
let me add sth else here
# exit with ctrl+d
```
##### Clear the contents of a file (e.g. filename)
@ -2773,7 +2960,7 @@ echo 'hihi' >>filename
##### Working with json data
```bash
#install the useful jq package
# Install the useful jq package
# sudo apt-get install jq
# e.g. to get all the values of the 'url' key, simply pipe the json to the following jq command(you can use .[]. to select inner json, i.e jq '.[].url')
cat file.json | jq '.url'
@ -2848,7 +3035,7 @@ while read a b; do yes $b |head -n $a ;done <test.txt
## Others
[[back to top](#handy-bash-one-liners)]
##### Describe the format and characteristics of image files.
##### Describe the format and characteristics of image files
```bash
identify myimage.png
#myimage.png PNG 1049x747 1049x747+0+0 8-bit sRGB 1.006MB 0.000u 0:00.000
@ -2877,6 +3064,13 @@ cal
# only display November
cal -m 11
```
##### Convert the hexadecimal MD5 checksum value into its base64-encoded format.
```bash
openssl md5 -binary /path/to/file| base64
# NWbeOpeQbtuY0ATWuUeumw==
```
##### Forces applications to use the default language for output
```bash
export LC_ALL=C
@ -3014,11 +3208,40 @@ rsync -av directory user@ip_address:/path/to/directory.bak
# skip files that are newer on receiver (i prefer this one!)
```
##### Create a temporary directory and `cd` into it
```bash
cd $(mktemp -d)
# for example, this will create a temporary directory "/tmp/tmp.TivmPLUXFT"
```
##### Make all directories at one time!
```bash
mkdir -p project/{lib/ext,bin,src,doc/{html,info,pdf},demo/stat}
# -p: make parent directory
# this will create project/doc/html/; project/doc/info; project/lib/ext ,etc
# this will create:
# project/
# project/bin/
# project/demo/
# project/demo/stat/
# project/doc/
# project/doc/html/
# project/doc/info/
# project/doc/pdf/
# project/lib/
# project/lib/ext/
# project/src/
#
# project/
# ├── bin
# ├── demo
# │ └── stat
# ├── doc
# │ ├── html
# │ ├── info
# │ └── pdf
# ├── lib
# │ └── ext
# └── src
```
##### Run command only if another command returns zero exit status (well done)
@ -3088,6 +3311,12 @@ scp -r directoryname user@ip:/path/to/send
# :(){:|:&};:
```
##### Trigger kernel crash
```bash
# Don't try this at home!
echo c > /proc/sysrq-trigger
```
##### Use the last argument
```bash
!$
@ -3098,7 +3327,7 @@ scp -r directoryname user@ip:/path/to/send
echo $?
```
##### Extract .xf
##### Extract .xz
```
unxz filename.tar.xz
# then
@ -3138,8 +3367,7 @@ yes n
# or 'anything':
yes anything
# For example:
```bash
# pipe yes to other command
yes | rm -r large_directory
```
@ -3150,9 +3378,9 @@ fallocate -l 10G 10Gigfile
##### Create dummy file of certain size (e.g. 200mb)
```bash
dd if=/dev/zero of=//dev/shm/200m bs=1024k count=200
dd if=/dev/zero of=/dev/shm/200m bs=1024k count=200
# or
dd if=/dev/zero of=//dev/shm/200m bs=1M count=200
dd if=/dev/zero of=/dev/shm/200m bs=1M count=200
# Standard output:
# 200+0 records in
@ -3165,9 +3393,29 @@ dd if=/dev/zero of=//dev/shm/200m bs=1M count=200
watch -n 1 wc -l filename
```
##### Use Bash Strict Mode
```bash
# These options can make your code safer but, depending on how your pipeline is written, it might be too aggressive
# or it might not catch the errors that you are interested in
# for reference see https://gist.github.com/mohanpedala/1e2ff5661761d3abd0385e8223e16425
# and https://mywiki.wooledge.org/BashPitfalls#set_-euo_pipefail
set -o errexit # exit immediately if a pipeline returns a non-zero status
set -o errtrace # trap ERR from shell functions, command substitutions, and commands from subshell
set -o nounset # treat unset variables as an error
set -o pipefail # pipe will exit with last non-zero status, if applicable
set -Eue -o pipefail # shorthand for above (pipefail has no short option)
```
##### Print commands and their arguments when execute (e.g. echo `expr 10 + 20 `)
```bash
set -x; echo `expr 10 + 20 `
# or
set -o xtrace; echo `expr 10 + 20 `
# to turn it off..
set +x
```
##### Print some meaningful sentences to you (install fortune first)
@ -3198,6 +3446,9 @@ q -d "," "select c3,c4,c5 from /path/to/file.txt where c3='foo' and c5='boo'"
# Create session and attach:
screen
# Create a screen and name it 'test'
screen -S test
# Create detached session foo:
screen -S foo -d -m
@ -3216,15 +3467,16 @@ screen -r foo
# Kill session foo:
screen -r foo -X quit
# Scroll:
Hit your screen prefix combination (C-a / control+A), then hit Escape.
Move up/down with the arrow keys (↑ and ↓).
# Hit your screen prefix combination (C-a / control+A), then hit Escape.
# Move up/down with the arrow keys (↑ and ↓).
# Redirect output of an already running process in Screen:
(C-a / control+A), then hit 'H'
# (C-a / control+A), then hit 'H'
# Store screen output for Screen:
Ctrl+A, Shift+H
# Ctrl+A, Shift+H
# You will then find a screen.log file under current directory.
```
@ -3351,6 +3603,8 @@ source .venv/bin/activate
type pip
# 5. Now you can install your pip package, here requirements.txt is simply a txt file containing all the packages you want. (e.g tornado==4.5.3).
pip install -r requirements.txt
# 6. Exit virtual environment
deactivate
```