mirror of
https://github.com/onceupon/Bash-Oneliner.git
synced 2025-07-05 20:48:48 +02:00
Compare commits
1 Commits
16197f576e
...
revert-16-
| Author | SHA1 | Date | |
|---|---|---|---|
| 6a12e9ce06 |
206
README.md
206
README.md
@ -1,7 +1,7 @@
|
||||
# Bash-Oneliner
|
||||
I am glad that you are here! I was working on bioinformatics a few years ago and was amazed by those single-word bash commands which are much faster than my dull scripts, time saved through learning command-line shortcuts and scripting. Recent years I am working on cloud computing and I keep recording those useful commands here. Not all of them is oneliner, but i put effort on making them brief and swift. I am mainly using Ubuntu, Amazon Linux, RedHat, Linux Mint, Mac and CentOS, sorry if the commands don't work on your system.
|
||||
|
||||
This blog will focus on simple bash commands for parsing data and Linux system maintenance that i acquired from work and LPIC exam. I apologize that there are no detailed citation for all the commands, but they are probably from dear Google and Stack Overflow.
|
||||
This blog will focus on simple bash commands for parsing data and Linux system maintenance that i acquired from work and LPIC exam. I apologize that there are no detailed citation for all the commands, but they are probably from dear Google and Stackoverflow.
|
||||
|
||||
English and bash are not my first language, please correct me anytime, thank you.
|
||||
If you know other cool commands, please teach me!
|
||||
@ -12,13 +12,13 @@ Here's a more stylish version of [Bash-Oneliner](https://onceupon.github.io/Bash
|
||||
|
||||
- [Terminal Tricks](#terminal-tricks)
|
||||
- [Variable](#variable)
|
||||
- [Math](#math)
|
||||
- [Grep](#grep)
|
||||
- [Sed](#sed)
|
||||
- [Awk](#awk)
|
||||
- [Xargs](#xargs)
|
||||
- [Find](#find)
|
||||
- [Condition and Loop](#condition-and-loop)
|
||||
- [Math](#math)
|
||||
- [Time](#time)
|
||||
- [Download](#download)
|
||||
- [Random](#random)
|
||||
@ -57,7 +57,7 @@ Esc + u
|
||||
Esc + l
|
||||
# converts text from cursor to the end of the word to lowercase.
|
||||
Esc + c
|
||||
# converts letter under the cursor to uppercase, rest of the word to lowercase.
|
||||
# converts letter under the cursor to uppercase.
|
||||
```
|
||||
##### Run history number (e.g. 53)
|
||||
```bash
|
||||
@ -69,6 +69,7 @@ Esc + c
|
||||
!!
|
||||
# run the previous command using sudo
|
||||
sudo !!
|
||||
# of course you need to enter your password
|
||||
```
|
||||
|
||||
##### Run last command and change some parameter using caret substitution (e.g. last command: echo 'aaa' -> rerun as: echo 'bbb')
|
||||
@ -97,16 +98,16 @@ sudo !!
|
||||
##### Bash globbing
|
||||
```bash
|
||||
# '*' serves as a "wild card" for filename expansion.
|
||||
/etc/pa*wd #/etc/passwd
|
||||
|
||||
# '?' serves as a single-character "wild card" for filename expansion.
|
||||
/b?n/?at #/bin/cat
|
||||
|
||||
# '[]' serves to match the character from a range.
|
||||
# '?' serves as a single-character "wild card" for filename expansion.
|
||||
/etc/pa*wd #/etc/passwd
|
||||
|
||||
# ‘[]’ serves to match the character from a range.
|
||||
ls -l [a-z]* #list all files with alphabet in its filename.
|
||||
|
||||
# '{}' can be used to match filenames with more than one patterns
|
||||
ls *.{sh,py} #list all .sh and .py files
|
||||
# ‘{}’ can be used to match filenames with more than one patterns
|
||||
ls {*.sh,*.py} #list all .sh and .py files
|
||||
```
|
||||
|
||||
##### Some handy environment variables
|
||||
@ -134,19 +135,9 @@ $HOSTNAME current hostname
|
||||
##### Variable substitution within quotes
|
||||
```bash
|
||||
# foo=bar
|
||||
echo $foo
|
||||
# bar
|
||||
echo "$foo"
|
||||
# bar
|
||||
# single quotes cause variables to not be expanded
|
||||
echo '$foo'
|
||||
# $foo
|
||||
# single quotes within double quotes will not cancel expansion and will be part of the output
|
||||
echo "'$foo'"
|
||||
#'bar'
|
||||
# doubled single quotes act as double quotes making variables expand
|
||||
echo ''$foo''
|
||||
# bar
|
||||
# double/single quotes around single quotes make the inner single quotes expand variables
|
||||
```
|
||||
##### Get the length of variable
|
||||
```bash
|
||||
@ -187,11 +178,9 @@ echo ${var[@]#0}
|
||||
```bash
|
||||
{var//a/,}
|
||||
```
|
||||
|
||||
##### Grep lines with strings from a file (e.g. lines with 'stringA or 'stringB' or 'stringC')
|
||||
```bash
|
||||
#with grep
|
||||
test="stringA stringB stringC"
|
||||
test="god the father"
|
||||
grep ${test// /\\\|} file.txt
|
||||
# turning the space into 'or' (\|) in grep
|
||||
```
|
||||
@ -216,9 +205,9 @@ echo "$bar" # foo
|
||||
```bash
|
||||
echo $(( 10 + 5 )) #15
|
||||
x=1
|
||||
echo $(( x++ )) #1 , notice that it is still 1, since it's post-increment
|
||||
echo $(( x++ )) #1 , notice that it is still 1, since it's post-incremen
|
||||
echo $(( x++ )) #2
|
||||
echo $(( ++x )) #4 , notice that it is not 3 since it's pre-increment
|
||||
echo $(( ++x )) #4 , notice that it is not 3 since it's pre-incremen
|
||||
echo $(( x-- )) #4
|
||||
echo $(( x-- )) #3
|
||||
echo $(( --x )) #1
|
||||
@ -276,7 +265,7 @@ echo "var=5;--var"| bc
|
||||
##### Type of grep
|
||||
```bash
|
||||
grep = grep -G # Basic Regular Expression (BRE)
|
||||
fgrep = grep -F # fixed text, ignoring meta-characters
|
||||
fgrep = grep -F # fixed text, ignoring meta-charachetrs
|
||||
egrep = grep -E # Extended Regular Expression (ERE)
|
||||
pgrep = grep -P # Perl Compatible Regular Expressions (PCRE)
|
||||
rgrep = grep -r # recursive
|
||||
@ -291,15 +280,15 @@ grep -c "^$"
|
||||
```bash
|
||||
grep -o '[0-9]*'
|
||||
#or
|
||||
grep -oP '\d*'
|
||||
grep -oP '\d'
|
||||
```
|
||||
##### Grep integer with certain number of digits (e.g. 3)
|
||||
```bash
|
||||
grep '[0-9]\{3\}'
|
||||
grep ‘[0-9]\{3\}’
|
||||
# or
|
||||
grep -E '[0-9]{3}'
|
||||
grep -E ‘[0-9]{3}’
|
||||
# or
|
||||
grep -P '\d{3}'
|
||||
grep -P ‘\d{3}’
|
||||
```
|
||||
|
||||
##### Grep only IP address
|
||||
@ -406,14 +395,14 @@ grep 'A\|B\|C\|D'
|
||||
grep 'A.*B'
|
||||
```
|
||||
|
||||
##### Regex any single character (e.g. ACB or AEB)
|
||||
##### Regex any singer character (e.g. ACB or AEB)
|
||||
```bash
|
||||
grep 'A.B'
|
||||
```
|
||||
|
||||
##### Regex with or without a certain character (e.g. color or colour)
|
||||
```bash
|
||||
grep 'colou\?r'
|
||||
grep ‘colou?r’
|
||||
```
|
||||
|
||||
##### Grep all content of a fileA from fileB
|
||||
@ -467,10 +456,9 @@ sed 1,100d filename
|
||||
##### Remove lines with string (e.g. 'bbo')
|
||||
```bash
|
||||
sed "/bbo/d" filename
|
||||
# case insensitive:
|
||||
- case insensitive:
|
||||
sed "/bbo/Id" filename
|
||||
```
|
||||
|
||||
##### Remove lines whose nth character not equal to a value (e.g. 5th character not equal to 2)
|
||||
```bash
|
||||
sed -E '/^.{5}[^2]/d'
|
||||
@ -522,7 +510,7 @@ sed -i '1s/^/[/' file
|
||||
|
||||
##### Add string at certain line number (e.g. add 'something' to line 1 and line 3)
|
||||
```bash
|
||||
sed -e '1isomething' -e '3isomething'
|
||||
sed -e '1isomething -e '3isomething'
|
||||
```
|
||||
|
||||
##### Add string to end of file (e.g. "]")
|
||||
@ -549,7 +537,7 @@ sed -e 's/$/\}\]/' filename
|
||||
sed 's/.\{4\}/&\n/g'
|
||||
```
|
||||
|
||||
##### Concatenate/combine/join files with a separator and next line (e.g separate by ",")
|
||||
##### Concatenate/combine/join files with a seperator and next line (e.g separate by ",")
|
||||
```bash
|
||||
sed -s '$a,' *.json > all.json
|
||||
```
|
||||
@ -786,7 +774,7 @@ awk '{printf("%s\t%s\n",NR,$0)}'
|
||||
|
||||
##### Break combine column data into rows
|
||||
```bash
|
||||
# For example, separate the following content:
|
||||
# For example, seperate the following content:
|
||||
# David cat,dog
|
||||
# into
|
||||
# David cat
|
||||
@ -906,7 +894,7 @@ find /dir/to/A -type f -name "*.py" -print 0| xargs -0 -r -I file cp -v -p file
|
||||
|
||||
##### With sed
|
||||
```bash
|
||||
ls |xargs -n1 -I file sed -i '/^Pos/d' file
|
||||
ls |xargs -n1 -I file sed -i '/^Pos/d' filename
|
||||
```
|
||||
|
||||
##### Add the file name to the first line of file
|
||||
@ -994,11 +982,6 @@ find . -type f -empty
|
||||
find . -type f -empty -delete
|
||||
```
|
||||
|
||||
##### Recursively count all the files in a directory
|
||||
```bash
|
||||
find . -type f | wc -l
|
||||
```
|
||||
|
||||
## Condition and loop
|
||||
[[back to top](#handy-bash-one-liners)]
|
||||
|
||||
@ -1008,7 +991,7 @@ find . -type f | wc -l
|
||||
if [[ "$c" == "read" ]]; then outputdir="seq"; else outputdir="write" ; fi
|
||||
|
||||
# Test if myfile contains the string 'test':
|
||||
if grep -q hello myfile; then echo -e "file contains the string!" ; fi
|
||||
if grep -q hello myfile; then …
|
||||
|
||||
# Test if mydir is a directory, change to it and do other stuff:
|
||||
if cd mydir; then
|
||||
@ -1018,12 +1001,9 @@ else
|
||||
fi
|
||||
|
||||
# if variable is null
|
||||
if [ ! -s "myvariable" ]; then echo -e "variable is null!" ; fi
|
||||
if [ ! -s "myvariable" ]
|
||||
#True of the length if "STRING" is zero.
|
||||
|
||||
# Using test command (same as []), to test if the length of variable is nonzero
|
||||
test -n "$myvariable" && echo myvariable is "$myvariable" || echo myvariable is not set
|
||||
|
||||
# Test if file exist
|
||||
if [ -e 'filename' ]
|
||||
then
|
||||
@ -1037,17 +1017,16 @@ then
|
||||
fi
|
||||
|
||||
# Test if the value of x is greater or equal than 5
|
||||
if [ "$x" -ge 5 ]; then echo -e "greater or equal than 5!" ; fi
|
||||
if [ "$x" -ge 5 ]; then …
|
||||
|
||||
# Test if the value of x is greater or equal than 5, in bash/ksh/zsh:
|
||||
if ((x >= 5)); then echo -e "greater or equal than 5!" ; fi
|
||||
if ((x >= 5)); then …
|
||||
|
||||
# Use (( )) for arithmetic operation
|
||||
if ((j==u+2)); then echo -e "j==u+2!!" ; fi
|
||||
if ((j==u+2))
|
||||
|
||||
# Use [[ ]] for comparison
|
||||
if [[ $age -gt 21 ]]; then echo -e "forever 21!!" ; fi
|
||||
|
||||
if [[ $age -gt 21 ]]
|
||||
```
|
||||
|
||||
[More if commands](http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_07_01.html)
|
||||
@ -1127,22 +1106,6 @@ date +%F
|
||||
# or
|
||||
date +'%d-%b-%Y-%H:%M:%S'
|
||||
#10-Apr-2020-21:54:40
|
||||
|
||||
# Returns the current time with nanoseconds.
|
||||
date +"%T.%N"
|
||||
# 11:42:18.664217000
|
||||
|
||||
# Get the seconds since epoch (Jan 1 1970) for a given date (e.g Mar 16 2021)
|
||||
date -d "Mar 16 2021" +%s
|
||||
# 1615852800
|
||||
# or
|
||||
date -d "Tue Mar 16 00:00:00 UTC 2021" +%s
|
||||
# 1615852800
|
||||
|
||||
# Convert the number of seconds since epoch back to date
|
||||
date --date @1615852800
|
||||
# Tue Mar 16 00:00:00 UTC 2021
|
||||
|
||||
```
|
||||
|
||||
##### wait for random duration (e.g. sleep 1-5 second, like adding a jitter)
|
||||
@ -1744,18 +1707,6 @@ du -h
|
||||
du -sk /var/log/* |sort -rn |head -10
|
||||
```
|
||||
|
||||
##### check the Inode utilization
|
||||
```
|
||||
df -i
|
||||
# Filesystem Inodes IUsed IFree IUse% Mounted on
|
||||
# devtmpfs 492652 304 492348 1% /dev
|
||||
# tmpfs 497233 2 497231 1% /dev/shm
|
||||
# tmpfs 497233 439 496794 1% /run
|
||||
# tmpfs 497233 16 497217 1% /sys/fs/cgroup
|
||||
# /dev/nvme0n1p1 5037976 370882 4667094 8% /
|
||||
# tmpfs 497233 1 497232 1% /run/user/1000
|
||||
```
|
||||
|
||||
##### Show all file system type
|
||||
```bash
|
||||
df -TH
|
||||
@ -2212,7 +2163,6 @@ sar -n ALL
|
||||
|
||||
# reading SAR log file using -f
|
||||
sar -f /var/log/sa/sa31|tail
|
||||
```
|
||||
|
||||
##### Reading from journal file
|
||||
```bash
|
||||
@ -2430,7 +2380,7 @@ nc -vw5 google.com 22
|
||||
# From server A:
|
||||
$ sudo nc -l 80
|
||||
# then you can connect to the 80 port from another server (e.g. server B):
|
||||
# e.g. telnet <server A IP address> 80
|
||||
# e.g. telent <server A IP address> 80
|
||||
# then type something in server B
|
||||
# and you will see the result in server A!
|
||||
```
|
||||
@ -2538,16 +2488,6 @@ curl -I http://example.com/
|
||||
# Vary: Accept-Encoding
|
||||
```
|
||||
|
||||
##### Find out the http status code of a URL
|
||||
```bash
|
||||
curl -s -o /dev/null -w "%{http_code}" https://www.google.com
|
||||
```
|
||||
|
||||
##### Unshorten a shortended URL
|
||||
```bash
|
||||
curl -s -o /dev/null -w "%{redirect_url}" https://bit.ly/34EFwWC
|
||||
```
|
||||
|
||||
##### Perform network throughput tests
|
||||
```bash
|
||||
# server side:
|
||||
@ -2572,16 +2512,16 @@ sudo iptables –A INPUT –s <IP> -p tcp –dport 80 –j DROP
|
||||
```bash
|
||||
# If file is not specified, the file /usr/share/dict/words is used.
|
||||
look phy|head -n 10
|
||||
# phycic
|
||||
# Phyciodes
|
||||
# phycite
|
||||
# Phycitidae
|
||||
# phycitol
|
||||
# phyco-
|
||||
# phycochrom
|
||||
# phycochromaceae
|
||||
# phycochromaceous
|
||||
# phycochrome
|
||||
# Phil
|
||||
# Philadelphia
|
||||
# Philadelphia's
|
||||
# Philby
|
||||
# Philby's
|
||||
# Philip
|
||||
# Philippe
|
||||
# Philippe's
|
||||
# Philippians
|
||||
# Philippine
|
||||
```
|
||||
|
||||
##### Repeat printing string n times (e.g. print 'hello world' five times)
|
||||
@ -2639,40 +2579,6 @@ sdiff fileA fileB
|
||||
diff fileA fileB --strip-trailing-cr
|
||||
```
|
||||
|
||||
##### Find common/differing lines
|
||||
```bash
|
||||
# having two sorted and uniqed files (for example after running `$ sort -uo fileA fileA` and same for fileB):
|
||||
# ------
|
||||
# fileA:
|
||||
# ------
|
||||
# joey
|
||||
# kitten
|
||||
# piglet
|
||||
# puppy
|
||||
# ------
|
||||
# fileB:
|
||||
# ------
|
||||
# calf
|
||||
# chick
|
||||
# joey
|
||||
# puppy
|
||||
#
|
||||
# Find lines in both files
|
||||
comm -12 fileA fileB
|
||||
# joey
|
||||
# puppy
|
||||
#
|
||||
# Find lines in fileB that are NOT in fileA
|
||||
comm -13 fileA fileB
|
||||
# calf
|
||||
# chick
|
||||
#
|
||||
# Find lines in fileA that are NOT in fileB
|
||||
comm -23 fileA fileB
|
||||
# kitten
|
||||
# piglet
|
||||
```
|
||||
|
||||
##### Number a file (e.g. fileA)
|
||||
|
||||
```bash
|
||||
@ -2961,13 +2867,6 @@ cal
|
||||
# only display November
|
||||
cal -m 11
|
||||
```
|
||||
|
||||
##### Convert the hexadecimal MD5 checksum value into its base64-encoded format.
|
||||
```bash
|
||||
openssl md5 -binary /path/to/file| base64
|
||||
# NWbeOpeQbtuY0ATWuUeumw==
|
||||
```
|
||||
|
||||
##### Forces applications to use the default language for output
|
||||
```bash
|
||||
export LC_ALL=C
|
||||
@ -3189,7 +3088,7 @@ scp -r directoryname user@ip:/path/to/send
|
||||
echo $?
|
||||
```
|
||||
|
||||
##### Extract .xz
|
||||
##### Extract .xf
|
||||
```
|
||||
unxz filename.tar.xz
|
||||
# then
|
||||
@ -3229,7 +3128,8 @@ yes n
|
||||
# or 'anything':
|
||||
yes anything
|
||||
|
||||
# pipe yes to other command
|
||||
# For example:
|
||||
```bash
|
||||
yes | rm -r large_directory
|
||||
```
|
||||
|
||||
@ -3288,9 +3188,6 @@ q -d "," "select c3,c4,c5 from /path/to/file.txt where c3='foo' and c5='boo'"
|
||||
# Create session and attach:
|
||||
screen
|
||||
|
||||
# Create a screen and name it 'test'
|
||||
screen -S test
|
||||
|
||||
# Create detached session foo:
|
||||
screen -S foo -d -m
|
||||
|
||||
@ -3309,16 +3206,15 @@ screen -r foo
|
||||
# Kill session foo:
|
||||
screen -r foo -X quit
|
||||
|
||||
|
||||
# Scroll:
|
||||
# Hit your screen prefix combination (C-a / control+A), then hit Escape.
|
||||
# Move up/down with the arrow keys (↑ and ↓).
|
||||
Hit your screen prefix combination (C-a / control+A), then hit Escape.
|
||||
Move up/down with the arrow keys (↑ and ↓).
|
||||
|
||||
# Redirect output of an already running process in Screen:
|
||||
# (C-a / control+A), then hit 'H'
|
||||
(C-a / control+A), then hit 'H'
|
||||
|
||||
# Store screen output for Screen:
|
||||
# Ctrl+A, Shift+H
|
||||
Ctrl+A, Shift+H
|
||||
# You will then find a screen.log file under current directory.
|
||||
```
|
||||
|
||||
@ -3445,8 +3341,6 @@ source .venv/bin/activate
|
||||
type pip
|
||||
# 5. Now you can install your pip package, here requirements.txt is simply a txt file containing all the packages you want. (e.g tornado==4.5.3).
|
||||
pip install -r requirements.txt
|
||||
# 6. Exit virtual environment
|
||||
deactivate
|
||||
```
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user