Eleven bash Tips You Might Want to Know
source link: https://www.tuicool.com/articles/hit/ZnUZNf3
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Here are some tips that might help you be more productive with bash.
1) ^x^y^
A gem I use all the time.
Ever typed anything like this?
$ grp somestring somefile
-bash: grp: command not found
Sigh. Hit ‘up’, ‘left’ until at the ‘p’ and type ‘e’ and return.
Or do this:
$ ^rp^rep^
grep 'somestring' somefile
$
One subtlety you may want to note though is:
$ grp rp somefile $ ^rp^rep^ $ grep rp somefile
If you wanted rep
to be searched for, then you’ll need to dig into the man page and use a more powerful history command:
$ grp rp somefile $ !!:gs/rp/rep grep rep somefile $
2) pushd / popd vs ‘cd -‘
This one comes in very handy for scripts, especially when operating within a loop.
Let’s say you’re in a for
loop moving in and out of folders like this:
for d1 in $(ls -d */) do # Store original working directory. original_wd="$(pwd)" cd "$d1" for d2 in $(ls -d */) do pushd "$d2" # Do something popd done # Return to original working directory cd "${original_wd}" done
You can rewrite the above using the pushd
stack like this:
for d1 in $(ls -d *) do pushd "$d1" for d2 in $(ls -d */) do pushd "$d2" # Do something popd done popd done
Which tracks the folders you’ve pushed and popped as you go.
Note that if there’s an error in a pushd
you may lose track of the stack and popd
too many time. You probably want to set -e
in your script as well (seeprevious post)
There’s also cd -
, but that doesn’t ‘stack’ – it just returns you to the previous folder:
cd ~ cd /tmp cd blah cd - # Back to /tmp cd - # Back to 'blah' cd - # Back to /tmp cd - # Back to 'blah' ...
Material here based on material from my book
Free preview available here .
3) shopt
vs set
This one bothered me for a while.
What’s the difference between set
and shopt
?
set
s we saw before, but shopt
s look very similar. Just inputting shopt
shows a bunch of options:
$ shopt cdable_vars off cdspell on checkhash off checkwinsize on cmdhist on compat31 off dotglob off
I found a set of answers here .
Essentially, it looks like it’s a consequence of bash (and other shells) being built on sh, and adding shopt
as another way to set extra shell options.
But I’m still unsure… if you know the answer, let me know.
4) Here Docs and Here Strings
‘Here docs’ are files created inline in the shell.
The ‘trick’ is simple. Define a closing word, and the lines between that word and when it appears alone on a line become a file.
Type this:
$ cat > afile << SOMEENDSTRING > here is a doc > it has three lines > SOMEENDSTRING alone on a line will save the doc > SOMEENDSTRING $ cat afile here is a doc it has three lines SOMEENDSTRING alone on a line will save the doc $
Notice that:
- the string could be included in the file if it was not ‘alone’ on the line
- the string
SOMEENDSTRING
is more normallyEND
, but that is just convention
Lesser known is the ‘here string’:
$ cat > asd <<< 'This file has one line'
5) String Variable Manipulation
You may have written code like this before, where you use tools like sed
to manipulate strings:
$ VAR='HEADERMy voice is my passwordFOOTER' $ PASS="$(echo $VAR | sed 's/^HEADER(.*)FOOTER/1/')" $ echo $PASS
But you may not be aware that this is possible natively in bash .
This means that you can dispense with lots of sed and awk shenanigans.
One way to rewrite the above is:
$ VAR='HEADERMy voice is my passwordFOOTER' $ PASS="${VAR#HEADER}" $ PASS="${PASS%FOOTER}" $ echo $PASS
# %
The second method is twice as fast as the first on my machine. And (to my surprise), it was roughly the same speed as a similar python script .
If you want to use glob patterns that are greedy (see globbinghere) then you double up:
$ VAR='HEADERMy voice is my passwordFOOTER' $ echo ${VAR##HEADER*} $ echo ${VAR%%*FOOTER}
6) Variable Defaults
These are very handy when you’re knocking up scripts quickly.
If you have a variable that’s not set, you can ‘default’ them by using this. Create a file called default.sh
with these contents
#!/bin/bash FIRST_ARG="${1:-no_first_arg}" SECOND_ARG="${2:-no_second_arg}" THIRD_ARG="${3:-no_third_arg}" echo ${FIRST_ARG} echo ${SECOND_ARG} echo ${THIRD_ARG}
Now run chmod +x default.sh
and run the script with ./default.sh first second
.
Observer how the third argument’s default has been assigned, but not the first two.
You can also assign directly with ${VAR:=defaultval}
(equals sign, not dash) but note that this won’t work with positional variables in scripts or functions. Try changing the above script to see how it fails.
7) Traps
The trap
builtin can be used to ‘catch’ when a signal is sent to your script.
Here’s an example I use in my own cheapci
script:
function cleanup() { rm -rf "${BUILD_DIR}" rm -f "${LOCK_FILE}" # get rid of /tmp detritus, leaving anything accessed 2 days ago+ find "${BUILD_DIR_BASE}"/* -type d -atime +1 | rm -rf echo "cleanup done" } trap cleanup TERM INT QUIT
Any attempt to CTRL-C
, CTRL-
or terminate the program using the TERM
signal will result in cleanup being called first.
Be aware:
- Trap logic can get very tricky (eg handling signal race conditions)
- The KILL signal can’t be trapped in this way
But mostly I’ve used this for ‘cleanups’ like the above, which serve their purpose.
8) Shell Variables
It’s well worth getting to know the standard shell variables available to you . Here are some of my favourites:
RANDOM
Don’t rely on this for your cryptography stack, but you can generate random numbers eg to create temporary files in scripts:
$ echo ${RANDOM} 16313 $ # Not enough digits? $ echo ${RANDOM}${RANDOM} 113610703 $ NEWFILE=/tmp/newfile_${RANDOM} $ touch $NEWFILE
REPLY
No need to give a variable name for read
…
$ read my input $ echo ${REPLY}
LINENO and SECONDS
Handy for debugging
$ echo ${LINENO} 115 $ echo ${SECONDS}; sleep 1; echo ${SECONDS}; echo $LINENO 174380 174381 116
Note that there are two ‘lines’ above, even though you used ;
to separate the commands.
TMOUT
You can timeout reads, which can be really handy in some scripts
#!/bin/bash TMOUT=5 echo You have 5 seconds to respond... read echo ${REPLY:-noreply}
9) Extglobs
If you’re really knee-deep in bash, then you might want to power up your globbing. You can do this by setting the extglob shell option. Here’s the setup:
shopt -s extglob A="12345678901234567890" B=" ${A} "
Now see if you can figure out what each of these does:
echo "B |${B}|" echo "B#+( ) |${B#+( )}|" echo "B#?( ) |${B#?( )}|" echo "B#*( ) |${B#*( )}|" echo "B##+( )|${B##+( )}|" echo "B##*( )|${B##*( )}|" echo "B##?( )|${B##?( )}|"
Now, potentially useful as it is, it’s hard to think of a situation where you’d absolutely want to do it this way. Normally you’d use a tool better suited to the task (like sed
) or just drop bash and go to a ‘proper’ programming language like python.
10) Associative Arrays
Talking of moving to other languages, a rule of thumb I use is that if I need arrays then I drop bash to go to python (I even created a Docker container for a tool to help with this here ).
What I didn’t know until I read up on it was that you can have associative arrays in bash.
Type this out for a demo:
$ declare -A MYAA=([one]=1 [two]=2 [three]=3) $ MYAA[one]="1" $ MYAA[two]="2" $ echo $MYAA $ echo ${MYAA[one]} $ MYAA[one]="1" $ WANT=two $ echo ${MYAA[$WANT]}
Note that this is only available in bashes 4.x+.
11) source vs ‘.’
This one confused me for a long time.
You can type:
$ cat > somescript.sh << END A=11 END $ source somescript.sh $ echo $A
which will run the script somescript.sh
and do so while retaining the environment changes in the script in your environment.
Try this to compare:
$ cat > somescript.sh << END A=12 END $ chmod +x somescript.sh $ ./somescript.sh $ echo $A
The dot (‘ .
‘) command does something similar, but what’s the difference? Why does it exist?
The answer is simple: in bash they are exactly the same. The ‘ .
‘ was the original command, and is more portable, since it works in the sh
shell as well as bash.
If you like this, you might like one of my books:
If you liked this post, you might also like these:
Ten Things I Wish I’d Known About bash
How (and Why) I Run My Own DNS Servers
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK