Terminal Utilities I Often Use
In this blog post, I just list a few Unixy terminal utilities I often use. If you are a terminal veteran this will be boring for you. If you rarely use the terminal, it might have a utility or two you are not aware of yet.
I’m not giving a tutorial on the commands. Google for more details or read the man pages =). It is just to let you know that these tools exist.
grep -v
I guess everyone knows grep. It is one of the tools you’ll use all the time. You are maybe not aware
of the negation filter argument -v. Instead of finding the term, it excludes it.
$ printf "abc\n123\nsomething\n" | grep 123
123
$ printf "abc\n123\nsomething\n" | grep -v 123
abc
somethingFor example, I use it to remove things except for the thing I want to keep. Like:
$ git branch
dev-branch-1
* master
some-other-branch
something-something
$ git branch | grep -v master | xargs git branch -D
Deleted branch dev-branch-1 (was 8998ebd).
Deleted branch some-other-branch (was 8998ebd).
Deleted branch something-something (was 8998ebd)xargs
xargs allows you to take a list of inputs and turn it into arguments. For example, take a list of git branches,
Docker images, etc and feed it into command line arguments. xargs expects the command and the list of flags for that
command. It will add the arguments as a list at the end.
$ git branch | grep -v master | xargs git branch -D
Deleted branch dev-branch-1 (was 8998ebd).
Deleted branch some-other-branch (was 8998ebd).
Deleted branch something-something (was 8998ebd)
$ docker ps -a | awk '{print $1}' | xargs docker rm
7eb755a161b5
91d655c80d1c
989efc0c7631
Error: No such container: CONTAINERSometimes the command you are using doesn’t accept multiple arguments or needs the input in a certain position.
For that you can use -I{placeholder} and then use that placeholder to tell where the argument should be placed.
$ ls | xargs -I@ echo Hello @ Directory
Hello file1.txt Directory
Hello file2.txt Directoryawk
awk is a text processing tool and is mighty powerful. It has its own text processing programming language.
I mostly use it to join and cut columns out, because I can’t remember more specific commands for it. Basically, each
column gets an implict variable starting from $1, $2, … $n. And $0 is the full line. With that, you can
print out reformated lines. Of course, awk supports way more, like matching witch regexes, do things at the start and
end of the processing, accumulating stuff, declare functions, etc. I’ll Google that on demand.
$ docker images | awk '{ print $1":"$2}'
REPOSITORY:TAG
ubuntu:latest
postgres:12.2I already explained how to use AWK to add extra information to logs here.
tee
You want to keep the output of an application and watch it at the same time? tee does store the standard out
a program and stores it to a file while also forwarding it to standard out. I guess the name comes from a the
T shape of a pipe?
$ ./run-a-build-script.sh | tee build.log Some build status info lines Pages and Pages of build log .... Build failed! $ grep -i error build.log
tmux
I rarely use tmux locally, but I use it all the time when I ssh into a machine. It allows
you to multiplex a terminal into multiple tabs and sections. That’s why I use it when ssh-ing to have a single
connection with multiple terminals. Even better, you can get disconnected, and when you reconnect
use tmux attach to reconnect to your old session. Everything will be there as you left it. Awesome!
watch
watch repeats a command over and over, so you can watch what is happening. Like xargs you add the command you
want to run.
$ watch -d ls -h blog-example/
Every 2.0s: ls -h blog-example/ gamlor-t470p: Sat May 16 07:45:03 2020
file1.txt
file2.txtnetstat, ss and others
Want to know what ports are open on your machine, and what process owns that port. There are many utilities doing that.
For example ss and netstat:
$ ss -l -p --tcp
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 0 127.0.0.1:33901 0.0.0.0:* users:(("java",pid=2095,fd=774))
LISTEN 0 0 127.0.0.1:63342 0.0.0.0:* users:(("java",pid=2095,fd=629))
LISTEN 0 0 0.0.0.0:41367 0.0.0.0:* users:(("java",pid=2095,fd=762))
LISTEN 0 0 127.0.0.1:ipp 0.0.0.0:*
LISTEN 0 0 0.0.0.0:db-lsp 0.0.0.0:* users:(("dropbox",pid=1857,fd=88))
LISTEN 0 0 127.0.0.1:6942 0.0.0.0:* users:(("java",pid=2095,fd=61))
LISTEN 0 0 127.0.0.1:12702 0.0.0.0:* users:(("octopi-notifier",pid=1023,fd=13))
LISTEN 0 0 127.0.0.1:46655 0.0.0.0:* users:(("java",pid=2095,fd=900))
LISTEN 0 0 127.0.0.1:17600 0.0.0.0:* users:(("dropbox",pid=1857,fd=111))
LISTEN 0 0 127.0.0.1:17603 0.0.0.0:* users:(("dropbox",pid=1857,fd=116))
$netstat -tulpn
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:33901 0.0.0.0:* LISTEN 2095/java
tcp 0 0 127.0.0.1:63342 0.0.0.0:* LISTEN 2095/java
tcp 0 0 0.0.0.0:41367 0.0.0.0:* LISTEN 2095/java
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:17500 0.0.0.0:* LISTEN 1857/dropbox
tcp 0 0 127.0.0.1:6942 0.0.0.0:* LISTEN 2095/java
tcp 0 0 127.0.0.1:12702 0.0.0.0:* LISTEN 1023/octopi-notifieThat’s It for Today
Maybe you saw a utility you didn’t know about.