Bash tips and tricks

Created by Jonathan Petitcolas / @Sethpolma

Base-64 encoding pictures

uuencode -m foo.png /dev/stdout
begin-base64 644 /dev/stdout
AAABAAEAEBAAAAEAIABoBAAAFgAAACgAAAAQAAAAIAAAAAEAIAAAAAAAAAQA
ABMLAAATCwAAAAAAAAAAAAAAAAAAAAAAADIvMQAyLzEIMi8xSzEuMKoyLzHk
Mi8x/TIvMf0yLzHlMi8xrDIvMU4yLzEJMi8xAAAAAAAAAAAAAAAAADIvMQAy
LzEGMi8xQzIvMZ4yLzHdMi8x+jIvMfoyLzHeMi8xoDIvMUUyLzEGMi8xAAAA
AAAAAAAA4AcAAMADAACAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAIABAADAAwAA4AcAAA==
====

Base-64 encoding pictures

uuencode -m foo.png /dev/stdout | sed '1d'
AAABAAEAEBAAAAEAIABoBAAAFgAAACgAAAAQAAAAIAAAAAEAIAAAAAAAAAQA
ABMLAAATCwAAAAAAAAAAAAAAAAAAAAAAADIvMQAyLzEIMi8xSzEuMKoyLzHk
Mi8x/TIvMf0yLzHlMi8xrDIvMU4yLzEJMi8xAAAAAAAAAAAAAAAAADIvMQAy
LzEGMi8xQzIvMZ4yLzHdMi8x+jIvMfoyLzHeMi8xoDIvMUUyLzEGMi8xAAAA
AAAAAAAA4AcAAMADAACAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAIABAADAAwAA4AcAAA==
====

Base-64 encoding pictures

uuencode -m foo.png /dev/stdout | sed '1d' | sed '$d'
AAABAAEAEBAAAAEAIABoBAAAFgAAACgAAAAQAAAAIAAAAAEAIAAAAAAAAAQA
ABMLAAATCwAAAAAAAAAAAAAAAAAAAAAAADIvMQAyLzEIMi8xSzEuMKoyLzHk
Mi8x/TIvMf0yLzHlMi8xrDIvMU4yLzEJMi8xAAAAAAAAAAAAAAAAADIvMQAy
LzEGMi8xQzIvMZ4yLzHdMi8x+jIvMfoyLzHeMi8xoDIvMUUyLzEGMi8xAAAA
AAAAAAAA4AcAAMADAACAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAIABAADAAwAA4AcAAA==

Base-64 encoding pictures

uuencode -m foo.png /dev/stdout | sed '1d' | sed '$d' | tr -d '\n'
AAABAAEAEBAAAAEAIABoBAAAFgAAACgAAAAQAAAAIAAAAAEAIAAAAAAAAAQAABMLAAATCwAAAAAAAAAAAAAAAAAAAAAAADIvMQAyLzEIMi8xSzEuMKoyLzHkMi8x/TIvMf0yLzHlMi8xrDIvMU4yLzEJMi8xAAAAAAAAAAAAAAAAADIvMQAyLzEGMi8xQzIvMZ4yLzHdMi8x+jIvMfoyLzHeMi8xoDIvMUUyLzEGMi8xAAAAAAAAAAAA4AcAAMADAACAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIABAADAAwAA4AcAAA==

Base-64 encoding pictures

uuencode -m foo.png /dev/stdout | sed '1d' | sed '$d' | tr -d '\n'
	| xclip -selection clipboard
AAABAAEAEBAAAAEAIABoBAAAFgAAACgAAAAQAAAAIAAAAAEAIAAAAAAAAAQAABMLAAATCwAAAAAAAAAAAAAAAAAAAAAAADIvMQAyLzEIMi8xSzEuMKoyLzHkMi8x/TIvMf0yLzHlMi8xrDIvMU4yLzEJMi8xAAAAAAAAAAAAAAAAADIvMQAyLzEGMi8xQzIvMZ4yLzHdMi8x+jIvMfoyLzHeMi8xoDIvMUUyLzEGMi8xAAAAAAAAAAAA4AcAAMADAACAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIABAADAAwAA4AcAAA==
is now in the clipboard!

Find biggest files

find / -type f -size +20M -exec ls -lh {} \;
-rw-r--r-- 1 root root  51M June  16 07:44 myfile.avi
-rw-r--r-- 1 root root  34M June  16 07:44 otherfile.avi
-rw-r--r-- 1 root root  24M June  16 06:14 anotherfile.avi
-rw-r--r-- 1 root root 314M June  16 18:39 stillanotherfile.avi
-rw-r--r-- 1 root root  79M June  16 11:41 alastfile.avi

Find biggest files

find / -type f -size +20M -exec ls -lh {} \;
	| awk '{ print $9 ": " $5 }'
myfile.avi: 51M
otherfile.avi: 34M
anotherfile.avi: 24M
stillanotherfile.avi: 314M
alastfile.avi: 79M

Send weekly report to your boss

git log --author="Jonathan Petitcolas" --format="- %B"
	--since=-2months --reverse
- a few bits tried to escape, but we caught them

- By works, I meant 'doesnt work'. Works now..

- more debug... who overwrote!

- Is there an achievement for this?

- Spinning up the hamster...

Send weekly report to your boss

git log --author="Jonathan Petitcolas" --format="- %B"
	--since=-2months --reverse | tr -s '\n'
- a few bits tried to escape, but we caught them
- By works, I meant 'doesnt work'. Works now..
- more debug... who overwrote!
- Is there an achievement for this?
- Spinning up the hamster...

Send weekly report to your boss

git log --author="Jonathan Petitcolas" --format="- %B"
	--since=-2months --reverse
	| tr -s '\n'
	| mail -s "Weekly report" [email protected]
- a few bits tried to escape, but we caught them
- By works, I meant 'doesnt work'. Works now..
- more debug... who overwrote!
- Is there an achievement for this?
- Spinning up the hamster...
is sent to your boss!

Download a whole site locally

wget -mrnp www.jonathan-petitcolas.com
--2013-06-20 06:18:44--  http://www.jonathan-petitcolas.com/
Résolution de www.jonathan-petitcolas.com... 204.232.175.78
Connexion 204.232.175.78|:80... connecté.
requête HTTP transmise, en attente de la réponse... 200 OK
Taille : 3399 (3,3K) [text/html]
Enregistre : «www.jonathan-petitcolas.com/index.html»

100%[==============================>] 3 399       --.-K/s   ds 0,1s

2013-06-20 06:18:44 (29,8 KB/s) - «www.jonathan-petitcolas.com/index.html» enregistré [3399/3399]

Chargement de robots.txt; svp ignorer les erreurs.
--2013-06-20 06:18:44--  http://www.jonathan-petitcolas.com/robots.txt
Réutilisation de la connexion vers www.jonathan-petitcolas.com:80.
requête HTTP transmise, en attente de la réponse... 404 Not Found
2013-06-20 06:18:44 ERREUR 404: Not Found.

--2013-06-20 06:18:44--  http://www.jonathan-petitcolas.com/atom.xml
Connexion vers www.jonathan-petitcolas.com:80... connecté.
requête HTTP transmise, en attente de la réponse... 200 OK
Taille : 110488 (108K) [text/xml]
Enregistre : «www.jonathan-petitcolas.com/atom.xml»

100%[==============================>] 110 488      127KB/s   ds 0,8s

Detecting broken links

wget -mrnp --spider www.jonathan-petitcolas.com
--2013-06-20 06:31:27--  http://www.jonathan-petitcolas.com/
requête HTTP transmise, en attente de la réponse... 200 OK
Taille : 3399 (3,3K) [text/html]
Le fichier doit contenir des liens vers d'autres ressources.
Récupération en cours.

--2013-06-20 06:31:28--  http://www.jonathan-petitcolas.com/
requête HTTP transmise, en attente de la réponse... 200 OK
Enregistre : «www.jonathan-petitcolas.com/index.html»

100%[==============================>] 3 399       --.-K/s   ds 0,1s
2013-06-20 06:31:28 (1,43 MB/s)
«www.jonathan-petitcolas.com/index.html» enregistré [3399/3399]

Chargement de robots.txt; svp ignorer les erreurs.
requête HTTP transmise, en attente de la réponse... 404 Not Found
2013-06-20 06:31:28 ERREUR 404: Not Found.
Destruction de www.jonathan-petitcolas.com/index.html.

--2013-06-20 06:31:28--  http://www.jonathan-petitcolas.com/atom.xml
requête HTTP transmise, en attente de la réponse... 200 OK
Taille : 110488 (108K) [text/xml]
Le fichier distant existe mais ne contient aucun lien.
Récupération abandonnée.

Detecting broken links

wget -mrnp --spider www.jonathan-petitcolas.com 2>&1
	| grep -B 2 "404 Not Found"
--2013-06-20 06:38:21--  http://www.jonathan-petitcolas.com/robots.txt
Réutilisation de la connexion vers www.jonathan-petitcolas.com:80.
requête HTTP transmise, en attente de la réponse... 404 Not Found
--
--2013-06-20 06:47:41--
http://www.jonathan-petitcolas.com/[email protected]
Réutilisation de la connexion vers www.jonathan-petitcolas.com:80.
requête HTTP transmise, en attente de la réponse... 404 Not Found

Detecting broken links

wget -mrnp --spider www.jonathan-petitcolas.com 2>&1
	| grep -B 2 "404 Not Found" | paste -d "  \n" -s 
--2013-06-20 06:38:21--  http://www.jonathan-petitcolas.com/robots.txt
Réutilisation de la connexion vers www.jonathan-petitcolas.com:80.
requête HTTP transmise, en attente de la réponse... 404 Not Found
# Breakline
-- --2013-06-20 06:47:41--
http://www.jonathan-petitcolas.com/[email protected]
Réutilisation de la connexion vers www.jonathan-petitcolas.com:80.
requête HTTP transmise, en attente de la réponse... 404 Not Found
# Breakline
no breaklines except if specified

Detecting broken links

wget -mrnp --spider www.jonathan-petitcolas.com 2>&1
	| grep -B 2 "404 Not Found" | paste -d "  \n" -s
	| sed -e 's/-- --/--/g'
2013-06-20 06:38:21--  http://www.jonathan-petitcolas.com/robots.txt
Réutilisation de la connexion vers www.jonathan-petitcolas.com:80.
requête HTTP transmise, en attente de la réponse... 404 Not Found
# Breakline
2013-06-20 06:47:41--
http://www.jonathan-petitcolas.com/[email protected]
Réutilisation de la connexion vers www.jonathan-petitcolas.com:80.
requête HTTP transmise, en attente de la réponse... 404 Not Found
# Breakline
no breaklines except if specified

Detecting broken links

wget -mrnp --spider www.jonathan-petitcolas.com 2>&1
	| grep -B 2 "404 Not Found" | paste -d "  \n" -s
	| sed -e 's/-- --/--/g' | awk '{ print $3 }'
http://www.jonathan-petitcolas.com/robots.txt
http://www.jonathan-petitcolas.com/[email protected]

Generating commit message

wget -O- whatthecommit.com 2>1
<!-- [...] -->
<body>
<div id="content">
<p>these confounded tests drive me nuts
</p>
<p class="permalink">
[<a href="/e22f11cc5818d6ffcb12702caae3728c">permalink</a>]
</p>
</div>
</body>
<!-- Like what you see? http://github.com/ngerakines/commitment -->
</html>

Generating commit message

wget -O- whatthecommit.com 2>1 | paste -s
[...]<div id="content">.*<p>these confounded tests drive me nuts</p>[...]
no more breaklines!

Generating commit message

wget -O- whatthecommit.com 2>1 | paste -s
	| sed 's/.*<div id="content"><p>//g'
these confounded tests drive me nuts</p>[...]

Generating commit message

wget -O- whatthecommit.com 2>1 | paste -s
	| sed 's/.*<div id="content"><p>//g' | sed 's/<\/p>.*//g'
these confounded tests drive me nuts

Generating commit message

wget -O- whatthecommit.com 2>1 | paste -s
	| sed 's/.*<div id="content"><p>//g' | sed 's/<\/p>.*//g'
	| xargs -0 git commit -m
[master 75f72f1] these confounded tests drive me nuts
3 files changed
it is now committed!

THE END

www.jonathan-petitcolas.com