Chapter 4

The shell as a language

The shell is not just a box you type names into. It has grammar. Once you see the grammar, long commands stop looking frightening — including the one you have run at the start of every step.

A command is a sentence

Every command has the same shape:

program   what to work on

The first word is always the name of a program. Everything after it is passed to that program as extra information.

$ cat hello.txt

Here cat is the program. hello.txt is an argument — a word handed to the program so it knows what to work on.

The shell does not read the file. The shell only starts cat and passes the word along. cat does the rest.

Options change how a program behaves

Some arguments are not things to work on. They are switches that change what the program does. These are called options, or flags.

They start with a dash, and come in two styles:

Style	Looks like	Notes
Short	`-l`	One dash, one letter. Quick to type.
Long	`--version`	Two dashes, a whole word. Easier to read later.

Short options can usually be joined together. These two lines do the same thing:

$ ls -l -a

$ ls -la

The dash is a habit, not a rule

Nothing forces a program to use dashes. It is a very old habit that nearly everyone follows. A few programs go their own way — tar and find are famous for it — and that is why they feel odd.

The shell cuts your line at every space

Before anything runs, the shell chops your line into words, using spaces as the scissors. Then it hands those words over.

So this command passes two arguments, not one:

$ cat my notes.txt

cat is told to open a file called my and a file called notes.txt. Both fail. This is the real reason chapter 2 warned you about spaces in file names.

Quotes glue words back together:

$ cat "my notes.txt"

Every program has three doors

A running program has three ways for text to come in and go out. They have plain names.

Door	Name	Used for
In	standard input	text going into the program
Out	standard output	the program's normal answers
Errors	standard error	complaints and warnings

Normally the last two both go to your screen, so they look like one thing. They are not. Keeping errors separate means you can save a program's answers to a file while still seeing its complaints.

You may see them written as numbers: 0 for in, 1 for out, 2 for errors.

Sending output to a file

You met > in step 1.5. Now it has a proper explanation: it takes the "out" door and points it at a file instead of the screen.

Written	Means
`> file`	send output to the file, throwing away what was there
`>> file`	add output to the end of the file
`2> file`	send only the errors to the file
`> file 2>&1`	send output to the file, and put the errors in with it

That last one looks strange. Read it as: send door 1 to the file, then send door 2 to wherever door 1 is now going.

Joining two programs with a pipe

A pipe, written |, connects the out door of one program to the in door of the next. The text never touches the screen or the disk. It goes straight across.

$ ls /usr/bin | wc -l

ls lists the programs. Instead of printing them, wc -l receives them and counts the lines. You get one number: how many programs are in that folder.

You have already used a pipe. It is how you installed step 2:

$ curl -fsSL https://aikaryashala.com/system_setup/scripts/install_cmds.sh | bash

curl fetches the text of a script. bash receives that text and runs it, line by line. Neither program knows the other exists. The shell joined them.

This is why small tools are useful

No single program here does very much. ls lists. wc counts. curl fetches. The power comes from joining them, and the pipe is the join.

Reading the command you have typed ten times

Now the payoff. Take that same line apart, piece by piece, with what you now know.

Piece	What it does
`curl`	the program: fetch something from the internet
`-f`	if the server sends an error page, do not print it as if it were the file
`-s`	silent: no progress bar
`-S`	but do still show real errors
`-L`	if the address has moved, follow it
the address	the argument: what to fetch
`\|`	send what came back into the next program
`bash`	the shell itself, reading those lines and running them

So the whole line means: fetch this text quietly, and run it as shell commands. Nothing more mysterious than that.

And this is why it deserves care

The last step runs whatever arrived. If the address were wrong, or the site were not trustworthy, you would be running someone else's instructions on your machine.

That is why every script on this site is plain text you can open in a browser first. Fetch it, read it, and run it as three separate commands whenever you are not sure.

Did it work? The exit code

When a program finishes it hands back one number, called the exit code. The rule is short:

0 means it worked
anything else means it did not

Zero for success feels backwards. The reason is that there is only one way to succeed, but many ways to fail, so the other numbers are free to describe which failure it was.

You can see the last one with $?:

$ ls /home
$ echo $?
0

$ ls /nothing-here
$ echo $?
2

You will rarely type $? yourself. But scripts read it after every important step, and that is how the installers in these steps know whether to carry on or to stop and tell you something went wrong.

Only if the last one worked

Two joins use the exit code to decide what happens next.

Written	Means
`a && b`	run b only if a worked
`a \|\| b`	run b only if a failed
`a ; b`	run b either way

This is why steps often show a line like this one:

$ cd ~/python-samples && python3 report.py

If the folder does not exist, cd fails and Python never runs. Without &&, Python would run in the wrong folder and give a confusing error instead of a clear one.

Stars stand for names

A * in a name means "anything". The shell replaces it with every matching file name before the program starts.

$ ls *.txt

If the folder holds a.txt and b.txt, the program actually receives ls a.txt b.txt. ls never sees the star.

This is why rm * is dangerous

The shell expands the star first, into every name in the folder. Then rm is handed all of them at once, and removes them all, immediately.

Before deleting with a star, run the same thing with ls first. What ls lists is exactly what rm would remove.

Try it

Count how many programs are in one folder:

$ ls /usr/bin | wc -l

Watch the two output doors behave differently. This prints an error to the screen:

$ ls /nothing-here

Now send the normal output to a file. The error still appears, because it uses the other door:

$ ls /nothing-here > out.txt

And now silence the error instead, by sending door 2 to a place that throws everything away:

$ ls /nothing-here 2> /dev/null

Look at an exit code after something that works, and something that does not:

$ true; echo $?

$ false; echo $?

See the difference between the two joins:

$ false && echo "you will not see this"

$ false || echo "but you will see this"

You can now explain

Why the first word of a command is always a program
The difference between an argument and an option
Why a space in a file name causes trouble
What standard input, output and error are
The difference between >, >> and 2>
What a pipe does, and why small tools are worth having
Every piece of curl -fsSL … | bash
What an exit code is, and why zero means success
Why rm * deserves a check with ls first