Page Menu
Home
c4science
Search
Configure Global Search
Log In
Files
F61105337
linux_2.html
No One
Temporary
Actions
Download File
Edit File
Delete File
View Transforms
Subscribe
Mute Notifications
Award Token
Subscribers
None
File Metadata
Details
File Info
Storage
Attached
Created
Sat, May 4, 14:01
Size
31 KB
Mime Type
text/html
Expires
Mon, May 6, 14:01 (2 d)
Engine
blob
Format
Raw Data
Handle
17432152
Attached To
R4368 ScITS UniBe Training
linux_2.html
View Options
<!DOCTYPE html>
<html>
<head>
<title>
Introduction to Linux, Part II
</title>
<meta
http-equiv=
"Content-Type"
content=
"text/html; charset=UTF-8"
/>
<link
rel=
"stylesheet"
type=
"text/css"
href=
"css/common.css"
>
</head>
<body>
<textarea
id=
"source"
>
class: center, middle
# Introduction to Linux
## Part II
`https://goo.gl/cypvZ1`
---
# Agenda
1. Linux resources you can use
2. Standard input/output and its redirection
3. UNIX pipelines
4. Background processes
5. Durable sessions with `screen`
6. File ownership and permissions
7. Shell scripting basics
8. Environment and customization
9. (Extra credits)
---
# What Linux resources can I use?
To do development and run light workloads:
* Your own computer may already run Linux.
* You can install Linux in a virtual machine.
I recommend Virtualbox for personal use.
* If you're running Windows 10, you can install
[Windows Subsystem for Linux](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux)
---
# What if it's not enough?
To create persistent services:
* Ask your group's sysadmin for servers/VM resources.
* UniBe Informatikdienste offers virtual machines.
* Cloud resources: [SWITCHengines](https://www.switch.ch/engines/), other cloud services.
To run heavy calculations:
* [UBELIX Linux cluster](https://docs.id.unibe.ch/ubelix/ubelix-101).
* Your group may have in-house infrastructure.
* Again, cloud services.
---
# Prepare for the tutorial
You should be either running Linux, or connected to a Linux system.
Log in to UBELIX, or inform me that you need access to another system.
Execute the following to set up training (if you haven't already):
```
$
wget https://scits.math.unibe.ch/script -O - | /bin/bash
```
---
# Processes and their input/output
When you're connected to a Linux system,
what you normally see is the shell prompt, awaiting input:
```
user@host:~
$
```
A .highlight[process] called shell is responsible for input/output at this moment.
By default, the input is either a keyboard connected to the system,
or your keypresses being relayed over the network.
The output, by default, is the screen connected to the system,
or text being relayed for display over the network.
---
# Processes and their input/output
Those are called .highlight[standard input] and .highlight[standard output],
or .highlight[STDIN/STDOUT].
.center[
![](assets/shell_stdio.svg)
]
--
If the user issues a command that calls another program, the shell
creates a .highlight[child process] and attaches the input/output to it.
.center[
![](assets/child_stdio.svg)
]
The shell will wait until the program terminates,
after which STDIN/STDOUT get reattached and a prompt is displayed.
---
# Redirecting output
Sometimes, we want to capture what a command is outputting to the screen.
For example, suppose that we want to save into a file the directory structure returned by `ls -R`:
```
user@host:~
$
ls -R ~/scits-training
scits-training:
animals moving numbers io scripts
[...]
user@host:~
$
```
We can instruct the shell to .highlight[redirect] the standard output:
.center[
![](assets/stdout_redirect.svg)
]
---
# Redirecting output
To redirect the output, we add .highlight[`> FILE`] to the command:
```
user@host:~
$
cd ~/scits-training/io
user@host:~/scits-training/io
$
ls -R ~/scits-training > listing
user@host:~/scits-training/io
$
cat listing
scits-training:
animals moving numbers io scripts
[...]
```
This will overwrite the contents of `FILE` (if any) with the output of the command.
The file will be created, if it does not exist yet.
---
# Appending output
Sometimes we don't want to overwrite (sometimes called "clobber") the file
with new contents and add them to the end instead.
To do that, use .highlight[`>> FILE`] instead of `> FILE`.
.exercise[
1. Try running the command `date` to see what it outputs.
2. Run `date` 3 times, appending the output to a file `date.log`.
3. Verify with `cat` that the file contains 3 records.
]
---
# Error output
Try saving the output of a command with errors
and you'll see that it still outputs to the screen:
```
user@host:~/scits-training/io
$
ls , > listing
ls: cannot access ,: No such file or directory
user@host:~/scits-training/io
$
```
--
This is intentional: Linux actually has two output streams for
its command line, STDOUT for normal data and .highlight[STDERR] for errors.
.center[
![](assets/stderr.svg)
]
This simplifies debugging: errors are separate from data.
---
# Error output
STDERR is not redirected when using `>` or `>>`:
.center[
![](assets/stderr_noredirect.svg)
]
--
It's possible to redirect it as well with .highlight[`2>`] or .highlight[`2>>`]:
```
user@host:~/scits-training/io
$
ls .. , > listing 2> errors
user@host:~/scits-training/io
$
cat errors
ls: cannot access ,: No such file or directory
user@host:~/scits-training/io
$
cat listing
..:
animals io moving numbers scripts
```
---
# Discarding output
Sometimes we don't need output at all.
In this case, we can redirect it to a special file, .highlight[`/dev/null`]
It's a .highlight[device] that will accept any input, discarding it immediately.
--
For example, one might want to silence errors:
```
user@host:~/scits-training/io
$
ls .. , 2> /dev/null
..:
animals io moving numbers scripts
```
---
# Interactive input
Most commands we've seen don't require any interactive input.
`tr` (for .highlight[translate]) is a command that transforms its input:
it substitutes some characters with others.
For example, `tr 'a-z' 'A-Z'` would translate all lowercase letters into uppercase.
--
Let's try this:
```
user@host:~/scits-training/io
$
tr 'a-z' 'A-Z'
Let's input some text
LET'S INPUT SOME TEXT
```
---
# Interactive input
```
user@host:~/scits-training/io
$
tr 'a-z' 'A-Z'
Let's input some text
LET'S INPUT SOME TEXT
But enter doesn't stop it!
BUT ENTER DOESN'T STOP IT!
```
A problem: text can contain many lines, and the program won't know when to stop.
--
We can terminate the program with Ctrl+C, but it actually expects an .highlight[end of input].
We can signal end of input with Ctrl+D on an empty line (or pressing it twice).
--
.exercise[
What happens if we press Ctrl+D while back at the shell prompt?
]
---
# Redirecting input
What if we want to use a file as an input in a command that doesn't accept files as arguments,
we need instruct the shell to use the file as the program's standard input:
.center[
![](assets/stdin_redirect.svg)
]
This is done with adding .highlight[`
< FILE
`]
to
the
command
.
```
user
@
host:
~/
scits-training
/
io
$
tr
'
a-z
'
'
A-Z
'
<
errors
LS:
CANNOT
ACCESS
,
:
NO
SUCH
FILE
OR
DIRECTORY
```
---
#
Redirects
.
exercise
[
1
.
Combine
input
and
output
redirection
to
save
the
output
of
last
`
tr
`
command
into
`
errors
.
uppercase
`
2
.
Use
`
cat
`
to
verify
the
saved
output
.
]
---
#
Pipelines
We
have
shown
how
to
save
outputs
to
a
file
,
and
further
process
files
as
inputs
.
Sometimes
,
we
don
'
t
need
to
save
this
intermediate
representation
.
In
that
case
,
we
can
directly
connect
the
output
of
one
program
to
the
input
of
another
with
.
highlight
[
pipes
].
To
do
so
,
separate
two
commands
with
.
highlight
[`|`]
:
```
user
@
host:
~/
scits-training
/
io
$
ls
.
|
tr
'
a-z
'
'
A-Z
'
DATE
.
LOG
ERRORS
ERRORS
.
UPPERCASE
LISTING
```
---
#
Pipelines
```
user
@
host:
~/
scits-training
/
io
$
ls
.
|
tr
'
a-z
'
'
A-Z
'
DATE
.
LOG
ERRORS
ERRORS
.
UPPERCASE
LISTING
```
Given
this
command
,
shell
starts
two
processes
in
parallel
and
ties
their
respective
output
and
input
together
.
Standard
input
/
output
is
connected
at
the
ends
of
the
chain:
.
center
[
![](
assets
/
pipe
.
svg
)
]
--
Such
.
highlight
[
pipelines
]
can
be
longer
than
two
commands
,
and
can
be
combined
with
file
redirects
.
---
#
Filtering
output
One
extremely
useful
command
used
in
pipes
is
`
grep
`.
It
allows
to
search
for
text
patterns
.
Example:
```
user
@
host:
~/
scits-training
/
io
$
ls
.
date
.
log
errors
errors
.
uppercase
listing
user
@
host:
~/
scits-training
/
io
$
ls
.
|
grep
log
date
.
log
```
`
grep
`
is
versatile:
*
Can
be
used
with
.
highlight
[
regular
expression
]
patterns
*
Can
search
for
non-matching
lines
(
with
`
-v
`)
*
Can
search
in
files
*
Can
print
where
the
match
happened
in
a
file
See
`
man
grep
`
or
Google
for
more
examples
.
---
#
Pipelines
and
errors
You
will
notice
that
all
errors
are
still
output
normally:
```
user
@
host:
~/
scits-training
/
io
$
ls
.
,
|
tr
'
a-z
'
'
A-Z
'
ls:
cannot
access
,
:
No
such
file
or
directory
.
:
DATE
.
LOG
ERRORS
ERRORS
.
UPPERCASE
LISTING
```
--
As
before
,
errors
are
not
normally
redirected
,
and
collected
from
all
processes
in
the
pipe:
.
center
[
![](
assets
/
pipe_err
.
svg
)
]
---
#
Background
jobs
Recall
that
,
when
running
a
command
,
the
shell
waits
until
it
is
terminated:
all
input
goes
to
the
program
(
or
nowhere
).
.
center
[
![](
assets
/
child_stdio
.
svg
)
]
Sometimes
,
we
don
'
t
need
to
wait
until
the
program
terminates
–
we
actually
want
it
running
in
background
.
---
#
Background
jobs
If
you
specify
.
highlight
[`&`]
at
the
end
of
the
command
,
the
shell
will
start
it
,
but
keep
control
of
STDIN:
.
center
[
![](
assets
/
job_stdio
.
svg
)
]
Instead
of
a
.
highlight
[
foreground
]
process
,
it
becomes
a
.
highlight
[
background
]
job
.
--
You
are
immediately
returned
to
the
shell
and
can
run
other
commands
while
the
job
executes
.
Note
that
both
the
shell
and
the
background
job
are
connected
to
STDOUT
.
Redirect
output
to
prevent
mix-ups
.
---
#
Background
jobs
Compare:
```
user
@
host:
~/
scits-training
/
io
$
sleep
3
user
@
host:
~/
scits-training
/
io
$
sleep
3
&
[
1
]
12231
user
@
host:
~/
scits-training
/
io
$
```
Here
,
`[
1
]`
is
the
.
highlight
[
job
number
],
and
`
12231
`
is
the
.
highlight
[
process
ID
],
or
PID
.
After
3
seconds
and
when
another
command
finishes
(
you
can
just
press
Enter
for
an
empty
command
),
you
'
ll
be
informed
that
the
job
terminated:
```
user
@
host:
~/
scits-training
/
io
$
[
1
]+
Done
sleep
3
user
@
host:
~/
scits-training
/
io
$
```
---
#
Listing
jobs
You
can
list
running
background
jobs
with
.
highlight
[`
jobs
`]
:
```
user
@
host:
~/
scits-training
/
io
$
sleep
100
&
[
1
]
12232
user
@
host:
~/
scits-training
/
io
$
sleep
0
&
[
2
]
12233
user
@
host:
~/
scits-training
/
io
$
jobs
[
1
]
-
Running
sleep
100
&
[
2
]+
Done
sleep
0
```
---
#
Terminating
jobs
You
can
forcibly
.
highlight
[
terminate
]
a
job
with
the
.
highlight
[`
kill
`]
command
,
which
accepts
either
PID
or
job
ID
(
with
`%`)
:
```
user
@
host:
~/
scits-training
/
io
$
sleep
100
&
[
1
]
12234
user
@
host:
~/
scits-training
/
io
$
kill
12234
user
@
host:
~/
scits-training
/
io
$
jobs
[
1
]+
Terminated
sleep
100
user
@
host:
~/
scits-training
/
io
$
sleep
100
&
[
1
]
12235
user
@
host:
~/
scits-training
/
io
$
kill
%
1
```
You
can
search
for
more
process
IDs
to
terminate
with
.
highlight
[`
ps
ax
`],
in
case
something
is
misbehaving
.
---
#
Stopped
jobs
Background
jobs
have
nothing
connected
to
their
standard
input
.
If
a
background
job
cannot
continue
without
user
input
,
it
will
.
highlight
[
stop
],
which
the
shell
will
signal
to
you:
```
user
@
host:
~/
scits-training
/
io
$
tr
'
a-z
'
'
A-Z
'
&
[
1
]
12236
user
@
host:
~/
scits-training
/
io
$
[
1
]+
Stopped
tr
/
a-z
/
/
A-Z
/
user
@
host:
~/
scits-training
/
io
$
```
--
You
can
bring
a
job
to
.
highlight
[
foreground
]
to
pass
STDIN
from
the
shell
to
the
running
job
with
`
fg
`
(
or
`
fg
%
N
`
for
a
specific
job
number
)
:
```
user
@
host:
~/
scits-training
/
io
$
fg
tr
/
a-z
/
/
A-Z
/
You
are
now
talking
to
the
job
YOU
ARE
NOW
TALKING
TO
THE
JOB
```
---
#
Stopping
and
resuming
programs
You
can
stop
most
currently-running
programs
with
`
Ctrl
`+`
Z
`
:
```
user
@
host:
~/
scits-training
/
io
$
sleep
100
^
Z
[
1
]+
Stopped
sleep
100
user
@
host:
~/
scits-training
/
io
$
```
--
From
there
,
you
can
use
`
fg
`
to
resume
normal
execution
of
the
program
,
or
use
.
highlight
[`
bg
`]
to
let
it
continue
to
run
in
the
background
.
```
user
@
host:
~/
scits-training
/
io
$
bg
[
1
]+
sleep
100
&
user
@
host:
~/
scits-training
/
io
$
jobs
[
1
]+
Running
sleep
100
&
```
---
#
Background
jobs
are
fragile
What
will
happen
if
you
start
a
background
job
,
and
then
close
the
terminal
?
--
Closing
the
terminal
(
or
disconnecting
the
SSH
session
)
kills
the
shell
you
were
talking
to
.
Since
the
job
was
a
.
highlight
[
child
process
]
of
that
shell
,
it
will
also
be
killed
.
--
A
minor
inconvenience
if
you
'
re
working
on
your
own
machine
(
you
can
just
leave
the
terminal
open
),
but
a
much
bigger
problem
with
remote
connections
.
If
the
connection
is
broken
,
the
shell
is
also
terminated
along
with
all
processes
launched
from
it
.
How
to
protect
against
it
?
---
#
`
screen
`
To
protect
your
session
,
you
can
use
.
highlight
[`
screen
`].
`
screen
`
starts
a
new
shell
that
exists
independently
of
your
current
one
.
Even
if
the
current
shell
dies
(
e
.
g
.
because
you
disconnected
),
the
shell
running
in
`
screen
`
will
continue
together
with
all
its
child
processes
.
--
Starting
a
new
`
screen
`
session
is
simple:
```
user
@
host:
~/
scits-training
/
io
$
screen
[
terminal
screen
is
cleared
]
user
@
host:
~/
scits-training
/
io
$
echo
"
Hello
,
I
'
m
in
a
screen
"
Hello
,
I
'
m
in
a
screen
!
user
@
host:
~/
scits-training
/
io
$
```
---
#
Reattaching
to
`
screen
`
Now
suppose
your
connection
was
terminated
.
Close
the
terminal
where
it
is
running
to
simulate
that
,
then
log
in
again
.
--
You
can
use
.
highlight
[`
screen
-ls
`]
to
list
active
sessions:
```
user
@
host:
~
$
screen
-ls
There
is
a
screen
on:
13383
.
pts-2
.
host
(
11
/
09
/
17
03:02:23
)
(
Detached
)
1
Socket
in
/
var
/
run
/
screen
/
S-user
.
```
--
You
can
attach
to
a
screen
session
(
possibly
detaching
it
first
,
if
it
'
s
being
used
somewhere
)
with
.
highlight
[`
-dR
`]
(
for
.
highlight
[
detach
,
reattach
])
```
user
@
host:
~
$
screen
-dR
[
terminal
screen
is
cleared
]
user
@
host:
~/
scits-training
/
io
$
echo
"
Hello
,
I
'
m
in
a
screen
"
Hello
,
I
'
m
in
a
screen
!
user
@
host:
~/
scits-training
/
io
$
```
---
#
Controlling
`
screen
`
`
screen
`
can
be
used
for
other
things
,
such
as
having
multiple
parallel
shell
sessions
open
.
Controlling
screen
consists
of
pressing
.
highlight
[`
Ctrl
`+`
A
`],
then
a
screen-specific
command
.
For
example
,
*
.
highlight
[`
c
`]
will
.
highlight
[
create
]
a
new
shell
within
screen
*
.
highlight
[`
n
`]
will
switch
to
the
.
highlight
[
next
]
shell
*
.
highlight
[`
d
`]
will
.
highlight
[
detach
]
from
`
screen
`,
returning
you
to
the
original
shell
Finally
,
you
can
use
.
highlight
[`?`]
to
access
built-in
help
,
or
use
`
man
screen
`
for
a
more
detailed
manual
.
--
Another
popular
alternative
to
`
screen
`
is
`
tmux
`.
It
will
not
be
covered
by
this
tutorial
,
but
is
worth
looking
into
.
---
#
Users
and
groups
Before
we
discuss
permissions
,
we
need
to
understand
users
and
groups
in
Linux
.
A
.
highlight
[
user
]
is
a
unit
of
access
control
;
it
has
a
set
of
credentials
to
access
the
system
and
.
highlight
[
owns
]
some
files
on
it
.
A
.
highlight
[
group
]
is
a
collection
of
users
to
facilitate
shared
access
to
resources
.
A
user
can
belong
to
many
groups
but
one
group
is
considered
primary
.
You
can
use
`
id
`
to
check
your
user
and
groups:
```
akashev
@
submit01:
~
$
id
uid=
7265(akashev)
gid=
1109(math)
groups=
1109(math),902(l_gaussian)
```
Here
,
`
akashev
`
is
my
user
,
`
math
`
is
my
primary
group
and
`
l_gaussian
`
is
another
group
I
belong
to
.
---
#
Permissions:
`
rwx
`
Each
file
and
directory
in
UNIX
filesystems
has
3
permissions
(
for
a
particular
user
).
Regular
files:
*
.
highlight
[`
r
`],
or
.
highlight
[
Read
],
means
that
you
can
read
the
contents
of
a
file
.
*
.
highlight
[`
w
`],
or
.
highlight
[
Write
],
means
that
you
can
modify
the
file
.
*
.
highlight
[`
x
`],
or
.
highlight
[
eXecute
],
means
that
the
file
may
be
launched
as
a
program
.
Directories:
*
.
highlight
[`
r
`]
means
that
you
can
read
the
list
of
files
within
the
directory
.
*
.
highlight
[`
w
`]
means
that
you
can
add
or
delete
files
from
the
directory
.
*
.
highlight
[`
x
`]
means
you
can
.
highlight
[
traverse
]
the
folder:
enter
it
with
`
cd
`
and
read
the
contents
of
its
files
.
---
#
Inspecting
permissions
Try
running
`
ls
-la
`
to
see
permissions
on
files
and
folders:
```
$
ls
-la
total
20
drwxrwxr-x
2
user
group
4096
Sep
11
01:26
.
drwxrwxr-x
6
user
group
4096
Sep
10
23:06
..
-rw-rw-r--
1
user
group
90
Sep
10
23:08
date
.
log
-rw-rw-r--
1
user
group
47
Sep
11
00:50
errors
-rw-rw-r--
1
user
group
30
Sep
11
01:09
listing
```
We
'
re
interested
in
the
first
column:
the
cryptic
`
drwxrwxr-x
`
and
`
-rw-rwr--
`,
which
are
called
.
highlight
[
mode
].
--
*
The
first
character
denotes
the
.
highlight
[
file
type
].
*
`
-
`
means
"
regular
file
".
*
`
d
`
means
"
directory
".
*
The
rest
is
divided
in
groups
of
three:
*
Access
for
the
owner
*
Access
for
the
group
*
Access
for
everyone
else
---
#
File
ownership
```
drwxrwxr-x
2
user
group
4096
Sep
11
01:26
.
-rw-rw-r--
1
user
group
90
Sep
10
23:08
date
.
log
```
Each
file
in
a
UNIX
filesystem
has
an
.
highlight
[
owner
]
and
a
.
highlight
[
group
]
attached
.
In
the
example
above
,
`
user
`
is
the
owner
and
`
group
`
is
the
designated
group
.
Note
that
the
user
.
highlight
[
doesn
'
t
have
to
be
in
the
assigned
group
].
---
#
Effective
permissions
```
-rwxr-x---
1
user
group
90
Sep
10
23:08
script
```
To
determine
which
permissions
apply
,
the
following
is
checked:
*
If
the
user
is
the
owner
,
the
first
set
applies
(`
rwx
`,
full
permissions
)
*
If
the
user
is
in
the
designated
group
,
the
second
set
applies
(`
r-x
`,
so
cannot
write
)
*
For
all
other
users
,
the
third
set
applies
(`
---
`,
so
cannot
do
anything
)
--
A
special
user
,
.
highlight
[
superuser
]
(
normally
called
.
highlight
[
root
]),
can
completely
disregard
permissions
and
do
anything
to
any
file
on
the
system
.
---
#
Permissions:
first
match
applies
Note
that
the
system
does
not
apply
"
best
"
permissions
–
only
the
first
set
that
matches
.
Let
'
s
reverse
the
situation:
```
----r-xrwx
1
user
group
90
Sep
10
23:08
script
```
For
this
file
,
the
owner
cannot
do
anything
to
the
file
,
anyone
in
`
group
`
cannot
modify
it
,
but
everyone
else
has
full
permissions
.
Note:
the
owner
can
always
change
a
file
'
s
permissions
.
---
#
Modifying
permissions
To
modify
a
file
'
s
permissions
,
use
.
highlight
[`
chmod
CHANGES
FILE
`]
.
mnemonic
[
`
chmod
`
stands
for
.
highlight
[
change
mode
].
]
Possible
changes:
*
.
highlight
[`+
r
`],
.
highlight
[`+
w
`],
.
highlight
[`+
x
`
add
]
permissions
.
Can
combine:
`+
rw
`
*
.
highlight
[`
-r
`
removes
]
permissions
.
*
.
highlight
[`=
r
`]
sets
pemissions
to
.
highlight
[
exactly
]
`
r--
`.
*
Prefix
.
highlight
[`
u
`]
changes
permissions
for
the
.
highlight
[
user
],
e
.
g
.
`
u
+
r
`.
*
Prefix
.
highlight
[`
g
`]
changes
permissions
for
the
.
highlight
[
group
],
e
.
g
.
`
g
+
rw
`.
*
Prefix
.
highlight
[`
o
`]
changes
permissions
for
.
highlight
[
others
],
e
.
g
.
`
o-w
`.
*
Prefix
.
highlight
[`
a
`]
or
no
prefix
changes
permissions
for
.
highlight
[
all
]
three
sets
.
*
An
.
highlight
[
octal
number
]
(
e
.
g
.
750
)
sets
permissions
to
a
specific
configuration
(
in
this
case
,
`
rwxr-x---
`).
---
#
Modifying
permissions
Several
changes
can
be
applied
at
once
,
separated
by
commas:
```
user
@
host:
~/
scits-training
/
io
$
ls
-la
date
.
log
-rwxrw-r--
1
user
group
90
Sep
10
23:08
date
.
log
user
@
host:
~/
scits-training
/
io
$
chmod
u
+
x
,
g=
rx,o-r
date
.
log
user
@
host:
~/
scits-training
/
io
$
ls
-la
date
.
log
-rwxr-x---
1
user
group
90
Sep
10
23:08
date
.
log
```
.
exercise
[
Modify
permissions
on
the
file
to
be
`
r-xr--rw-
`
]
---
#
Changing
ownership
Similarly
to
`
chmod
`,
the
.
highlight
[`
chown
`]
command
allows
changing
a
file
'
s
owner
and
group
.
*
`
chown
USER
FILE
`
changes
the
owner
*
`
chown
:GROUP
FILE
`
changes
the
group
*
`
chown
USER:GROUP
FILE
`
changes
both
Note:
once
the
owner
is
changed
,
the
old
owner
no
longer
can
modify
access
to
the
file
.
.
highlight
[
For
this
reason
,
only
administrators
can
change
the
file
owner
,
or
assign
a
group
the
owner
is
not
part
of
.]
.
exercise
[
Use
`
groups
`
to
list
groups
you
belong
to
.
Change
a
file
'
s
group
to
one
of
them
,
and
then
back
to
the
original
one
.
]
---
#
Shell
scripting
Shell
is
not
just
an
interface
to
launch
other
programs
;
it
comes
with
its
own
scripting
language
to
automate
complex
tasks
.
You
can
have
variables
,
loops
,
conditionals
–
a
full-featured
programming
language
.
We
will
only
show
the
very
basics
.
.
exercise
[
Navigate
to
`~/
scits-training
/
scripts
`
and
open
`
boom
.
sh
`
in
your
favourite
editor
(
nano
,
vim
)
]
---
layout:
true
#
Shell
scripting
```
bash
#!/
bin
/
bash
#
I
hope
you
get
the
reference
echo
"
Someone
set
up
us
the
bomb
."
for
i
in
{
5
..
1
}
do
echo
"$
i
.."
sleep
1
done
explosion=
"Boom!"
echo
$
explosion
```
---
The
first
line
of
the
script
is
special:
```
bash
#!/
bin
/
bash
```
It
'
s
called
a
.
highlight
["
shebang
"]
(
for
.
highlight
[
shell
]
and
"!"
.
highlight
[
bang
]).
It
tells
the
shell
what
to
execute
the
rest
of
the
script
with
.
Since
we
'
re
writing
a
`
bash
`
shell
script
,
we
put
there
the
path
to
`/
bin
/
bash
`
itself
.
---
Other
lines
starting
with
.
highlight
[`#`]
are
.
highlight
[
comments
]
```
bash
#
I
hope
you
get
the
reference
```
They
are
ignored
by
`
bash
`
and
are
used
to
leave
notes
to
yourself
or
others
.
---
.
highlight
[`
echo
`]
command
outputs
its
arguments
to
STDIN
.
```
bash
echo
"
Someone
set
up
us
the
bomb
."
```
Quotes
are
used
to
make
text
with
spaces
in
it
a
single
argument
;
here
,
they
are
optional
.
---
.
highlight
[`
for
`]
designates
a
.
highlight
[
loop
]
:
a
.
highlight
[
variable
`
i
`]
will
change
from
5
to
1
.
```
bash
for
i
in
{
5
..
1
}
do
#
something
done
```
The
code
in
`#
something
`
will
repeat
with
`
i
`
as
5
,
4
,
3
,
2
and
1
.
`
do
`
and
`
done
`
delimit
the
bounds
of
the
loop
.
---
One
can
use
the
variable
in
expressions
.
highlight
[
prefixed
by
`$`],
i
.
e
.
.
highlight
[`$
i
`]
:
```
bash
echo
"$
i
.."
```
If
there
is
ambiguity
as
to
where
a
variable
name
ends
,
use
braces:
.
highlight
[`
${
i
}
`],
e
.
g
.
`"
Sample
${
i
}
A
"`
for
"
Sample
1A
",
etc
.
---
Variables
can
also
be
simply
.
highlight
[
assigned
to
]
:
```
bash
explosion=
"Boom!"
echo
$
explosion
```
The
lack
of
spaces
around
`=`
is
.
highlight
[
significant
].
Otherwise
Bash
will
try
to
execute
`
explosion
`
as
a
command
.
---
layout:
false
#
Running
a
script
OK
,
suppose
we
wrote
the
above
script
.
How
to
execute
it
?
1
.
We
need
to
make
sure
that
it
'
s
allowed
to
execute:
```
user
@
host:
~/
scits-training
/
scripts
$
chmod
+
x
boom
.
sh
```
2
.
For
security
reasons
,
the
current
directory
is
not
automatically
considered
when
starting
other
programs
.
We
need
to
explicitly
refer
to
it:
```
user
@
host:
~/
scits-training
/
scripts
$
./
boom
.
sh
```
.
exercise
[
1
.
Execute
the
script
,
saving
its
output
to
a
file
.
2
.
Modify
the
script
to
count
down
from
10
.
]
---
#
Scripting
,
take
two
The
next
script
you
will
type
out
yourselves
.
Open
`
beer
.
sh
`
in
your
favourite
editor
.
We
'
ll
write
a
simple
script
to
determine
if
a
user
is
old
enough
to
drink
beer
.
---
#
Scripting
,
take
two
```
bash
#!/
bin
/
bash
```
Any
bash
script
should
start
with
an
appropriate
shebang
.
We
want
to
ask
the
user
for
his
/
her
age
;
we
can
use
the
.
highlight
[`
read
`]
command
.
```
bash
#
-n
prevents
a
line
break
,
and
note
the
extra
space
echo
-n
"
What
'
s
your
age
?
"
read
age
```
This
will
display
a
promt
for
the
user
and
wait
for
input
.
The
result
is
then
stored
in
the
variable
`$
age
`.
For
simplicity
,
we
will
not
check
that
the
input
is
indeed
a
valid
number
.
---
#
Scripting
,
take
two
```
bash
#!/
bin
/
bash
echo
-n
"
What
'
s
your
age
?
"
read
age
```
We
need
to
make
a
decision
based
on
age
;
we
need
an
if-then-else
construct
.
```
bash
if
[
$
age
-lt
16
]
then
echo
"
You
'
re
too
young
to
drink
!"
else
echo
"
You
'
re
old
enough
,
have
a
beer
!"
fi
```
.
highlight
[`
fi
`]
here
is
`
if
`
reversed
,
to
close
the
.
highlight
[`
if
`]
statement
.
Conditionals
in
bash
are
a
bit
clunky
,
but
.
highlight
[`
-lt
`]
here
stands
for
.
highlight
[
less
than
].
Again
,
the
whitespace
here
is
.
highlight
[
significant
].
---
#
Scripting
,
take
two
```
bash
#!/
bin
/
bash
echo
-n
"
What
'
s
your
age
?
"
read
age
if
[
$
age
-lt
16
]
then
echo
"
You
'
re
too
young
to
drink
!"
else
echo
"
You
'
re
old
enough
,
have
a
beer
!"
fi
```
.
exercise
[
1
.
Save
this
script
to
`
beer
.
sh
`.
2
.
Change
the
file
'
s
mode
to
allow
execution
.
3
.
Test
the
script
with
different
values
.
]
---
#
Scripting
improvements
Let
'
s
add
a
little
personal
touch
.
`
whoami
`
is
a
command
that
returns
the
username
.
Let
'
s
edit
`
beer
.
sh
`
to
use
it:
```
bash
then
echo
"$(
whoami
),
you
'
re
too
young
to
drink
!"
else
echo
"$(
whoami
),
you
'
re
old
enough
,
have
a
beer
!"
fi
```
.
highlight
[`$(
something
)`]
allows
you
to
execute
a
command
and
substitute
the
result
within
another
command
.
.
exercise
[
Test
the
new
additions
.
]
---
#
Scripting
improvements
Let
'
s
read
the
age
from
the
command
line
arguments
.
`
bash
`
automatically
populates
.
highlight
[`$
0
`]
with
the
.
highlight
[
name
of
the
executable
],
and
.
highlight
[`$
1
`],
.
highlight
[`$
2
`]
and
so
on
with
.
highlight
[
arguments
].
Let
'
s
use
`$
1
`
as
age
if
it
'
s
defined:
```
bash
if
[
$
1
]
then
age=
$
1
else
echo
-n
"
What
'
s
your
age
?
"
read
age
fi
```
.
exercise
[
Test
that
`./
beer
.
sh
`
now
automatically
gets
the
age
from
its
first
argument
,
and
still
asks
if
no
argument
is
provided
.
]
---
#
Return
values
Whenever
a
program
terminates
,
it
returns
a
single
integer
to
the
shell
that
called
it
;
it
'
s
called
the
.
highlight
[
return
value
].
By
convention:
*
.
highlight
[
0
]
means
"
no
error
".
*
any
.
highlight
[
non-zero
value
]
means
"
some
kind
of
error
".
Let
'
s
return
appropriate
values:
```
bash
then
echo
"$(
whoami
),
you
'
re
too
young
to
drink
!"
exit
1
else
echo
"$(
whoami
),
you
'
re
old
enough
,
have
a
beer
!"
exit
0
fi
```
---
#
Chaining
commands
You
can
chain
commands
in
shell
with
.
highlight
[`;`]
or
.
highlight
[`&&`].
.
highlight
[`;`]
will
execute
commands
one
by
one
,
regardless
of
errors
.
```
$
command1
;
command2
```
.
highlight
[`&&`]
will
only
execute
the
next
command
only
if
the
previous
one
returned
0
,
i
.
e
.
finished
without
errors
.
```
$
command1
&&
command2
```
.
exercise
[
1
.
Apply
the
return
value
changes
to
`
beer
.
sh
`
2
.
Test
it
with
`./
beer
.
sh
&&
echo
'
Cheers
!'`
]
---
#
Environment
variables
Your
profile
files
can
set
various
.
highlight
[
environment
variables
]
:
snippets
of
data
inherited
by
programs
running
from
shell
.
You
can
see
your
current
environment
variables
with:
```
$
env
|
less
```
--
Some
programs
rely
on
environmental
variables
to
change
their
behavior
.
Example:
```
bash
#
Will
replace
the
default
editor
with
vim
in
some
commands
export
EDITOR=
vim
```
---
#
Environment
variables
Environment
variables
work
similarly
to
variables
in
a
script
,
except
for
the
extra
command
.
highlight
[
export
],
which
propagates
this
variable
to
child
processes
.
By
convention
,
environment
variables
are
UPPERCASE
.
--
.
exercise
[
1
.
Set
an
ordinary
varable
A
(`
A=
'Hello'
`)
and
environment
variable
B
(`
export
B=
'World'
`).
2
.
Use
`
echo
$
A
`
and
`
echo
$
B
`
to
display
them
.
3
.
Add
`
echo
$
A
`
and
`
echo
$
B
`
to
a
script
and
execute
it
.
What
changes
?
]
---
#
`$
PATH
`
variable
An
important
variable
is
`$
PATH
`.
It
'
s
a
colon-separated
list
of
directories
which
are
searched
when
you
try
to
run
a
program
by
name
.
Notably
,
the
current
directory
is
not
in
`$
PATH
`.
If
you
have
created
some
own
scripts
/
programs
and
want
them
to
be
available
by
name
from
anywhere
,
you
can
put
them
in
a
folder
(
e
.
g
.
`~/
bin
`)
and
add
it
to
`$
PATH
`
:
```
bash
export
PATH=
"
$PATH:$
HOME/bin"
```
---
#
`$
PATH
`
variable
,
example
.
exercise
[
1
.
Try
running
`
beer
.
sh
`
directly
y
name
.
It
fails
.
2
.
Add
the
folder
that
holds
it
to
`$
PATH
`
:
```
bash
export
PATH=
"
$
PATH:~/scits-training/scripts"
```
3
.
Try
running
`
beer
.
sh
`
now
.
4
.
Try
going
somewhere
else
(`
cd
~`)
and
run
it
.
]
---
#
Aliases
If
you
use
a
certain
command
often
,
you
can
define
a
short
name
for
it
.
For
example
,
if
you
want
a
shorter
name
for
`
ls
-lh
`
because
you
always
want
to
see
human-readable
sizes
,
you
can
make
an
alias:
```
$
alias
lh=
"ls -lh"
$
lh
total
26M
-rw-r--r--
1
user
group
25M
Sep
11
07:22
big_file
-rw-r--r--
1
user
group
735
Sep
11
07:22
description
-rw-r--r--
1
user
group
0
Sep
11
07:22
empty_file
-rw-r--r--
1
user
group
551
Sep
11
07:22
naming
drwxr-xr-x
0
user
group
512
Sep
11
07:22
subfolder
```
---
#
Making
customizations
permanent
To
make
above
tweaks
permanent
,
they
need
to
be
added
either
to
`.
bash_profile
`
or
`.
bashrc
`.
Then
they
will
apply
on
each
opened
shell
.
*
`.
bash_profile
`
is
.
highlight
[
sourced
]
at
most
once
.
Put
things
there
that
shouldn
'
t
be
called
multiple
times
.
*
`.
bashrc
`
is
sourced
almost
every
time
`
bash
`
is
called
,
except
for
initial
SSH
shell
.
To
be
safe
,
you
can
"
include
"
`.
bashrc
`
into
`.
bash_profile
`
like
this:
```
bash
#
In
.
bash_profile
#
-f
tests
that
file
exists
#
source
executes
commands
in
the
current
shell
if
[
-f
~/.
bashrc
];
then
source
~/.
bashrc
fi
```
---
class:
center
,
middle
#
Extra
credits
---
#
Searching
through
history
There
'
s
a
way
to
quickly
search
through
previous
commands
.
`
Ctrl
`+`
R
`
opens
"
reverse
search
"
mode
.
Enter
some
pattern
and
the
closest
command
in
history
that
matches
will
be
shown
.
To
look
into
older
commands
,
press
`
Ctrl
`+`
R
`
again
,
or
`
Ctrl
`+`
C
`
to
abort
.
---
#
Custom
shell
prompt
The
variable
`$
PS1
`
contains
the
format
template
for
your
shell
prompt
.
Throughout
this
training
,
you
saw
the
following
prompt:
```
user
@
host:
~
$
```
You
can
customize
it
!
For
example:
```
user
@
host:
~
$
export
PS1=
"[\t] \u@\h:\w\\n\\
$
"
[
16:40:00
]
user
@
host:
~
$
```
Want
to
control
that
precisely
?
Want
to
add
color
?
There
'
s
[
a
guide
](
https:
//
www
.
digitalocean
.
com
/
community
/
tutorials
/
how-to-customize-your-bash-prompt-on-a-linux-vps
)
for
that
.
---
#
Finding
files
The
`
find
PATH
`
command
looks
through
the
filesystem
at
PATH
to
find
files
.
One
can
then
filter
the
output
with
`
grep
`,
or
use
`
find
`'
s
own
keys
for
sophisticated
filtering
.
```
user
@
host:
~
$
find
~/
scits-training
-name
'*.
sh
'
/
home
/
user
/
scits-training
/
scripts
/
boom
.
sh
/
home
/
user
/
scits-training
/
scripts
/
beer
.
sh
```
---
#
`
xargs
`
The
`
xargs
`
command
can
be
used
to
convert
output
into
arguments
of
another
command
.
`
xargs
COMMAND
`
will
take
input
and
pass
it
as
separate
arguments
after
`
COMMAND
`
:
```
user
@
host:
~
$
find
~/
scits-training
-name
'*.
sh
'
|
xargs
cat
[
contents
of
both
.
sh
files
]
```
This
is
equivalent
to
```
user
@
host:
~
$
cat
/
home
/
user
/
scits-training
/
scripts
/
boom
.
sh
\
/
home
/
user
/
scits-training
/
scripts
/
beer
.
sh
```
---
#
Public
key
authentication:
theory
It
can
be
useful
to
use
key
authentication
instead
of
standard
password
authentication
.
*
Far
more
secure
—
suitable
for
internet-facing
computers
.
*
May
be
required
in
cloud
environments
to
set
new
VMs
.
*
Allows
passwordless
authentication
for
more
convenience
.
--
It
is
based
on
modern
cryptography
and
consists
of
pairs
of
keys:
.
highlight
[
public
],
which
you
can
give
to
others
,
and
.
highlight
[
private
],
that
you
keep
yourself
(
preferably
encrypted
with
a
.
highlight
[
passphrase
]).
Having
the
private
key
allows
you
to
prove
that
you
own
the
keypair
to
anyone
having
your
public
key
,
without
disclosing
the
private
key
itself
.
---
#
Public
key
authentication:
workflow
When
setting
up
public
key
authentication
on
Linux
,
here
'
s
the
workflow:
1
.
You
generate
a
keypair:
private
key
and
public
key
files
.
On
Linux
/
Mac
,
`
ssh-keygen
`
is
used
.
On
Windows
,
PuTTYgen
can
be
used
.
2
.
You
copy
the
public
to
the
remote
system
.
3
.
You
connect
,
instructing
SSH
to
use
the
private
key
.
If
it
'
s
encrypted
,
you
'
re
asked
for
the
passphrase
(
and
may
be
cached
in
an
SSH
agent
after
that
).
---
#
Public
key
authentication:
workflow
On
most
Linux
systems
,
it
is
sufficient
to
have
your
public
key
in
`~/.
ssh
/
authorized_keys
`
file
.
To
copy
the
keypair
to
the
remote
system
,
`
ssh-copy-id
`
script
can
be
used
.
--
Otherwise
,
you
need
to
create
that
file
/
folder
yourself
,
and
make
sure
they
have
proper
permissions
.
```
user
@
remote:
~$
ls
-l
~/.
ssh
total
12
drwx------
2
user
user
4096
Sep
13
2017
.
drwxr-xr-x
13
user
user
4096
Sep
2
21:47
..
-rw-------
1
user
user
1159
Jan
21
2018
authorized_keys
user
@
remote:
~$
cat
.
ssh
/
authorized_keys
ssh-rsa
AAAAB3NzaC1yc2EAAA
[...]
j6aKfAUoXOE=
some
comment
```
--
To
use
a
key
,
one
can
use
`
-i
PRIVATE_KEY_FILE
`
flag
for
`
ssh
`,
or
Pageant
on
Windows
.
</
textarea
>
<script
src=
"js/vendor/remark.min.js"
></script>
<script
src=
"js/vendor/jquery-3.2.1.min.js"
></script>
<script
src=
"js/terminal.language.js"
></script>
<script
src=
"js/common.js"
></script>
</body>
</html>
Event Timeline
Log In to Comment