We followed the intro SCITAS Course to connect to the Clusters
Documentation is here:
https://scitas-data.epfl.ch/public/training/using_the_clusters.pdf
https://scitas-data.epfl.ch/kb/File+systems to get some information on FIleSystem
= Useful UNIX commands =
* `ls` list content of directory
* `id` returns all groups you belong to
* `pwd` displays current directory
* `mkdir` makes a folder
* `wget` download a web page
* `cat` displays the content of a file in the terminal
* `curl` (`curl www.perdu.com`) write the content of a web page ?
* `man` get help on a linux command `man wget` for instance
To execute a file:
* Add a point before the file : `.`
=Structure of cluster =
`/home/username` your personal area, shared among all 3 clusters
`/work/ptbiop` where biop software are installed, shared among all 3 clusters
`/scratch/username` is a high performance file system where we put the data we need to work on, it is not shared and depends on the cluster you are connected to
=How to connect=
Scitas documentation : https://scitas-data.epfl.ch/kb/Connecting%2Bto%2Bthe%2Bclusters
There are 3 clusters at EPFL named `deneb1`, `deneb2`, `fidis`.
* Linux: Type `ssh gasparlogin@clustername.epfl.ch` and give your password. For instance if Nico wants to connect to the deneb1 cluster (there's also deneb2 and fidis) : `ssh chiarutt@deneb1.epfl.ch`.
* Windows:
** On windows 10, there's a ssh client: so execute `cmd.exe` and just type `ssh gasparlogin@deneb1.epfl.ch` in your windows terminal
** Alternative method: install and use [[ https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html | PuttY ]]
=How to transfer data into the server=
== Terminal command line ==
It can be useful to transfer directly from the internet :
* From the internet : `wget http://linkToMyFile.what` for instance to download ilastik: `wget http://files.ilastik.org/ilastik-1.3.0-Linux.tar.bz2`
* From a windows shared server (svfas6): `smbclient '//svfas6.epfl.ch/biop/c$' -c 'lcd /home/chiarutt/examples; cd public; get test.txt' -U intranet/chiarutt`. This rather unintuitive command copies the file `//svfas6.epfl.ch/biop/public/test.txt` to the folder `/home/chiarutt/examples`. You'll need to enter your gaspar password. Type `cat test.txt` in the correct file to see that you've indeed downloaded this file.
* From your local computer
** Windows users:
** Linux users: in nautilus, go to `ssh://gasparlogin@deneb1.epfl.ch`
== Where ==
For temporary jobs, put to `\scratch\gasparlogin`. For instance Nico can type `cd \scratch\chiarutt` then `ls` to list the contents of this folder within a terminal.
=How to install software=
You can install either software (unix executable / java applications) into your folder.
=How to install software within your folder =
=How to install software within the BIOP work folder =
= Software available =
In the work folder. Fiji is available. As several FIJI may be required depending on the configuration, the first one, which do not have any plugin installed, is the default one.
== DEFAULT FIJI ==
No update site installed. Just a raw updated fiji.
* Location folder: ` /work/ptbiop/DefaultFiji/Fiji.app/`
== Ilaslik ==
= Testing simple tasks on the cluster =
By default you are connected to the so-called login node, which is not a processing node. You can browse, copy files, do your stuff, even launch ImageJ / Ilastik / Whateve(r) , but this is not where you are supposed to do heavy processing!
Described below:
* how to launch process in the login node (in order to understand a bit about unix commands)
* how to launch a JOB, i.e. which is calling a request to launch a process.
== How to launch a process ==
== Example Ilastik Launch ==
`./run_ilastik.sh --headless --project=/scratch/chiarutt/ilps/MyProject.ilp /scratch/chiarutt/images_in/Vesicles.tif `
`./run_ilastik.sh --headless`
== Example CellProfiler Launch ==
== Example Fiji Launch ==
To launch Fiji, just type:
` /work/ptbiop/DefaultFiji/Fiji.app/ImageJ-linux64 --ij2 --headless`
This launches a Fiji instance, which is doing nothing special.
There is the possibility to run a simple script. For instance go to `/work/ptbiop/samplescripts/ij/` ant type `nano hello.py`. This script has one parameter and simply greets the user by its name. Type 'Ctrl+X' to exit nano, and then type:
` /work/ptbiop/DefaultFiji/Fiji.app/ImageJ-linux64 --ij2 --headless --run /work/ptbiop/samplescripts/ij/hello.py 'name="Patrick"' `
You've said Hello to Patrick with Fiji on the cluster. That's quite an achievement. Congrats!