We followed the intro SCITAS Course to connect to the Clusters
Documentation is here:
https://scitas-data.epfl.ch/public/training/using_the_clusters.pdf
https://scitas-data.epfl.ch/kb/File+systems to get some information on FIleSystem
= Useful UNIX commands =
* `ls` list content of directory
* `id` returns all groups you belong to
* `pwd` displays current directory
* `mkdir` makes a folder
* `wget` download a web page
* `cat` displays the content of a file in the terminal
* `curl` (`curl www.perdu.com`) write the content of a web page ?
* `man` get help on a linux command `man wget` for instance
To execute a file:
* Add a point before the file : `.`
=Structure of cluster =
`/home/username` your personal area, shared among all 3 clusters
`/work/ptbiop` where biop software are installed, shared among all 3 clusters
`/scratch/username` is a high performance file system where we put the data we need to work on, it is not shared and depends on the cluster you are connected to
=How to connect=
Scitas documentation : https://scitas-data.epfl.ch/kb/Connecting%2Bto%2Bthe%2Bclusters
There are 3 clusters at EPFL named `deneb1`, `deneb2`, `fidis`.
* Linux: Type `ssh gasparlogin@clustername.epfl.ch` and give your password. For instance if Nico wants to connect to the deneb1 cluster (there's also deneb2 and fidis) : `ssh chiarutt@deneb1.epfl.ch`.
* Windows:
** On windows 10, there's a ssh client: so execute `cmd.exe` and just type `ssh gasparlogin@deneb1.epfl.ch` in your windows terminal
** Alternative method: install and use [[ https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html | PuttY ]]
=How to transfer data into the server=
== Terminal command line ==
It can be useful to transfer directly from the internet :
* From the internet : `wget http://linkToMyFile.what` for instance to download ilastik: `wget http://files.ilastik.org/ilastik-1.3.0-Linux.tar.bz2`
* From a windows shared server (svfas6): `smbclient '//svfas6.epfl.ch/biop/c$' -c 'lcd /home/chiarutt/examples; cd public; get test.txt' -U intranet/chiarutt`. This rather unintuitive command copies the file `//svfas6.epfl.ch/biop/public/test.txt` to the folder `/home/chiarutt/examples`. You'll need to enter your gaspar password. Type `cat test.txt` in the correct folder to see that you've indeed downloaded this file.
== From your local computer with a Graphical User Interface ==
* Windows users: This can be done via [[ https://filezilla-project.org/ | FileZilla ]], but take care, this soft now installs unwanted software during the installation.
* Linux users: in nautilus, go to `ssh://gasparlogin@deneb1.epfl.ch`
== Where ==
For temporary jobs, put to `\scratch\gasparlogin`. For instance Nico can type `cd \scratch\chiarutt` then `ls` to list the contents of this folder within a terminal.
=How to install software=
= Software available =
You can install either software (unix executable / java applications) into your folder.
=How to install software within your folder =In the work folder. Fiji is available. As several FIJI may be required depending on the configuration, the first one, which do not have any plugin installed, is the default one.
=== How to install software within the BIOP work folder =
= Snew software available ===
In the work folder. Fiji is available. As several FIJI may be required depending on the configuration, the first one, which do not have any plugin installed, is the default oneYou can install either software (unix executable / java applications) into your folder or in the shared `/work/ptbiop/` folder.
== DEFAULT FIJI ==
No update site installed. Just a raw updated fiji.
* Location folder: ` /work/ptbiop/DefaultFiji/Fiji.app/`
* Fiji can be updated using command lines:
** Update FIJI by
== Ilaslik ==
= Testing simple tasks on the cluster =
By default you are connected to the so-called login node, which is not a processing node. You can browse, copy files, do your stuff, even launch ImageJ / Ilastik / Whateve(r) , but this is not where you are supposed to do heavy processing!
Described below:
* how to launch process in the login node (in order to understand a bit about unix commands)
* how to launch a JOB, i.e. which is calling a request to launch a process.
== How to launch a process ==
== Example Ilastik Launch ==
`./run_ilastik.sh --headless --project=/scratch/chiarutt/ilps/MyProject.ilp /scratch/chiarutt/images_in/Vesicles.tif `
`./run_ilastik.sh --headless`
== Example CellProfiler Launch ==
== Example Fiji Launch ==
To launch Fiji, just type:
` /work/ptbiop/DefaultFiji/Fiji.app/ImageJ-linux64 --ij2 --headless`
This launches a Fiji instance, which is doing nothing special. Some sparse documentation can be found on the ImageJ.net website:
* [[https://imagej.net/Headless | Headless ]]
* [[https://imagej.net/Scripting_Headless| Scripting Headless ]]
There is the possibility to run a simple script. For instance go to `/work/ptbiop/samplescripts/ij/` ant type `nano hello.py`. This script has one parameter and simply greets the user by its name. Type 'Ctrl+X' to exit nano, and then type:
` /work/ptbiop/DefaultFiji/Fiji.app/ImageJ-linux64 --ij2 --headless --run /work/ptbiop/samplescripts/ij/hello.py 'name="Patrick"' `
You've said Hello to Patrick with Fiji on the cluster. That's quite an achievement. Congrats!