This page aims to contain the setup in order to be able to run the deep learning tools that are currently being used at the BIOP = Setup for Windows = NOTE: This setup is intended for a workstation with a good GPU and Windows 10 Installed. It has been tested with the following cards: **GTX Titan, GTX 1080, GTX 1080 Ti, RTX 2080 Ti** == Software to Install == - [[ https://www.nvidia.com/Download/index.aspx | Download Latest NVIDIA Drivers ]] - We use the `Studio Drivers` - [[ https://visualstudio.microsoft.com/downloads/#build-tools-for-visual-studio-2019 |Download Build Tools for Visual Studio 2019 ]] - Install the `Build tools for C++`, namely - `MSVC v142 - VS 2019 C++ x64/x86 build tools` - `Windows 10 SDK (10.0.18362.0)` - [[ https://developer.nvidia.com/cuda-10.1-download-archive-update2?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exenetwork | Download CUDA Toolkit 10.1 Update 2]] - Install just the `Developer` and `Runtime` parts {F14514741, width=300, layout=center} - [[ https://developer.nvidia.com/compute/machine-learning/cudnn/secure/7.6.5.32/Production/10.1_20191031/cudnn-10.1-windows10-x64-v7.6.5.32.zip | Download CuDNN 7.6.5 for CUDA 10.1]] - Unzip into `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\` - [[https://www.python.org/downloads/release/python-379/| Install Python 3.7]] - We install it for all users and add Python to the `PATH` == Environment Variables == NOTE: Environment Variables are file and folder locations your operating system looks into when you call a program or library. It is like the index of a library, telling you where to look for different things. In Windows, we can create and edit Environment Variables graphically. # Start typing "Environment" in the Windows searchbar until it suggests "Edit the system environment variables". # Click on {nav Environment Variables...} and use the {nav New...} button all the way down (The one that corresponds to the "System Variables") to create the two following environment variables. These are needed for compiling PyOpenCL, which is a dependency of `gpuools` for StarDist, but is beneficial for any other tools that use the GPU and may need compiling. |VARIABLE NAME | Path (Default) | |-------------------|-----------------| |INCLUDE| `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include`| |LIB| `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\lib\x64`| NOTE: You will need to click on {nav New...} for each Environment Variable you create. In the end it should look like this: {F15073749, width=300, layout=center} Note how there is also a CUDA 10.0 version installed. This is OK and will be made clear below if you install Noise2Void. == Creating Virtual Environments == NOTE: Consider upgrading `pip` to the latest vesion before starting: `python -m pip install --upgrade pip` === Windows Command Prompt `cmd` === IMPORTANT: We are running all the commands below from the **Windows Command Prompt**. Access it using {key Win R}, type `cmd` and hit enter. To get started, we need to install `virtualenv` to manage our environments ``` pip install virtualenv ``` Now we can create the `virtualenv`s for our favorite deep learning tools! === StarDist === We are going to create a `stardist-tf2` environment in the `D:\environments` folder From your command prompt: ``` d: mkdir environments python -m venv environments\stardist-tf2 environments\stardist-tf2\Scripts\activate ``` (WARNING) **Checkpoint**: Ensure the right python version is installed using `where python` and you should have a result like below ``` D:\environments\stardist-tf2\Scripts\python.exe C:\Users\oburri\AppData\Local\Programs\Python\Python37\python.exe C:\Users\oburri\AppData\Local\Microsoft\WindowsApps\python.exe ``` Finally, we can install all of StarDist using the `stardist.txt` file below `pip install -r stardist.txt` {F15067175} === Noise2Void === Noise2Void currently needs a lower version of CUDA to function. Installations - [[ https://developer.nvidia.com/cuda-10.0-download-archive?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exenetwork | Download CUDA Toolkit 10.0 ]] - [[ https://developer.nvidia.com/compute/machine-learning/cudnn/secure/7.6.5.32/Production/10.0_20191031/cudnn-10.0-windows10-x64-v7.6.5.32.zip | Download CuDNN 7.6.5 for CUDA 10.0]] - Unzip into `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\` We are going to create a `n2v` environment in the `D:\environments` folder Start a command prompt and type ``` d: mkdir environments python -m venv environments\n2v environments\n2v\Scripts\activate ``` Finally, we can install all of Noise2Void using the `n2v.txt` file below `pip install -r n2v.txt` {F15067171} === CellPose === Installations: - [[ https://developer.nvidia.com/cuda-10.1-download-archive-update2?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exenetwork | Download CUDA Toolkit 10.1 Update 2]] - Install just the `Developer` and `Runtime` parts - [[ https://developer.nvidia.com/compute/machine-learning/cudnn/secure/7.6.5.32/Production/10.1_20191031/cudnn-10.1-windows10-x64-v7.6.5.32.zip | Download CuDNN 7.6.5 for CUDA 10.1]] - Unzip into `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\` - [[https://www.python.org/downloads/release/python-379/| Install Python 3.7]] - We install it for all users and add Python to the `PAT We are going to create a `cellpose` environment in the `D:\environments` folder Start a command prompt and type ``` d: mkdir environments python -m venv environments\cellpose environments\cellpose\Scripts\activate ``` Finally, we can install all of Cellpose using the `cellpose.txt` file below ``` pip install -r cellpose.txt pip uninstall mxnet-mkl -y pip install mxnet-cu101 ``` {F15073960} = Setup for SCITAS GPU Clusters = Romain can tell us this :) = 'virtualenv' Information & Examples = IMPORTANT: Notice how we always use `Scripts\activate` **before **running any pip commands. The `activate` command ensures that we are running within the `virtualenv`. To confirm you are running in the virtualenv, its name should be in parentheses on the left of the command prompt. {F15073659, width=700} IMPORTANT: `virtualenv` are unique to each machine (potentially user) and cannot be duplicated. It is not enough to copy the `stardist-tf` environment to another location or another computer to use it. This is by construction. File paths and file permissions are linked to the location the virtual environment was created in. IMPORTANT: You should **not **have your Notebooks in the same folder as your `virtualenv`. A `virtualenv` is a folder which contains the libraries and executables to create an //environment// for you to run your code. It is independent. It would be like storing your Excel results in `C:\Program Files\Microscoft Office`. == Example: Activating `stardist-tf` and running JupyterLab in your project folder == Suppose we have the following setup: {F15073616, width=400} Note how the `My Project` folder is not even on the same disk as the `virtualenv`. To start JupyterLab in your folder, do the following from the command prompt ``` d:\environments\stardist-tf2\Scripts\activate cd /d "E:\My Project" jupyter lab ``` = Create a shortcut to activate your `virtualenv` = === Example with Noise2Void === You can create a `Run Noise2Void.bat` file that you can keep somethere in your project folder that does this automatically. The syntax is a little different than above. Copy paste the code below into the file you just created. Adjust paths as needed. ``` call d:\environments\n2v\scripts\activate cd /d "E:\My Project" call jupyter lab ``` === Example with StarDist=== You can create a `Run StarDist-TF2.bat` file that you can keep somethere in your project folder that does this automatically. Copy paste the code below into the file you just created. Adjust paths as needed. ``` call d:\environments\stardist-tf2\scripts\activate cd /d "E:\My Project" call jupyter lab ```