Install on HPC cluster

HPC cluster often do not have a desktop environment, meaning that you need to install the headless version of SpotMAX.

Since most of the HPC clusters run on some Linux-based OS, we recommend using conda not only to manage the environments, but also to install the dependencies.

That means installing all the dependencies first and then install Cell-ACDC and SpotMAX without dependencies.

In the Environments folder, you will find the following files:

  • conda_env_headless.yml to install dependencies with conda.

  • requirements_headless.txt to install dependencies with pip. Note that pip can also be used within a conda environment.

Follow these steps to install SpotMAX headless:

  1. Copy environment file(s)

    Copy the files above on a folder on the cluster or download them automatically from the terminal with the following commands:

    curl -O https://github.com/ElpadoCan/SpotMAX/blob/main/envs/conda_env_headless.yml
curl -O https://github.com/ElpadoCan/SpotMAX/blob/main/envs/requirements_headless.txt

  2. Install the package manager

    If conda is not already installed on the cluster, install Miniconda by following this guide Install Miniconda.

  3. Create environment and install dependencies

    Navigate in the terminal to the folder where you downloaded the environment files and run the following command(s):

    conda env create -f conda_env_headless.yml

  4. Install additional dependencies from pip:

    Some of SpotMAX dependencies are available only from pip which means you need to install them manually with the following commands:

    pip install --no-deps "git+https://github.com/SchmollerLab/Cell_ACDC.git"
pip install "git+https://github.com/ElpadoCan/pytorch3dunet.git"

  5. Install SpotMAX from pip:

    Install SpotMAX with the following command:

    pip install "git+https://github.com/ElpadoCan/SpotMAX.git"

  6. Install PyTorch (optional):

    To install PyTorch follow this guide Install PyTorch. If you have an NVIDIA GPU and a CUDA-capable system, make sure to install the correct CUDA drivers for your GPU by following this guide Install CUDA drivers.

    If you are using a cloud computing cluster with Ubuntu (e.g., on de.NBI cloud) and you need to setup PyTorch with NVIDIA GPU see this guide Install PyTorch with GPU support (Linux)

    Note

    PyTorch is needed only if you plan to use the SpotMAX AI method for spot segmentation. See the parameter Spots segmentation method for more details.

  7. Download SpotMAX AI model weights (optional):

    Download the model weights from here 3D model and here 2D model to these paths:

    ~/spotmax_appdata/unet_checkpoints/unet2D/unet_best.pth
~/spotmax_appdata/unet_checkpoints/unet3D/normal_30_250_250_20_100_100/best_checkpoint.pytorch

    Alternatively, you can download them automatically with the following commands:

    curl --create-dirs -o ~/spotmax_appdata/unet_checkpoints/unet2D/unet_best.pth https://hmgubox2.helmholtz-muenchen.de/index.php/s/4dxeHSLDfAbC8dA/download/unet_best.pth
curl --create-dirs -o ~/spotmax_appdata/unet_checkpoints/unet3D/normal_30_250_250_20_100_100/best_checkpoint.pytorch https://hmgubox2.helmholtz-muenchen.de/index.php/s/eoeFcgsAMDsgTgw/download/best_checkpoint.pytorch

Optional: install Numba

SpotMAX can take advantage of multiple CPU cores using the package numba. For more details, see the Numba documentation.

This typically increases execution speed. After installing SpotMAX, consider installing numba with the following command:

conda install numba

Note

You can set the number of CPU cores used by numba in the INI configuration file using the Number of threads used by numba parameter as follows:

Note

If any of the packages’ installation fails, it is worth trying installing that package with pip (or with conda if it fails with pip). In this case you will have to install the packages manually one by one. However, this strategy should be used as a very last resort.