How to use my conda environment on SLURM sbatch?

changed the description

By ployplearn.ravivanpong on 2022-07-22T16:41:30 (imported from GitLab)

changed the description

By ployplearn.ravivanpong on 2022-07-22T16:43:12 (imported from GitLab)

changed the description

By ployplearn.ravivanpong on 2022-07-22T16:45:03 (imported from GitLab)

mentioned in issue #54

By ployplearn.ravivanpong on 2022-07-22T16:50:59 (imported from GitLab)

added help needed label

removed help needed label

1. In the folder where your notebook is saved, create a shell file, e.g. run_my_python.sh
2. In side the shell file, type the following lines

#!/bin/bash

#SBATCH --mail-user=ployplearn.ravivanpong@kit.edu  # User e-mail for reporting
#SBATCH --mail-type=END,FAIL                        # Send e-mail when the process ends or an error occurs
#SBATCH --export=ALL                                # Export all environment variables 
#SBATCH --time=08:00:00                             # Wall time 8h
#SBATCH --partition=sdil                            # Use SDIL cluster
#SBATCH --cpus-per-task=40                          # Request 40 cpus
#SBATCH --mem=80gb                                  # 80GB RAM

# Execute jupyter notebook. A new notebook with outputs will be created.
jupyter nbconvert --to notebook --execute 1_0_1_topic_model_bigram.ipynb

NOTE

• The file must start with #!/bin/bash. Otherwise, your script may not run.

3. In a terminal, activate your environment with conda my_env_name

4. Then run your SLURM batch file e.g. sbatch run_my_python.sh

By ployplearn.ravivanpong on 2023-02-27T08:59:21 (imported from GitLab)