setup.rst

Setup

Prerequisites

The tutorial might still work with Python 2 but it is recommended to use Python 3.

If you already have Python on your computer you can skip the next step but you should first test your Python version with:

python --version

Install Python

It is recommended to use the anaconda python distribution. Download anaconda (Python 3 version) from https://www.anaconda.com/distribution/ and follow the installation instructions on the download page.

After the installation you should make sure that python from this installation is in your $PATH.

Virtual Environment

If you do not want to modify your python installation or if you use an installation for which you do not have write permissions you can set up a virtual environment and activate it:

python -m venv $HOME/python3-fireworks-tutorial
. $HOME/python3-fireworks-tutorial/bin/activate

Install packages

The following python packages must be installed:

pip install --upgrade pip
pip install pjson
pip install pyaml
pip install python-igraph
pip install fireworks

NOTE: For the igraph package a C++ compiler and some additional development packages must be installed on your computer. For CentOS and Red Hat Linux these can be installed with:

sudo yum install gcc-c++
sudo yum install libxml2-devel
sudo yum install python3-devel

Setting up a FireServer

The tutorial requires a FireServer which is accessible from your computer and from your computing resource (e.g. an HPC cluster).

Remote FireServer

If you intend to run the rocket launchers on a remote computing cluster then you can configure a launchpad file for an existing FireServer. In this case you should create a launchpad configuration file named launchpad.yaml in the folder $HOME/.fireworks (create this folder if necessary). The file must have the following contents:

host: <hostname of the FireServer>
port: 27017
name: <database name>
username: <your username>
password: <your password>
mongoclient_kwargs:
  tls: true
  tlsCAFile: <absolute path to CA certificate>
  tlsCertificateKeyFile: <absolute path to client certificate>

You will get the two certificates, hostname, username and password from the instructors. In the folder $HOME/.fireworks the fireworks configuration file FW_config.yaml must be created with the following command:

echo LAUNCHPAD_LOC: $HOME/.fireworks/launchpad.yaml >> $HOME/.fireworks/FW_config.yaml

This configuration has to be made available on all resources that will use the FireServer, i.e. your local computer, login nodes and compute nodes of the clusters etc. If these resources share the same $HOME file system then the configuration has to be done only once.

Local FireServer

If you have Linux and administrator permissions on your computer you can install MongoDB system-wide, e.g. on Ubuntu with the following command:

sudo apt-get install mongodb

The server is automatically started after the installation is completed. In this case you do not need a launchpad configuration file. However, you cannot launch fireworks on a remote FireWorker machine in this mode.

For general instructions please read the installation manual https://docs.mongodb.com/manual/administration/install-on-linux/

Install the tutorial

The tutorial is available for download on the SCC Gitlab repository:

cd $HOME
wget https://git.scc.kit.edu/jk7683/grk2450-fireworks-tutorial/-/archive/master/grk2450-fireworks-tutorial-master.zip
unzip grk2450-fireworks-tutorial-master.zip
mv grk2450-fireworks-tutorial-master grk2450-fireworks-tutorial
cd grk2450-fireworks-tutorial
export PYTHONPATH=$PWD/lib:$PYTHONPATH
export PATH=$PWD/bin:$PATH

NOTE: The folder grk2450-fireworks-tutorial/lib must also be available in $PYTHONPATH on the FireWorker, i.e. the machine where fireworks will be launched.

Install further packages

In order to visualize the workflows graphically two packages must be installed:

sudo apt-get install graphviz evince

The packages ImageMagick and Eye of GNOME (eog) are necessary for Exercise 3:

sudo apt-get install imagemagick eog

If editors like vi or nano are not preferred, more advanced editors may be installed, e.g.:

sudo apt-get install gedit emacs