Skip to content
Snippets Groups Projects
Commit ed4801ee authored by Ivan Kondov's avatar Ivan Kondov
Browse files

Update basic.rst

parent 5b4a17e7
No related branches found
No related tags found
No related merge requests found
Basic procedures
================
* Compose fireworks and workflows
* Validate workflows
* View workflows
* Add fireworks to LaunchPad: lpad add
* Execution: rlaunch
* Monitoring: lpad get_*, lpad webgui
Basic procedures
================
Compose fireworks and workflows
-------------------------------
Fireworks and Workflows can be defined in three different general-purpose
languages: 1) Python 2) JSON and 3) YAML. There is no domain-specific language
for FireWorks and thus no specialized editor. Thus, a normal text editor is
sufficient and an IDE supporting editing and validation of these three languages
is "nice to have".
All workflow definitions in all exercises will be based on JSON and YAML.
Exercise 5 will introduce to writing a custom Firetask for which
Python will be used. Again, JSON and YAML will be used to define the workflows
using the custom Firetask. For each exercise there are one or more initial
examples in the ``exercises/demos`` folder. We recommend trying these examples
before starting solving the problems.
A firework consists of <describe the constituting parts: Firetask, Fireworks
and relevant attributes>
Add fireworks to LaunchPad
--------------------------
The LaunchPad is a database where the workflows are stored during their full
life cycle.
When used productively the LaunchPad contains many workflows in different states.
To distinguish between different workflows, the query commands can specify e.g.
the firework ID from the relevant workflow on the LaunchPad or perform a
mongo-like queries. To avoid the need apply filters to the queries, at the
beginning of each exercise in this tutorial we will clean up the LaunchPad from
previous fireworks with this command::
lpad reset
To add a workflow to the LaunchPad::
lpad add workflow.yaml
Alternatively in JSON format::
lpad add workflow.json
Validate workflows
------------------
Formal verification is done with adding a workflow to the LaunchPad. However,
missing links or data dependencies, and circular dependencies are not detected
at this stage and the errors appear at run time. To also check for such errors
the *-c* or *--check* flags can be used when adding the workflow to the
LaunchPad::
lpad add -c workflow.json
If a workflow has been added without check it can be check later with::
lpad check_wflow -i <firework ID>
Note: The correctness check is recommended for all exercises in the tutorial.
View workflows
--------------
Already added workflows can be converted into DOT format and viewed graphically
as PDF:
lpad check_wflow -i <firework ID> [--view_control_flow] [--view_data_flow] [-f <DOT_FILE>]
Execute workflows
-----------------
The workflow engine of FireWorks is called FireWorker. Multiple FireWorkers can be
runninng on different resources where idividual fireworks can be executed by the rocket
launcher ``rlaunch`` which has three modes of operation: *singleshot*, *rapidfire*
and *multi*.
To only execute one firework from the LaunchPad which is in *READY* state and
exit the following command is used::
rlaunch singleshot
To run all fireworks in *READY* state in a sequence::
rlaunch rapidfire
Note: Every firework changes its state to *READY* after all its dependency fireworks
are completed (state *COMPLETED*) and the states of linked fireworks are updated as soon as a firework
if completed. This means that any workflow will be continuous executed until there
are no fireworks in *READY* state.
To suppress verbose information on the screen the *-s* flag can be added::
rlaunch -s rapidfire
Monitor workflows
-----------------
To query workflows available on the LaunchPad use the command ``get_wflows``::
lpad get_wflows [-d <more|all>]
To query individual fireworks use the command::
lpad get_fws [-i <firework ID> [-d <more|all>]]
Note: The query from the command line is recommended for thi tutorial.
Alternatively the web GUI can be used::
lpad webgui
Note: Make sure that a web browser is configured in your terminal session and an
X server is running on your machine and it is configured and tunneled properly
in your terminal session.
\ No newline at end of file
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment