Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
G
grk2450-fireworks-tutorial
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Requirements
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Locked files
Deploy
Releases
Package registry
Model registry
Operate
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
Repository analytics
Code review analytics
Issue analytics
Insights
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Ivan Kondov
grk2450-fireworks-tutorial
Commits
ed4801ee
Commit
ed4801ee
authored
7 years ago
by
Ivan Kondov
Browse files
Options
Downloads
Patches
Plain Diff
Update basic.rst
parent
5b4a17e7
No related branches found
Branches containing commit
No related tags found
No related merge requests found
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
docs/basic.rst
+0
-9
0 additions, 9 deletions
docs/basic.rst
docs/basics.rst
+122
-0
122 additions, 0 deletions
docs/basics.rst
with
122 additions
and
9 deletions
docs/basic.rst
deleted
100644 → 0
+
0
−
9
View file @
5b4a17e7
Basic procedures
================
* Compose fireworks and workflows
* Validate workflows
* View workflows
* Add fireworks to LaunchPad: lpad add
* Execution: rlaunch
* Monitoring: lpad get_*, lpad webgui
This diff is collapsed.
Click to expand it.
docs/basics.rst
0 → 100644
+
122
−
0
View file @
ed4801ee
Basic procedures
================
Compose fireworks and workflows
-------------------------------
Fireworks and Workflows can be defined in three different general-purpose
languages: 1) Python 2) JSON and 3) YAML. There is no domain-specific language
for FireWorks and thus no specialized editor. Thus, a normal text editor is
sufficient and an IDE supporting editing and validation of these three languages
is "nice to have".
All workflow definitions in all exercises will be based on JSON and YAML.
Exercise 5 will introduce to writing a custom Firetask for which
Python will be used. Again, JSON and YAML will be used to define the workflows
using the custom Firetask. For each exercise there are one or more initial
examples in the ``exercises/demos`` folder. We recommend trying these examples
before starting solving the problems.
A firework consists of <describe the constituting parts: Firetask, Fireworks
and relevant attributes>
Add fireworks to LaunchPad
--------------------------
The LaunchPad is a database where the workflows are stored during their full
life cycle.
When used productively the LaunchPad contains many workflows in different states.
To distinguish between different workflows, the query commands can specify e.g.
the firework ID from the relevant workflow on the LaunchPad or perform a
mongo-like queries. To avoid the need apply filters to the queries, at the
beginning of each exercise in this tutorial we will clean up the LaunchPad from
previous fireworks with this command::
lpad reset
To add a workflow to the LaunchPad::
lpad add workflow.yaml
Alternatively in JSON format::
lpad add workflow.json
Validate workflows
------------------
Formal verification is done with adding a workflow to the LaunchPad. However,
missing links or data dependencies, and circular dependencies are not detected
at this stage and the errors appear at run time. To also check for such errors
the *-c* or *--check* flags can be used when adding the workflow to the
LaunchPad::
lpad add -c workflow.json
If a workflow has been added without check it can be check later with::
lpad check_wflow -i <firework ID>
Note: The correctness check is recommended for all exercises in the tutorial.
View workflows
--------------
Already added workflows can be converted into DOT format and viewed graphically
as PDF:
lpad check_wflow -i <firework ID> [--view_control_flow] [--view_data_flow] [-f <DOT_FILE>]
Execute workflows
-----------------
The workflow engine of FireWorks is called FireWorker. Multiple FireWorkers can be
runninng on different resources where idividual fireworks can be executed by the rocket
launcher ``rlaunch`` which has three modes of operation: *singleshot*, *rapidfire*
and *multi*.
To only execute one firework from the LaunchPad which is in *READY* state and
exit the following command is used::
rlaunch singleshot
To run all fireworks in *READY* state in a sequence::
rlaunch rapidfire
Note: Every firework changes its state to *READY* after all its dependency fireworks
are completed (state *COMPLETED*) and the states of linked fireworks are updated as soon as a firework
if completed. This means that any workflow will be continuous executed until there
are no fireworks in *READY* state.
To suppress verbose information on the screen the *-s* flag can be added::
rlaunch -s rapidfire
Monitor workflows
-----------------
To query workflows available on the LaunchPad use the command ``get_wflows``::
lpad get_wflows [-d <more|all>]
To query individual fireworks use the command::
lpad get_fws [-i <firework ID> [-d <more|all>]]
Note: The query from the command line is recommended for thi tutorial.
Alternatively the web GUI can be used::
lpad webgui
Note: Make sure that a web browser is configured in your terminal session and an
X server is running on your machine and it is configured and tunneled properly
in your terminal session.
\ No newline at end of file
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment