Extend description of custom qadapters in documentation of tools
This issue stems from merge request !118 (merged), and issue #185 (closed). See this comment
Once a model has been added to the database, through workflow execution, it is possible to employ the interactive session, in asynchronous mode, to perform the evaluation of such model.
By default the session manager, with -a
option, will create a custom and unique wfengine object file within the user's ~/.fireworks/
directory.
This wfengine file contains threen main objects that are essential for the execution of a Fireworks' workflow:
-
launchpad
: provides data and credentials of the database hosting the workflows; -
qdapter
: provides the details about the computing resources required to run a job (firework) such as batch system type, ammount of memory, number of processors, nodes, queue type, etc. The qadapter should specify any pre- and post-run requirements (pre- and post-rocket), as well as the execution of the main task (rocket_launch); -
fworker
: specification about which computing resource (fireworker) can run what type of job. For example, jobs with a "batch" type label, can only be performed by a fireworker which has been given also such label.
Currently, automatic generation of the qadapter object relies mainly on interpreting resource annotations within the model scripts. It is also assumed that no further pre- nor- post-requisites should be satisfied. This is of course a problem for more complex tasks that would require, for example, loading of additional modules, or setting environment variables and paths within the computing node (all of these are platform dependent).
The qadapter can be an extremely complex object/file because of the many different custom parameters that can be included.
This is why fully automating the generation of the qadapter
is not practically feasible.
Unfortunately this is one thing that cannot be breached, yet, by the vre-language, and direct input from the user will be needed in the foreseeable future.
However, one way to make things more transparent is to provide some notes and examples about custom qadapters and ,especially, the pre_rocket
option in the tools documentation.
The addition should be done to this section because it already documents the arguments for the script and session executors, which inclulde a -q
(qadapter file) option.