Mongomock support for fireworks not working

Related to Merge Request !205 (merged)

The error messages

When using texts in session mode the following error message is produced:

$> texts session -r
Error: Neither default Launchpad file configured nor custom Launchpad file is specified.

The error is also produced with texts script -m workflow -f tst_file.vm.

Manually providing a Launchpad file does not help:

$> texts session -r -l ~/.fireworks/mongomock.json
Traceback (most recent call last):
  File "/home/rodri/Programs/miniconda3/envs/vrel-mmock/lib/python3.9/site-packages/virtmat/language/utilities/errors.py", line 281, in wrapper
    return func(*args, **kwargs)
  [...]
  File "/home/rodri/Programs/miniconda3/envs/vrel-mmock/lib/python3.9/site-packages/fireworks/core/launchpad.py", line 284, in from_dict
    d["host"],
KeyError: 'host'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/rodri/Programs/miniconda3/envs/vrel-mmock/bin/texts", line 8, in <module>
    sys.exit(texts())
  [...]
  File "/home/rodri/Programs/miniconda3/envs/vrel-mmock/lib/python3.9/site-packages/virtmat/language/utilities/errors.py", line 307, in wrapper
    raise RuntimeError('non-handled exception') from err
RuntimeError: non-handled exception

TEST-1: Basic installation

The vre-language package was installed from an up-to-date, local copy of the master branch of the project's repository. Installation of the project was both tested in conda and normal python virtual environments. The installation command employed was python -m pip install .[mongomock].

Line 20 setup.cfg file was commented out:

20     #fireworks @ git+https://github.com/materialsproject/fireworks.git@main

Otherwise the following version conflict error messages are issued:

INFO: pip is looking at multiple versions of vre-language[mongomock] to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install None and vre-language[mongomock]==0.3.1 because these package versions have conflicting dependencies.

The conflict is caused by:
    vre-language 0.3.1 depends on fireworks 2.0.3 (from git+https://github.com/materialsproject/fireworks.git@main)
    vre-language[mongomock] 0.3.1 depends on fireworks 2.0.3 (from git+https://github.com/materialsproject/fireworks.git@main)
    vre-language[mongomock] 0.3.1 depends on fireworks 2.0.3 (from git+https://github.com/ikondov/fireworks.git@mongomock)

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

The fireworks package used for the [mongomock] version of the vre-language is that indicated by line 42 of setup.cfg:

42     fireworks @ git+https://github.com/ikondov/fireworks.git@mongomock

Inspection of the fireworks package in the environment's installation path (for example /home/python_VirtEnvs/vrel-br277mmock/lib/python3.9/site-packages/fireworks) shows that the installed package is consistent with the latest commits of the mongomock branch of the fireworks fork

This installation method produces the same errors in session and script-workflow modes:

$> texts session -r
Error: Neither default Launchpad file configured nor custom Launchpad file is specified.

TEST-2: Manually installing mongomock branch of fireworks branch

This process is a bit more involved, but ensures the correct branch of the fireworks fork is used.

Install vre-language with mongomock support in a clean python environment python -m pip install .
Remove the fireworks package python -m pip uninstall fireworks
Clone the forked fireworks repository with mongomock support git clone https://github.com/ikondov/fireworks.git
Go to the fireworks repo path and switch (checkout) to the mongomock branch git switch mongomock
Install the fireworks-mongomock fork python -m pip install .[mongomock]

This also produces the Launchpad errors in both session and script-workflow modes:

$> texts session -r
Error: Neither default Launchpad file configured nor custom Launchpad file is specified..`

TEST-3: custom branches of fireworks and mongomock

This third test builds on TEST-2. The difference is that the persistence branch of IK's mongomock fork is manually installed. In TEST-1 and TEST-2 the following were installed:

mongomock             4.1.2

This changes in TEST-3:

mongomock             0.0.1.dev1126

The setup for testing goes as follows:

Install vre-language with mongomock support in a clean python environment python -m pip install .
Remove the "default" fireworks package python -m pip uninstall fireworks
Clone the forked fireworks repository with mongomock support git clone https://github.com/ikondov/fireworks.git
Go to the fireworks repo path and switch (checkout) to the mongomock branch git switch mongomock
Install the fireworks-mongomock fork python -m pip install .[mongomock]
Remove the "default" mongomock package python -m pip uninstall mongomock
Clone the forked mongomock repository with persistence support git clone https://github.com/ikondov/mongomock.git
Go to the path of the cloned mongomock repo and switch to the persistence branch git switch persistence
Install mongomock-persistence package ``python -m pip install .`

Once more, the Launchpad file error is printed in either session or script+workflow modes:

$> texts session -r
Error: Neither default Launchpad file configured nor custom Launchpad file is specified.`

One additional question is whether the mongomock fork should be added to the setup.cfg file, similar to the fireworks-mongomock fork.

mongomock-fireworks interface works fine

After installation, regular fireworks opperations work as normal, for example lpad reset, lpad get_wflows, and lpad add -c sample_wflow.yaml If a workflow is added, the changes are reflected in the storage file ~/.fireworks/mongomock.json, proving that mongomock is indeed working with fireworks.

The problem is related to the layers on top of the mongomock-fireworks interface.

Edited Jun 13, 2024 by Rodrigo Cortes Mejia