Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • kit/virtmat-tools/vre-language
1 result
Show changes
Commits on Source (22)
Hide whitespace changes
Inline Side-by-side
Showing
with 370 additions and 138 deletions
docs/amml.md
View file @ 5a6bd887
...@@ -186,7 +186,7 @@ h2_fix0 = FixedAtoms where (fix: true, false) # fix first atom in H2 ...@@ -186,7 +186,7 @@ h2_fix0 = FixedAtoms where (fix: true, false) # fix first atom in H2
## Algorithm ## Algorithm
The AMML `Algorithm` parameter describes operations that are either available as algorithms in the [Atomic Simulation Environment (ASE)](https://wiki.fysik.dtu.dk/ase/) or are user-defined routines. For example, the `BFGS` structure optimization is an algorithm. Further available algorithms are: `BFGSLineSearch`, `LBFGS`, `LBFGSLineSearch`, `GPMin`, `MDMin`, `QuasiNewton`, `FIRE`, `VelocityVerlet`, `Langevin`, `Andersen`, `NVTBerendsen`, `NPTBerendsen`, `NPT`, `RDF`, `RMSD`, `EquationOfState`, `NEB`, `Dimer`. Not implemented: `BFGSClimbFixInternals`, `BasinHopping`, `MinimaHopping`, `GA`, `PourbaixDiagram`, `PhaseDiagram`, `Inertia`. The AMML `Algorithm` parameter describes operations that are either available as algorithms in the [Atomic Simulation Environment (ASE)](https://wiki.fysik.dtu.dk/ase/) or are user-defined routines. For example, the `BFGS` structure optimization is an algorithm. Further available algorithms are summarized in the table below.
The general syntax of `Algorithm` is: The general syntax of `Algorithm` is:
...@@ -289,8 +289,7 @@ As for the calculators, the numerical parameters of algorithms must have explici ...@@ -289,8 +289,7 @@ As for the calculators, the numerical parameters of algorithms must have explici
| | adjust | Boolean | true | --- | --- | | | adjust | Boolean | true | --- | --- |
| [RDF](https://wiki.fysik.dtu.dk/ase/ase/geometry.html#ase.geometry.analysis.Analysis.get_rdf) | rmax | Float | null [Å] | [length] | [Å] | | [RDF](https://wiki.fysik.dtu.dk/ase/ase/geometry.html#ase.geometry.analysis.Analysis.get_rdf) | rmax | Float | null [Å] | [length] | [Å] |
| | nbins | Integer | 40 | [dimensionless]| --- | | | nbins | Integer | 40 | [dimensionless]| --- |
| | neighborlist | NeighborList | null | --- | --- | | | neighborlist | Table | null | --- | --- |
| | neighborlist_pars | Table | () | --- | --- |
| | elements | String | null | --- | --- | | | elements | String | null | --- | --- |
| [VelocityDistribution](https://wiki.fysik.dtu.dk/ase/ase/md.html#velocity-distributions) | distribution | String | 'maxwell-boltzmann' | --- | --- | | [VelocityDistribution](https://wiki.fysik.dtu.dk/ase/ase/md.html#velocity-distributions) | distribution | String | 'maxwell-boltzmann' | --- | --- |
| | temperature_K | Float | --- | [temperature] | [K] | | | temperature_K | Float | --- | [temperature] | [K] |
...@@ -377,12 +376,40 @@ As for the calculators, the numerical parameters of algorithms must have explici ...@@ -377,12 +376,40 @@ As for the calculators, the numerical parameters of algorithms must have explici
| | fmax | Float | 0.05 [eV Å{sup}`-1`] | [length mass time{sup}`-2`] | [eV Å{sup}`-1`] | | | fmax | Float | 0.05 [eV Å{sup}`-1`] | [length mass time{sup}`-2`] | [eV Å{sup}`-1`] |
| | trajectory | Boolean | null | --- | --- | | | trajectory | Boolean | null | --- | --- |
| | logfile | Boolean | null | --- | --- | | | logfile | Boolean | null | --- | --- |
| | eigenmode_logfile | String | null | --- | --- |
| | eigenmode_method | String | 'dimer' | --- | --- |
| | f_rot_min | Float | 0.1 [eV Å{sup}`-1`] | [length mass time{sup}`-2`] | [eV Å{sup}`-1`] |
| | f_rot_max | Float | 1.0 [eV Å{sup}`-1`] | [length mass time{sup}`-2`] | [eV Å{sup}`-1`] |
| | max_num_rot | Integer | 1 | [dimensionless] | --- |
| | trial_angle | Float | 0.25 $\pi$ [radians] | [angle] | [radians] |
| | trial_trans_step | Float | 0.001 [Å] | [length] | [Å] |
| | maximum_translation | Float | 0.1 [Å] | [length] | [Å] |
| | cg_translation | Boolean | true | --- | --- |
| | use_central_forces | Boolean | true | --- | --- |
| | dimer_separation | Float | 0.0001 [Å] | [length] | [Å] |
| | initial_eigenmode_method | String | 'gauss' | --- | --- |
| | extrapolate_forces | Boolean | False | --- | --- |
| | displacement_method | String | 'gauss' | --- | --- |
| | gauss_std | String | 0.1 [Å] | [length] | [Å] |
| | order | Integer | 1 | [dimensionless] | --- |
| | mask | BoolArray | null | --- | --- |
| | displacement_center | FloatArray | null | [length] | [Å] |
| | displacement_radius | Float | null | [length] | [Å] |
| | number_of_displacement_atoms | Integer | null | [dimensionless] | --- |
| [Vibrations](https://wiki.fysik.dtu.dk/ase/ase/vibrations/modes.html) | delta | Float | 0.01 [Å] | [lenght] | [Å] | | [Vibrations](https://wiki.fysik.dtu.dk/ase/ase/vibrations/modes.html) | delta | Float | 0.01 [Å] | [lenght] | [Å] |
| | nfree | Integer | 2 | [dimensionless] | --- | | | nfree | Integer | 2 | [dimensionless] | --- |
| | method | String | 'standard' | --- | --- | | | method | String | 'standard' | --- | --- |
| | direction | String | 'central' | --- | --- | | | direction | String | 'central' | --- | --- |
| | all_atoms | Boolean | false | --- | --- | | | all_atoms | Boolean | false | --- | --- |
| | imag_tol | Float | 1e-5 [eV] | [length{sup}`2` mass time{sup}`-2`] | [eV] | | | imag_tol | Float | 1e-5 [eV] | [length{sup}`2` mass time{sup}`-2`] | [eV] |
| [NeighborList](https://wiki.fysik.dtu.dk/ase/ase/neighborlist.html) | cutoffs | FloatArray | null | [length] | [Å] |
| | skin | Float | 0.3 [Å] | [length] | [Å] |
| | sorted | Boolean | false | --- | ---|
| | self_interaction | Boolean | true | --- | ---|
| | bothways | Boolean | false | --- | ---|
| | method | String | 'quadratic' | --- | ---|
**NOTE:** If the default is `---` then the parameter must be specified in the input. If the default is `null` then the parameter is not required in the input and will be set automatically by the algorithm.
**NOTE:** If `trajectory` / `logfile` are set to `true` a trajectory file / logfile will be created. If `logfile` is set to `false` then the step logging will be written on the screen. By default, `logfile` is set to `null`, i.e. step logging is turned off completely. **NOTE:** If `trajectory` / `logfile` are set to `true` a trajectory file / logfile will be created. If `logfile` is set to `false` then the step logging will be written on the screen. By default, `logfile` is set to `null`, i.e. step logging is turned off completely.
...@@ -486,6 +513,16 @@ The `Property` parameter is evaluated in the following way. The input [`Structur ...@@ -486,6 +513,16 @@ The `Property` parameter is evaluated in the following way. The input [`Structur
For example, if two geometries of the same molecule have to be processed by the same calculator with three different parameter sets, then the `results` Table will contain six tuples. The computed constraints and the properties are the same for all structure-calculator-algorithm combinations. The results can be accessed via subscripting the `Property` parameter, as it is shown in the examples above. For example, if two geometries of the same molecule have to be processed by the same calculator with three different parameter sets, then the `results` Table will contain six tuples. The computed constraints and the properties are the same for all structure-calculator-algorithm combinations. The results can be accessed via subscripting the `Property` parameter, as it is shown in the examples above.
### Computational resources for Property
Very often `Property` parameters require multiple processor cores and multiple computing nodes and long computing times (up to several days) to evaluate. In only a few exceptional cases, e.g. computing small molecules or periodic cells with a few atoms, one can evaluate a property in an [interactive](resources.md#policies-for-interactive-and-batch-execution) statement. To enable [batch](resources.md#policies-for-interactive-and-batch-execution) evaluation one has to add resource requirements at the end of the statement as introduced [here](resources.md#resource-annotation).
### Storage resources for Property
In many use cases, the evaluation of `Property` parameters yields large output data. This data does not fit into a single document in the database. Therefore a mechanism is in place that stores data with excessive volumes in a separate datastore. Typical use cases where the mechanism gets activated are storing trajectories from molecular dynamics and Monte Carlo simulations, and charge densities from density functional calculations. See [this section](io.md#background-io-operations) to learn more about this mechanism and how to customize it, i.e. to change the threshold volume, type of store etc.
## Using AMML parameters as table-like iterables ## Using AMML parameters as table-like iterables
As seen above, some AMML parameters behave like iterables. All operations defined for the [Table](scl.md#table) type can be applied to the following AMML parameters: `Structure`, `Calculator`, `Algorithm` and `Property`. These operations include, e.g. subscripting, slicing, filter function and expression, map and reduce. These operations base on a table attribute in each of these parameters: As seen above, some AMML parameters behave like iterables. All operations defined for the [Table](scl.md#table) type can be applied to the following AMML parameters: `Structure`, `Calculator`, `Algorithm` and `Property`. These operations include, e.g. subscripting, slicing, filter function and expression, map and reduce. These operations base on a table attribute in each of these parameters:
......
docs/io.md
View file @ 5a6bd887
...@@ -65,34 +65,32 @@ The workflow executor interprets the model and stores the model instance as a wo ...@@ -65,34 +65,32 @@ The workflow executor interprets the model and stores the model instance as a wo
## Background I/O operations ## Background I/O operations
In the workflow executor, sometimes a computed parameter value allocates too much memory and cannot be stored into the node / launch documents on the database. Then a fully transparent mechanism is automatically triggered to store the value in a file. To this end, a threshold for the maximum size in memory is set; the default is 1 million bytes. In addition, one can set different store types (local file system or GridFS in MongoDB), different file formats (JSON is the only implemented currently), and data compression. Obviously, all these settings have effect on performance and storage volumes but do not change the results or any other behavior. In the workflow executor, sometimes a computed parameter value allocates too much memory and cannot be stored into the node / launch documents on the database. Then a fully transparent mechanism is automatically triggered to store the value in a file. To this end, a threshold for the maximum size in memory is set; the default is 100,000 bytes. In addition, one can set different store types (local file system or GridFS in MongoDB), different file formats (JSON is the only one implemented currently), and data compression. Obviously, all these settings have effect on performance and storage volumes but do not change the results or any other behavior.
Typical use cases where the mechanism gets activated are storing trajectories from molecular dynamics and Monte Carlo simulations, and charge densities from density functional calculations. If you wish to change these settings you can create a custom datastore configuration file with these contents:
If you wish to change these settings you should create a datastore configuration file, e.g. `/path/to/datastore_config.yaml`, with these contents:
```yaml ```yaml
inline-threshold: 1000000 # threshold for offloading data inline-threshold: 100000 # threshold for offloading data, in bytes
type: gridfs # can be 'file' for local filesystem type: file # can be 'gridfs' for database file object storage
path: /path/to/local/workspace # directory used if type is 'file'
name: vre_language_datastore # collection name used if type is 'gridfs' name: vre_language_datastore # collection name used if type is 'gridfs'
launchpad: /path/to/launchpad.yaml # path to custom launchpad file used if type is 'gridfs' launchpad: /path/to/launchpad.yaml # path to custom launchpad file used if type is 'gridfs'
path: /path/to/local/workspace # directory used if type is 'file'
format: json # 'yaml' and 'hdf5' not implemented format: json # 'yaml' and 'hdf5' not implemented
compress: true # use compression compress: true # use compression
``` ```
To activate your custom datastore configuration you must set the environment variable The type `file` triggers storage in local files in `path`. The default `path` is `$HOME/.fireworks/vre-language-datastore`. The `path` will be created automatically if it does not exist. The default `launchpad` is `LAUNCHPAD_LOC` as provided by FireWorks. All other default settings are shown in the example above.
The default path of the datastore configuration file is `$HOME/.fireworks/datastore_config.yaml`. It will be automatically loaded, if the file exists. If your datastore configuration has a different location then you must set the environment variable
```bash ```bash
export DATASTORE_CONFIG=/path/to/datastore_config.yaml export DATASTORE_CONFIG=/path/to/datastore_config.yaml
``` ```
before running the `texts` tool. before running the `texts` tool. If the variable `$DATASTORE_CONFIG` should be used then it has to also be added to the `default_envvars` list in the relevant worker as described (here)[https://vre-middleware.readthedocs.io/en/latest/resconfig.html#configuration-of-environment-variables].
It is recommended to specify only these parameters that you want to change. The default `path` is `$HOME/.fireworks/vre-language-datastore`. The path will be created automatically if it does not exist. The default `launchpad` is `LAUNCHPAD_LOC` as provided by FireWorks. All other default settings are shown in the example above.
It is not recommended to set `inline-threshold` to very small values, unless you are testing. It is not recommended to set `inline-threshold` to very small values, unless you are testing.
If your model has a large number of parameters with moderate storage requirements it is recommended to use `gridfs` (default) and adapt `inline-threshold` as needed. If your model has a small number of parameters with large storage requirements then it is recommended to use a workspace in the local file system (set `type: file`). If your model has a large number of parameters with moderate storage requirements it is recommended to use `type: gridfs` and adapt `inline-threshold` as needed. If your model has a small number of parameters with large storage requirements then it is recommended to use a workspace in the local file system (`type: file`).
The datastore metadata, i.e. `type`, `name`, `launchpad`, `path`, `format` and `compress` are stored permanently per individual object and not for all objects in a model. Change of datastore configuration is possible at any time but effective only for the objects that are created as this configuration is active. The external resources, i.e. the file paths, databases, collection names, etc., used for the storage should be available permanently to ensure that the model data can be used later. The datastore metadata, i.e. `type`, `name`, `launchpad`, `path`, `format` and `compress` are stored permanently per individual object and not for all objects in a model. Change of datastore configuration is possible at any time but effective only for the objects that are created as this configuration is active. The external resources, i.e. the file paths, databases, collection names, etc., used for the storage should be available permanently to ensure that the model data can be used later.
\ No newline at end of file
examples/neighborlist.vm 0 → 100644
View file @ 5a6bd887
"""use the neighborlist algorithm to calculate connectivity matrix"""
h2o = Structure water (
(atoms: ((symbols: 'O', 'H', 'H'),
(x: 0., 0., 0.) [nm],
(y: 0., 0.763239, -0.763239) [angstrom],
(z: 0.119262, -0.477047, -0.477047) [angstrom]
),
((symbols: 'O', 'H', 'H'),
(x: 0., 0., 0.) [nm],
(y: 0., 0.763239, -0.763239) [angstrom],
(z: 0.119262, -0.477047, -0.477047) [angstrom]
)
),
(cell: [4., 4., 4.] [angstrom], [4., 4., 4.][angstrom]),
(pbc: [true, true, true], [true, true, true])
)
algo_nl = Algorithm NeighborList ((self_interaction: false), (bothways: true),
(method: 'linear'), (sorted: true))
prop_nl = Property neighbors,
neighbor_offsets,
connectivity_matrix,
connected_components ((algorithm: algo_nl), (structure: h2o))
print(algo_nl)
print(prop_nl.neighbors)
print(prop_nl.neighbor_offsets)
print(prop_nl.connectivity_matrix)
print(prop_nl.connected_components)
examples/rdf_with_neighborlist.vm 0 → 100644
View file @ 5a6bd887
"""use radial distribution function with custom neighbor list parameters"""
h2o = Structure water (
(atoms: ((symbols: 'O', 'H', 'H'),
(x: 0., 0., 0.) [nm],
(y: 0., 0.763239, -0.763239) [angstrom],
(z: 0.119262, -0.477047, -0.477047) [angstrom]
),
((symbols: 'O', 'H', 'H'),
(x: 0., 0., 0.) [nm],
(y: 0., 0.763239, -0.763239) [angstrom],
(z: 0.119262, -0.477047, -0.477047) [angstrom]
)
),
(cell: [4., 4., 4.] [angstrom], [4., 4., 4.][angstrom]),
(pbc: [true, true, true], [true, true, true])
)
"""
algo_rdf = Algorithm RDF ((rmax: 1.7) [angstrom], (nbins: 5),
(neighborlist: ((name: 'NeighborList'),
(parameters: ((self_interaction: false),
(bothways: true))))))
"""
algo_rdf = Algorithm RDF, many_to_one ((nbins: 2),
(neighborlist: ((name: 'NeighborList'),
(parameters: ((self_interaction: false),
(bothways: true))))))
print(algo_rdf)
prop_rdf = Property rdf, rdf_distance ((structure: h2o), (algorithm: algo_rdf))
print(prop_rdf)
print(prop_rdf.rdf)
print(prop_rdf.rdf_distance)
requirements.txt
View file @ 5a6bd887
textx textx
numpy numpy
scipy
pandas >=2.2.0 pandas >=2.2.0
pint >=0.24.3 pint >=0.24.3
pint-pandas ==0.6.2 pint-pandas ==0.6.2
......
setup.cfg
View file @ 5a6bd887
...@@ -3,7 +3,7 @@ name = vre-language ...@@ -3,7 +3,7 @@ name = vre-language
version = 0.5.6 version = 0.5.6
author = The Virtmat Tools Team author = The Virtmat Tools Team
author_email = virtmat-tools@lists.kit.edu author_email = virtmat-tools@lists.kit.edu
description = VRE Language description = Domain-specific languages and supporting tools for scientific computing
long_description = file: README.md long_description = file: README.md
long_description_content_type = text/markdown long_description_content_type = text/markdown
keywords = domain-specific language, scientific computing, workflow, Jupyter keywords = domain-specific language, scientific computing, workflow, Jupyter
...@@ -26,6 +26,15 @@ classifiers = ...@@ -26,6 +26,15 @@ classifiers =
Topic :: Software Development :: Interpreters Topic :: Software Development :: Interpreters
Topic :: Software Development :: User Interfaces Topic :: Software Development :: User Interfaces
Topic :: Other/Nonlisted Topic Topic :: Other/Nonlisted Topic
project_urls =
Homepage = https://gitlab.kit.edu/kit/virtmat-tools/vre-language
Documentation = https://vre-language.readthedocs.io
Download = https://pypi.org/project/vre-language/#files
Source Code = https://gitlab.kit.edu/kit/virtmat-tools/vre-language
Bug Reports = https://gitlab.kit.edu/kit/virtmat-tools/vre-language/-/issues
Changelog = https://gitlab.kit.edu/kit/virtmat-tools/vre-language/-/blob/master/CHANGELOG.md
Credits = https://gitlab.kit.edu/kit/virtmat-tools/vre-language/-/blob/master/CREDITS.md
Contributing = https://gitlab.kit.edu/kit/virtmat-tools/vre-language/-/blob/master/CONTRIBUTING.md
[options] [options]
zip_safe = false zip_safe = false
...@@ -36,6 +45,7 @@ python_requires = >=3.9 ...@@ -36,6 +45,7 @@ python_requires = >=3.9
install_requires = install_requires =
textx textx
numpy numpy
scipy
pandas >=2.2.0 pandas >=2.2.0
pint >=0.24.3 pint >=0.24.3
pint-pandas ==0.7.1 pint-pandas ==0.7.1
......
src/virtmat/language/grammar/virtmat.tx
View file @ 5a6bd887
/* grammar for virtmat modeling language */ /* grammar for virtmat modeling language */
/* grammar version 30 */ /* grammar version 31 */
import imports import imports
import identifiers import identifiers
...@@ -479,8 +479,7 @@ Comment[noskipws]: ...@@ -479,8 +479,7 @@ Comment[noskipws]:
/* atomic and molecular modeling language */ /* atomic and molecular modeling language */
AMMLObject: AMMLObject:
AMMLStructure | AMMLAlgorithm | AMMLCalculator | AMMLProperty | AMMLConstraint | AMMLStructure | AMMLAlgorithm | AMMLCalculator | AMMLProperty | AMMLConstraint
AMMLNeighborList
; ;
AMMLStructure: AMMLStructure:
...@@ -512,7 +511,7 @@ AMMLAlgorithmName: ...@@ -512,7 +511,7 @@ AMMLAlgorithmName:
'Langevin' | 'Andersen' | 'NVTBerendsen' | 'NPTBerendsen' | 'NPT' | 'NEB' | 'Langevin' | 'Andersen' | 'NVTBerendsen' | 'NPTBerendsen' | 'NPT' | 'NEB' |
'Dimer' | 'BasinHopping' | 'MinimaHopping' | 'GA' | 'PourbaixDiagram' | 'Dimer' | 'BasinHopping' | 'MinimaHopping' | 'GA' | 'PourbaixDiagram' |
'PhaseDiagram' | 'RDF' | 'RMSD' | 'EquationOfState' | 'DensityOfStates' | 'PhaseDiagram' | 'RDF' | 'RMSD' | 'EquationOfState' | 'DensityOfStates' |
'BandStructure' | 'Vibrations' | 'VelocityDistribution' 'BandStructure' | 'Vibrations' | 'VelocityDistribution' | 'NeighborList'
; ;
AMMLAlgorithm: AMMLAlgorithm:
...@@ -526,7 +525,8 @@ AMMLPropertyName: ...@@ -526,7 +525,8 @@ AMMLPropertyName:
'rmsd' | 'rdf_distance' | 'rdf' | 'trajectory' | 'transition_state' | 'rmsd' | 'rdf_distance' | 'rdf' | 'trajectory' | 'transition_state' |
'minimum_energy' | 'optimal_volume' | 'bulk_modulus' | 'eos_volume' | 'eos_energy' | 'minimum_energy' | 'optimal_volume' | 'bulk_modulus' | 'eos_volume' | 'eos_energy' |
'dos_energy' | 'dos' | 'band_structure' | 'activation_energy' | 'reaction_energy' | 'dos_energy' | 'dos' | 'band_structure' | 'activation_energy' | 'reaction_energy' |
'maximum_force' | 'velocity' | 'vdf' 'maximum_force' | 'velocity' | 'vdf' | 'neighbors' | 'neighbor_offsets' |
'connectivity_matrix' | 'connected_components'
; ;
AMMLProperty: AMMLProperty:
...@@ -545,11 +545,6 @@ AMMLConstraint: ...@@ -545,11 +545,6 @@ AMMLConstraint:
( name = 'FixedPlane' 'normal to' direction = IntArray 'where' fixed = IterableParameter ) ( name = 'FixedPlane' 'normal to' direction = IntArray 'where' fixed = IterableParameter )
; ;
AMMLNeighborList:
( 'NeighborList' 'from' tab = Table ) |
( 'NeighborList' 'from' ( 'url' url = STRING | 'file' filename = STRING ) )
;
/* chemistry extensions */ /* chemistry extensions */
ChemObject: ChemObject:
......
src/virtmat/language/interpreter/session_manager.py
View file @ 5a6bd887
...@@ -101,6 +101,10 @@ def expand_query_prefix(query): ...@@ -101,6 +101,10 @@ def expand_query_prefix(query):
class SessionManager(InteractiveConsole): class SessionManager(InteractiveConsole):
"""session manager for basic interactive work using a text terminal""" """session manager for basic interactive work using a text terminal"""
options = ['%exit', '%bye', '%close', '%quit', '%start', '%stop', '%new',
'%vary', '%history', '%hist', '%tag', '%uuid', '%sleep',
'%rerun', '%find', '%help', 'use', 'print', 'view', 'vary', 'tag']
def __init__(self, lpad, **kwargs): def __init__(self, lpad, **kwargs):
super().__init__() super().__init__()
self.lpad = lpad self.lpad = lpad
...@@ -119,12 +123,9 @@ class SessionManager(InteractiveConsole): ...@@ -119,12 +123,9 @@ class SessionManager(InteractiveConsole):
obj_processors = {'Null': null_processor, 'Number': number_processor, obj_processors = {'Null': null_processor, 'Number': number_processor,
'Table': table_processor} 'Table': table_processor}
self.metamodel.register_obj_processors(obj_processors) self.metamodel.register_obj_processors(obj_processors)
options = ['%exit', '%bye', '%close', '%quit', '%start', '%stop', '%new',
'%vary', '%history', '%hist', '%tag', '%uuid', '%sleep',
'%rerun', '%find', 'use', 'print', 'view', 'vary', 'tag']
ids = [i.name for i in get_identifiers(self.session.model)]
model_grammar = GrammarString(self.grammar_path).string model_grammar = GrammarString(self.grammar_path).string
self.completer = TextXCompleter(model_grammar, options, ids, console=self) self.completer = TextXCompleter(model_grammar, self.options, console=self)
self._set_completer_ids()
readline.set_completer(self.completer.complete) readline.set_completer(self.completer.complete)
readline.set_completer_delims('') readline.set_completer_delims('')
readline.parse_and_bind('tab: complete') readline.parse_and_bind('tab: complete')
...@@ -158,6 +159,10 @@ class SessionManager(InteractiveConsole): ...@@ -158,6 +159,10 @@ class SessionManager(InteractiveConsole):
return need_inp return need_inp
return False return False
def _set_completer_ids(self):
"""set completer IDs, such as variables, imports, ..."""
self.completer.ids = [i.name for i in get_identifiers(self.session.model)]
@error_handler @error_handler
def get_model_value(self, *args, **kwargs): def get_model_value(self, *args, **kwargs):
"""wrapped and evaluated version of get_model() of the Session class""" """wrapped and evaluated version of get_model() of the Session class"""
...@@ -190,7 +195,7 @@ class SessionManager(InteractiveConsole): ...@@ -190,7 +195,7 @@ class SessionManager(InteractiveConsole):
output = self.get_model_value(model_str=input_str) output = self.get_model_value(model_str=input_str)
if output: if output:
print('Output >', output) print('Output >', output)
self.completer.ids = [i.name for i in get_identifiers(self.session.model)] self._set_completer_ids()
return False return False
return True return True
...@@ -218,6 +223,7 @@ class SessionManager(InteractiveConsole): ...@@ -218,6 +223,7 @@ class SessionManager(InteractiveConsole):
self.uuid = self.session.uuids[0] self.uuid = self.session.uuids[0]
if start_thread: if start_thread:
self.session.start_runner() self.session.start_runner()
self._set_completer_ids()
print(f'Started new session with uuids {formatter(self.session.uuids)}') print(f'Started new session with uuids {formatter(self.session.uuids)}')
elif model.com == 'uuid': elif model.com == 'uuid':
if model.arg is None: if model.arg is None:
...@@ -308,3 +314,4 @@ class SessionManager(InteractiveConsole): ...@@ -308,3 +314,4 @@ class SessionManager(InteractiveConsole):
self.uuid = new_uuid self.uuid = new_uuid
if start_thread: if start_thread:
self.session.start_runner() self.session.start_runner()
self._set_completer_ids()
src/virtmat/language/interpreter/workflow_executor.py
View file @ 5a6bd887
...@@ -11,6 +11,7 @@ graph analysis. ...@@ -11,6 +11,7 @@ graph analysis.
import base64 import base64
import traceback import traceback
import uuid import uuid
import sys
from functools import cached_property from functools import cached_property
import dill import dill
import pandas import pandas
...@@ -42,17 +43,10 @@ from .instant_executor import program_value, plain_type_value ...@@ -42,17 +43,10 @@ from .instant_executor import program_value, plain_type_value
def get_input_name(par): def get_input_name(par):
"""return the input name and input value of a named object""" """return the input name and input value of a named object"""
inp_value = None if isinstance_m(par, ['Variable']):
if isinstance_m(par, ['ObjectImport']): return par.name, None
inp_name = par.name assert isinstance_m(par, ['ObjectImport'])
inp_value = FWDataObject.from_obj(par.value) return par.name, FWDataObject.from_obj(par.value)
elif isinstance_m(par, ['Variable']):
inp_name = par.name
else:
msg = 'parameter must be variable or imported object'
get_logger(__name__).error('get_input_name: error: %s', msg)
raise RuntimeError(msg)
return inp_name, inp_value
def get_par_value(par): def get_par_value(par):
...@@ -69,13 +63,7 @@ def get_par_value(par): ...@@ -69,13 +63,7 @@ def get_par_value(par):
def get_fstr(func): def get_fstr(func):
"""Return a pickle-serialized function as a Python3 string""" """Return a pickle-serialized function as a Python3 string"""
try: return base64.b64encode(dill.dumps(func)).decode('utf-8')
f_str = base64.b64encode(dill.dumps(func)).decode('utf-8')
except dill.PicklingError:
get_logger(__name__).error('get_fstr: pickling error')
raise
get_logger(__name__).debug('get_fstr: normal run')
return f_str
def get_fws(self): def get_fws(self):
...@@ -97,6 +85,7 @@ def get_fws(self): ...@@ -97,6 +85,7 @@ def get_fws(self):
spec_rsc['_source_code'] = self.source_code spec_rsc['_source_code'] = self.source_code
spec_rsc['_grammar_version'] = get_model(self).grammar_version spec_rsc['_grammar_version'] = get_model(self).grammar_version
spec_rsc['_data_schema_version'] = DATA_SCHEMA_VERSION spec_rsc['_data_schema_version'] = DATA_SCHEMA_VERSION
spec_rsc['_python_version'] = sys.version
if self.resources is None: if self.resources is None:
spec_rsc['_category'] = 'interactive' spec_rsc['_category'] = 'interactive'
spec_rsc['_fworker'] = get_model(self).worker_name spec_rsc['_fworker'] = get_model(self).worker_name
...@@ -129,6 +118,7 @@ def _get_fws_parallel(self, inputs, spec_rsc): ...@@ -129,6 +118,7 @@ def _get_fws_parallel(self, inputs, spec_rsc):
spec['_source_code'] = spec_rsc['_source_code'] spec['_source_code'] = spec_rsc['_source_code']
spec['_grammar_version'] = get_model(self).grammar_version spec['_grammar_version'] = get_model(self).grammar_version
spec['_data_schema_version'] = DATA_SCHEMA_VERSION spec['_data_schema_version'] = DATA_SCHEMA_VERSION
spec['_python_version'] = sys.version
fw1 = Firework([tsk], spec=spec, name=uuid.uuid4().hex) fw1 = Firework([tsk], spec=spec, name=uuid.uuid4().hex)
if isinstance_m(self.parameter, ('Map', 'Filter')): if isinstance_m(self.parameter, ('Map', 'Filter')):
fstr = get_fstr(lambda *x: pandas.concat(x)) fstr = get_fstr(lambda *x: pandas.concat(x))
...@@ -151,7 +141,8 @@ def get_fw_object_to(self): ...@@ -151,7 +141,8 @@ def get_fw_object_to(self):
spec = {'_source_code': self.source_code, '_category': 'interactive', spec = {'_source_code': self.source_code, '_category': 'interactive',
'_fworker': get_model(self).worker_name, '_fworker': get_model(self).worker_name,
'_grammar_version': get_model(self).grammar_version, '_grammar_version': get_model(self).grammar_version,
'_data_schema_version': DATA_SCHEMA_VERSION} '_data_schema_version': DATA_SCHEMA_VERSION,
'_python_version': sys.version}
tsk = ExportDataTask(varname=self.ref.name, filename=self.filename, url=self.url) tsk = ExportDataTask(varname=self.ref.name, filename=self.filename, url=self.url)
self.__fw_name = uuid.uuid4().hex self.__fw_name = uuid.uuid4().hex
return Firework([tsk], spec=spec, name=self.__fw_name) return Firework([tsk], spec=spec, name=self.__fw_name)
...@@ -202,8 +193,9 @@ def get_wf(self): ...@@ -202,8 +193,9 @@ def get_wf(self):
for statement in ['ObjectImports', 'FunctionDefinition']: for statement in ['ObjectImports', 'FunctionDefinition']:
for obj in get_children_of_type(statement, self): for obj in get_children_of_type(statement, self):
meta_src.extend(obj.source_code) meta_src.extend(obj.source_code)
root_spec = {'_category': 'interactive', '_fworker': self.worker_name} root_spec = {'_python_version': sys.version,
meta_spec = {'_source_code': meta_src, '_category': 'interactive', '_fworker': self.worker_name}
meta_spec = {'_python_version': sys.version, '_source_code': meta_src,
'_grammar_version': get_model(self).grammar_version, '_grammar_version': get_model(self).grammar_version,
'_data_schema_version': DATA_SCHEMA_VERSION} '_data_schema_version': DATA_SCHEMA_VERSION}
meta_spec.update(root_spec) meta_spec.update(root_spec)
...@@ -249,15 +241,15 @@ def variable_value(self): ...@@ -249,15 +241,15 @@ def variable_value(self):
logger.error('variable_value:%s evaluation error', repr(self)) logger.error('variable_value:%s evaluation error', repr(self))
exception_dct = launch_dct['action']['stored_data']['_exception'] exception_dct = launch_dct['action']['stored_data']['_exception']
trace = exception_dct['_stacktrace'] trace = exception_dct['_stacktrace']
if exception_dct.get('_details') is None: if exception_dct.get('_details') is None: # not covered
raise EvaluationError(f'No details found. Stacktrace:\n{trace}') raise EvaluationError(f'No details found. Stacktrace:\n{trace}')
pkl = exception_dct['_details']['pkl'] pkl = exception_dct['_details']['pkl']
exc = dill.loads(base64.b64decode(pkl.encode())) exc = dill.loads(base64.b64decode(pkl.encode()))
if isinstance(exc, TEXTX_WRAPPED_EXCEPTIONS): if isinstance(exc, TEXTX_WRAPPED_EXCEPTIONS):
raise exc raise exc
lst = traceback.format_exception(type(exc), exc, exc.__traceback__) lst = traceback.format_exception(type(exc), exc, exc.__traceback__) # not covered
raise EvaluationError(''.join(lst)) from exc raise EvaluationError(''.join(lst)) from exc
raise EvaluationError('state FIZZLED but no exception found') raise EvaluationError('state FIZZLED but no exception found') # not covered
if fw_dct['state'] == 'WAITING': if fw_dct['state'] == 'WAITING':
parents = get_ancestors(model.lpad, fw_dct['fw_id']) parents = get_ancestors(model.lpad, fw_dct['fw_id'])
fw_q = {'fw_id': {'$in': parents}, 'state': 'FIZZLED'} fw_q = {'fw_id': {'$in': parents}, 'state': 'FIZZLED'}
...@@ -278,8 +270,8 @@ def variable_value(self): ...@@ -278,8 +270,8 @@ def variable_value(self):
raise NonCompletedException raise NonCompletedException
if fw_dct['state'] in ['READY', 'RESERVED', 'RUNNING']: if fw_dct['state'] in ['READY', 'RESERVED', 'RUNNING']:
raise NonCompletedException raise NonCompletedException
logger.critical('variable_value:%s unknown node state', repr(self)) logger.critical('variable_value:%s unknown node state', repr(self)) # not covered
raise RuntimeError('Unknown node state') raise RuntimeError('Unknown node state') # not covered
@error_handler @error_handler
...@@ -324,7 +316,7 @@ def type_value(self): ...@@ -324,7 +316,7 @@ def type_value(self):
var = next(v for v in var_list if v.name == par.ref.name) var = next(v for v in var_list if v.name == par.ref.name)
proj_keys = ['fw_id', 'name', 'state', 'created_on', 'updated_on', proj_keys = ['fw_id', 'name', 'state', 'created_on', 'updated_on',
'launches', 'archived_launches', 'spec._category', 'launches', 'archived_launches', 'spec._category',
'spec._worker', 'spec._grammar_version', 'spec._worker', 'spec._grammar_version', 'spec._python_version',
'spec._data_schema_version', 'spec._dupefinder'] 'spec._data_schema_version', 'spec._dupefinder']
wfs = get_nodes_info(model.lpad, {'metadata.uuid': model.uuid}, wfs = get_nodes_info(model.lpad, {'metadata.uuid': model.uuid},
{'name': var.__fw_name}, {k: True for k in proj_keys}, {'name': var.__fw_name}, {k: True for k in proj_keys},
...@@ -346,8 +338,9 @@ def type_value(self): ...@@ -346,8 +338,9 @@ def type_value(self):
'node state': node['state'], 'node state': node['state'],
'created on': get_iso_datetime(node['created_on']), 'created on': get_iso_datetime(node['created_on']),
'updated on': get_iso_datetime(node['updated_on']), 'updated on': get_iso_datetime(node['updated_on']),
'grammar_version': node['spec'].get('_grammar_version'), 'grammar version': node['spec'].get('_grammar_version'),
'data_schema_version': node['spec'].get('_data_schema_version'), 'data schema version': node['spec'].get('_data_schema_version'),
'python version': node['spec'].get('_python_version'),
'category': node['spec'].get('_category'), 'category': node['spec'].get('_category'),
'fworker': node['spec'].get('_worker'), 'fworker': node['spec'].get('_worker'),
'dupefinder': bool(node['spec'].get('_dupefinder')), 'dupefinder': bool(node['spec'].get('_dupefinder')),
...@@ -383,7 +376,7 @@ def get_g_uuid(self): ...@@ -383,7 +376,7 @@ def get_g_uuid(self):
"""return group uuid of the model""" """return group uuid of the model"""
muuid = self._tx_model_params['model_instance']['uuid'] muuid = self._tx_model_params['model_instance']['uuid']
g_uuid = self._tx_model_params['model_instance'].get('g_uuid') g_uuid = self._tx_model_params['model_instance'].get('g_uuid')
if muuid is not None: if muuid is not None: # not covered
wf_q = {'metadata.uuid': self.uuid} wf_q = {'metadata.uuid': self.uuid}
wfl = self.lpad.workflows.find_one(wf_q, projection={'metadata': True}) wfl = self.lpad.workflows.find_one(wf_q, projection={'metadata': True})
assert wfl is not None assert wfl is not None
...@@ -531,7 +524,7 @@ def append_output_nodes(model): ...@@ -531,7 +524,7 @@ def append_output_nodes(model):
if nodes: if nodes:
assert len(nodes) == 1 assert len(nodes) == 1
obj_to.__fw_name = next(n['name'] for n in nodes) obj_to.__fw_name = next(n['name'] for n in nodes)
else: else: # not covered
parents = get_nodes_providing(model.lpad, model.uuid, obj_to.ref.name) parents = get_nodes_providing(model.lpad, model.uuid, obj_to.ref.name)
model.lpad.append_wf(Workflow([obj_to.firework]), fw_ids=parents) model.lpad.append_wf(Workflow([obj_to.firework]), fw_ids=parents)
logger.debug('added output node for var %s', obj_to.ref.name) logger.debug('added output node for var %s', obj_to.ref.name)
......
src/virtmat/language/metamodel/amml.py
View file @ 5a6bd887
"""processors for AMML objects""" """processors for AMML objects"""
from virtmat.language.utilities.errors import raise_exception, StaticValueError from virtmat.language.utilities.errors import raise_exception, StaticValueError
from virtmat.language.utilities.textx import isinstance_m
from virtmat.language.utilities.ase_params import spec, par_units from virtmat.language.utilities.ase_params import spec, par_units
...@@ -29,7 +30,6 @@ def amml_algorithm_processor(obj): ...@@ -29,7 +30,6 @@ def amml_algorithm_processor(obj):
msg = f'Algorithm {obj.name} is not implemented' msg = f'Algorithm {obj.name} is not implemented'
raise_exception(obj, NotImplementedError, msg) raise_exception(obj, NotImplementedError, msg)
params = spec[obj.name]['params'] params = spec[obj.name]['params']
par_mandatory = [k for k, v in params.items() if 'default' not in v]
mt1 = spec[obj.name].get('many_to_one') mt1 = spec[obj.name].get('many_to_one')
if mt1 is not None and obj.many_to_one is not mt1: if mt1 is not None and obj.many_to_one is not mt1:
msg = f'Incorrect many-to-one relationship for algorithm {obj.name}' msg = f'Incorrect many-to-one relationship for algorithm {obj.name}'
...@@ -39,10 +39,23 @@ def amml_algorithm_processor(obj): ...@@ -39,10 +39,23 @@ def amml_algorithm_processor(obj):
inv_pars = tuple(p for p in par_names if p not in params) inv_pars = tuple(p for p in par_names if p not in params)
msg = f'Invalid parameters used in algorithm {obj.name}: {inv_pars}' msg = f'Invalid parameters used in algorithm {obj.name}: {inv_pars}'
raise_exception(obj.parameters or obj, StaticValueError, msg) raise_exception(obj.parameters or obj, StaticValueError, msg)
par_mandatory = [k for k, v in params.items() if 'default' not in v]
if not all(p in par_names for p in par_mandatory): if not all(p in par_names for p in par_mandatory):
inv_pars = tuple(p for p in par_mandatory if p not in par_names) inv_pars = tuple(p for p in par_mandatory if p not in par_names)
msg = f'Mandatory parameters missing in algorithm {obj.name}: {inv_pars}' msg = f'Mandatory parameters missing in algorithm {obj.name}: {inv_pars}'
raise_exception(obj.parameters or obj, StaticValueError, msg) raise_exception(obj.parameters or obj, StaticValueError, msg)
# check string and integer parameters with finite number of valid choices
par_choices = {p: v['choices'] for p, v in params.items() if 'choices' in v}
par_names = [p for p in par_names if p in par_choices]
for pname in par_names:
col = obj.parameters.get_column(pname)
if col is not None:
for elem in col.elements:
if isinstance_m(elem, ['String', 'Number']):
if elem.value not in par_choices[pname]:
msg = (f'Parameter "{pname}" should be one of '
f'{par_choices[pname]} but is {elem.value}')
raise_exception(col, StaticValueError, msg)
def amml_property_processor(obj): def amml_property_processor(obj):
......
src/virtmat/language/utilities/amml.py
View file @ 5a6bd887
...@@ -751,7 +751,3 @@ class Trajectory(AMMLObject): ...@@ -751,7 +751,3 @@ class Trajectory(AMMLObject):
self.properties.iloc[key], self.constraints[key], self.properties.iloc[key], self.constraints[key],
self.filename) self.filename)
raise TypeError(f'unknown key type {type(key)}') raise TypeError(f'unknown key type {type(key)}')
class NeighborList(AMMLObject):
"""custom AMML NeighborList class"""
src/virtmat/language/utilities/ase_handlers.py
View file @ 5a6bd887
...@@ -410,6 +410,32 @@ ase_p_df = pandas.DataFrame([ ...@@ -410,6 +410,32 @@ ase_p_df = pandas.DataFrame([
'property': 'energy_minimum', 'property': 'energy_minimum',
'handler': lambda x: pandas.Series(i['algo'].results['energy_minimum'] for i in x) 'handler': lambda x: pandas.Series(i['algo'].results['energy_minimum'] for i in x)
}, },
{
'method': 'NeighborList',
'property': 'neighbors',
'handler': lambda x: pandas.Series(tuple(ureg.Quantity(j)
for j in i['algo'].results['neighbors'])
for i in x)
},
{
'method': 'NeighborList',
'property': 'neighbor_offsets',
'handler': lambda x: pandas.Series(tuple(ureg.Quantity(j, 'angstrom')
for j in i['algo'].results['neighbor_offsets'])
for i in x)
},
{
'method': 'NeighborList',
'property': 'connectivity_matrix',
'handler': lambda x: pandas.Series(ureg.Quantity(i['algo'].results['connectivity_matrix'])
for i in x)
},
{
'method': 'NeighborList',
'property': 'connected_components',
'handler': lambda x: pandas.Series(ureg.Quantity(i['algo'].results['connected_components'])
for i in x)
},
{ {
'method': 'lj', 'method': 'lj',
'property': 'energy', 'property': 'energy',
......
src/virtmat/language/utilities/ase_params.py
View file @ 5a6bd887
...@@ -378,6 +378,7 @@ spec = { ...@@ -378,6 +378,7 @@ spec = {
}, },
'update_prior_strategy': { 'update_prior_strategy': {
'default': 'maximum', 'default': 'maximum',
'choices': ['maximum', 'init', 'average'],
'type': str, 'type': str,
'units': None, 'units': None,
'method': 'class', 'method': 'class',
...@@ -565,13 +566,8 @@ spec = { ...@@ -565,13 +566,8 @@ spec = {
}, },
'neighborlist': { 'neighborlist': {
'default': None, 'default': None,
'type': object, # cannot be checked, always true
'units': None,
'method': 'run'
},
'neighborlist_pars': {
'default': pandas.DataFrame(),
'type': pandas.DataFrame, 'type': pandas.DataFrame,
'otype': 'algorithm',
'units': None, 'units': None,
'method': 'run' 'method': 'run'
}, },
...@@ -592,6 +588,7 @@ spec = { ...@@ -592,6 +588,7 @@ spec = {
'params': { 'params': {
'distribution': { 'distribution': {
'default': 'maxwell-boltzmann', # phonon_harmonics not implemented 'default': 'maxwell-boltzmann', # phonon_harmonics not implemented
'choices': ['maxwell-boltzmann', 'phonon_harmonics'],
'type': str, 'type': str,
'units': None, 'units': None,
'method': 'class' 'method': 'class'
...@@ -650,6 +647,8 @@ spec = { ...@@ -650,6 +647,8 @@ spec = {
}, },
'eos': { 'eos': {
'default': 'sjeos', 'default': 'sjeos',
'choices': ['sjeos', 'taylor', 'murnaghan', 'birch', 'birchmurnaghan'
'pouriertarantola', 'vinet', 'antonschmidt', 'p3'],
'type': str, 'type': str,
'units': None, 'units': None,
'method': 'run' 'method': 'run'
...@@ -689,8 +688,8 @@ spec = { ...@@ -689,8 +688,8 @@ spec = {
'method': 'run' 'method': 'run'
}, },
'spin': { 'spin': {
# None, 0 or 1
'default': None, 'default': None,
'choices': [None, 0, 1],
'type': numbers.Integral, 'type': numbers.Integral,
'units': 'dimensionless', 'units': 'dimensionless',
'method': 'run' 'method': 'run'
...@@ -1089,6 +1088,7 @@ spec = { ...@@ -1089,6 +1088,7 @@ spec = {
}, },
'interpolate_method': { 'interpolate_method': {
'default': 'linear', 'default': 'linear',
'choices': ['linear', 'idpp'],
'type': str, 'type': str,
'units': None, 'units': None,
'method': 'class', 'method': 'class',
...@@ -1137,6 +1137,7 @@ spec = { ...@@ -1137,6 +1137,7 @@ spec = {
}, },
'method': { 'method': {
'default': 'aseneb', 'default': 'aseneb',
'choices': ['aseneb', 'improvedtangent', 'eb', 'spline', 'string'],
'type': str, 'type': str,
'units': None, 'units': None,
'method': 'class', 'method': 'class',
...@@ -1189,7 +1190,8 @@ spec = { ...@@ -1189,7 +1190,8 @@ spec = {
'method': 'class', 'method': 'class',
}, },
'eigenmode_method': { 'eigenmode_method': {
'default': 'dimer', # only 'dimer' possible 'default': 'dimer',
'choices': ['dimer'],
'type': str, 'type': str,
'units': None, 'units': None,
'method': 'class', 'method': 'class',
...@@ -1249,7 +1251,8 @@ spec = { ...@@ -1249,7 +1251,8 @@ spec = {
'method': 'class', 'method': 'class',
}, },
'initial_eigenmode_method': { 'initial_eigenmode_method': {
'default': 'gauss', # 'gauss' or 'displacement' 'default': 'gauss',
'choices': ['gauss', 'displacement'],
'type': str, 'type': str,
'units': None, 'units': None,
'method': 'class', 'method': 'class',
...@@ -1261,7 +1264,8 @@ spec = { ...@@ -1261,7 +1264,8 @@ spec = {
'method': 'class', 'method': 'class',
}, },
'displacement_method': { 'displacement_method': {
'default': 'gauss', # 'gauss' or 'vector' 'default': 'gauss',
'choices': ['gauss', 'vector'],
'type': str, 'type': str,
'units': None, 'units': None,
'method': 'class', 'method': 'class',
...@@ -1339,12 +1343,14 @@ spec = { ...@@ -1339,12 +1343,14 @@ spec = {
}, },
'method': { 'method': {
'default': 'standard', 'default': 'standard',
'choices': ['standard', 'frederiksen'],
'type': str, 'type': str,
'units': None, 'units': None,
'method': 'run', 'method': 'run',
}, },
'direction': { 'direction': {
'default': 'central', 'default': 'central',
'choices': ['central', 'forward', 'backward'],
'type': str, 'type': str,
'units': None, 'units': None,
'method': 'run', 'method': 'run',
...@@ -1363,6 +1369,53 @@ spec = { ...@@ -1363,6 +1369,53 @@ spec = {
}, },
} }
}, },
'NeighborList': {
'module': 'virtmat.language.utilities.ase_wrappers',
'class': 'NeighborListWrapper',
'requires_dof': False,
'many_to_one': False,
'properties': ['neighbors', 'neighbor_offsets', 'connectivity_matrix',
'connected_components'],
'params': {
'cutoffs': {
'default': None,
'type': numpy.ndarray,
'units': 'angstrom',
'method': 'class',
},
'skin': {
'default': ureg.Quantity(0.3, 'angstrom'),
'type': numbers.Real,
'units': 'angstrom',
'method': 'class',
},
'sorted': {
'default': False,
'type': bool,
'units': None,
'method': 'class',
},
'self_interaction': {
'default': True,
'type': bool,
'units': None,
'method': 'class',
},
'bothways': {
'default': False,
'type': bool,
'units': None,
'method': 'class',
},
'method': {
'default': 'quadratic',
'choices': ['quadratic', 'linear'],
'type': str,
'units': None,
'method': 'class',
},
}
},
} }
for calc in ('vasp', 'turbomole', 'lj', 'lennardjones', 'emt', 'free_electrons'): for calc in ('vasp', 'turbomole', 'lj', 'lennardjones', 'emt', 'free_electrons'):
......
src/virtmat/language/utilities/ase_wrappers.py
View file @ 5a6bd887
...@@ -3,6 +3,7 @@ import importlib ...@@ -3,6 +3,7 @@ import importlib
from contextlib import contextmanager from contextlib import contextmanager
from tempfile import TemporaryDirectory from tempfile import TemporaryDirectory
import numpy import numpy
from scipy import sparse
from matplotlib import pyplot from matplotlib import pyplot
from ase import Atoms from ase import Atoms
from ase.utils import IOContext from ase.utils import IOContext
...@@ -17,6 +18,8 @@ from ase.vibrations import Vibrations ...@@ -17,6 +18,8 @@ from ase.vibrations import Vibrations
from ase.calculators.singlepoint import SinglePointCalculator from ase.calculators.singlepoint import SinglePointCalculator
from ase.calculators.vasp import Vasp from ase.calculators.vasp import Vasp
from ase.md import velocitydistribution from ase.md import velocitydistribution
from ase.neighborlist import NeighborList, natural_cutoffs
from ase.neighborlist import PrimitiveNeighborList, NewPrimitiveNeighborList
from virtmat.language.utilities.errors import RuntimeValueError from virtmat.language.utilities.errors import RuntimeValueError
from virtmat.language.utilities.ase_params import spec from virtmat.language.utilities.ase_params import spec
from virtmat.language.utilities.warnings import warnings, TextSUserWarning from virtmat.language.utilities.warnings import warnings, TextSUserWarning
...@@ -67,11 +70,10 @@ class RDF(IOContext): ...@@ -67,11 +70,10 @@ class RDF(IOContext):
msg = 'the structure cell must have at least one non-zero vector' msg = 'the structure cell must have at least one non-zero vector'
raise RuntimeValueError(msg) raise RuntimeValueError(msg)
def run(self, rmax=None, nbins=40, neighborlist=None, neighborlist_pars=None, def run(self, rmax=None, nbins=40, neighborlist=None, elements=None):
elements=None):
"""Calculate the radial distribution function for a structure""" """Calculate the radial distribution function for a structure"""
neighborlist_pars = neighborlist_pars or {} neighborlist_pars = neighborlist and neighborlist['parameters'] or {}
analysis = Analysis(self.atoms_list, neighborlist, **neighborlist_pars) analysis = Analysis(self.atoms_list, **neighborlist_pars)
rmax = rmax or 0.49*max(max(a.cell.lengths()) for a in self.atoms_list) rmax = rmax or 0.49*max(max(a.cell.lengths()) for a in self.atoms_list)
ret = analysis.get_rdf(rmax, nbins, elements=elements, return_dists=True) ret = analysis.get_rdf(rmax, nbins, elements=elements, return_dists=True)
self.results = {'rdf': numpy.mean([a for a, b in ret], axis=0), self.results = {'rdf': numpy.mean([a for a, b in ret], axis=0),
...@@ -291,6 +293,35 @@ class VibrationsWrapper(IOContext): ...@@ -291,6 +293,35 @@ class VibrationsWrapper(IOContext):
return True return True
class NeighborListWrapper(IOContext):
"""a wrapper class for the NeighborList algorithms in ASE"""
results = None
def __init__(self, atoms, cutoffs=None, method='quadratic', **kwargs):
assert isinstance(atoms, Atoms)
self.atoms = atoms
if cutoffs is None:
cutoffs = natural_cutoffs(atoms)
assert method in ('quadratic', 'linear')
pclass = PrimitiveNeighborList if method == 'quadratic' else NewPrimitiveNeighborList
self.nl = NeighborList(cutoffs, primitive=pclass, **kwargs)
def run(self):
"""update neghbor list and get the properties"""
self.nl.update(self.atoms)
neighbors = []
neighbor_offsets = []
for atom in self.atoms:
neighbors_ = self.nl.get_neighbors(atom.index)
neighbors.append(neighbors_[0])
neighbor_offsets.append(neighbors_[1])
matrix = self.nl.get_connectivity_matrix(sparse=False)
_, components = sparse.csgraph.connected_components(matrix)
self.results = {'neighbors': neighbors, 'neighbor_offsets': neighbor_offsets,
'connectivity_matrix': matrix, 'connected_components': components}
return True
def vasp_calculate_decorator(func): def vasp_calculate_decorator(func):
"""perform modifications of Vasp.calculate's arguments""" """perform modifications of Vasp.calculate's arguments"""
def calculate_wrapper(self, atoms, *args, **kwargs): def calculate_wrapper(self, atoms, *args, **kwargs):
......
src/virtmat/language/utilities/compatibility.py
View file @ 5a6bd887
"""define/check grammar and data schema versions compatible to the interpreter""" """define/check grammar and data schema versions compatible to the interpreter"""
import re import re
from virtmat.language.utilities.logging import get_logger from virtmat.language.utilities.logging import get_logger
from virtmat.language.utilities.errors import CompatibilityError
versions = {'grammar': [14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 27, 28, versions = {'grammar': [14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 27, 28,
29, 30], 29, 30, 31],
'data_schema': [6, 7]} 'data_schema': [6, 7]}
class CompatibilityError(Exception):
"""raise this exception if the grammar or data schema are incompatible"""
def get_grammar_version(grammar_str): def get_grammar_version(grammar_str):
"""extract the version number from the grammar""" """extract the version number from the grammar"""
regex = re.compile(r'\/\*\s*grammar version\s+(\d+)\s*\*\/', re.MULTILINE) regex = re.compile(r'\/\*\s*grammar version\s+(\d+)\s*\*\/', re.MULTILINE)
......
src/virtmat/language/utilities/errors.py
View file @ 5a6bd887
...@@ -6,7 +6,6 @@ from pint.errors import PintError, DimensionalityError, UndefinedUnitError ...@@ -6,7 +6,6 @@ from pint.errors import PintError, DimensionalityError, UndefinedUnitError
from pint.errors import OffsetUnitCalculusError from pint.errors import OffsetUnitCalculusError
from ase.calculators.calculator import CalculatorSetupError from ase.calculators.calculator import CalculatorSetupError
from virtmat.middleware.exceptions import ResourceConfigurationError from virtmat.middleware.exceptions import ResourceConfigurationError
from virtmat.language.utilities.compatibility import CompatibilityError
from virtmat.language.utilities.textx import get_location_context from virtmat.language.utilities.textx import get_location_context
FILE_READ_EXCEPTION_IMPORTS = {'ruamel.yaml.parser': 'ParserError', FILE_READ_EXCEPTION_IMPORTS = {'ruamel.yaml.parser': 'ParserError',
...@@ -37,6 +36,10 @@ for mod, exc in MONGODB_EXCEPTION_IMPORTS.items(): ...@@ -37,6 +36,10 @@ for mod, exc in MONGODB_EXCEPTION_IMPORTS.items():
MONGODB_EXCEPTIONS.append(class_) MONGODB_EXCEPTIONS.append(class_)
class CompatibilityError(Exception):
"""raise this exception if grammar, data schema or python versions are incompatible"""
class InvalidUnitError(RuntimeError): class InvalidUnitError(RuntimeError):
"""raise this exception if an invalid unit is detected""" """raise this exception if an invalid unit is detected"""
......
src/virtmat/language/utilities/firetasks.py
View file @ 5a6bd887
...@@ -7,6 +7,18 @@ from fireworks import Firework ...@@ -7,6 +7,18 @@ from fireworks import Firework
from fireworks.core.firework import FWAction, FireTaskBase from fireworks.core.firework import FWAction, FireTaskBase
from virtmat.language.utilities.serializable import FWDataObject from virtmat.language.utilities.serializable import FWDataObject
from virtmat.language.utilities.ioops import store_value from virtmat.language.utilities.ioops import store_value
from virtmat.language.utilities.errors import CompatibilityError
def get_exception_serializable(exc):
"""make an exception fireworks-serializable
https://materialsproject.github.io/fireworks/failures_tutorial.html
"""
cls = exc.__class__
dct = {'name': cls.__name__, 'module': cls.__module__, 'msg': str(exc),
'pkl': base64.b64encode(dill.dumps(exc)).decode('utf-8')}
exc.to_dict = lambda: dct
return exc
class FunctionTask(FireTaskBase): class FunctionTask(FireTaskBase):
...@@ -22,14 +34,16 @@ class FunctionTask(FireTaskBase): ...@@ -22,14 +34,16 @@ class FunctionTask(FireTaskBase):
params = [fw_spec[i].value for i in inputs] params = [fw_spec[i].value for i in inputs]
func = dill.loads(base64.b64decode(self['func'].encode())) func = dill.loads(base64.b64decode(self['func'].encode()))
f_output = func(*params) f_output = func(*params)
except SystemError as err: # not covered
if 'unknown opcode' in str(err):
python = fw_spec.get('_python_version') or 'unknown'
msg = (f'This statement has been compiled with incompatible python '
f'version: {python}.\nEither rerun and use the same version'
f' or use variable update ":=" to re-compile the statement.')
raise get_exception_serializable(CompatibilityError(msg)) from err
raise get_exception_serializable(err) from err
except BaseException as err: except BaseException as err:
# serialize and raise the exception raise get_exception_serializable(err) from err
# https://materialsproject.github.io/fireworks/failures_tutorial.html
cls = err.__class__
dct = {'name': cls.__name__, 'module': cls.__module__, 'msg': str(err),
'pkl': base64.b64encode(dill.dumps(err)).decode('utf-8')}
err.to_dict = lambda: dct
raise err
return self.get_fw_action(f_output) return self.get_fw_action(f_output)
def get_fw_action(self, output): def get_fw_action(self, output):
...@@ -37,15 +51,11 @@ class FunctionTask(FireTaskBase): ...@@ -37,15 +51,11 @@ class FunctionTask(FireTaskBase):
outputs = self.get('outputs', []) outputs = self.get('outputs', [])
assert isinstance(outputs, list) assert isinstance(outputs, list)
assert all(isinstance(o, str) for o in outputs) assert all(isinstance(o, str) for o in outputs)
actions = {}
if len(outputs) == 1: if len(outputs) == 1:
actions['update_spec'] = {outputs[0]: FWDataObject.from_obj(output)} update_dct = {outputs[0]: FWDataObject.from_obj(output)}
elif len(outputs) > 1: return FWAction(update_spec=update_dct)
assert isinstance(output, (list, tuple, set)) assert len(outputs) == 0 and output is None
assert len(output) == len(outputs) return FWAction()
output_data = (FWDataObject.from_obj(o) for o in output)
actions['update_spec'] = dict(zip(outputs, output_data))
return FWAction(**actions)
class ExportDataTask(FireTaskBase): class ExportDataTask(FireTaskBase):
...@@ -82,7 +92,7 @@ class ScatterTask(FireTaskBase): ...@@ -82,7 +92,7 @@ class ScatterTask(FireTaskBase):
for dct, chunk in zip(dcts, chunks): for dct, chunk in zip(dcts, chunks):
dct[inp] = FWDataObject.from_obj(chunk) dct[inp] = FWDataObject.from_obj(chunk)
for inp in self['inputs']: for inp in self['inputs']:
if inp not in self['split']: if inp not in self['split']: # not covered
for dct in dcts: for dct in dcts:
dct[inp] = fw_spec[inp] dct[inp] = fw_spec[inp]
fireworks = [] fireworks = []
......
src/virtmat/language/utilities/ioops.py
View file @ 5a6bd887
...@@ -14,6 +14,15 @@ from fireworks.utilities.fw_serializers import load_object, recursive_dict ...@@ -14,6 +14,15 @@ from fireworks.utilities.fw_serializers import load_object, recursive_dict
from virtmat.language.utilities.errors import RuntimeTypeError, RuntimeValueError from virtmat.language.utilities.errors import RuntimeTypeError, RuntimeValueError
from virtmat.language.utilities.amml import AMMLStructure from virtmat.language.utilities.amml import AMMLStructure
FW_CONFIG_DIR = os.path.join(os.path.expanduser('~'), '.fireworks')
DATASTORE_CONFIG_DEFAULT = {'path': os.path.join(FW_CONFIG_DIR, 'vre-language-datastore'),
'type': 'file', # in ['file', 'gridfs', 'url', None]
'format': 'json', # in ['json', 'yaml', 'hdf5']
'name': 'vre_language_datastore',
'launchpad': LAUNCHPAD_LOC,
'compress': True,
'inline-threshold': 100000}
def load_value(url=None, filename=None): def load_value(url=None, filename=None):
"""load data from file or from URL using the GET method""" """load data from file or from URL using the GET method"""
...@@ -50,18 +59,14 @@ def store_value(val, url=None, filename=None): ...@@ -50,18 +59,14 @@ def store_value(val, url=None, filename=None):
def get_datastore_config(**kwargs): def get_datastore_config(**kwargs):
"""update, set globally and return the data store configuration""" """update, set globally and return the data store configuration"""
fw_config = os.path.join(os.path.expanduser('~'), '.fireworks') config = DATASTORE_CONFIG_DEFAULT
config = {'path': os.path.join(fw_config, 'vre-language-datastore'),
'type': 'gridfs', # in ['file', 'gridfs', 'url', None]
'format': 'json', # in ['json', 'yaml', 'hdf5']
'name': 'vre_language_datastore',
'launchpad': LAUNCHPAD_LOC,
'compress': True,
'inline-threshold': 1000000}
if 'DATASTORE_CONFIG' in os.environ: if 'DATASTORE_CONFIG' in os.environ:
conf_path = os.environ['DATASTORE_CONFIG'] conf_path = os.environ['DATASTORE_CONFIG']
if not os.path.exists(conf_path):
msg = f'The config file {conf_path} does not exist.'
raise FileNotFoundError(msg)
else: else:
conf_path = os.path.join(fw_config, 'datastore_config.yaml') conf_path = os.path.join(FW_CONFIG_DIR, 'datastore_config.yaml')
if os.path.exists(conf_path): if os.path.exists(conf_path):
with open(conf_path, 'r', encoding='utf-8') as inp: with open(conf_path, 'r', encoding='utf-8') as inp:
custom_config = yaml.safe_load(inp) custom_config = yaml.safe_load(inp)
......
src/virtmat/language/utilities/typemap.py
View file @ 5a6bd887
...@@ -30,7 +30,6 @@ typemap = { ...@@ -30,7 +30,6 @@ typemap = {
'AMMLProperty': amml.Property, 'AMMLProperty': amml.Property,
'AMMLConstraint': amml.Constraint, 'AMMLConstraint': amml.Constraint,
'AMMLTrajectory': amml.Trajectory, 'AMMLTrajectory': amml.Trajectory,
'AMMLNeighborList': amml.NeighborList,
'ChemReaction': chemistry.ChemReaction, 'ChemReaction': chemistry.ChemReaction,
'ChemSpecies': chemistry.ChemSpecies 'ChemSpecies': chemistry.ChemSpecies
} }
...@@ -228,6 +227,14 @@ specs = [ ...@@ -228,6 +227,14 @@ specs = [
'typespec': {'datatype': ('FloatArray', 'FloatArray')}}, 'typespec': {'datatype': ('FloatArray', 'FloatArray')}},
{'typ': 'AMMLProperty', 'id': 'vdf', 'basetype': 'Series', {'typ': 'AMMLProperty', 'id': 'vdf', 'basetype': 'Series',
'typespec': {'datatype': ('FloatArray', 'FloatArray')}}, 'typespec': {'datatype': ('FloatArray', 'FloatArray')}},
{'typ': 'AMMLProperty', 'id': 'neighbors', 'basetype': 'Series',
'typespec': {'datatype': ('IntArray', 'IntArray')}},
{'typ': 'AMMLProperty', 'id': 'neighbor_offsets', 'basetype': 'Series',
'typespec': {'datatype': ('FloatArray', 'FloatArray')}},
{'typ': 'AMMLProperty', 'id': 'connectivity_matrix', 'basetype': 'Series',
'typespec': {'datatype': ('IntArray', 'IntArray')}},
{'typ': 'AMMLProperty', 'id': 'connected_components', 'basetype': 'Series',
'typespec': {'datatype': ('IntArray', 'IntArray')}},
{'typ': 'BSTable', 'basetype': 'Table', 'typespec': {'datatype': None}}, {'typ': 'BSTable', 'basetype': 'Table', 'typespec': {'datatype': None}},
{'typ': 'AMMLConstraint', 'basetype': 'AMMLConstraint', 'typespec': {}}, {'typ': 'AMMLConstraint', 'basetype': 'AMMLConstraint', 'typespec': {}},
{'typ': 'AMMLTrajectory', 'basetype': 'AMMLTrajectory', 'typespec': {}}, {'typ': 'AMMLTrajectory', 'basetype': 'AMMLTrajectory', 'typespec': {}},
......
src/virtmat/language/utilities/types.py
View file @ 5a6bd887
...@@ -22,23 +22,13 @@ scalar_type = (str, *scalar_booltype, numbers.Number) ...@@ -22,23 +22,13 @@ scalar_type = (str, *scalar_booltype, numbers.Number)
def is_numeric_type(type_): def is_numeric_type(type_):
"""check if the type_ is numeric""" """check if type_ is a numeric type"""
if issubclass(type_, ScalarNumerical) and not issubclass(type_, bool): if issubclass(type_, scalar_booltype):
return True return False
if issubclass(type_, ureg.Quantity): if issubclass(type_, (ScalarNumerical, ureg.Quantity)):
return True
if issubclass(type_, numpy.ndarray) and issubclass(type_.datatype, ScalarNumerical):
return True return True
if issubclass(type_, pandas.Series): if issubclass(type_, (pandas.Series, numpy.ndarray)):
if type_.datatype is None: return getattr(type_, 'datatype', None) and is_numeric_type(type_.datatype)
return None
if issubclass(type_.datatype, scalar_booltype):
return False
if issubclass(type_.datatype, (ScalarNumerical, ureg.Quantity)):
return True
if issubclass(type_.datatype, numpy.ndarray):
if issubclass(type_.datatype.datatype, ScalarNumerical):
return True
return False return False
...@@ -52,7 +42,7 @@ def is_scalar_type(type_): ...@@ -52,7 +42,7 @@ def is_scalar_type(type_):
def is_numeric_scalar_type_of(type_, numtype): def is_numeric_scalar_type_of(type_, numtype):
"""check if type_ is numerical scalar type of numtype""" """check if type_ is numeric scalar sub-type of numtype"""
if not is_scalar_type(type_) or issubclass(type_, scalar_booltype): if not is_scalar_type(type_) or issubclass(type_, scalar_booltype):
return False return False
if issubclass(type_, ureg.Quantity) and type_.datatype: if issubclass(type_, ureg.Quantity) and type_.datatype:
...@@ -61,35 +51,35 @@ def is_numeric_scalar_type_of(type_, numtype): ...@@ -61,35 +51,35 @@ def is_numeric_scalar_type_of(type_, numtype):
def is_scalar_inttype(type_): def is_scalar_inttype(type_):
"""check if type_ is scalar integer type""" """check if type_ is a scalar integer type"""
return is_numeric_scalar_type_of(type_, ScalarInteger) return is_numeric_scalar_type_of(type_, ScalarInteger)
def is_scalar_realtype(type_): def is_scalar_realtype(type_):
"""check if type_ is scalar real type""" """check if type_ is a scalar real type"""
return (is_numeric_scalar_type_of(type_, ScalarReal) and not return (is_numeric_scalar_type_of(type_, ScalarReal) and not
is_numeric_scalar_type_of(type_, ScalarInteger)) is_numeric_scalar_type_of(type_, ScalarInteger))
def is_scalar_complextype(type_): def is_scalar_complextype(type_):
"""check if type_ is scalar complex type""" """check if type_ is a scalar complex type"""
return (is_numeric_scalar_type_of(type_, ScalarComplex) and not return (is_numeric_scalar_type_of(type_, ScalarComplex) and not
is_numeric_scalar_type_of(type_, ScalarReal)) is_numeric_scalar_type_of(type_, ScalarReal))
def is_array_type(type_): def is_array_type(type_):
"""check if type_ is array type""" """check if type_ is an array type"""
return ((hasattr(type_, 'arraytype') and type_.arraytype) or return ((hasattr(type_, 'arraytype') and type_.arraytype) or
(hasattr(type_, 'datatype') and is_array_type(type_.datatype))) (hasattr(type_, 'datatype') and is_array_type(type_.datatype)))
def is_numeric_scalar_type(type_): def is_numeric_scalar_type(type_):
"""check if the type_ is numeric scalar type""" """check if type_ is a numeric scalar type"""
return is_numeric_type(type_) and is_scalar_type(type_) return is_numeric_type(type_) and is_scalar_type(type_)
def is_numeric_array_type(type_): def is_numeric_array_type(type_):
"""check if the type is numeric array type""" """check if type is a numeric array type"""
return is_numeric_type(type_) and is_array_type(type_) return is_numeric_type(type_) and is_array_type(type_)
......

Help | Imprint | Privacy policy | Accessibility | Contact