Cross-compatibility of models with different grammar versions

This issue has the purpose of documenting the testing of cross-compatibility between grammar versions of old and new releases of the vre-language. The question of whether there is new-old grammar cross-compatibility is addressed though the following points:

1. Can models with different grammars be grouped by the vary method?
2. Is submodel reuse possible between models with different grammar versions?
3. Can duplicate variables be detected between models with different grammars?

The releases used for these tests are v0.2.0, v0.3.0, and v0.3.1. texts is preferred over textm since the latter has had more potentially model-breaking changes in the latest releases. The docker-containerized mogodb database is used.

Test No.1: Can models with different grammars be grouped by the vary method?

Test approach:

1. Create 1 model (with only one group= no vary). Process (execute and evaluate) with one grammar version.
2. Using a branch with a different grammar try to recover the model, and then create a group using the vary method.
   • Does this work? What happens?
   • Do we end up with a 2-model group?

These tests are motivated by future development of migration tools so older models are compatible with newer releases of the language.

Results

The grammars of the tested versions of the vre-language are not cross-compatible at the stage of model recovery. For example attempting to recover a model using v0.3.1, which was initially processed with v0.2.0 will print the error message

ValueError: cannot find version tag in grammar

since the grammar version tagging was not yet implemented.

Another example comes from the recovery with v0.3.0 of a model originally processed with v0.3.1, which results in the message:

Compatibility error: Provided grammar has version 6 but the supported versions are [2, 3]

If the grammar version tag is available in the model, the parser will catch the incompatibility and stop the session. Since it is not possible to load a model to a session with conflicting grammar versions, variation won't be possible either with such setup.

Test No.2: Is submodel reuse possible between models with different grammar versions?

This test tries to proof whether a model can be extended by sub-model reuse of variables from a model with older (or newer) grammar version than the model being extended.

Testing approach:

1. Create a simple model script (model_a.vm) and execute script in workflow mode.
   • var1 = 2
2. Run the model's evaluation employing the interactive session (uuid).
3. Use the submodel reuse method to recover such variable in a new model:
   • 3.1 Start a new clean session (-w -r flags);
   • 3.2 Create a new variable (var2);
   • 3.3 Test switching between models (%uuid magic);
   • 3.4 Add the variable from the model in step 1 (var1@uuid) to the new variable

Results

The results list the specific versions of the language employed for the execution (exec), evaluation (eval), and reuse (reuse) of the models. These are followed by the outcomes.

	exec	eval	reuse	Outcome
tst1	0.3.1	0.3.1	0.3.1	Sub-model reuse works as expected: variables can be called by %uuid without problems as long as target and source models have compatible grammar versions.
tst2	0.3.1	0.3.1	0.3.0	Loading the model prints the following error message and the session crashes:Compatibility error: Provided grammar has version 6 but the supported versions are [2, 3] // Calling a 0.3.1 variable for reuse in a v0.3.0 session, a non-compatibility-related error message is printed, and the session continues.
tst3	0.3.1	0.3.0	0.3.1	Compatibility error: Provided grammar has version 6 but the supported versions are [2, 3]
tst4	0.3.0	0.3.1	0.3.1	Compatibility error: Provided grammar has version 3 but the supported versions are [5, 6]
tst5	0.3.1	0.3.0	0.3.0	Compatibility error: Provided grammar has version 6 but the supported versions are [2, 3]
tst6	0.3.0	0.3.1	0.3.0	Compatibility error: Provided grammar has version 3 but the supported versions are [5, 6]
tst7	0.3.0	0.3.0	0.3.1	Loading the model prints the following error message and the session crashes:Compatibility error: Provided grammar has version 3 but the supported versions are [5, 6] // It is possible to reuse a variable from interpreter v0.3.0 (grammar v3) in a v0.3.1 session (grammar v6). Grammar version check seems to be working different compared to model reloading (-u flag).
tst8	0.3.0	0.3.0	0.3.0	Model reuse is not working properly, as the @ symbol is not being recognized as part of the syntax. It is possible to switch between models via the %uuid magic, but this does not change the fact that the model to reuse is not loaded due to syntax problems.
tst9	0.2.0	0.2.0	0.3.1	Calling a 0.2.0 variable for reuse in a v0.3.1 session, a syntax error message is printed Syntax error: None:2:20 --> r2 + var1@*a6ed6cf2-e <-- Expected '[a-f\d]{32}\b', and the session continues. // Loading a 0.2.0 model in a 0.3.1 session crashes with a nnon-wrapped error ValueError: cannot find version tag in grammar

The parser will block the reuse of variables from models with grammar versions non compatible with the target models.This has to be confirmed after issue #262 (closed) is fixed.

It should be noted that it suffices to call only once a variable from the target model (e.g. only one my_var@uuid_of_model). More than one call will cause a re-use error:

Input > var2 = 5
Input > print(var1@uuid_of_var1s_model)
Output > 2
Input > suma = var2 +var1
Input > print(suma)
Output > 7
Input>
Input > prodb = var2 * var1@uuid_of_var1s_model
Reuse error: name conflicts between source and target models: {'var1'}

Test No.3. Can duplicate variables be detected between models with different grammars?

Testing approach:

1. Initiate an interactive session with interpreter v0.3.0 (grammar v3) texts session -w -r
2. Create a set of new variables var1 = 5[meter], var2 = 5
3. Print history (%hist) and UUID (`uuid) of the session for later use. Then exit.
4. Change vre-language branch to interpreter v0.3.1 (grammar v6).
5. Start a new interactive session.
6. Create a new set of variables var1 = 10[m], var3 = 10.
7. Check also %hist and %uuid of this session.
8. Test reuse of the variables with non-duplicate names var2 and var3, for example:

Input > suma = var3 + var2@uuid_of_model_1
Input > print(suma)
Output > 15

Input> %hist
COMPLETED  2024-05-07T16:25:01+02:00 var1 = 10[cm]
COMPLETED  2024-05-07T16:25:07+02:00 var3 = 10
COMPLETED  2024-05-07T16:27:23+02:00 var2 = 5 <-! Notice how var2 has been added to model_2

9. Test reuse of the var1 variables, which have have duplicate names:

Input > sumb = var1 + var1@muuid_of_model_1
Reuse error: name conflicts between source and target models: {'var1'}                                 

10. Switch the model in the active session to model_1:

Input > %uuid uuid_of_model_1
Compatibility error: Provided grammar has version 3 but the supported versions are [5, 6]

Results

There is no cross compatibility for direct loading via the texts session manager (texts session ...-u some_uuid). In-session switching between these models is also not possible. If the grammar versions are queried, one can see that model_1 and model_2 have respectively:

• "grammar_str": "/* grammar for virtmat modeling language */\n/* grammar version 3
• "grammar_str": "/* grammar for virtmat modeling language */\n/* grammar version 6 So indeed these grammar versions are being recognized as incompatible.

However, as long as there are no name conflicts/duplicates (see var1 case), the variables from grammar v3 (interpreter v0.3.0) can be re-used in a model with grammar v6 (interpreter v0.3.1). A naming conflict will rise a "Reuse error" message. In any case, the newest interpreter (v0.3.1) can detect naming conflicts/duplicates of variables when re-using models with grammar version 3 at least.

It appears as if the grammar detection for submodel re-use works differently to the check done by the interpreter when loading a model directly to a session. Maybe this method is looking only for duplicates? Is the behaviour the same in script runs?