Current conda installation fails to build vre-language package
Currently, the creation of the environment and installation of the vre-language package (and dependencies) with conda
is done using the following conda-setup.yml
:
name: vre-language
dependencies:
- python=3.9 # Python version must be 3.7 or newer
- pip
- pip:
- -r requirements_test.txt # install core and test features
There are no error messages from the setup employing conda env create -f conda-setup.yml
.
If the installation is verified with pip list
all packages and requirements seem to be present.
HOWEVER, when I tried to install the VirtMat kernel I got the following error message:
ModuleNotFoundError: No module named 'virtmat'
If the output of pip list
is compared between the conda
and pip
installations of the vre-language, one can find that the pip installation has this line
[...]
vre-language 0.1.0 /home/rodri/vre-language
[...]
which is missing in the conda
installation.
I tested further by doing clean installations with both methods.
I collected the output from the terminal in log files and compared them.
I found that the pip
installation has direct references to the vre-language package:
[…]
Successfully built vre-language
Installing collected packages: […] vre-language
Successfully installed […] vre-language-0.1.0 […]
These references are completely missing in the output of the conda
installation, meaning that though the requirements and requirements_test are installed, the vre-language package is not installed.
Further testing showed that it is possible to overcome this problem by modifying the conda-setup.yml
file as follows:
name: vre-language
dependencies:
- python=3.9 # Python version must be 3.7 or newer
- pip
- pip:
- -e .[test]
The -e
option inside the pip dictionary enables installation in "editable mode", which allows pip
to install a package from a local project (in this case the project's main directory).
Such package installations require specifications about the build system, as well as information regarding metadata, contents, dependencies, etc.
The vre-language uses setuptools as build system, which is specified within a pyproyect.toml
file.
The package's metadata and other options are defined within the setup.cfg file.
The setup.cfg
file can be set up, such that pip
, via setuptools
, looks for all packages and scripts in the project's source directory, including subfolders.
This can be specified through the following lines in the setup.cfg
file :
[…]
package_dir =
= src
packages = find:
[…]
[options.packages.find]
where = src
[…]
However it seems like conda
on its own does not care about setup.cfg
and setup.py
files.
It uses different format to record dependencies.
By ignoring setup.cfg
it will not look inside the folder tree of the source directory, thus a "more complex" package (meaning it is composed by other packages/scripts in several directories in the src folder) will simply not be built by conda
.
This results in the pervious installation outcome with a missing vre-language package.
If the setup method with the pip -e
option is employed the installation log prints the following lines confirming the installation of the vre-language package:
[…]
Successfully built vre-language
Installing collected packages: […] vre-language
Successfully installed […] vre-language-0.1.0 […]
[…]
Once the environment is activated, running pip list
(or conda list
) does show the package:
vre-language 0.1.0 /home/rodri/vre-language ---> pip list
vre-language 0.1.0 pypi_0 pypi ---> conda list
The conda-setup.yml file for the conda
installation must be fixed so that the vre-language package is build properly.