Installation of matlabengine with pip fails since --build-dir is not available anymore

[samfux84]

Description

Hi,

I try to build matlabengine with pip on our HPC cluster. The matlabengine is usually installed by entering the corresponding matlab directory

$MATLAB_BASEDIR/extern/engines/python

and then running

pip install --user .

The installation tries to create some build directory, which does not work as users don't have write access to the Matlab installation directory.

[sfux@eu-login-22 ~]$ module load gcc/6.3.0 matlab/R2022b python/3.8.5

The following have been reloaded with a version change:
  1) gcc/4.8.5 => gcc/6.3.0

[sfux@eu-login-22 ~]$ cd $MATLAB_BASEDIR/extern/engines/python
[sfux@eu-login-22 python]$ pip install -v --user .
Using pip 23.2.1 from /cluster/apps/nss/gcc-6.3.0/python/3.8.5/x86_64/lib64/python3.8/site-packages/pip (python 3.8)
Processing /cluster/apps/nss/matlab/R2022b/extern/engines/python
  Running command pip subprocess to install build dependencies
  Collecting setuptools>=42
    Obtaining dependency information for setuptools>=42 from https://files.pythonhosted.org/packages/bb/26/7945080113158354380a12ce26873dd6c1ebd88d47f5bc24e2c5bb38c16a/setuptools-68.2.2-py3-none-any.whl.metadata
    Using cached setuptools-68.2.2-py3-none-any.whl.metadata (6.3 kB)
  Collecting wheel
    Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/b8/8b/31273bf66016be6ad22bb7345c37ff350276cfd46e389a0c2ac5da9d9073/wheel-0.41.2-py3-none-any.whl.metadata
    Using cached wheel-0.41.2-py3-none-any.whl.metadata (2.2 kB)
  Using cached setuptools-68.2.2-py3-none-any.whl (807 kB)
  Using cached wheel-0.41.2-py3-none-any.whl (64 kB)
  Installing collected packages: wheel, setuptools
  Successfully installed setuptools-68.2.2 wheel-0.41.2
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  error: could not create 'dist/matlabengineforpython.egg-info': Permission denied
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /cluster/apps/nss/gcc-6.3.0/python/3.8.5/x86_64/bin/python3.8 /cluster/apps/nss/gcc-6.3.0/python/3.8.5/x86_64/lib64/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpphv11src
  cwd: /cluster/apps/nss/matlab/R2022b/extern/engines/python
  Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
[sfux@eu-login-22 python]$

So far our solution was to specify a different build directory with the pip option -b/--build-dir, like

pip install --user -b $SCRATCH .

But now I noticed that the -b/--build-dir option has been removed from pip and I could not find another way to specify another build directory.

What is the replacement for the removed -b/--build-dir option? Or is it no longer possible to specify a different build directory?

Best regards

Sam

Expected behavior

No response

pip version

23.2.1

Python version

3.8.5

OS

CentOS 7.9

How to Reproduce

1.) module load gcc/6.3.0 matlab/R2022b python/3.8.5
2.) cd $MATLAB_BASEDIR/extern/engines/python
3.) pip install -v --user .

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[sbidoul]

@samfux84 indeed building with setuptools in a read-only directory is not supported anymore at the moment.

For now the only solution is to copy the source to a read-write temporary directory before installing.

This is tracked in the setuptools repo: pypa/setuptools#3237 where there is also background information on the topic, so I'm closing this as there is nothing we can do in pip.

[samfux84]

@sbidoul Does that mean you confirm that there is no replacement in pip for -b/--build-dir?

[sbidoul]

That is correct, yes. That is something that can only be implemented correctly by the build backend (setuptools in this case).