Missing SOABI in some cross-compiled builds on conda-forge

[minrk]

Apologies if this is the wrong place. My guess is something needs amending in conda-forge's cross-python or crossenv, but I think I need some help here to point me in the right direction of what needs fixing.

Short summary: How and where do I diagnose and fix missing SOABI (or EXT_SUFFIX?) in a cross-compiled build?

Context:

I'm working on migrating pzymq to scikit-build-core. As a test, I decided to try building the prerelease on conda-forge.

Everything is working (yay!) except for one: pypy3.9 cross-compiled to linux-ppc64le.

Interestingly, the two most similar builds both work without error:

• cross-compile CPython to linux-ppc64le, and
• cross-compile pypy3.9 to linux-aarch64

The ppc64le build is failing because the extension is missing the SOABI, building _cffi.so instead of _cffi.pypy39-pp73-ppc_64-linux-gnu.so (everything compiles and links, but the module fails to import because the filename is not in the import extension list).

The older setuptools builds also get the right SOABI for the same configuration, so there must be some difference in how this filename is derived.

I didn't find any warnings or messages related to failing to derive SOABI in the verbose build output.

So the question is: how do I debug why SOABI is not found, so I can possibly fix it? Presumably in cross-python or crossenv, but I'm not sure where to begin. My hunch is it has to do with the fact that ppc64le has 3 names:

• ppc64le
• powerpc64le
• ppc_64 (first in _imp.extension_suffixes(), which populates SOABI, EXT_SUFFIX on PyPy)

I tried to debug this by calling $PYTHON -m sysconfig, but that fails on all of the passing ppc64le builds with:

ModuleNotFoundError: No module named '_sysconfigdata__linux_powerpc64le-linux-gnu'

and passes on the failing one, yielding:

	EXT_SUFFIX = ".pypy39-pp73-x86_64-linux-gnu.so"

which isn't correct, but doesn't explain why the EXT_SUFFIX is empty when it is built with scikit-build-core.

[henryiii]

Quick debugging question: what is SKBUILD_SOABI? That might be more reliable than CMake's Python_SOABI value (and we likely can more easily fix it if not). We compute it like this: https://github.com/scikit-build/scikit-build-core/blob/73fcab8bc009db723c3c5164f7a65396f2b86317/src/scikit_build_core/builder/sysconfig.py#L144. FindPython's SOABI and cross compiling has been discussed in the issue tracker, see https://gitlab.kitware.com/cmake/cmake/-/issues/25001, though this sounds like it might be an edge case bug that could be fixed.

My guess is that because the sysconfig import fails, it's falling back on a different path (probably distutils related), while when it passes, it's triggering the wrong computation for cross compiling. IMO, if $PYTHON -m sysconfig is broken or especially returning the wrong value, the cross-compiling environment may be set up incorrectly (I'd guess that distutils.sysconfig is correctly patched).

[minrk]

SKBUILD_SOABI has the correct value, Python_SOABI is empty. Adding set(Python_SOABI ${SKBUILD_SOABI}) results in a successful build. Thank you!

What makes me think it can be fixed is that it's correct for the aarch64 cross-compiles, just not ppc64le. This sounds like a bug in conda-forge's cross-python. Moving over there. Thanks again for your help.