Build Python 3.13 wheels #461
Conversation
edgarrmondragon
changed the title
|
Thanks. However, I don't plan to merge this without free threading handled as well, as that would just make more work for me later on to try to figure out how to upload a new build for the same MarkupSafe and Python version, something my publish automation is not set up for. From the linked documentation in #460, it doesn't seem like much additional work, but as I said there I won't have time to work on it soon. So if you can get it in here, that's the way to move this forward. |
Gotcha, thanks for replying! I'll try to get the free-threaded stuff in this PR and ping you back when it's ready. |
edgarrmondragon
force-pushed
the
cp313
branch
4 times, most recently
from
7bb911f to
714c768
Compare
edgarrmondragon
force-pushed
the
cp313
branch
from
714c768 to
5408006
Compare
edgarrmondragon
changed the title
|
@davidism I've added free-threaded Python 3.13 to the test matrix, and set |
There's more to it than that: Any extension modules should declare whether they support running with the GIL disabled. Our extension module is I have attempted that in #462. That PR also enables CI tests under 3.13t both on Linux and Windows. (It is still unclear — to me at least — how to test under 3.13t on the macOS runner.) |
|
Thanks @dairiki, I've brought in the initialization change. Let's indeed hope someone gets to actions/setup-python#771 🤞. |
tests/conftest.py
Outdated
| @@ -14,6 +15,11 @@ | |||
| _speedups = None # type: ignore | |||
|
|
|||
|
|
|||
| def pytest_report_header() -> list[str]: | |||
| """Return a list of strings to be displayed in the header of the report.""" | |||
| return [f"Free-threaded: {bool(sysconfig.get_config_var('Py_GIL_DISABLED'))}"] | |||
There was a problem hiding this comment.
Could sys._is_gil_disabled() be of more interest here?
As I understand it Py_GIL_DISABLED tells whether --disable-gil was set at python build time.
Even for --disable-gil builds, the PYTHONGIL environment variable or loading of extension modules without a valid Py_mod_gil slot can cause the GIL to be enabled. _is_gil_disabled() tells whether the GIL is actually in use or not. (This seems more of interest when interpreting test results.)
|
I can't get this to do the right thing on my local Mac. I have 3.13.0rc3 installed with the official installer, including the free-threaded build. I needed to add the following to tox to get it to recognize a py313t env: outputYou can see in the output that free threading is enabled. However, this fails to build the speedups correctly, the wheel gets tagged as Now clearly this is working in the CI tests, but I can't figure out why it's completely different than local. First, you don't specify a separate |
|
cc @hugovk since you seem to know a lot about testing the free-threaded build: https://dev.to/hugovk/help-us-test-free-threaded-python-without-the-gil-1hgf 🤔 any chance you can explain what's going on? |
|
What version of setuptools is the wheel build using? I think it got support for the |
I think it's better to release regular 3.13 wheels as normal, to unblock regular use, and only add free-threading later. Python 3.13.0 final is scheduled for tomorrow, so it would be really good if binary wheels are available. Free-threading is still experimental, so a delay for free-threading is okay.
Hmm, I think it's something to do with the tox config. On macOS, I can install and test okay without tox: ❯ python3.13t --version --version
Python 3.13.0rc3 experimental free-threading build (v3.13.0rc3:fae84c70fbd, Oct 1 2024, 00:22:06) [Clang 15.0.0 (clang-1500.3.9.4)]
❯ python3.13t -m pip install -e .
Obtaining file:///private/tmp/markupsafe
Installing build dependencies ... done
Checking if build backend supports build_editable ... done
Getting requirements to build editable ... done
Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: MarkupSafe
Building editable for MarkupSafe (pyproject.toml) ... done
Created wheel for MarkupSafe: filename=MarkupSafe-3.0.0.dev0-0.editable-cp313-cp313t-macosx_10_13_universal2.whl size=4013 sha256=32b458f7421af07da58ea4219f0d96cff0088ffbfbc0be7fb54fc154345a4ca5
Stored in directory: /private/var/folders/p6/lf2s1s5d4kb335g2n1td8z8c0000gn/T/pip-ephem-wheel-cache-um2nxb9l/wheels/fe/79/54/456dc68def78a5f343e828d62061f1d8f5ca9416f55591f24e
Successfully built MarkupSafe
Installing collected packages: MarkupSafe
Successfully installed MarkupSafe-3.0.0.dev0
❯ python3.13t -m pytest -v
===================================================================== test session starts =====================================================================
platform darwin -- Python 3.13.0rc3, pytest-8.3.3, pluggy-1.5.0 -- /usr/local/bin/python3.13t
cachedir: .pytest_cache
Free-threaded: True
rootdir: /private/tmp/markupsafe
configfile: pyproject.toml
testpaths: tests
collected 74 items
tests/test_escape.py::test_escape[markupsafe._native--] PASSED [ 1%]
tests/test_escape.py::test_escape[markupsafe._native-abcd&><'"efgh-abcd&><'"efgh] PASSED [ 2%]
tests/test_escape.py::test_escape[markupsafe._native-&><'"efgh-&><'"efgh] PASSED [ 4%]
tests/test_escape.py::test_escape[markupsafe._native-abcd&><'"-abcd&><'"] PASSED [ 5%]
tests/test_escape.py::test_escape[markupsafe._native-\u3053\u3093\u306b\u3061\u306f&><'"\u3053\u3093\u3070\u3093\u306f-\u3053\u3093\u306b\u3061\u306f&><'"\u3053\u3093\u3070\u3093\u306f] PASSED [ 6%]
tests/test_escape.py::test_escape[markupsafe._native-&><'"\u3053\u3093\u3070\u3093\u306f-&><'"\u3053\u3093\u3070\u3093\u306f] PASSED [ 8%]
tests/test_escape.py::test_escape[markupsafe._native-\u3053\u3093\u306b\u3061\u306f&><'"-\u3053\u3093\u306b\u3061\u306f&><'"] PASSED [ 9%]
tests/test_escape.py::test_escape[markupsafe._native-\U0001f363\U0001f362&><'"\U0001f37a xyz-\U0001f363\U0001f362&><'"\U0001f37a xyz] PASSED [ 10%]
tests/test_escape.py::test_escape[markupsafe._native-&><'"\U0001f37a xyz-&><'"\U0001f37a xyz] PASSED [ 12%]
tests/test_escape.py::test_escape[markupsafe._native-\U0001f363\U0001f362&><'"-\U0001f363\U0001f362&><'"] PASSED [ 13%]
tests/test_exception_custom_html.py::test_exception_custom_html[markupsafe._native] PASSED [ 14%]
tests/test_leak.py::test_markup_leaks[markupsafe._native] PASSED [ 16%]
tests/test_markupsafe.py::test_adding[markupsafe._native] PASSED [ 17%]
tests/test_markupsafe.py::test_string_interpolation[markupsafe._native-<em>%s</em>-<bad user>-<em><bad user></em>] PASSED [ 18%]
tests/test_markupsafe.py::test_string_interpolation[markupsafe._native-<em>%(username)s</em>-data1-<em><bad user></em>] PASSED [ 20%]
tests/test_markupsafe.py::test_string_interpolation[markupsafe._native-%i-3.14-3] PASSED [ 21%]
tests/test_markupsafe.py::test_string_interpolation[markupsafe._native-%.2f-3.14-3.14] PASSED [ 22%]
tests/test_markupsafe.py::test_type_behavior[markupsafe._native] PASSED [ 24%]
tests/test_markupsafe.py::test_html_interop[markupsafe._native] PASSED [ 25%]
tests/test_markupsafe.py::test_missing_interpol[markupsafe._native-foo] PASSED [ 27%]
tests/test_markupsafe.py::test_missing_interpol[markupsafe._native-42] PASSED [ 28%]
tests/test_markupsafe.py::test_missing_interpol[markupsafe._native-args2] PASSED [ 29%]
tests/test_markupsafe.py::test_tuple_interpol[markupsafe._native] PASSED [ 31%]
tests/test_markupsafe.py::test_dict_interpol[markupsafe._native] PASSED [ 32%]
tests/test_markupsafe.py::test_escaping[markupsafe._native] PASSED [ 33%]
tests/test_markupsafe.py::test_unescape[markupsafe._native] PASSED [ 35%]
tests/test_markupsafe.py::test_format[markupsafe._native] PASSED [ 36%]
tests/test_markupsafe.py::test_format_map[markupsafe._native] PASSED [ 37%]
tests/test_markupsafe.py::test_formatting_empty[markupsafe._native] PASSED [ 39%]
tests/test_markupsafe.py::test_custom_formatting[markupsafe._native] PASSED [ 40%]
tests/test_markupsafe.py::test_complex_custom_formatting[markupsafe._native] PASSED [ 41%]
tests/test_markupsafe.py::test_formatting_with_objects[markupsafe._native] PASSED [ 43%]
tests/test_markupsafe.py::test_escape_silent[markupsafe._native] PASSED [ 44%]
tests/test_markupsafe.py::test_splitting[markupsafe._native] PASSED [ 45%]
tests/test_markupsafe.py::test_mul[markupsafe._native] PASSED [ 47%]
tests/test_markupsafe.py::test_escape_return_type[markupsafe._native] PASSED [ 48%]
tests/test_markupsafe.py::test_soft_str[markupsafe._native] PASSED [ 50%]
tests/test_escape.py::test_escape[markupsafe._speedups--] PASSED [ 51%]
tests/test_escape.py::test_escape[markupsafe._speedups-abcd&><'"efgh-abcd&><'"efgh] PASSED [ 52%]
tests/test_escape.py::test_escape[markupsafe._speedups-&><'"efgh-&><'"efgh] PASSED [ 54%]
tests/test_escape.py::test_escape[markupsafe._speedups-abcd&><'"-abcd&><'"] PASSED [ 55%]
tests/test_escape.py::test_escape[markupsafe._speedups-\u3053\u3093\u306b\u3061\u306f&><'"\u3053\u3093\u3070\u3093\u306f-\u3053\u3093\u306b\u3061\u306f&><'"\u3053\u3093\u3070\u3093\u306f] PASSED [ 56%]
tests/test_escape.py::test_escape[markupsafe._speedups-&><'"\u3053\u3093\u3070\u3093\u306f-&><'"\u3053\u3093\u3070\u3093\u306f] PASSED [ 58%]
tests/test_escape.py::test_escape[markupsafe._speedups-\u3053\u3093\u306b\u3061\u306f&><'"-\u3053\u3093\u306b\u3061\u306f&><'"] PASSED [ 59%]
tests/test_escape.py::test_escape[markupsafe._speedups-\U0001f363\U0001f362&><'"\U0001f37a xyz-\U0001f363\U0001f362&><'"\U0001f37a xyz] PASSED [ 60%]
tests/test_escape.py::test_escape[markupsafe._speedups-&><'"\U0001f37a xyz-&><'"\U0001f37a xyz] PASSED [ 62%]
tests/test_escape.py::test_escape[markupsafe._speedups-\U0001f363\U0001f362&><'"-\U0001f363\U0001f362&><'"] PASSED [ 63%]
tests/test_exception_custom_html.py::test_exception_custom_html[markupsafe._speedups] PASSED [ 64%]
tests/test_leak.py::test_markup_leaks[markupsafe._speedups] PASSED [ 66%]
tests/test_markupsafe.py::test_adding[markupsafe._speedups] PASSED [ 67%]
tests/test_markupsafe.py::test_string_interpolation[markupsafe._speedups-<em>%s</em>-<bad user>-<em><bad user></em>] PASSED [ 68%]
tests/test_markupsafe.py::test_string_interpolation[markupsafe._speedups-<em>%(username)s</em>-data1-<em><bad user></em>] PASSED [ 70%]
tests/test_markupsafe.py::test_string_interpolation[markupsafe._speedups-%i-3.14-3] PASSED [ 71%]
tests/test_markupsafe.py::test_string_interpolation[markupsafe._speedups-%.2f-3.14-3.14] PASSED [ 72%]
tests/test_markupsafe.py::test_type_behavior[markupsafe._speedups] PASSED [ 74%]
tests/test_markupsafe.py::test_html_interop[markupsafe._speedups] PASSED [ 75%]
tests/test_markupsafe.py::test_missing_interpol[markupsafe._speedups-foo] PASSED [ 77%]
tests/test_markupsafe.py::test_missing_interpol[markupsafe._speedups-42] PASSED [ 78%]
tests/test_markupsafe.py::test_missing_interpol[markupsafe._speedups-args2] PASSED [ 79%]
tests/test_markupsafe.py::test_tuple_interpol[markupsafe._speedups] PASSED [ 81%]
tests/test_markupsafe.py::test_dict_interpol[markupsafe._speedups] PASSED [ 82%]
tests/test_markupsafe.py::test_escaping[markupsafe._speedups] PASSED [ 83%]
tests/test_markupsafe.py::test_unescape[markupsafe._speedups] PASSED [ 85%]
tests/test_markupsafe.py::test_format[markupsafe._speedups] PASSED [ 86%]
tests/test_markupsafe.py::test_format_map[markupsafe._speedups] PASSED [ 87%]
tests/test_markupsafe.py::test_formatting_empty[markupsafe._speedups] PASSED [ 89%]
tests/test_markupsafe.py::test_custom_formatting[markupsafe._speedups] PASSED [ 90%]
tests/test_markupsafe.py::test_complex_custom_formatting[markupsafe._speedups] PASSED [ 91%]
tests/test_markupsafe.py::test_formatting_with_objects[markupsafe._speedups] PASSED [ 93%]
tests/test_markupsafe.py::test_escape_silent[markupsafe._speedups] PASSED [ 94%]
tests/test_markupsafe.py::test_splitting[markupsafe._speedups] PASSED [ 95%]
tests/test_markupsafe.py::test_mul[markupsafe._speedups] PASSED [ 97%]
tests/test_markupsafe.py::test_escape_return_type[markupsafe._speedups] PASSED [ 98%]
tests/test_markupsafe.py::test_soft_str[markupsafe._speedups] PASSED [100%]
===================================================================== 74 passed in 0.15s ======================================================================But with the above with tox: ❯ PYTHON_GIL=0 tox r -e py313t
Fatal Python error: config_read_gil: Disabling the GIL is not supported by this build
Python runtime state: preinitialized
❯ tox r -e py313t
.pkg: _optional_hooks> python /Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: get_requires_for_build_wheel> python /Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: build_wheel> python /Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/pyproject_api/_backend.py True setuptools.build_meta
py313t: install_package> python -I -m pip install --force-reinstall --no-deps /private/tmp/markupsafe/.tox/.tmp/package/8/MarkupSafe-3.0.0.dev0-cp313-cp313-macosx_10_13_universal2.whl
ERROR: MarkupSafe-3.0.0.dev0-cp313-cp313-macosx_10_13_universal2.whl is not a supported wheel on this platform.
py313t: exit 1 (0.88 seconds) /private/tmp/markupsafe> python -I -m pip install --force-reinstall --no-deps /private/tmp/markupsafe/.tox/.tmp/package/8/MarkupSafe-3.0.0.dev0-cp313-cp313-macosx_10_13_universal2.whl pid=91457
py313t: FAIL code 1 (1.64 seconds)
evaluation failed :( (1.80 seconds)Without tox we get With tox we get |
|
Yeah, that's what I saw too, but weirdly tox does work in CI. I do plan to release something today, I know that 3.13 releases tomorrow. I think I might go with this PR since it does seem to work. I confirmed that tests for 3.13t pass manually on Linux, Mac, and Windows. |
|
Yep, not supported by tox yet, waiting for virtualenv support first:
|
|
Never mind, it seems to just segfault if I try |
|
#462 (comment) I tried installing the wheel that cibuildwheel created, and then |
|
By the way, I've run out of time today, so I will either get to this tonight, or tomorrow morning. |
|
I pushed a temporary tag so that the cibuildwheel job will run. You should be able to download the artifacts when they're done and install them/see the tests pass. |
|
Sigh, I meant to do a squash merge so there wasn't a bunch of work commits in there, and also credit @dairiki with co-authored-by. Sorry about that, working too quickly right now. |
Thank you. Don't sweat it! |
|
This is now available on PyPI with MarkupSafe 3.0.0. I confirmed that the tests pass on Linux, Mac, and Windows with the cibuildwheel 313t builds, even though a local build crashes on Windows. We'll have to figure that out at some point. |
|
Thank you David. A |
Python 3.13 RC1 was released recently1:
It'd be really nice to have wheels to start testing applications that use MarkupSafe before the eventual final release.
Thanks!
PS: This PR is only for GIL-enabled 3.13 wheels and not free-threaded ones2, which will certainly involve a higher level of effort.Related: #460
Footnotes
https://discuss.python.org/t/python-3-13-0-release-candidate-1-released/59703 ↩
https://docs.python.org/3.13/whatsnew/3.13.html#free-threaded-cpython ↩