Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allocating extra data after an object instance #103743

Closed
jbradaric opened this issue Apr 24, 2023 · 0 comments
Closed

Allocating extra data after an object instance #103743

jbradaric opened this issue Apr 24, 2023 · 0 comments
Labels
topic-C-API type-feature A feature request or enhancement

Comments

@jbradaric
Copy link
Contributor

jbradaric commented Apr 24, 2023 •
edited by bedevere-bot
Loading

Feature or enhancement

Add an unstable (in PEP 689 sense) function that allocates memory for a GC object and allows the caller to specify the size of additional memory that should be allocated after the object. Similar to what could be done using _PyObject_GC_Malloc function in Python <= 3.10.

Pitch

Currently there is no way of allocating extra data after an object and still being able to have a standard Python class that supports GC, allows inheriting and doesn't come with limitations of PyVarObject (see #103740).

See the linked discussion for more details, but in essence the use case is the following. We have a base class and metaclass that support defining "slots" on classes (similar to Python __slots__, but with a different behavior, more on that later). The data for the slots is stored after the object, in the same block of memory. When inheriting from a class that defines such "slots", the subclass gets all the slots from its base classes + slots defined on it. The data for all those slots is always stored after the object, so the slot storage size doesn't really come into calculation for tp_basicsize.

A nice effect of this behavior is that a subclass can inherit from multiple classes with each of the bases defining its own slots. The subclass then gets all those slots, as if they were directly defined on it (in contrast with Python __slots__, which prevents such inheritance).

CC: @encukou

Previous discussion

See this discussion for more details.

Linked PRs
@jbradaric jbradaric added the type-feature A feature request or enhancement label Apr 24, 2023
@bedevere-bot bedevere-bot mentioned this issue Apr 24, 2023
Merged
jbradaric added a commit to jbradaric/cpython that referenced this issue Apr 24, 2023
@jbradaric
pythongh-103743: Add PyUnstable_Object_GC_NewWithExtraData (python#10…
d300004 
…3743)
jbradaric added a commit to jbradaric/cpython that referenced this issue Apr 24, 2023
@jbradaric
pythongh-103743: Add PyUnstable_Object_GC_NewWithExtraData (python#10…
8bf01ab 
…3743)
jbradaric added a commit to jbradaric/cpython that referenced this issue Apr 24, 2023
@jbradaric
pythongh-103743: Add PyUnstable_Object_GC_NewWithExtraData (python#10…
aa3716f 
…3743)
jbradaric added a commit to jbradaric/cpython that referenced this issue Apr 24, 2023
@jbradaric
pythongh-103743: Add PyUnstable_Object_GC_NewWithExtraData (python#10…
cc46476 
…3743)
jbradaric added a commit to jbradaric/cpython that referenced this issue Apr 24, 2023
@jbradaric
pythongh-103743: Add PyUnstable_Object_GC_NewWithExtraData (python#10…
d644206 
…3743)
encukou added a commit that referenced this issue May 2, 2023
@jbradaric @encukou @erlend-aasland
gh-103743: Add PyUnstable_Object_GC_NewWithExtraData (GH-103744)
87223f3 
Co-authored-by: Petr Viktorin <[email protected]>
Co-authored-by: Erlend E. Aasland <[email protected]>
@encukou encukou closed this as completed May 2, 2023
carljm added a commit to carljm/cpython that referenced this issue May 2, 2023
@carljm
Merge branch 'main' into inlinecomp2
0727d6f 
* main:
  pythongh-103822: [Calendar] change return value to enum for day and month APIs (pythonGH-103827)
  pythongh-65022: Fix description of tuple return value in copyreg (python#103892)
  pythonGH-103525: Improve exception message from `pathlib.PurePath()` (pythonGH-103526)
  pythongh-84436: Add integration C API tests for immortal objects (pythongh-103962)
  pythongh-103743: Add PyUnstable_Object_GC_NewWithExtraData (pythonGH-103744)
  pythongh-102997: Update Windows installer to SQLite 3.41.2. (python#102999)
  pythonGH-103484: Fix redirected permanently URLs (python#104001)
  Improve assert_type phrasing (python#104081)
  pythongh-102997: Update macOS installer to SQLite 3.41.2. (pythonGH-102998)
  pythonGH-103472: close response in HTTPConnection._tunnel (python#103473)
  pythongh-88496: IDLE - fix another test on macOS (python#104075)
  pythongh-94673: Hide Objects in PyTypeObject Behind Accessors (pythongh-104074)
  pythongh-94673: Properly Initialize and Finalize Static Builtin Types for Each Interpreter (pythongh-104072)
  pythongh-104016: Skip test for deeply neste f-strings on wasi (python#104071)
carljm added a commit to carljm/cpython that referenced this issue May 2, 2023
@carljm
Merge branch 'main' into cached_property_docs
b9c39e3 
* main: (760 commits)
  pythonGH-104102: Optimize `pathlib.Path.glob()` handling of `../` pattern segments (pythonGH-104103)
  pythonGH-104104: Optimize `pathlib.Path.glob()` by avoiding repeated calls to `os.path.normcase()` (pythonGH-104105)
  pythongh-103822: [Calendar] change return value to enum for day and month APIs (pythonGH-103827)
  pythongh-65022: Fix description of tuple return value in copyreg (python#103892)
  pythonGH-103525: Improve exception message from `pathlib.PurePath()` (pythonGH-103526)
  pythongh-84436: Add integration C API tests for immortal objects (pythongh-103962)
  pythongh-103743: Add PyUnstable_Object_GC_NewWithExtraData (pythonGH-103744)
  pythongh-102997: Update Windows installer to SQLite 3.41.2. (python#102999)
  pythonGH-103484: Fix redirected permanently URLs (python#104001)
  Improve assert_type phrasing (python#104081)
  pythongh-102997: Update macOS installer to SQLite 3.41.2. (pythonGH-102998)
  pythonGH-103472: close response in HTTPConnection._tunnel (python#103473)
  pythongh-88496: IDLE - fix another test on macOS (python#104075)
  pythongh-94673: Hide Objects in PyTypeObject Behind Accessors (pythongh-104074)
  pythongh-94673: Properly Initialize and Finalize Static Builtin Types for Each Interpreter (pythongh-104072)
  pythongh-104016: Skip test for deeply neste f-strings on wasi (python#104071)
  pythongh-104057: Fix direct invocation of test_super (python#104064)
  pythongh-87092: Expose assembler to unit tests (python#103988)
  pythongh-97696: asyncio eager tasks factory (python#102853)
  pythongh-84436: Immortalize in _PyStructSequence_InitBuiltinWithFlags() (pythongh-104054)
  ...
facebook-github-bot pushed a commit to facebookincubator/cinderx that referenced this issue Aug 5, 2024
@jbower-fb @facebook-github-bot
Switch from using _PyObject_GC_Malloc to PyUnstable_Object_GC_NewWith…
9a8300b 
…ExtraData

Summary: `_PyObject_GC_Malloc` has gone away in 3.12, but I think the reason we're using it - adding extra data to a GC-managed `PyObject*` - is covered by the recently added `PyUnstable_Object_GC_NewWithExtraData`. The future of this [is a little uncertain](python/cpython#103743) but for now it seems to do what we want.

Reviewed By: alexmalyshev

Differential Revision: D60684943

fbshipit-source-id: d6b8fb001502c68805bd7edd31fdec61eb9389bb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
topic-C-API type-feature A feature request or enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jbradaric @encukou @arhadthedev