-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
LibraryImport generated code feedback #69608
Comments
I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label. |
Tagging subscribers to this area: @dotnet/interop-contrib Issue DetailsI spent a little time looking over the code generated for the LibraryImports in CoreLib. Some random feedback / comments / questions... Performance
__lpData_gen_native__marshaller = new(lpData, new System.Span<byte>(__lpData_gen_native__marshaller__stackptr, 512), sizeof(ushort));
{
System.ReadOnlySpan<char> __lpData_gen_native__marshaller__managedSpan = __lpData_gen_native__marshaller.GetManagedValuesSource();
System.Span<ushort> __lpData_gen_native__marshaller__nativeSpan = System.Runtime.InteropServices.MemoryMarshal.Cast<byte, ushort>(__lpData_gen_native__marshaller.GetNativeValuesDestination());
for (int __i0 = 0; __i0 < __lpData_gen_native__marshaller__managedSpan.Length; ++__i0)
{
__lpData_gen_native__marshaller__nativeSpan[__i0] = __lpData_gen_native__marshaller__managedSpan[__i0];
}
} That copy loop could be fairly inefficient. Can't we use MemoryMarshal.Cast in a case like this to treat the char input as a ushort and then use the vectorized ReadOnlySpan.CopyTo?
internal static partial int GetCalendars(string localeName, global::System.Globalization.CalendarId[] calendars, int calendarsCapacity)
{
int __retVal;
//
// Marshal
//
ref global::System.Globalization.CalendarId __byref_calendars = ref (calendars == null ? ref *(global::System.Globalization.CalendarId*)0 : ref System.Runtime.InteropServices.MemoryMarshal.GetArrayDataReference(calendars));
//
// Pin
//
fixed (void* __localeName_gen_native__pinned = localeName)
{
ushort* __localeName_gen_native = (ushort*)__localeName_gen_native__pinned;
fixed (byte* __calendars_gen_native = &System.Runtime.CompilerServices.Unsafe.As<global::System.Globalization.CalendarId, byte>(ref __byref_calendars))
{
__retVal = __PInvoke__(__localeName_gen_native, __calendars_gen_native, calendarsCapacity);
}
}
return __retVal;
//
// Local P/Invoke
//
[System.Runtime.InteropServices.DllImportAttribute("System.Globalization.Native", EntryPoint = "GlobalizationNative_GetCalendars", ExactSpelling = false)]
extern static unsafe int __PInvoke__(ushort* localeName, byte* calendars, int calendarsCapacity);
} Why is it getting a ref to the array via that complicated null check + GetArrayDataReference followed later by using fixed on an Unsafe.As cast in order to get a byte*, rather than just using fixed with the array to get a void* and then casting that pointer to byte*?
__retVal = new global::Microsoft.Win32.SafeHandles.SafeFindHandle();
try
{
//
// Pin
//
fixed (void* __lpFileName_gen_native__pinned = lpFileName)
{
ushort* __lpFileName_gen_native = (ushort*)__lpFileName_gen_native__pinned;
fixed (global::Interop.Kernel32.WIN32_FIND_DATA* __lpFindFileData_gen_native = &lpFindFileData)
{
{
System.Runtime.InteropServices.Marshal.SetLastSystemError(0);
__retVal_gen_native = __PInvoke__(__lpFileName_gen_native, fInfoLevelId, __lpFindFileData_gen_native, fSearchOp, lpSearchFilter, dwAdditionalFlags);
__lastError = System.Runtime.InteropServices.Marshal.GetLastSystemError();
}
}
}
__invokeSucceeded = true;
}
finally
{
if (__invokeSucceeded)
{
//
// GuaranteedUnmarshal
//
System.Runtime.InteropServices.Marshal.InitHandle(__retVal, __retVal_gen_native);
}
} It's using a try/finally and __invokeSucceeded, so it's trying to accomodate something in that try region throwing. But if something there does throw, the allocated SafeHandle won't be disposed, which will in turn leave it for finalization. Should there be a catch block? catch
{
__retVal.Dispose();
throw;
} or potentially instead a catch with an exception filter that does the disposal and returns false to avoid interrupting the unwind?
internal static partial int CLSIDFromProgID(string lpszProgID, out global::System.Guid lpclsid)
{
lpclsid = default;
... Does the built-in marshaling do this, or is this done to work around C# definite assignment errors? If we're doing it just for the latter, we could instead use Style
int __throwOnError_gen_native;
int __ignoreCase_gen_native;
//
// Marshal
//
__throwOnError_gen_native = (int)(throwOnError ? 1 : 0);
__ignoreCase_gen_native = (int)(ignoreCase ? 1 : 0);
//
// Pin
//
fixed (void* __name_gen_native__pinned = name)
{
ushort* __name_gen_native = (ushort*)__name_gen_native__pinned;
{
__PInvoke__(assembly, __name_gen_native, __throwOnError_gen_native, __ignoreCase_gen_native, type, keepAlive, assemblyLoadContext);
}
} Could we add a blank line, purely for readability, before each big comment block that's preceded by statements? Also, do we need the extra "//" lines, or could these just be single-line comments? It takes up a lot of vertical real estate, and I'm not sure what it helps.
//
// Pin
//
fixed (void* __resourceName_gen_native__pinned = resourceName)
{
ushort* __resourceName_gen_native = (ushort*)__resourceName_gen_native__pinned;
{
__retVal = __PInvoke__(assembly, __resourceName_gen_native, assemblyRef, retFileName);
}
} Why?
fixed (void* __assemblyName_gen_native__pinned = assemblyName)
{
ushort* __assemblyName_gen_native = (ushort*)__assemblyName_gen_native__pinned;
fixed (void* __handlerName_gen_native__pinned = handlerName)
{
ushort* __handlerName_gen_native = (ushort*)__handlerName_gen_native__pinned;
fixed (void* __alcName_gen_native__pinned = alcName)
{
ushort* __alcName_gen_native = (ushort*)__alcName_gen_native__pinned;
fixed (void* __resultAssemblyName_gen_native__pinned = resultAssemblyName)
{
ushort* __resultAssemblyName_gen_native = (ushort*)__resultAssemblyName_gen_native__pinned;
fixed (void* __resultAssemblyPath_gen_native__pinned = resultAssemblyPath)
{
ushort* __resultAssemblyPath_gen_native = (ushort*)__resultAssemblyPath_gen_native__pinned;
{
__retVal_gen_native = __PInvoke__(__assemblyName_gen_native, __handlerName_gen_native, __alcName_gen_native, __resultAssemblyName_gen_native, __resultAssemblyPath_gen_native);
}
}
}
}
}
} It's creating new locals just to store the void* cast to a ushort*, which is then only used as an argument. Could it instead be: fixed (void* __assemblyName_gen_native = assemblyName)
fixed (void* __handlerName_gen_native = handlerName)
fixed (void* __alcName_gen_native = alcName)
fixed (void* __resultAssemblyName_gen_native = resultAssemblyName)
fixed (void* __resultAssemblyPath_gen_native = resultAssemblyPath)
{
__retVal_gen_native = __PInvoke__((ushort*)__assemblyName_gen_native, (ushort*)__handlerName_gen_native, (ushort*)__alcName_gen_native, (ushort*)__resultAssemblyName_gen_native, (ushort*)__resultAssemblyPath_gen_native);
} or even declare PInvoke to take void* instead of ushort* and then the casts wouldn't be needed at all? fixed (void* __assemblyName_gen_native = assemblyName)
fixed (void* __handlerName_gen_native = handlerName)
fixed (void* __alcName_gen_native = alcName)
fixed (void* __resultAssemblyName_gen_native = resultAssemblyName)
fixed (void* __resultAssemblyPath_gen_native = resultAssemblyPath)
{
__retVal_gen_native = __PInvoke__(__assemblyName_gen_native, __handlerName_gen_native, __alcName_gen_native, __resultAssemblyName_gen_native, __resultAssemblyPath_gen_native);
}
private static partial bool LaunchInternal()
{
bool __retVal;
int __retVal_gen_native;
__retVal_gen_native = __PInvoke__();
//
// Unmarshal
//
__retVal = __retVal_gen_native != 0;
return __retVal;
//
// Local P/Invoke
//
[System.Runtime.InteropServices.DllImportAttribute("QCall", EntryPoint = "DebugDebugger_Launch", ExactSpelling = false)]
extern static unsafe int __PInvoke__();
} If someone were writing this by hand, it would just be: private static partial bool LaunchInternal()
{
return __PInvoke__() != 0;
[System.Runtime.InteropServices.DllImportAttribute("QCall", EntryPoint = "DebugDebugger_Launch", ExactSpelling = false)]
extern static unsafe int __PInvoke__();
} Have we considered enabling such simplified emitted code when we can detect it?
|
@stephentoub could you point us at an example for this? We should already be doing this (and should have tests for it), so it's definitely a bug if it is not happening. I believe most of the runtime libraries already have it set it at the module level, so we don't add it on the method. |
@stephentoub Using I appreciate the aspirational aspect of the style issues and I think we can move closer, but for our first public version we are only going to focus on the easy ones. Updated comments on the stages - good idea and will help with design and education. Modifier order is an oops on our part, good idea. The "__" suffix is superfluous and agree that we can remove it. The rest are really about simplifying code. This gets really complicated fast because it involves going back through the Roslyn syntax trees and updating them. Seems I misinterpreted the stated cost for some of the style issues. Will handle on a case by case basis. |
You're right, it's behaving as I'd expect. I'd tried it in a separate project but must have been doing something wrong. I see it now doing what I expect it to be doing.
Thanks.
Yes. Though sometimes stylistic improvements also map to smaller IL and asm, e.g. here's (6) on the style list, before and after:
I'm curious why we're using Roslyn's syntax trees to generate the resulting code, especially if it makes it harder to generate the desired output. We can also just write out the C# directly, which is what all of the other generators in dotnet/runtime do (the logging generator, the json generator, and the regex generator).
Sounds good. It's feedback; address it as you see fit. |
We can't use a runtime/src/libraries/System.Private.CoreLib/src/System/Globalization/CompareInfo.Icu.cs Line 733 in 01b7e73
runtime/src/coreclr/vm/ilmarshalers.cpp Line 3967 in 01b7e73
Our SafeHandle marshalling was designed to match the built-in marshalling as closely as possible, as SafeHandle has particular guarantees in interop that we didn't want to break. Changing this portion of the behavior seems doable, so we'll consider it.
As we designed our generator, we decided that using at least some of the Roslyn syntax APIs would help make our code easier to maintain as we'd know if a particular API returned a statement, expression, type name, etc. just by looking at the signature. Additionally, it handles indentation for us, which is quite a convenient feature. Honestly I feel like just using string concatenation would make our generator harder to maintain and extend as we'd have to heavily document what each method returns as the type system wouldn't be able to help us, and analyzing the syntax after generation to optimize code after generation would actually be even more difficult than using the Roslyn APIs. We don't want to special-case or overcomplicate things in our generator too much as it can quickly make the system unmaintainable (i.e. MCG).
|
Can this complex "GetArrayDataReference" expression be made into a helper method in the |
@GSPP that change was handled with the V2 marshaller design and the new I've extracted performance suggestion 4 (safehandle) to #73496 as that's the last feedback item that we plan to address. I'll close this as completed in the meantime. |
I spent a little time looking over the code generated for the LibraryImports in CoreLib. Some random feedback / comments / questions...
Performance
That copy loop could be fairly inefficient. Can't we use MemoryMarshal.Cast in a case like this to treat the char input as a ushort and then use the vectorized ReadOnlySpan.CopyTo?
[Already the case] SkipLocalsInit. Some of the generated code emits fairly large stackallocs, e.g.
stackalloc byte[512]
. That can add measurable overhead if the user's code hasn't set SkipLocalsInit. Can we add [SkipLocalsInit] to the generated method if there isn't already one provided by the user at a level that would encompass the generated method?GetArrayDataReference. The GetCalendars method in corelib is being generated like this:
Why is it getting a ref to the array via that complicated null check + GetArrayDataReference followed later by using fixed on an Unsafe.As cast in order to get a byte*, rather than just using fixed with the array to get a void* and then casting that pointer to byte*?
It's using a try/finally and __invokeSucceeded, so it's trying to accomodate something in that try region throwing. But if something there does throw, the allocated SafeHandle won't be disposed, which will in turn leave it for finalization. Should there be a catch block?
or potentially instead a catch with an exception filter that does the disposal and returns false to avoid interrupting the unwind?
Does the built-in marshaling do this, or is this done to work around C# definite assignment errors? If we're doing it just for the latter, we could instead use
Unsafe.SkipInit(out ...)
, and avoid the extra zero'ing / indirect write.Style
Could we add a blank line, purely for readability, before each big comment block that's preceded by statements? Also, do we need the extra "//" lines, or could these just be single-line comments? It takes up a lot of vertical real estate, and I'm not sure what it helps.
[Addressed: Libraryimport src gen audit #69619] Comments. Instead of "Marshal", could we make it a bit more descriptive, e.g. "Marshal the input arguments.", and instead of "Pin", it could be "Pin the input parameters". Similarly for "Unmarshal", which could be "Marshal the output values" or something like that.
[Addressed: Style improvements for generated p/invokes #69638] Variable naming. Lots of variables include "_gen_native"... presumably the "_native" part is meant to mean that's the variable that's going to be passed to the P/Invoke... what's the meaning / value-add of the "_gen"? The variables are already prefixed with "__", presumably to differentiate them from the values being used?
[Addressed: Libraryimport src gen audit #69619] Modifier ordering. IDE0036 has a preference for the order of modifiers. Since we have to choose some ordering, it'd be nice to snap to its preference (https://docs.microsoft.com/en-us/dotnet/fundamentals/code-analysis/style-rules/ide0036#overview), e.g. today we might output "extern static unsafe void PInvoke", but following its ordering it would be "static extern unsafe void PInvoke"
[Addressed: Style improvements for generated p/invokes #69638] Braces. The actual PInvoke call often ends up being surrounded by an extra level of braces, e.g.
Why?
It's creating new locals just to store the void* cast to a ushort*, which is then only used as an argument. Could it instead be:
or even declare PInvoke to take void* instead of ushort* and then the casts wouldn't be needed at all?
If someone were writing this by hand, it would just be:
Have we considered enabling such simplified emitted code when we can detect it?
"__"
prefix to avoid conflict with user-defined names, like for arguments? Why does the"__PInvoke__"
also have a"__"
suffix?The text was updated successfully, but these errors were encountered: