[One .NET] improve native library ABI detection at build time

[jonathanpeppers]

Context: #5386

To determine the Android ABI of native libraries, we currently:

• Look for %(Abi) metadata
• Look for a directory named: armeabi-v7a, arm64-v8a, x86, x86_64

This will not work in .NET 6, if the native library is coming from a NuGet package with a path such as:

packages\sqlitepclraw.lib.e_sqlite3.android\1.1.11\runtimes\android-arm64\native\libe_sqlite3.so

Instead of doing more inference with directory names, we should actually look at the contents of the .so file and:

• Give a warning if it is not an Android library and ignore it.
• Determine the ABI. If none found, emit a warning and ignore it.

We should still use the %(Abi) metadata if specified, for backwards compatibility.

We could try using the ElfSharp library to do this. We would need to make sure the code path is fast, or we have some mechanism for making incremental builds fast.

Useful links:

• https://www.nuget.org/packages/ELFSharp/
• https://stackoverflow.com/a/47539958
• https://android.googlesource.com/platform/ndk/+/master/parse_elfnote.py

[jonathanpeppers]

I wrote a sample program using ElfSharp: ElfSharpTest.zip

It looks for the equivalent of readelf -d | grep NEEDED | grep ld-linux in about 47ms for 10 native libraries.

However one native library breaks the pattern:

$ readelf -d linux-x86/libe_sqlite3.so | grep NEEDED
 0x00000001 (NEEDED)                     Shared library: [libc.so.6]

We may need to just check the file path and not use ElfSharp at all.

[mattleibow]

We could just add another rule and see if it came from a nuget and was in the runtimes, then look at the value after the android-.

[jonathanpeppers]

I just haven't gotten around to doing this yet, but I think the plan will be:

• If the directory name is an Android RuntimeIdentifier, so you could put android-arm64/libfoo.so in an app
• If the directory name two levels above is an Android RuntimeIdentifier, so NuGet packages will work.

android-arm64/foo/libbar.so would also work in an app, but I think that would be fine.

[mattleibow]

I am looking at the targets here now and I think this code already works. When the packages restore, there is metadata on the .so files RuntimeIdentifier=android-arm64. There is also the code that detects and converts from this RID to the ABI in the ProcessNativeLibraries task in the _IncludeNativeSystemLibraries target.

However, the files that this task uses is obtained via the _ResolveAssemblies target, which calls msbuild with the _ComputeFilesToPublishForRuntimeIdentifiers target. The result of this operation does not include the RID metadata.

I got a bit of a hack to add it back...

    <Target Name="_IncludeNativeSystemLibrariesFixes" BeforeTargets="_IncludeNativeSystemLibraries">
        <ItemGroup>
            <_FixedResolvedFileToPublish
                Include="@(ResolvedFileToPublish)"
                Condition=" 
                    '%(ResolvedFileToPublish.Extension)' == '.so' and 
                    '%(ResolvedFileToPublish.NuGetPackageId)' != '' and 
                    '%(ResolvedFileToPublish.RuntimeIdentifier)' == '' and 
                    '%(ResolvedFileToPublish.PathInPackage)' != '' " />
            <ResolvedFileToPublish Remove="@(_FixedResolvedFileToPublish)" />
            <ResolvedFileToPublish Include="@(_FixedResolvedFileToPublish)">
                <RuntimeIdentifier>$([System.Text.RegularExpressions.Regex]::Match('%(_FixedResolvedFileToPublish.PathInPackage)', 'runtimes/([^/]+)/native/.*').Groups[1].Value)</RuntimeIdentifier>
            </ResolvedFileToPublish>
        </ItemGroup>
    </Target>

[jonathanpeppers]

The dotnet/sdk creates @(ResolvedFileToPublish), and I've noticed that %(RuntimeIdentifier) is blank when the file is coming from a regular NuGet package.

For some reason, it's there for "runtime" packs that are defined such as:

https://github.com/xamarin/xamarin-android/blob/02ab8f723b6b8e327a3d8a6fd99192598ca94668/build-tools/create-packs/Microsoft.Android.Sdk.proj#L103-L114

Native libraries from Microsoft.Android.Runtime.android-arm has RuntimeIdentifier=android-arm filled out for us.