Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get Linux build working. #9

Merged
merged 1 commit into from
Apr 27, 2016
Merged

Get Linux build working. #9

merged 1 commit into from
Apr 27, 2016

Conversation

atsushieno
Copy link
Contributor

You know what happened if I tried to use property to just change file
extensions? Property references in item metadata like this didn't work...

You know what happened if I tried to use property to just change file
extensions? Property references in item metadata like this didn't work...
@jonpryor jonpryor merged commit 323f432 into dotnet:master Apr 27, 2016
radical pushed a commit that referenced this pull request May 8, 2018
When `JniRuntime.CreationOptions.DestroyRuntimeOnDispose` is true,
`JavaVM::DestroyJavaVM()` will be invoked when the `JniRuntime`
instance is disposed *or* finalized.

`JreRuntime.CreateJreVM()` would *always* set
`DestroyRuntimeOnDispose` to true, because it called
`JNI_CreateJavaVM()`, so *of course* you'd want to destroy the
Java VM, right?

Which brings us to unit tests. I don't know of any "before all test
fixtures run" and "after all test fixtures run" extension points,
which means:

1. The JVM needs to be created implicitly, "on demand."
2. There's no good way to destroy the JVM created in (1) after all
    tests have finished executing.

Which *really* means that the `JreRuntime` instance is *finalized*,
which sets us up for the unholy trifecta of AppDomain unloads,
finalizers, and JVM shutdown:

For unknown reasons, ~randomly, when running the unit tests (e.g.
`make run-tests`), the test runner will *hang*, indefinitely.

Attaching `lldb` and triggering a backtrace shows the unholy trifecta:

Finalization:

	thread #4: tid = 0x403831, 0x00007fff9656bdb6 libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'tid_1403'
	  ...
	  frame #10: 0x00000001001ccb4a mono64`mono_gc_run_finalize(obj=<unavailable>, data=<unavailable>) + 938 at gc.c:256 [opt]
	  frame #11: 0x00000001001cdd4a mono64`finalizer_thread [inlined] finalize_domain_objects + 51 at gc.c:681 [opt]
	  frame #12: 0x00000001001cdd17 mono64`finalizer_thread(unused=<unavailable>) + 295 at gc.c:730 [opt]

JVM destruction:

	thread #4: tid = 0x403831, 0x00007fff9656bdb6 libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'tid_1403'
	  frame #0: 0x00007fff9656bdb6 libsystem_kernel.dylib`__psynch_cvwait + 10
	  frame #1: 0x00007fffa04d4728 libsystem_pthread.dylib`_pthread_cond_wait + 767
	  frame #2: 0x000000010ba5bc76 libjvm.dylib`os::PlatformEvent::park() + 192
	  frame #3: 0x000000010ba38e32 libjvm.dylib`ParkCommon(ParkEvent*, long) + 42
	  frame #4: 0x000000010ba39708 libjvm.dylib`Monitor::IWait(Thread*, long) + 168
	  frame #5: 0x000000010ba398f0 libjvm.dylib`Monitor::wait(bool, long, bool) + 246
	  frame #6: 0x000000010bb3dca2 libjvm.dylib`Threads::destroy_vm() + 80
	  frame #7: 0x000000010b8fd665 libjvm.dylib`jni_DestroyJavaVM + 254

AppDomain unload:

	thread #37: tid = 0x4038fb, 0x00007fff9656bdb6 libsystem_kernel.dylib`__psynch_cvwait + 10
	  frame #0: 0x00007fff9656bdb6 libsystem_kernel.dylib`__psynch_cvwait + 10
	  frame #1: 0x00007fffa04d4728 libsystem_pthread.dylib`_pthread_cond_wait + 767
	  frame #2: 0x0000000100234a7f mono64`mono_os_cond_timedwait [inlined] mono_os_cond_wait(cond=0x0000000102016e50, mutex=0x0000000102016e10) + 11 at mono-os-mutex.h:105 [opt]
	  frame #3: 0x0000000100234a74 mono64`mono_os_cond_timedwait(cond=0x0000000102016e50, mutex=0x0000000102016e10, timeout_ms=<unavailable>) + 164 at mono-os-mutex.h:120 [opt]
	  frame #4: 0x0000000100234828 mono64`_wapi_handle_timedwait_signal_handle(handle=0x0000000000000440, timeout=4294967295, alertable=1, poll=<unavailable>, alerted=0x0000700000a286f4) + 536 at handles.c:1554 [opt]
	  frame #5: 0x0000000100246370 mono64`wapi_WaitForSingleObjectEx(handle=<unavailable>, timeout=<unavailable>, alertable=<unavailable>) + 592 at wait.c:189 [opt]
	  frame #6: 0x00000001001c832e mono64`mono_domain_try_unload [inlined] guarded_wait(timeout=4294967295, alertable=1) + 30 at appdomain.c:2390 [opt]
	  frame #7: 0x00000001001c8310 mono64`mono_domain_try_unload(domain=0x000000010127ccb0, exc=0x0000700000a287a0) + 416 at appdomain.c:2482 [opt]
	  frame #8: 0x00000001001c7db2 mono64`ves_icall_System_AppDomain_InternalUnload [inlined] mono_domain_unload(domain=<unavailable>) + 20 at appdomain.c:2379 [opt]
	  frame #9: 0x00000001001c7d9e mono64`ves_icall_System_AppDomain_InternalUnload(domain_id=<unavailable>) + 46 at appdomain.c:2039 [opt]

This randomly results in deadlock, and hung Jenkins bots.

Fix this behavior by altering `JreRuntime.CreateJreVM()` to *not*
override the value of
`JniRuntime.CreationOptions.DestroyRuntimeOnDispose`. This allows the
constructor of the `JreRuntime` instance to decide whether or not the
JVM is destroyed.

In the case of TestJVM, we *don't* want to destroy the JVM.

This prevents the JVM from being destroyed, which in turn prevents the
hang during process shutdown.
grendello added a commit to grendello/xamarin-android that referenced this pull request Jul 18, 2018
Xamarin.Android uses the ILRepack
task (https://www.nuget.org/packages/ILRepack.Lib.MSBuild.Task/) to repack/merge
the `Xamarin.Android.Build.Tasks` assembly, a step that is performed after the
`Xamarin.Android.Build.Tasks` project build completes. The step takes the
project's output assembly and several other dependencies and merges them into
one, replacing the original output assembly. For the majority of time it works
flawlessly, however the scenario described below causes a Mono runtime
crash (similar to this one mono/mono#9613, although
not identical).

The crash causes aren't yet know in every detail, but the high level overview of
the issue is as follows. Mono runtime uses the `mmap(2)` system call on Unix to
map the on-disk image of the assembly into memory. The mmapped area is used
whenever the same assembly is loaded (e.g. via `Assembly.LoadFrom`) into the
application to save time and memory. The memory is mapped as private which means
that anything written to the mapped area by the Mono process is effectively
discarded, that is not reflected in the file on disk. The protection, however,
doesn't work in the other direction - when an external process (or even the same
Mono process) writes to the file that was mmapped. Unfortunately, the OS kernel
behavior in such instance is left undefined by the POSIX standard - the kernel
can choose to reflect the on-disk change in the mapping or ignore it. On macOS
the write is ignored, on Linux, however, the write leads either to the SIGBUS
signal being sent to the process or corruption to the mmapped memory
area (details of how this happens are, as of yet, unknown - it's just a theory
at this point). The problem occurs in our case when the `ILRepacker` task
overwrites the `Xamarin.Android.Build.Tasks.dll` assembly with its
merged/repackaged form and that, somehow, corrupts Mono runtime's memory mapping
of that assembly, eventually leading to a segfault:

  #0  0x00007f974fd1b23a in __waitpid (pid=pid@entry=120561, stat_loc=stat_loc@entry=0x7f9712162b9c, options=options@entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
  #1  0x000055b471f36c96 in dump_native_stacktrace (ctx=0x7f97121637c0, signal=0x55b472147cd3 "SIGSEGV") at mini-posix.c:719
  #2  0x000055b471f36df2 in mono_dump_native_crash_info (signal=signal@entry=0x55b472147cd3 "SIGSEGV", ctx=ctx@entry=0x7f97121637c0, info=info@entry=0x7f97121638f0) at mini-posix.c:742
  #3  0x000055b471ecc620 in mono_handle_native_crash (signal=signal@entry=0x55b472147cd3 "SIGSEGV", ctx=ctx@entry=0x7f97121637c0, info=info@entry=0x7f97121638f0) at mini-exceptions.c:2938
  dotnet#4  0x000055b471e4c82c in mono_sigsegv_signal_handler (_dummy=7, _info=0x7f97121638f0, context=0x7f97121637c0) at mini-runtime.c:3441
  dotnet#5  <signal handler called>
  dotnet#6  __strcasecmp_l_avx () at ../sysdeps/x86_64/multiarch/strcmp-sse42.S:199
  dotnet#7  0x000055b471fb2105 in mono_assembly_names_equal_flags (l=l@entry=0x7f96e0004370, r=r@entry=0x7f97307edb60, flags=flags@entry=MONO_ANAME_EQ_IGNORE_CASE) at assembly.c:663
  dotnet#8  0x000055b471fa7baf in mono_domain_assembly_search (aname=0x7f96e0004370, user_data=0x0) at appdomain.c:2242
  dotnet#9  0x000055b471fb10f7 in mono_assembly_invoke_search_hook_internal (aname=aname@entry=0x7f96e0004370, requesting=requesting@entry=0x0, refonly=0, postload=postload@entry=0) at assembly.c:1755
  dotnet#10 0x000055b471fb4526 in mono_assembly_load_from_predicate (image=image@entry=0x7f96e001aae0, fname=fname@entry=0x7f96e000bc50 "/home/grendel/vc/xamarin/monodroid/monodroid/external/xamarin-android/src/Xamarin.Android.Build.Tasks/../../packages/ILRepack.Lib.MSBuild.Task.2.0.15.4/build/ILRepack.Lib.MSBuild.Task.dll", asmctx=asmctx@entry=MONO_ASMCTX_LOADFROM, predicate=predicate@entry=0x0, user_data=user_data@entry=0x0, status=status@entry=0x7f9712163fd4) at assembly.c:2638
  dotnet#11 0x000055b471fb635c in mono_assembly_open_predicate (filename=<optimized out>, filename@entry=0x7f96e000ae40 "/home/grendel/vc/xamarin/monodroid/monodroid/external/xamarin-android/src/Xamarin.Android.Build.Tasks/../../packages/ILRepack.Lib.MSBuild.Task.2.0.15.4/build/ILRepack.Lib.MSBuild.Task.dll", asmctx=MONO_ASMCTX_LOADFROM, predicate=predicate@entry=0x0, user_data=user_data@entry=0x0, status=status@entry=0x7f9712163fd4) at assembly.c:2206
  dotnet#12 0x000055b471fabd42 in ves_icall_System_Reflection_Assembly_LoadFrom (fname=..., refOnly=<optimized out>, error=0x7f9712164050) at appdomain.c:2272
  dotnet#13 0x00000000412f4007 in ?? ()
  dotnet#14 0x00007f97440e1918 in ?? ()
  dotnet#15 0x00007f9749bd1800 in ?? ()
  dotnet#16 0x00000000e36e75af in ?? ()
  dotnet#17 0x00007f974434fda8 in ?? ()
  dotnet#18 0x00007f9749bd18e0 in ?? ()
  dotnet#19 0x00007f96e0002820 in ?? ()
  dotnet#20 0x00007f9712164160 in ?? ()
  dotnet#21 0x00007f9712164010 in ?? ()
  dotnet#22 0x0000000000000000 in ?? ()

The crash occurs in frame 7, because the `r` parameter (an instance of the
`MonoAssembly` structure) contains pointers that used to point to valid places
in the `Xamarin.Android.Build.Tasks` assembbly's mmaped area, but now point to
garbage data. One of those corrupted pointers is the ASCII assembly name - which
is used by the function in frame 7.

The situation can be detected by the runtime by using advisory locks and
reporting the attempt to overwrite the mmapped area, however a fix would require
creating a copy of all the assembly's in-memory data before the write is allowed
and that might be too expensive. None of those measures are currently
implemented by Mono (and it isn't sure if any of them will ever be implemented).

The crash this commit fixes occurs in a very specific scenario - when building
the `OpenTK.csproj` project *directly* (that is passing it as parameter to
`msbuild`), which causes a the `Xamarin.Android.Build.Tasks` assembly to be
loaded for use in the `<UsingTask/>` msbuild statement, but at the same time
`OpenTK.csproj` references the `Xamarin.Android.Build.Tasks` *project* which
causes the ILRepacker task to run unconditionally, attempting to repack the
assemblies again and leading to the crash described above.

The fix, courtesy of @jonpryor, is to make the ILRepacker target not run every
time but only whenver the assembly has to be re-packed because it has just been
rebuilt.
grendello added a commit to grendello/xamarin-android that referenced this pull request Jul 19, 2018
Xamarin.Android uses the ILRepack
task (https://www.nuget.org/packages/ILRepack.Lib.MSBuild.Task/) to repack/merge
the `Xamarin.Android.Build.Tasks` assembly, a step that is performed after the
`Xamarin.Android.Build.Tasks` project build completes. The step takes the
project's output assembly and several other dependencies and merges them into
one, replacing the original output assembly. For the majority of time it works
flawlessly, however the scenario described below causes a Mono runtime
crash (similar to this one mono/mono#9613, although
not identical).

The crash causes aren't yet know in every detail, but the high level overview of
the issue is as follows. Mono runtime uses the `mmap(2)` system call on Unix to
map the on-disk image of the assembly into memory. The mmapped area is used
whenever the same assembly is loaded (e.g. via `Assembly.LoadFrom`) into the
application to save time and memory. The memory is mapped as private which means
that anything written to the mapped area by the Mono process is effectively
discarded, that is not reflected in the file on disk. The protection, however,
doesn't work in the other direction - when an external process (or even the same
Mono process) writes to the file that was mmapped. Unfortunately, the OS kernel
behavior in such instance is left undefined by the POSIX standard - the kernel
can choose to reflect the on-disk change in the mapping or ignore it. On macOS
the write is ignored, on Linux, however, the write leads either to the SIGBUS
signal being sent to the process or corruption to the mmapped memory
area (details of how this happens are, as of yet, unknown - it's just a theory
at this point). The problem occurs in our case when the `ILRepacker` task
overwrites the `Xamarin.Android.Build.Tasks.dll` assembly with its
merged/repackaged form and that, somehow, corrupts Mono runtime's memory mapping
of that assembly, eventually leading to a segfault:

  #0  0x00007f974fd1b23a in __waitpid (pid=pid@entry=120561, stat_loc=stat_loc@entry=0x7f9712162b9c, options=options@entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
  #1  0x000055b471f36c96 in dump_native_stacktrace (ctx=0x7f97121637c0, signal=0x55b472147cd3 "SIGSEGV") at mini-posix.c:719
  #2  0x000055b471f36df2 in mono_dump_native_crash_info (signal=signal@entry=0x55b472147cd3 "SIGSEGV", ctx=ctx@entry=0x7f97121637c0, info=info@entry=0x7f97121638f0) at mini-posix.c:742
  #3  0x000055b471ecc620 in mono_handle_native_crash (signal=signal@entry=0x55b472147cd3 "SIGSEGV", ctx=ctx@entry=0x7f97121637c0, info=info@entry=0x7f97121638f0) at mini-exceptions.c:2938
  dotnet#4  0x000055b471e4c82c in mono_sigsegv_signal_handler (_dummy=7, _info=0x7f97121638f0, context=0x7f97121637c0) at mini-runtime.c:3441
  dotnet#5  <signal handler called>
  dotnet#6  __strcasecmp_l_avx () at ../sysdeps/x86_64/multiarch/strcmp-sse42.S:199
  dotnet#7  0x000055b471fb2105 in mono_assembly_names_equal_flags (l=l@entry=0x7f96e0004370, r=r@entry=0x7f97307edb60, flags=flags@entry=MONO_ANAME_EQ_IGNORE_CASE) at assembly.c:663
  dotnet#8  0x000055b471fa7baf in mono_domain_assembly_search (aname=0x7f96e0004370, user_data=0x0) at appdomain.c:2242
  dotnet#9  0x000055b471fb10f7 in mono_assembly_invoke_search_hook_internal (aname=aname@entry=0x7f96e0004370, requesting=requesting@entry=0x0, refonly=0, postload=postload@entry=0) at assembly.c:1755
  dotnet#10 0x000055b471fb4526 in mono_assembly_load_from_predicate (image=image@entry=0x7f96e001aae0, fname=fname@entry=0x7f96e000bc50 "/home/grendel/vc/xamarin/monodroid/monodroid/external/xamarin-android/src/Xamarin.Android.Build.Tasks/../../packages/ILRepack.Lib.MSBuild.Task.2.0.15.4/build/ILRepack.Lib.MSBuild.Task.dll", asmctx=asmctx@entry=MONO_ASMCTX_LOADFROM, predicate=predicate@entry=0x0, user_data=user_data@entry=0x0, status=status@entry=0x7f9712163fd4) at assembly.c:2638
  dotnet#11 0x000055b471fb635c in mono_assembly_open_predicate (filename=<optimized out>, filename@entry=0x7f96e000ae40 "/home/grendel/vc/xamarin/monodroid/monodroid/external/xamarin-android/src/Xamarin.Android.Build.Tasks/../../packages/ILRepack.Lib.MSBuild.Task.2.0.15.4/build/ILRepack.Lib.MSBuild.Task.dll", asmctx=MONO_ASMCTX_LOADFROM, predicate=predicate@entry=0x0, user_data=user_data@entry=0x0, status=status@entry=0x7f9712163fd4) at assembly.c:2206
  dotnet#12 0x000055b471fabd42 in ves_icall_System_Reflection_Assembly_LoadFrom (fname=..., refOnly=<optimized out>, error=0x7f9712164050) at appdomain.c:2272
  dotnet#13 0x00000000412f4007 in ?? ()
  dotnet#14 0x00007f97440e1918 in ?? ()
  dotnet#15 0x00007f9749bd1800 in ?? ()
  dotnet#16 0x00000000e36e75af in ?? ()
  dotnet#17 0x00007f974434fda8 in ?? ()
  dotnet#18 0x00007f9749bd18e0 in ?? ()
  dotnet#19 0x00007f96e0002820 in ?? ()
  dotnet#20 0x00007f9712164160 in ?? ()
  dotnet#21 0x00007f9712164010 in ?? ()
  dotnet#22 0x0000000000000000 in ?? ()

The accompanying managed stacktrace looks as follows:

  Stacktrace:

  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) System.Reflection.Assembly.LoadFrom (string,bool) [0x0001b] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Reflection.Assembly.LoadFrom (string) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/corlib/System.Reflection/Assembly.cs:539
  at System.Reflection.Assembly.UnsafeLoadFrom (string) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/corlib/System.Reflection/Assembly.cs:571
  at Microsoft.Build.Shared.TypeLoader.LoadAssembly (Microsoft.Build.Shared.AssemblyLoadInfo) [0x0001e] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Shared.TypeLoader/AssemblyInfoToLoadedTypes.ScanAssemblyForPublicTypes () [0x00007] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Shared.TypeLoader/AssemblyInfoToLoadedTypes/<>c__DisplayClass8_0.<GetLoadedTypeByTypeName>b__0 (string) [0x000b4] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Collections.Concurrent.ConcurrentDictionary`2<TKey_REF, TValue_REF>.GetOrAdd (TKey_REF,System.Func`2<TKey_REF, TValue_REF>) [0x00034] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.Shared.TypeLoader/AssemblyInfoToLoadedTypes.GetLoadedTypeByTypeName (string) [0x0003c] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Shared.TypeLoader.GetLoadedType (System.Collections.Concurrent.ConcurrentDictionary`2<System.Func`3<System.Type, object, bool>, System.Collections.Concurrent.ConcurrentDictionary`2<Microsoft.Build.Shared.AssemblyLoadInfo, Microsoft.Build.Shared.TypeLoader/AssemblyInfoToLoadedTypes>>,string,Microsoft.Build.Shared.AssemblyLoadInfo) [0x00042] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Shared.TypeLoader.Load (string,Microsoft.Build.Shared.AssemblyLoadInfo) [0x00008] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.AssemblyTaskFactory.InitializeFactory (Microsoft.Build.Shared.AssemblyLoadInfo,string,System.Collections.Generic.IDictionary`2<string, Microsoft.Build.Framework.TaskPropertyInfo>,string,System.Collections.Generic.IDictionary`2<string, string>,bool,Microsoft.Build.BackEnd.Logging.TargetLoggingContext,Microsoft.Build.Construction.ElementLocation,string) [0x00058] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Execution.TaskRegistry/RegisteredTaskRecord.GetTaskFactory (Microsoft.Build.BackEnd.Logging.TargetLoggingContext,Microsoft.Build.Construction.ElementLocation,string) [0x000b8] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Execution.TaskRegistry/RegisteredTaskRecord.CanTaskBeCreatedByFactory (string,string,System.Collections.Generic.IDictionary`2<string, string>,Microsoft.Build.BackEnd.Logging.TargetLoggingContext,Microsoft.Build.Construction.ElementLocation) [0x0003b] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Execution.TaskRegistry.GetMatchingRegistration (string,System.Collections.Generic.List`1<Microsoft.Build.Execution.TaskRegistry/RegisteredTaskRecord>,string,System.Collections.Generic.IDictionary`2<string, string>,Microsoft.Build.BackEnd.Logging.TargetLoggingContext,Microsoft.Build.Construction.ElementLocation) [0x0001a] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Execution.TaskRegistry.GetTaskRegistrationRecord (string,string,System.Collections.Generic.IDictionary`2<string, string>,bool,Microsoft.Build.BackEnd.Logging.TargetLoggingContext,Microsoft.Build.Construction.ElementLocation,bool&) [0x00169] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Execution.TaskRegistry.GetRegisteredTask (string,string,System.Collections.Generic.IDictionary`2<string, string>,bool,Microsoft.Build.BackEnd.Logging.TargetLoggingContext,Microsoft.Build.Construction.ElementLocation) [0x00010] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.TaskExecutionHost.FindTaskInRegistry (System.Collections.Generic.IDictionary`2<string, string>) [0x00038] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.FindTask (System.Collections.Generic.IDictionary`2<string, string>) [0x0000b] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.TaskBuilder/<ExecuteBucket>d__19.MoveNext () [0x0011e] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<Microsoft.Build.BackEnd.WorkUnitResult>.Start<Microsoft.Build.BackEnd.TaskBuilder/<ExecuteBucket>d__19> (Microsoft.Build.BackEnd.TaskBuilder/<ExecuteBucket>d__19&) [0x0002c] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:471
  at Microsoft.Build.BackEnd.TaskBuilder.ExecuteBucket (Microsoft.Build.BackEnd.TaskHost,Microsoft.Build.BackEnd.ItemBucket,Microsoft.Build.BackEnd.TaskExecutionMode,System.Collections.Generic.Dictionary`2<string, string>) [0x00048] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.TaskBuilder/<ExecuteTask>d__18.MoveNext () [0x0016a] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<Microsoft.Build.BackEnd.WorkUnitResult>.Start<Microsoft.Build.BackEnd.TaskBuilder/<ExecuteTask>d__18> (Microsoft.Build.BackEnd.TaskBuilder/<ExecuteTask>d__18&) [0x0002c] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:471
  at Microsoft.Build.BackEnd.TaskBuilder.ExecuteTask (Microsoft.Build.BackEnd.TaskExecutionMode,Microsoft.Build.BackEnd.Lookup) [0x00037] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.TaskBuilder/<ExecuteTask>d__13.MoveNext () [0x00193] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<Microsoft.Build.BackEnd.WorkUnitResult>.Start<Microsoft.Build.BackEnd.TaskBuilder/<ExecuteTask>d__13> (Microsoft.Build.BackEnd.TaskBuilder/<ExecuteTask>d__13&) [0x0002c] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:471
  at Microsoft.Build.BackEnd.TaskBuilder.ExecuteTask (Microsoft.Build.BackEnd.Logging.TargetLoggingContext,Microsoft.Build.BackEnd.BuildRequestEntry,Microsoft.Build.BackEnd.ITargetBuilderCallback,Microsoft.Build.Execution.ProjectTargetInstanceChild,Microsoft.Build.BackEnd.TaskExecutionMode,Microsoft.Build.BackEnd.Lookup,Microsoft.Build.BackEnd.Lookup,System.Threading.CancellationToken) [0x0006c] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.TargetEntry/<ProcessBucket>d__52.MoveNext () [0x00091] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<Microsoft.Build.BackEnd.WorkUnitResult>.Start<Microsoft.Build.BackEnd.TargetEntry/<ProcessBucket>d__52> (Microsoft.Build.BackEnd.TargetEntry/<ProcessBucket>d__52&) [0x0002c] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:471
  at Microsoft.Build.BackEnd.TargetEntry.ProcessBucket (Microsoft.Build.BackEnd.ITaskBuilder,Microsoft.Build.BackEnd.Logging.TargetLoggingContext,Microsoft.Build.BackEnd.TaskExecutionMode,Microsoft.Build.BackEnd.Lookup,Microsoft.Build.BackEnd.Lookup) [0x00051] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.TargetEntry/<ExecuteTarget>d__45.MoveNext () [0x002bb] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start<Microsoft.Build.BackEnd.TargetEntry/<ExecuteTarget>d__45> (Microsoft.Build.BackEnd.TargetEntry/<ExecuteTarget>d__45&) [0x0002c] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:316
  at Microsoft.Build.BackEnd.TargetEntry.ExecuteTarget (Microsoft.Build.BackEnd.ITaskBuilder,Microsoft.Build.BackEnd.BuildRequestEntry,Microsoft.Build.BackEnd.Logging.ProjectLoggingContext,System.Threading.CancellationToken) [0x00048] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.TargetBuilder/<ProcessTargetStack>d__21.MoveNext () [0x003a5] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<System.Threading.Tasks.VoidTaskResult>.TrySetResult (System.Threading.Tasks.VoidTaskResult) [0x0004f] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:433
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<System.Threading.Tasks.VoidTaskResult>.SetResult (System.Threading.Tasks.VoidTaskResult) [0x0003d] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:608
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<System.Threading.Tasks.VoidTaskResult>.SetResult (System.Threading.Tasks.Task`1<System.Threading.Tasks.VoidTaskResult>) [0x00010] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:636
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.SetResult () [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:378
  at Microsoft.Build.BackEnd.TargetEntry/<ExecuteTarget>d__45.MoveNext () [0x00720] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<TResult_REF>.TrySetResult (TResult_REF) [0x0004f] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<TResult_REF>.SetResult (TResult_REF) [0x0003d] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.BackEnd.TargetEntry/<ProcessBucket>d__52.MoveNext () [0x001d5] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<TResult_REF>.TrySetResult (TResult_REF) [0x0004f] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<TResult_REF>.SetResult (TResult_REF) [0x0003d] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.BackEnd.TaskBuilder/<ExecuteTask>d__13.MoveNext () [0x0021e] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<TResult_REF>.TrySetResult (TResult_REF) [0x0004f] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<TResult_REF>.SetResult (TResult_REF) [0x0003d] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.BackEnd.TaskBuilder/<ExecuteTask>d__18.MoveNext () [0x002be] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<TResult_REF>.TrySetResult (TResult_REF) [0x0004f] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<TResult_REF>.SetResult (TResult_REF) [0x0003d] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.BackEnd.TaskBuilder/<ExecuteBucket>d__19.MoveNext () [0x003e5] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<TResult_REF>.TrySetResult (TResult_REF) [0x0004f] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<TResult_REF>.SetResult (TResult_REF) [0x0003d] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.BackEnd.TaskBuilder/<InitializeAndExecuteTask>d__24.MoveNext () [0x0012d] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<TResult_REF>.TrySetResult (TResult_REF) [0x0004f] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<TResult_REF>.SetResult (TResult_REF) [0x0003d] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.BackEnd.TaskBuilder/<ExecuteInstantiatedTask>d__26.MoveNext () [0x0065e] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<bool>.TrySetResult (bool) [0x0004f] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:433
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<bool>.SetResult (bool) [0x0003d] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:608
  at Microsoft.Build.BackEnd.MSBuild/<ExecuteInternal>d__76.MoveNext () [0x004dd] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<bool>.TrySetResult (bool) [0x0004f] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:433
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<bool>.SetResult (bool) [0x0003d] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:608
  at Microsoft.Build.BackEnd.MSBuild/<BuildProjectsInParallel>d__77.MoveNext () [0x0014a] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<bool>.TrySetResult (bool) [0x0004f] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:433
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<bool>.SetResult (bool) [0x0003d] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:608
  at Microsoft.Build.BackEnd.MSBuild/<ExecuteTargets>d__80.MoveNext () [0x00713] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<Microsoft.Build.Framework.BuildEngineResult>.TrySetResult (Microsoft.Build.Framework.BuildEngineResult) [0x0004f] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:433
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<Microsoft.Build.Framework.BuildEngineResult>.SetResult (Microsoft.Build.Framework.BuildEngineResult) [0x0003d] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:608
  at Microsoft.Build.BackEnd.TaskHost/<InternalBuildProjects>d__49.MoveNext () [0x0025c] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<Microsoft.Build.Framework.BuildEngineResult>.TrySetResult (Microsoft.Build.Framework.BuildEngineResult) [0x0004f] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:433
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<Microsoft.Build.Framework.BuildEngineResult>.SetResult (Microsoft.Build.Framework.BuildEngineResult) [0x0003d] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:608
  at Microsoft.Build.BackEnd.TaskHost/<BuildProjectFilesInParallelAsync>d__53.MoveNext () [0x004fb] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<TResult_REF>.TrySetResult (TResult_REF) [0x0004f] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<TResult_REF>.SetResult (TResult_REF) [0x0003d] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.BackEnd.RequestBuilder/<BuildProjects>d__33.MoveNext () [0x00263] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<TResult_REF>.TrySetResult (TResult_REF) [0x0004f] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<TResult_REF>.SetResult (TResult_REF) [0x0003d] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.BackEnd.RequestBuilder/<StartNewBuildRequests>d__51.MoveNext () [0x0020e] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<int>.TrySetResult (int) [0x0004f] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:433
  at System.Threading.Tasks.TaskCompletionSource`1<int>.TrySetResult (int) [0x00000] in /home/grendel/vc/mono/mono/external/corefx/src/Common/src/CoreLib/System/Threading/Tasks/TaskCompletionSource.cs:274
  at Microsoft.Build.Shared.AwaitExtensions/<>c__DisplayClass7_2.<ToTask>b__0 (object,bool) [0x00038] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Threading.RegisteredWaitHandle.DoCallBack (object) [0x00008] in /home/grendel/vc/mono/mono/mcs/class/corlib/System.Threading/RegisteredWaitHandle.cs:103
  at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context (object) [0x0000d] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:1308
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x00021] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:1285
  at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00074] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:858
  at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:1213
  at (wrapper runtime-invoke) <Module>.runtime_invoke_bool (object,intptr,intptr,intptr) [0x0001f] in <79002cfb0e6e431d8465d6dcea08995e>:0

The crash occurs in frame 7, because the `r` parameter (an instance of the
`MonoAssembly` structure) contains pointers that used to point to valid places
in the `Xamarin.Android.Build.Tasks` assembbly's mmaped area, but now point to
garbage data. One of those corrupted pointers is the ASCII assembly name - which
is used by the function in frame 7.

The situation can be detected by the runtime by using advisory locks and
reporting the attempt to overwrite the mmapped area, however a fix would require
creating a copy of all the assembly's in-memory data before the write is allowed
and that might be too expensive. None of those measures are currently
implemented by Mono (and it isn't sure if any of them will ever be implemented).

The crash this commit fixes occurs in a very specific scenario - when building
the `OpenTK.csproj` project *directly* (that is passing it as parameter to
`msbuild`), which causes a the `Xamarin.Android.Build.Tasks` assembly to be
loaded for use in the `<UsingTask/>` msbuild statement, but at the same time
`OpenTK.csproj` references the `Xamarin.Android.Build.Tasks` *project* which
causes the ILRepacker task to run unconditionally, attempting to repack the
assemblies again and leading to the crash described above.

The fix, courtesy of @jonpryor, is to make the ILRepacker target not run every
time but only whenver the assembly has to be re-packed because it has just been
rebuilt.
jonpryor pushed a commit that referenced this pull request Jul 19, 2018
Xamarin.Android uses the [ILRepack task][0] to repack/merge the
`Xamarin.Android.Build.Tasks.dll` assembly (610ade7), a step that is
performed after the `Xamarin.Android.Build.Tasks.csproj` project's
`Build` target completes.  The step takes the project's output
assembly -- `Xamarin.Android.Build.Tasks.dll` -- and several other
dependencies, then merges them into one, *replacing* the original
output assembly.

The majority of time it works flawlessly, however when building the
`monodroid` repo we observe a Mono runtime crash, similar-but-not-
identical to [mono issue 9613][m9613].  Not all details are
understood, but the high level overview of the issue is as follows:
Mono runtime uses the **mmap**(2) system call on Unix to map the
on-disk image of the assembly into memory.  The `mmapp`ed area is
used whenever the same assembly is loaded (e.g.
`Assembly.LoadFrom()`) into the application to save time and memory.
The memory is mapped as private which means that anything written to
the mapped area by the Mono process is effectively discarded, that is
not reflected in the file on disk.  The protection, however, doesn't
work in the other direction: when an external process (or even the
same Mono process!) writes to the file that was `mmapp`ed.
Unfortunately, the OS kernel behavior in such instance is left
undefined by the POSIX standard: the kernel can choose to reflect the
on-disk change in the mapping or ignore it.  On macOS the write is
ignored; on Linux, the write leads either to the `SIGBUS` signal
being sent to the process or corruption to the `mmap`ped memory area
(details of how this happens are, as of yet, unknown; it's just a
theory at this point).  The problem occurs in our case when the
`<ILRepacker/>` task overwrites the `Xamarin.Android.Build.Tasks.dll`
assembly with its merged/repackaged form and that, somehow, corrupts
Mono runtime's memory mapping of that assembly, eventually crashing:

        ## native crash
        #0  0x00007f974fd1b23a in __waitpid (pid=pid@entry=120561, stat_loc=stat_loc@entry=0x7f9712162b9c, options=options@entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
        #1  0x000055b471f36c96 in dump_native_stacktrace (ctx=0x7f97121637c0, signal=0x55b472147cd3 "SIGSEGV") at mini-posix.c:719
        #2  0x000055b471f36df2 in mono_dump_native_crash_info (signal=signal@entry=0x55b472147cd3 "SIGSEGV", ctx=ctx@entry=0x7f97121637c0, info=info@entry=0x7f97121638f0) at mini-posix.c:742
        #3  0x000055b471ecc620 in mono_handle_native_crash (signal=signal@entry=0x55b472147cd3 "SIGSEGV", ctx=ctx@entry=0x7f97121637c0, info=info@entry=0x7f97121638f0) at mini-exceptions.c:2938
        #4  0x000055b471e4c82c in mono_sigsegv_signal_handler (_dummy=7, _info=0x7f97121638f0, context=0x7f97121637c0) at mini-runtime.c:3441
        #5  <signal handler called>
        #6  __strcasecmp_l_avx () at ../sysdeps/x86_64/multiarch/strcmp-sse42.S:199
        #7  0x000055b471fb2105 in mono_assembly_names_equal_flags (l=l@entry=0x7f96e0004370, r=r@entry=0x7f97307edb60, flags=flags@entry=MONO_ANAME_EQ_IGNORE_CASE) at assembly.c:663
        #8  0x000055b471fa7baf in mono_domain_assembly_search (aname=0x7f96e0004370, user_data=0x0) at appdomain.c:2242
        #9  0x000055b471fb10f7 in mono_assembly_invoke_search_hook_internal (aname=aname@entry=0x7f96e0004370, requesting=requesting@entry=0x0, refonly=0, postload=postload@entry=0) at assembly.c:1755
        #10 0x000055b471fb4526 in mono_assembly_load_from_predicate (image=image@entry=0x7f96e001aae0, fname=fname@entry=0x7f96e000bc50 "/home/grendel/vc/xamarin/monodroid/monodroid/external/xamarin-android/src/Xamarin.Android.Build.Tasks/../../packages/ILRepack.Lib.MSBuild.Task.2.0.15.4/build/ILRepack.Lib.MSBuild.Task.dll", asmctx=asmctx@entry=MONO_ASMCTX_LOADFROM, predicate=predicate@entry=0x0, user_data=user_data@entry=0x0, status=status@entry=0x7f9712163fd4) at assembly.c:2638
        #11 0x000055b471fb635c in mono_assembly_open_predicate (filename=<optimized out>, filename@entry=0x7f96e000ae40 "/home/grendel/vc/xamarin/monodroid/monodroid/external/xamarin-android/src/Xamarin.Android.Build.Tasks/../../packages/ILRepack.Lib.MSBuild.Task.2.0.15.4/build/ILRepack.Lib.MSBuild.Task.dll", asmctx=MONO_ASMCTX_LOADFROM, predicate=predicate@entry=0x0, user_data=user_data@entry=0x0, status=status@entry=0x7f9712163fd4) at assembly.c:2206
        #12 0x000055b471fabd42 in ves_icall_System_Reflection_Assembly_LoadFrom (fname=..., refOnly=<optimized out>, error=0x7f9712164050) at appdomain.c:2272
        ...

        ## managed stacktrace
        at <unknown> <0xffffffff>
        at (wrapper managed-to-native) System.Reflection.Assembly.LoadFrom (string,bool) [0x0001b] in <79002cfb0e6e431d8465d6dcea08995e>:0
        at System.Reflection.Assembly.LoadFrom (string) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/corlib/System.Reflection/Assembly.cs:539
        at System.Reflection.Assembly.UnsafeLoadFrom (string) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/corlib/System.Reflection/Assembly.cs:571
        at Microsoft.Build.Shared.TypeLoader.LoadAssembly (Microsoft.Build.Shared.AssemblyLoadInfo) [0x0001e] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
        ...

The crash occurs in frame 7, because the `r` parameter (an instance
of the `MonoAssembly` structure) contains pointers that used to point
to valid places in the `Xamarin.Android.Build.Tasks.dll`s `mmap`ed
area, but now point to garbage data.  One of those corrupted pointers
is the ASCII assembly name - which is used by the function in frame 7.

The crash this commit fixes occurs in a very specific scenario: when
building the `OpenTK.csproj` project *directly*
(e.g. `msbuild OpenTK.csproj`), which causes the
`Xamarin.Android.Build.Tasks.dll` assembly to be loaded for use in
the `<UsingTask/>` MSBuild statement, but at the same time 
`OpenTK.csproj` has a `@(ProjectReference)` to
`Xamarin.Android.Build.Tasks.csproj`, which caused the
`<ILRepacker/>` task to run unconditionally, attempting to repack the
assemblies again and leading to the crash described above.

The fix, courtesy of @jonpryor, is to make the `ILRepacker` target
not run every time but only whenever the assembly has to be re-packed
because it has just been rebuilt.

This also improves rebuild times.

[0]: https://www.nuget.org/packages/ILRepack.Lib.MSBuild.Task
[m9613]: mono/mono#9613
jonpryor pushed a commit that referenced this pull request Aug 3, 2018
Xamarin.Android uses the [ILRepack task][0] to repack/merge the
`Xamarin.Android.Build.Tasks.dll` assembly (610ade7), a step that is
performed after the `Xamarin.Android.Build.Tasks.csproj` project's
`Build` target completes.  The step takes the project's output
assembly -- `Xamarin.Android.Build.Tasks.dll` -- and several other
dependencies, then merges them into one, *replacing* the original
output assembly.

The majority of time it works flawlessly, however when building the
`monodroid` repo we observe a Mono runtime crash, similar-but-not-
identical to [mono issue 9613][m9613].  Not all details are
understood, but the high level overview of the issue is as follows:
Mono runtime uses the **mmap**(2) system call on Unix to map the
on-disk image of the assembly into memory.  The `mmapp`ed area is
used whenever the same assembly is loaded (e.g.
`Assembly.LoadFrom()`) into the application to save time and memory.
The memory is mapped as private which means that anything written to
the mapped area by the Mono process is effectively discarded, that is
not reflected in the file on disk.  The protection, however, doesn't
work in the other direction: when an external process (or even the
same Mono process!) writes to the file that was `mmapp`ed.
Unfortunately, the OS kernel behavior in such instance is left
undefined by the POSIX standard: the kernel can choose to reflect the
on-disk change in the mapping or ignore it.  On macOS the write is
ignored; on Linux, the write leads either to the `SIGBUS` signal
being sent to the process or corruption to the `mmap`ped memory area
(details of how this happens are, as of yet, unknown; it's just a
theory at this point).  The problem occurs in our case when the
`<ILRepacker/>` task overwrites the `Xamarin.Android.Build.Tasks.dll`
assembly with its merged/repackaged form and that, somehow, corrupts
Mono runtime's memory mapping of that assembly, eventually crashing:

        ## native crash
        #0  0x00007f974fd1b23a in __waitpid (pid=pid@entry=120561, stat_loc=stat_loc@entry=0x7f9712162b9c, options=options@entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
        #1  0x000055b471f36c96 in dump_native_stacktrace (ctx=0x7f97121637c0, signal=0x55b472147cd3 "SIGSEGV") at mini-posix.c:719
        #2  0x000055b471f36df2 in mono_dump_native_crash_info (signal=signal@entry=0x55b472147cd3 "SIGSEGV", ctx=ctx@entry=0x7f97121637c0, info=info@entry=0x7f97121638f0) at mini-posix.c:742
        #3  0x000055b471ecc620 in mono_handle_native_crash (signal=signal@entry=0x55b472147cd3 "SIGSEGV", ctx=ctx@entry=0x7f97121637c0, info=info@entry=0x7f97121638f0) at mini-exceptions.c:2938
        #4  0x000055b471e4c82c in mono_sigsegv_signal_handler (_dummy=7, _info=0x7f97121638f0, context=0x7f97121637c0) at mini-runtime.c:3441
        #5  <signal handler called>
        #6  __strcasecmp_l_avx () at ../sysdeps/x86_64/multiarch/strcmp-sse42.S:199
        #7  0x000055b471fb2105 in mono_assembly_names_equal_flags (l=l@entry=0x7f96e0004370, r=r@entry=0x7f97307edb60, flags=flags@entry=MONO_ANAME_EQ_IGNORE_CASE) at assembly.c:663
        #8  0x000055b471fa7baf in mono_domain_assembly_search (aname=0x7f96e0004370, user_data=0x0) at appdomain.c:2242
        #9  0x000055b471fb10f7 in mono_assembly_invoke_search_hook_internal (aname=aname@entry=0x7f96e0004370, requesting=requesting@entry=0x0, refonly=0, postload=postload@entry=0) at assembly.c:1755
        #10 0x000055b471fb4526 in mono_assembly_load_from_predicate (image=image@entry=0x7f96e001aae0, fname=fname@entry=0x7f96e000bc50 "/home/grendel/vc/xamarin/monodroid/monodroid/external/xamarin-android/src/Xamarin.Android.Build.Tasks/../../packages/ILRepack.Lib.MSBuild.Task.2.0.15.4/build/ILRepack.Lib.MSBuild.Task.dll", asmctx=asmctx@entry=MONO_ASMCTX_LOADFROM, predicate=predicate@entry=0x0, user_data=user_data@entry=0x0, status=status@entry=0x7f9712163fd4) at assembly.c:2638
        #11 0x000055b471fb635c in mono_assembly_open_predicate (filename=<optimized out>, filename@entry=0x7f96e000ae40 "/home/grendel/vc/xamarin/monodroid/monodroid/external/xamarin-android/src/Xamarin.Android.Build.Tasks/../../packages/ILRepack.Lib.MSBuild.Task.2.0.15.4/build/ILRepack.Lib.MSBuild.Task.dll", asmctx=MONO_ASMCTX_LOADFROM, predicate=predicate@entry=0x0, user_data=user_data@entry=0x0, status=status@entry=0x7f9712163fd4) at assembly.c:2206
        #12 0x000055b471fabd42 in ves_icall_System_Reflection_Assembly_LoadFrom (fname=..., refOnly=<optimized out>, error=0x7f9712164050) at appdomain.c:2272
        ...

        ## managed stacktrace
        at <unknown> <0xffffffff>
        at (wrapper managed-to-native) System.Reflection.Assembly.LoadFrom (string,bool) [0x0001b] in <79002cfb0e6e431d8465d6dcea08995e>:0
        at System.Reflection.Assembly.LoadFrom (string) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/corlib/System.Reflection/Assembly.cs:539
        at System.Reflection.Assembly.UnsafeLoadFrom (string) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/corlib/System.Reflection/Assembly.cs:571
        at Microsoft.Build.Shared.TypeLoader.LoadAssembly (Microsoft.Build.Shared.AssemblyLoadInfo) [0x0001e] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
        ...

The crash occurs in frame 7, because the `r` parameter (an instance
of the `MonoAssembly` structure) contains pointers that used to point
to valid places in the `Xamarin.Android.Build.Tasks.dll`s `mmap`ed
area, but now point to garbage data.  One of those corrupted pointers
is the ASCII assembly name - which is used by the function in frame 7.

The crash this commit fixes occurs in a very specific scenario: when
building the `OpenTK.csproj` project *directly*
(e.g. `msbuild OpenTK.csproj`), which causes the
`Xamarin.Android.Build.Tasks.dll` assembly to be loaded for use in
the `<UsingTask/>` MSBuild statement, but at the same time 
`OpenTK.csproj` has a `@(ProjectReference)` to
`Xamarin.Android.Build.Tasks.csproj`, which caused the
`<ILRepacker/>` task to run unconditionally, attempting to repack the
assemblies again and leading to the crash described above.

The fix, courtesy of @jonpryor, is to make the `ILRepacker` target
not run every time but only whenever the assembly has to be re-packed
because it has just been rebuilt.

This also improves rebuild times.

[0]: https://www.nuget.org/packages/ILRepack.Lib.MSBuild.Task
[m9613]: mono/mono#9613
lewurm added a commit to lewurm/xamarin-android that referenced this pull request Nov 22, 2018
grendello added a commit to grendello/xamarin-android that referenced this pull request Nov 22, 2022
It won't fix the failure, but extra logging might be useful at some
point. The current failure is:

    droid.NET_Test: java_vm_ext.cc:570] JNI DETECTED ERROR IN APPLICATION: use of invalid jobject 0x7100941bb830
    backtrace:
           #00 pc 00000000000943f8  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+24) (BuildId: a08a19770d6696739c847e29c3f5f650)
           #1 pc 0000000000097146  /apex/com.android.runtime/lib64/bionic/libc.so (abort+182) (BuildId: a08a19770d6696739c847e29c3f5f650)
           #2 pc 000000000055321f  /apex/com.android.runtime/lib64/libart.so (art::Runtime::Abort(char const*)+2399) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           #3 pc 000000000000c873  /system/lib64/libbase.so (android::base::LogMessage::~LogMessage()+611) (BuildId: 40d2b536dbf0730fdc31abd2b469f94f)
           dotnet#4 pc 00000000003ede64  /apex/com.android.runtime/lib64/libart.so (art::JavaVMExt::JniAbort(char const*, char const*)+1604) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#5 pc 00000000003ee16a  /apex/com.android.runtime/lib64/libart.so (art::JavaVMExt::JniAbortF(char const*, char const*, ...)+234) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#6 pc 00000000005adf7b  /apex/com.android.runtime/lib64/libart.so (art::Thread::DecodeJObject(_jobject*) const+875) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#7 pc 00000000003def7b  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::ScopedCheck::CheckInstance(art::ScopedObjectAccess&, art::(anonymous namespace)::ScopedCheck::InstanceKind, _jobject*, bool)+91) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#8 pc 00000000003de1e5  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::ScopedCheck::CheckPossibleHeapValue(art::ScopedObjectAccess&, char, art::(anonymous namespace)::JniValueType)+485) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#9 pc 00000000003de2d4  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::ScopedCheck::CheckPossibleHeapValue(art::ScopedObjectAccess&, char, art::(anonymous namespace)::JniValueType)+724) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#10 pc 00000000003dd712  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::ScopedCheck::Check(art::ScopedObjectAccess&, bool, char const*, art::(anonymous namespace)::JniValueType*)+690) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#11 pc 00000000003e28c0  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::CheckJNI::CheckCallArgs(art::ScopedObjectAccess&, art::(anonymous namespace)::ScopedCheck&, _JNIEnv*, _jobject*, _jclass*, _jmethodID*, art::InvokeType, art::(anonymous namespace)::VarArgs const*)+160) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#12 pc 00000000003e1a9e  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::CheckJNI::CallMethodV(char const*, _JNIEnv*, _jobject*, _jclass*, _jmethodID*, __va_list_tag*, art::Primitive::Type, art::InvokeType)+910) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#13 pc 00000000003cf551  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::CheckJNI::CallObjectMethod(_JNIEnv*, _jobject*, _jmethodID*, ...)+177) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#14 pc 00000000000013ec  /data/app/Mono.Android.NET_Tests--O9vgexkYeCx3nX-AuvLTQ==/split_config.x86_64.apk!libreuse-threads.so (offset 0xa8d000) (BuildId: 562d86d81ebdd3bb6b7528e2a9235ff84827294e)
           dotnet#15 pc 0000000000100fce  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+30) (BuildId: a08a19770d6696739c847e29c3f5f650)
           dotnet#16 pc 0000000000098fe7  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+55) (BuildId: a08a19770d6696739c847e29c3f5f650)
jonpryor pushed a commit that referenced this pull request Feb 28, 2023
…7759)

Fixes? #6067

Context: https://gist.github.com/grendello/55b9a74c80c611dd48b726ee6f16d3c9
Context: dotnet/android-libzipsharp#125

Context: #7622

Changes: http://github.com/xamarin/monodroid/compare/50faac94c6a0c27864564829ac83f3988c82f8ef...602aca98245744882a129206b79b5a5e093dae44

  * xamarin/monodroid@602aca982: Bump androidtools for new LibZipSharp (xamarin/monodroid#1287)
  * xamarin/monodroid@1f52d5873: [tools/msbuild] Deploy *resources.dll from PackageReference (xamarin/monodroid#1276)

Changes: xamarin/androidtools@f11d163...c0bcb66

  * xamarin/androidtools@c0bcb66: Bump xamarin-android-tools for new LibZipSharp (xamarin/androidtools#377)

Changes: dotnet/android-tools@099fd95...dbe42bf

  * dotnet/android-tools@dbe42bf: Bump LibZipSharp version (dotnet/android-tools#204)

Sometimes in our tests we see an error from LibZipSharp which claims
a file cannot be opened as a valid ZIP archive:

	error ANDZA0000: Unable to open '…/example.apk' as zip archive

The reason for this is ZIP format corruption happening sometimes in
the `<BuildApk/>` task.

The way the process works is that we first run `aapt2` to produce
an APK archive named `packaged_resources`, which we then copy to the
destination APK (using standard `File.Copy()`) and open it to append
our content.  The first item we always append is the `classes.dex`
file which is also the **only** corrupted entry in the broken APK
files.

ZIP files contain two records describing each entry in the archive:

 1. in the Central Directory (located at the end of the file)

 2. local header in the ZIP data stream, offset of which is contained
    in the above Central Directory record.

After close examination, it appears that the broken APK files contain
(1) for `classes.dex` but not (2).  Instead of (2) we see a *copy*
of `packaged_resource` package's Central Directory record:

	% zipinfo -vv example.apk
	…
	Central directory entry #9:
	---------------------------

	  There are an extra 2 bytes preceding this file.

	  classes.dex

	  offset of local header from start of archive:   21680
	…

In particular, "There are an extra 2 bytes preceding this file" is a
sign that something "isn't right".  If we use `hexdump` to look at
the 128 bytes starting at offset 21680:

	% hexdump -C -s 21680 -n 128 example.apk 
	000054b0  50 4b 01 02 00 00 00 00  00 00 08 00 00 00 21 00  |PK............!.|
	000054c0  5e b2 00 5e cc 03 00 00  5c 0b 00 00 13 00 00 00  |^..^....\.......|
	000054d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 41 6e  |..............An|
	000054e0  64 72 6f 69 64 4d 61 6e  69 66 65 73 74 2e 78 6d  |droidManifest.xm|
	000054f0  6c 50 4b 01 02 00 00 00  00 00 00 00 00 00 00 21  |lPK............!|
	00005500  00 5c 71 d9 26 d2 05 00  00 d2 05 00 00 1d 00 00  |.\q.&...........|
	00005510  00 00 00 00 00 00 00 00  00 00 00 fd 03 00 00 72  |...............r|
	00005520  65 73 2f 64 72 61 77 61  62 6c 65 2d 6d 64 70 69  |es/drawable-mdpi|

we see that it starts with "PK", the common magic signature for zip
archives, and the following bytes { 0x1 0x2 } indicate that this is a
*central directory* entry, while it *should* be { 0x3, 0x4 } for a
*local header* entry.

The corruption may be caused by invalid stream position after we open
the `packaged_resources` copy and append `classes.dex` entry to it;
see dotnet/android-libzipsharp#125 for details.

We believe that the changes to `ZipArchive.stream_callback()` in
dotnet/android-libzipsharp#125 alongside additional calls to `Flush()` after
adding `classex.dex` **may** fix the problem.

If it doesn't, we should consider copying data from
`packaged_resources` to the final APK entry by entry by decompressing
them from the original archive and adding them to the destination one.
jonpryor added a commit that referenced this pull request Jan 26, 2024
Context: dotnet/java-interop@6b3637d

The story so far is that some of our unit tests are crashing, and
have been in one form or another since 4332ea0
(the bump to dotnet/java-interop@def5bc0).

The current crash, from the PR build for 7b46391:

	D monodroid-assembly: typemap: assembly 'Java.Interop-Tests' hasn't been loaded yet, attempting a full load
	W monodroid-assembly: typemap: failed to load managed assembly 'Java.Interop-Tests.dll'. No such file or directory
	E monodroid-assembly: typemap: unable to load assembly 'Java.Interop-Tests' when looking up managed type corresponding to Java type 'java/lang/Object'
	I monodroid-timing: [1/5] Typemap.java_to_managed: end, total time; elapsed: 0:0::260000
	W monodroid-assembly: typemap: called from
	W monodroid-assembly: at Java.Interop.TypeManager.GetJavaToManagedType(String )
	W monodroid-assembly:    at Java.Interop.TypeManager.CreateInstance(IntPtr , JniHandleOwnership , Type )
	W monodroid-assembly:    at Java.Lang.Object.GetObject(IntPtr , JniHandleOwnership , Type )
	W monodroid-assembly:    at Java.Lang.Object._GetObject[IIterator](IntPtr , JniHandleOwnership )
	W monodroid-assembly:    at Java.Lang.Object.GetObject[IIterator](IntPtr handle, JniHandleOwnership transfer)
	W monodroid-assembly:    at Android.Runtime.JavaSet.Iterator()
	W monodroid-assembly:    at Android.Runtime.JavaSet`1[[System.String, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].GetEnumerator()
	W monodroid-assembly:    at Xamarin.Android.UnitTests.TestInstrumentation`1[[Xamarin.Android.UnitTests.NUnit.NUnitTestRunner, TestRunner.NUnit.NET, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].ProcessArguments()
	W monodroid-assembly:    at Xamarin.Android.UnitTests.TestInstrumentation`1[[Xamarin.Android.UnitTests.NUnit.NUnitTestRunner, TestRunner.NUnit.NET, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].OnCreate(Bundle arguments)
	W monodroid-assembly:    at Android.App.Instrumentation.n_OnCreate_Landroid_os_Bundle_(IntPtr jnienv, IntPtr native__this, IntPtr native_arguments)
	W monodroid-assembly:    at Android.Runtime.JNINativeWrapper.Wrap_JniMarshal_PPL_V(_JniMarshal_PPL_V callback, IntPtr jnienv, IntPtr klazz, IntPtr p0)
	E droid.NET_Test: JNI ERROR (app bug): accessed stale Local 0x75  (index 7 in a table of size 7)
	F droid.NET_Test: java_vm_ext.cc:570] JNI DETECTED ERROR IN APPLICATION: use of deleted local reference 0x75

There are *three* "concerning" items here:

 1. typemaps are trying to load `Java.Interop-Tests`, and failing:

        typemap: failed to load managed assembly 'Java.Interop-Tests.dll'. No such file or directory

    @grendello is looking into this.

 2. The binding for `java/lang/Object` is coming from
    Java.Interop-Tests, not Mono.Android (?!)

        typemap: unable to load assembly 'Java.Interop-Tests' when looking up managed type corresponding to Java type 'java/lang/Object'

    dotnet/java-interop#1181 has a fix for this, and we're not
    applying the fix yet because we believe that it will hide (1).

 3. The JNI error, which crashes the process:

        F droid.NET_Test: java_vm_ext.cc:570] JNI DETECTED ERROR IN APPLICATION: use of deleted local reference 0x75
        F droid.NET_Test: java_vm_ext.cc:570]     from void crc643df67da7b13bb6b1.TestInstrumentation_1.n_onCreate(android.os.Bundle)
        F droid.NET_Test: runtime.cc:630] Runtime aborting...
        F droid.NET_Test: runtime.cc:630] Dumping all threads without mutator lock held
        F droid.NET_Test: runtime.cc:630] All threads:
        F droid.NET_Test: runtime.cc:630] DALVIK THREADS (14):
        F droid.NET_Test: runtime.cc:630] "main" prio=5 tid=1 Runnable
        F droid.NET_Test: runtime.cc:630]   | group="" sCount=0 dsCount=0 flags=0 obj=0x729e9d98 self=0x7567e0f51000
        F droid.NET_Test: runtime.cc:630]   | sysTid=9143 nice=0 cgrp=default sched=0/0 handle=0x7567e14daed8
        F droid.NET_Test: runtime.cc:630]   | state=R schedstat=( 1270418000 334229000 139 ) utm=16 stm=110 core=0 HZ=100
        F droid.NET_Test: runtime.cc:630]   | stack=0x7ffcbb3e4000-0x7ffcbb3e6000 stackSize=8192KB
        F droid.NET_Test: runtime.cc:630]   | held mutexes= "abort lock" "mutator lock"(shared held)
        F droid.NET_Test: runtime.cc:630]   native: #00 pc 000000000048df4e  /apex/com.android.runtime/lib64/libart.so (art::DumpNativeStack(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, int, BacktraceMap*, char const*, art::ArtMethod*, void*, bool)+126)
        F droid.NET_Test: runtime.cc:630]   native: #1 pc 00000000005a77c3  /apex/com.android.runtime/lib64/libart.so (art::Thread::DumpStack(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, bool, BacktraceMap*, bool) const+675)
        F droid.NET_Test: runtime.cc:630]   native: #2 pc 00000000005c49cb  /apex/com.android.runtime/lib64/libart.so (art::DumpCheckpoint::Run(art::Thread*)+859)
        F droid.NET_Test: runtime.cc:630]   native: #3 pc 00000000005bcf28  /apex/com.android.runtime/lib64/libart.so (art::ThreadList::RunCheckpoint(art::Closure*, art::Closure*)+456)
        F droid.NET_Test: runtime.cc:630]   native: #4 pc 00000000005bc2e1  /apex/com.android.runtime/lib64/libart.so (art::ThreadList::Dump(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, bool)+1601)
        F droid.NET_Test: runtime.cc:630]   native: #5 pc 0000000000552eb9  /apex/com.android.runtime/lib64/libart.so (art::Runtime::Abort(char const*)+1529)
        F droid.NET_Test: runtime.cc:630]   native: #6 pc 000000000000c873  /system/lib64/libbase.so (android::base::LogMessage::~LogMessage()+611)
        F droid.NET_Test: runtime.cc:630]   native: #7 pc 00000000003ede84  /apex/com.android.runtime/lib64/libart.so (art::JavaVMExt::JniAbort(char const*, char const*)+1604)
        F droid.NET_Test: runtime.cc:630]   native: #8 pc 00000000003ee18a  /apex/com.android.runtime/lib64/libart.so (art::JavaVMExt::JniAbortF(char const*, char const*, ...)+234)
        F droid.NET_Test: runtime.cc:630]   native: #9 pc 00000000005adf31  /apex/com.android.runtime/lib64/libart.so (art::Thread::DecodeJObject(_jobject*) const+785)
        F droid.NET_Test: runtime.cc:630]   native: #10 pc 00000000003def9b  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::ScopedCheck::CheckInstance(art::ScopedObjectAccess&, art::(anonymous namespace)::ScopedCheck::InstanceKind, _jobject*, bool)+91)
        F droid.NET_Test: runtime.cc:630]   native: #11 pc 00000000003de205  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::ScopedCheck::CheckPossibleHeapValue(art::ScopedObjectAccess&, char, art::(anonymous namespace)::JniValueType)+485)
        F droid.NET_Test: runtime.cc:630]   native: #12 pc 00000000003dd732  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::ScopedCheck::Check(art::ScopedObjectAccess&, bool, char const*, art::(anonymous namespace)::JniValueType*)+690)
        F droid.NET_Test: runtime.cc:630]   native: #13 pc 00000000003ce865  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::CheckJNI::GetObjectClass(_JNIEnv*, _jobject*)+837)
        F droid.NET_Test: runtime.cc:630]   native: #14 pc 0000000000017196  /data/app/Mono.Android.NET_Tests-LUUW792fOvX745oAS4jeRQ==/split_config.x86_64.apk (offset 331000) (???)
        F droid.NET_Test: runtime.cc:630]   at crc643df67da7b13bb6b1.TestInstrumentation_1.n_onCreate(Native method)

As per `native #13`, we're (somehow) passing an invalid JNI reference
to `JNIEnv::GetObjectClass()`.

***HOW*** are we passing an invalid JNI reference to
`JNIEnv::GetObjectClass()`?

Attempt to investigate (3) further, by:

 1. Reviewing all calls to `JNIEnv::GetObjectClass()` within this
    repo to see if we could potentially be passing an invalid value.
    The "most obvious" candidate is `TypeManager.CreateInstance()`,
    which calls `JNIEnv::GetObjectClass()` in a loop.

    I'm still not sure if that could possibly be the cause, but
    Just In Case™…

    "Cleanup" some C++ code so that calls to
    `JNIEnv::DeleteLocalReference()` are closer to the
    `JNIEnv::GetObjectClass()` calls.

 2. Update
    `tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj`
    to add an `@(AndroidEnvironment)` item, which sets
    `debug.mono.log=gref+,lref+`.  This will enable LREF and GREF
    logging within `adb logcat`, which *may* allow us to track down
    where "local reference 0x75" came from.
@github-actions github-actions bot locked and limited conversation to collaborators Feb 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants