MSBuild:UpdateGeneratedFiles takes 40 seconds for a single layout file change

[cosminstirbu]

Hello,

We have a relatively large Xamarin.Android project (it is approx. 2 years old and we have approx. 270 .xml resource files as it can be seen in the attached Diagnostics log).

The issue is that when we update a layout file, 'MSBuild:UpdateGeneratedFiles' takes approximately 40 seconds to run, mainly due to Aapt2Compile task.

Can you please have a look at the build log and investigate if you can make any improvements around this area?

This is my local setup:

Visual Studio Enterprise 2019 for Mac
Version 8.4.2 (build 59)
Installation UUID: b0ff9aab-fcf9-40b6-89ce-1a0b501be149
	GTK+ 2.24.23 (Raleigh theme)
	Xamarin.Mac 5.16.1.25 (issue-7441-d16-3-vsmac / 881172e73)


	Package version: 606000155


Mono Framework MDK
Runtime:
	Mono 6.6.0.155 (2019-08/296a9afdb24) (64-bit)
	Package version: 606000155


Roslyn (Language Service)
3.4.0-beta4-19562-05+ff930dec4565e2bc424ad3bf3e22ecb20542c87d


NuGet
Version: 5.3.0.6192


.NET Core SDK
SDK: /usr/local/share/dotnet/sdk/3.1.101/Sdks
SDK Versions:
	3.1.101
	3.1.100
	3.0.101
	3.0.100
	3.0.100-preview9-014004
	3.0.100-preview8-013656
	2.1.701
	2.1.700
	2.1.505
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/6.6.0/lib/mono/msbuild/Current/bin/Sdks


.NET Core Runtime
Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
	3.1.1
	3.1.0
	3.0.1
	3.0.0
	3.0.0-preview9-19423-09
	2.1.15
	2.1.14
	2.1.13
	2.1.12
	2.1.11
	2.1.9


Xamarin.Profiler
Version: 1.6.13.11
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler


Updater
Version: 11


Apple Developer Tools
Xcode 11.3.1 (15715)
Build 11C505


Xamarin.Mac
Version: 6.10.0.17 (Visual Studio Enterprise)
Hash: 5f802ef53
Branch: xcode11.3
Build date: 2020-01-07 11:53:06-0500


Xamarin.iOS
Version: 13.10.0.17 (Visual Studio Enterprise)
Hash: 5f802ef53
Branch: xcode11.3
Build date: 2020-01-07 11:53:06-0500


Xamarin Designer
Version: 16.4.0.478
Hash: 95f0ab363
Branch: remotes/origin/d16-4
Build date: 2020-01-08 23:59:46 UTC


Xamarin.Android
Version: 10.1.3.7 (Visual Studio Enterprise)
Commit: xamarin-android/d16-4/d66aed0
Android SDK: /Users/cosminstirbu/Library/Developer/Xamarin/android-sdk-macosx
	Supported Android versions:
		5.1 (API level 22)
		7.1 (API level 25)
		8.1 (API level 27)


SDK Tools Version: 26.1.1
SDK Platform Tools Version: 29.0.5
SDK Build Tools Version: 29.0.2


Build Information: 
Mono: fd9f379
Java.Interop: xamarin/java.interop/d16-4@c4e569f
ProGuard: xamarin/proguard/master@905836d
SQLite: xamarin/sqlite/3.28.0@46204c4
Xamarin.Android Tools: xamarin/xamarin-android-tools/master@9f4ed4b


Microsoft Mobile OpenJDK
Java SDK: /Users/cosminstirbu/Library/Developer/Xamarin/jdk/microsoft_dist_openjdk_1.8.0.25
1.8.0-25
Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL


Android SDK Manager
Version: 16.4.0.9
Hash: 3f7256f
Branch: remotes/origin/d16-4
Build date: 2020-01-14 22:19:04 UTC


Android Device Manager
Version: 16.4.0.30
Hash: f9172e2
Branch: remotes/origin/d16-4
Build date: 2020-01-14 22:19:24 UTC


Xamarin Inspector
Version: 1.4.3
Hash: db27525
Branch: 1.4-release
Build date: Mon, 09 Jul 2018 21:20:18 GMT
Client compatibility: 1


Build Information
Release ID: 804020059
Git revision: eb6fcdd83a227678e487aa733df3c8745f54fafc
Build date: 2020-01-17 12:12:02+00
Build branch: release-8.4
Xamarin extensions: ec32c90978c94f03d72f507b01f5aee70952ce87


Operating System
Mac OS X 10.14.5
Darwin 18.6.0 Darwin Kernel Version 18.6.0
    Thu Apr 25 23:16:27 PDT 2019
    root:xnu-4903.261.4~2/RELEASE_X86_64 x86_64


Enabled user installed extensions
DeepClean 1.2.5
Straight8's SpecFlow Integration 1.11.3.0
FileNesting 0.1.2
StyleCop Support 1.1.0.4
NuGet Package Management Extensions 0.20

P.S. - since the build log contains sensitive information, please use this ticket to download it: https://developercommunity.visualstudio.com/content/problem/896179/android-msbuildupdategeneratedfiles-takes-40-secon.html

If you don't have access to that ticket, please give me an email address that I can use to share the log.

Thanks,
Cosmin

[grendello]

@cosminstirbu at the first sight, it seems your issue might be similar to #4095, can you check if the information provided there helps you?

[cosminstirbu]

I had a look at that issue which suggests that there might be some missing AndroidResource file in .csproj.

However, if I search the log of MSBuild:UpdateGeneratedFiles after making a layout change (which when saved triggers MSBuild:UpdateGeneratedFiles automatically) I can't find any occurrences "similar" to

Building target "_CompileResources" completely.
Input file "Resources\layout\noexist.xml" does not exist.

Just to make it as clear as possible - the issue happens when I update a layout file and save the change (which triggers "Updating resources ..." MSBuild:UpdateGeneratedFiles).

[cjrvdev]

I feel you.. It is a real pain compared to how gradle works for example. I feel like this is not something that can be fixed for larger projects unless they change the tool..

[cosminstirbu]

@grendello - any news on this issue? I know you guys are busy with plenty of stuff but I feel that build performance (and other developer productivity areas) should be high priority.

Thanks,
Cosmin

[grendello]

@cosminstirbu I'm sorry, we really have a lot on our hands right now, but it's not an excuse to not take care of all the issues. Can you attach a binlog of the build which takes such a long time for you? (build with msbuild /bl from command line and attach the msbuild.binlog file which will be produced by the command). This will help our msbuild folks to see all the details of the build process and hopefully figure out what's going on. Thanks!

[cosminstirbu]

I've attached a (private) verbose tex log in the initial ticket.

If you don't have access to it, then please give me an email address where I can send you the verbose text log as it contains private information.

Also please notice that I'm not invoking an actual build action - I'm simply editing a layout .xml file and UpdateGeneratedFiles is automatically invoked - so I'm not sure exactly how to obtain a binary log giving that information - I'd probably have to invoke UpdateGeneratedFiles myself with some parameters from the command line.

I hope the text log is good enough, if not, if you can provide some guidance on how I can invoke UpdateGeneratedFiles from the command line, I'll try to get the binary log.

[dellis1972]

This issue will be fixed via #4190

One of the problems was we were not using the incremental capabilities of aapt2 but building a flata archive. This meant we had to build the entire archive from scratch each time. PR #4190 uses a .flat file for each resource, allowing us to partially build the _CompileResources target. Which means we will only need to compile the flat file for the resources that change, not the entire resource tree.

[cosminstirbu]

Thank you for your updates. Looking forward to see this fix in a near future stable version of Xamarin.Android.

[brendanzagaeski]

Release status update

A new Preview version has now been published that includes the fix for this item. The fix is not yet included in a Release version. I will update this again when a Release version is available that includes the fix.

Fix included in Xamarin.Android 10.3.99.230.

Fix included on Windows in Visual Studio 2019 version 16.7 Preview 1. To try the Preview version that includes the fix, check for the latest updates in Visual Studio Preview.

Fix included on macOS in Visual Studio 2019 for Mac version 8.7 Preview 1. To try the Preview version that includes the fix, check for the latest updates on the Preview updater channel.

[brendanzagaeski]

Release status update

A new Release version of Xamarin.Android has now been published that includes the fix for this issue.

Fix included in Xamarin.Android SDK version 11.0.0.3.

Fix included on Windows in Visual Studio 2019 version 16.7. To get the new version that includes the fix, check for the latest updates or install the most recent release from https://visualstudio.microsoft.com/downloads/.

Fix included on macOS in Visual Studio 2019 for Mac version 8.7. To get the new version that includes the fix, check for the latest updates on the Stable updater channel.