Make XmlCommentHelper faster

[ninedan]

1. Use StringBuilder only when there are multiple lines
2. In white space normalization, return original string when no change is needed.
3. When changes are really needed, calculate exact size, allocate a char buffer for the changed text, then allocate string.

[codecov]

Codecov Report

Merging #3651 (03341fa) into master (5d98796) will decrease coverage by 0.01%.
The diff coverage is 92.06%.

❗ Current head 03341fa differs from pull request most recent head 589f040. Consider uploading reports for the commit 589f040 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3651      +/-   ##
==========================================
- Coverage   93.26%   93.26%   -0.01%     
==========================================
  Files        1080     1080              
  Lines      113902   113950      +48     
  Branches     4035     4043       +8     
==========================================
+ Hits       106229   106271      +42     
- Misses       6636     6641       +5     
- Partials     1037     1038       +1     

[danmoseley]

just curious, @stephentoub on the latest .NET (of course this is broader targeted) would you expect regex replace using pre generated regex to have comparable performance to open coded?

I didn't measure, but fwiw

        [GeneratedRegex(@"[\s^ ]+")]
        private static partial Regex GetRegex();

generates code in part like

			private bool TryFindNextPossibleStartingPosition(ReadOnlySpan<char> inputSpan)
			{
				int pos = runtextpos;
				if ((uint)pos < (uint)inputSpan.Length)
				{
					ReadOnlySpan<char> span = inputSpan.Slice(pos);
					for (int i = 0; i < span.Length; i++)
					{
						char ch;
						if (((ch = span[i]) < '\u0080') ? ((byte)("㸀\0\u0001\0\0䀀\0\0"[(int)ch >> 4] & (1 << (ch & 0xF))) != 0) : RegexRunner.CharInClass(ch, "\0\u0004\u0001 !^_d"))
						{
							runtextpos = pos + i;
							return true;
						}
					}
				}
				runtextpos = inputSpan.Length;
				return false;

the code below hides some of this inside char.IsWhiteSpace

[danmoseley]

and will return the same string if there are no non-space whitespace chars.

[ninedan]

After addressing the single line issue and regular expression issue, more optimizations could be having much less return.

[stephentoub]

just curious, @stephentoub on the latest .NET (of course this is broader targeted) would you expect regex replace using pre generated regex to have comparable performance to open coded?

Yes.

Note that in .NET 8, the generated code now looks like this:

                private bool TryFindNextPossibleStartingPosition(ReadOnlySpan<char> inputSpan)
                {
                    int pos = base.runtextpos;
                    
                    // Empty matches aren't possible.
                    if ((uint)pos < (uint)inputSpan.Length)
                    {
                        // The pattern begins with a whitespace character.
                        // Find the next occurrence. If it can't be found, there's no match.
                        int i = inputSpan.Slice(pos).IndexOfAnyWhiteSpace();
                        if (i >= 0)
                        {
                            base.runtextpos = pos + i;
                            return true;
                        }
                    }
                    
                    // No match found.
                    base.runtextpos = inputSpan.Length;
                    return false;
                }

where that IndexOfAnyWhitespace is emitted as:

        /// <summary>Finds the next index of any character that matches a whitespace character.</summary>
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        internal static int IndexOfAnyWhiteSpace(this ReadOnlySpan<char> span)
        {
            int i = span.IndexOfAnyExcept(Utilities.s_asciiExceptWhiteSpace);
            if ((uint)i < (uint)span.Length)
            {
                if (char.IsAscii(span[i]))
                {
                    return i;
                }
        
                do
                {
                    if (char.IsWhiteSpace(span[i]))
                    {
                        return i;
                    }
                    i++;
                }
                while ((uint)i < (uint)span.Length);
            }
        
            return -1;
        }
        
        /// <summary>Supports searching for characters in or not in "\0\u0001\u0002\u0003\u0004\u0005\u0006\a\b\u000e\u000f\u0010\u0011\u0012\u0013\u0014\u0015\u0016\u0017\u0018\u0019\u001a\u001b\u001c\u001d\u001e\u001f!\"#$%&amp;'()*+,-./0123456789:;&lt;=&gt;?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~\u007f".</summary>
        internal static readonly IndexOfAnyValues<char> s_asciiExceptWhiteSpace = IndexOfAnyValues.Create("\0\u0001\u0002\u0003\u0004\u0005\u0006\a\b\u000e\u000f\u0010\u0011\u0012\u0013\u0014\u0015\u0016\u0017\u0018\u0019\u001a\u001b\u001c\u001d\u001e\u001f!\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~\u007f");

such that the search for the whitespace is vectorized. Then the matching logic for finding all the contiguous whitespace currently looks like:

                    // Match a whitespace character atomically at least once.
                    {
                        int iteration = 0;
                        while ((uint)iteration < (uint)slice.Length && char.IsWhiteSpace(slice[iteration]))
                        {
                            iteration++;
                        }
                        
                        if (iteration == 0)
                        {
                            return false; // The input didn't match.
                        }
                        
                        slice = slice.Slice(iteration);
                        pos += iteration;
                    }

(though we're flirting with a change that'll result in that loop also changing to similarly be vectorized with IndexOfAnyExcept).

Replace itself also gets better in .NET 8.

[danmoseley]

as you probably know, if this code was targeting .NET Core (or conditionally compiled thus) you could eliminate this char[] allocation by using string.Create( ...)

[ninedan]

Yes, but this project is targeting multiple runtimes.

[ninedan]

Can someone please review the changes? Thanks.

[sharwell]

❔ Can we reasonably omit the normalizeWhitespace parameter, and update callers that passed false to call Append instead?

[sharwell]

❔ Is it common for this line to be hit when lastWhitespace is true and item is pure whitespace?