[Android] Fix issue#11068 of duplicating characters when replacing letters to lowercase or uppercase in TextInput

[fknives]

Summary

These changes are intended to resolve #11068.

Changelog:

[Android] [Fixed] - Fix letters duplication when using autoCapitalize

Test Plan

I took the RewriteExample from TextInputSharedExamples.js duplicated it, updated the labels, attached to the same screen. Modified its onChangeText function, from text = text.replace(/ /g, '_'); to text = text.toLowerCase(); then tested via rn-tester as shown in the video:

• No duplicate characters
• Characters are updated to be lowercase
• Long pressing delete properly deletes, doesn’t stop after deleting one character
• Suggestions (selected from keyboard) work and are updated to lowercase when it becomes part of the input text
• Moving the cursor and typing works, cursor position is kept as it should
• Moving the cursor and deleting works
• Selection portion and deleting it works, cursor position is kept as it should

demo_of_lowercase_no_duplicates.mp4

Note: I have tested manually with 0.67.4, because later versions failed on my machine with cmake and argument errors when building the rn-tester from Android Studio to any device.
Help regarding that would be appreciated.

Possible Future Improvements

As it can be seen the video, the letter duplication is resolved, however since the lowercase modification happens on the Javascript side, it takes a couple milliseconds and the Uppercase can still be shown momentarily while typing.

Technical details, why the solution works

I've debugged a simple AppCompatEditText with calling the same getText().replace in doAfterTextChanged with a bit of delay and noticed a difference to the ReactEditText.

The ReactEditText removes the android.view.inputmethod.ComposingText Span in manageSpans before calling replace (ComposingText is Spanned.SPAN_EXCLUSIVE_EXCLUSIVE).
That ComposingText Span is used in android.view.inputmethod.BaseInputConnection private replaceText to find from what point the text should be replaced from when applying suggestions or typing new letters. Without that Span, it defaults to the selection position, which is usually the end of the text causing duplication of the old "word".

In simple terms, while typing with suggestions on the keyboard, each new letter is handled similarly as clicking a suggestion would be, aka replacing the current "word" with the new "word". (let's say "Ar" word with "Are" word)

Another way to describe it:
While typing with suggestions, with the ComposingText Span the TextView keeps track of what word completions are suggested for on the keyboard UI. When receiving a new letter input, it replaces the current "word" with a new "word", and without the Span, it replaces nothing at the end (selection point) with the new word which results in character duplication.

It also seems to make sense then why without suggestions (like password-visible and secureTextEntry) the issue hasn't occurred.

Examples

How the issue happened:

• User types: A (ComposingText Span becomes (0,1), composingWord: "A")
• Javascript replaced A with a, ComposingText Span was removed from getText()
• User types a new character: r (ComposingText, defaulting to selection, from selection, empty string is replaced with word "Ar")
  => Complete text: aAr => letter duplication.

How it works with the changes applied:

• User types: A (ComposingText Span becomes (0,1), composingWord: "A")
• Javascript replaces A with a, (ComposingText Span (0,1) is re-set after replace)
• User types a new character: r (ComposingText (0,1), "a" word is replaced with word "Ar". ComposingText becomes (0,2) "Ar")
• Javascript replaced Ar with ar, (ComposingText Span (0,2) is re-set after replace)
  => Complete text: ar => no letter duplication
• User selects suggestion "Are" (ComposingText Span (0,2) "ar" is replaced with new word and space "Are ")
• CompleteText: "Are "
• Javascript replaces "Are " with "are " (ComposingText Span doesn't exist, no string after space " ")

Note: the Editable.replace also removes the ComposingText, if it doesn't cover the whole text, that's why we need to re-setSpan them even if we wouldn't remove them in manageSpans.

Note

This is my first attempt to contribute so if I missed something or messed up, please point out and I will try to adapt.

[analysis-bot]

Platform	Engine	Arch	Size (bytes)	Diff
android	hermes	arm64-v8a	9,001,731	+14
android	hermes	armeabi-v7a	8,255,794	+30
android	hermes	x86	9,510,884	+25
android	hermes	x86_64	9,356,745	+17
android	jsc	arm64-v8a	9,614,865	+133
android	jsc	armeabi-v7a	8,741,499	+135
android	jsc	x86	9,701,838	+146
android	jsc	x86_64	9,948,381	+137

Base commit: f071616
Branch: main

[MaeIg]

Note: I have tested manually with 0.67.4, because later versions failed on my machine with cmake and argument errors when building the rn-tester from Android Studio to any device.
Help regarding that would be appriciated.

Hello @fknives,

If you have this kind of error:

You should install cmake '3.22.1' in Android Studio:

It worked for me!

[fknives]

hello @MaeIg,
Thank you for helping out and for the clear instructions, I am not that familiar with CMake to be honest. I have checked the sdk tools and CMake 3.22.1, 3.18.1 and 3.10.2 are all installed.

I get the following error when calling ./gradlew :ReactAndroid:assemble

> Task :ReactAndroid:generateCodegenSchemaFromJavaScript FAILED
/{..}/react-native/node_modules/flow-parser/flow_parser.js:807
throw a}function
^

Error: ENOENT: no such file or directory, lstat 'android'
ENOENT: no such file or directory, lstat 'android'

// later
Node.js v19.4.0

Execution failed for task ':ReactAndroid:generateCodegenSchemaFromJavaScript'.
> Process 'command 'node'' finished with non-zero exit value 7

When trying to build packages.rn-tester.android.app to a device/emulator, the error is same as above (second), plus:

> Task :ReactAndroid:prepareBoost

FAILURE: Build completed with 3 failures.

// first
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':packages:rn-tester:android:app:generateCodegenSchemaFromJavaScript'.
> Process 'command 'node'' finished with non-zero exit value 7

// second is the same as for ./gradlew assemble :ReactAndroid:assemble

// third
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':ReactAndroid:hermes-engine:configureBuildForHermes'.
> A problem occurred starting process 'command 'cmake''

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':ReactAndroid:hermes-engine:configureBuildForHermes'.

Caused by: net.rubygrapefruit.platform.NativeException: Could not start 'cmake'

	at net.rubygrapefruit.platform.internal.DefaultProcessLauncher.start(DefaultProcessLauncher.java:25)
	... 9 more
Caused by: java.io.IOException: error=2, No such file or directory
	at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
	at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:340)
Caused by: java.io.IOException: Cannot run program "cmake" (in directory "{..}/react-native/sdks/hermes"): error=2, No such file or directory

But apparently no issues on CI, so must be just some local config issue on my machine

[fknives]

@MaeIg Sorry, if I am spamming you, I would just like to ask, what suppose to happen now?
I should just wait for someone else to comment or is there something else I suppose to do?

[MaeIg]

@MaeIg Sorry, if I am spamming you, I would just like to ask, what suppose to happen now? I should just wait for someone else to comment or is there something else I suppose to do?

Hello,
Someone from meta should review the PR. When everything is ok, they will import it on Phabricator to do an internal code review and merge it.
So, you just have to wait for now :)

For your problem, I've never had this problem so I don't know how to help you. Sorry :/

[fknives]

^ Had to rebase because of merge conflict

[efstathiosntonas]

Hey folks, is this PR affecting this PR?

[fknives]

Hey folks, is this PR affecting this PR?

Hello, it should have no correlation.

This PR is working with Spans with Spanned.SPAN_COMPOSING flag, while the ReactAbsoluteSizeSpan has no such flag. ReactAbsoluteSizeSpan's flags are setup in SetSpanOperation, which has no SPAN_COMPOSING flag in it.

The SPAN_COMPOSING flags should come only from input methods for temporary styling.

[efstathiosntonas]

@fknives thanks, I've just wanted to bring this to your attention since the other PR is a critical fix. Congrats for fixing this, it's been around for ages.

[fknives]

@fknives thanks, I've just wanted to bring this to your attention since the other PR is a critical fix. Congrats for fixing this, it's been around for ages.

Thank you for pointing me to that PR, it was good idea to have me check out for potential conflicts 👍.

It is only fixed if it gets merged :), but I am hopeful.

One question if I may, the ci/circleci: test_windows seems to have failed on something unrelated (since I modified only Android part, nothing in JS). What should I do in such a case? since I cannot retry it 😅

[efstathiosntonas]

@fknives I have no clue about the ci, I'm not a contributor, sorry. Don't worry about them tho, Meta team will take care of them.

[MarcoScabbiolo]

If this solves #11068 we are seeing history in the making. I'm having goosebumps already and its not even merged.

[fknives]

@lunaleaps 👋 I see you have self-assigned #11068, so I thought maybe you could take a look at this PR? I would greatly appreciate it.
I hope I am not inconveniencing you, I am just a bit unsure about the process and it has been hanging for a while.

[TreeOfLearning]

Astonishing that this issue has existed for at least 7 years, and yet a PR which fixes it has gone unnoticed!

[facebook-github-bot]

@lunaleaps has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[lunaleaps]

@fknives Sorry for the delay and thank you so much for the fix! Importing and testing it

[NickGerleman]

When I read through this at first, the name composite confused me, as meaning multiple spans together (e.g. nested spans or something else).

Could we consistently use composing instead?

[NickGerleman]

A second question, can there be multiple spans? It looks like we iterate through all of them, so maybe this should be named something like shouldRestoreComposingSpans.

[fknives]

Yes, that makes a lot of sense, I should have used composing, I will adjust!

During my testing I haven't seen multiple composing spans, but it's added by auto complete, so I wouldn't take that as granted. So yes, I agree, the naming would make more sense.

I will adjust and reply on this thread once that's done.

[fknives]

@NickGerleman Adjusted naming as suggested, thank you!

[NickGerleman]

#11068 describes an issue where autoCapitalize causes duplicated text. autoCapitalize today at least works by changing Android IME settings, on the component itself on the UI thread.

The repro describes here is around RewriteExample, which happens on the JS side via a controlled component, so a different mechanism.

Are these addressing the same issue?

[NickGerleman]

This means that we only retain composing spans if JS wants to set a new string, and the string is the same length as the one currently held in native.

Is this a heuristic to try to preserve location of the spans? I could imagine some real-world cases where this would bail (e.g. we add a dash in a phone number), where in the problem described, we may still want to preserve the composing span. But I understand this gets a bit hairy.

[fknives]

Yes, it's a heuristic way of preserving the span.
I agree this could be improved to handle more cases, I just tried to keep focus on this specific issue since I am not that familiar with the codebase.

Note that, even if this solution loses the span, the heuristic fails, the code will be working just as it is working currently without the changes introduced

[NickGerleman]

My overall comment here is that the PR may be addressing a different issue than the one described in the linked issue, since controlled text updates, and autoComplete, go through different mechanisms.

Because this PR was made targeting 0.67, there have been quite a few changes to this logic over time. So it would be helpful to understand:

1. Does duplicated text repro for controlled inputs which make modifications?
2. Does this resolve or interact with the text duplication from autoCapitalize

[fknives]

Hello @NickGerleman, first of all, let me thank you for looking into this, I appreciate this a lot!

#11068 describes an issue where autoCapitalize causes duplicated text. autoCapitalize today at least works by changing Android IME settings, on the component itself on the UI thread.

The repro describes here is around RewriteExample, which happens on the JS side via a controlled component, so a different mechanism.

Are these addressing the same issue?

I might misunderstand the linked issue, but as I see it:
The reporter tries to ensure that all the characters are uppercase in an input. They first used autoCapitalize, so the keyboard shows uppercase letters. However, the user is still able to write lowercase letters, by pressing "shift" on the virtual keyboard.
So next they tried to use a controlled text input with toUpperCase, which causes the letter duplication.
=> This is why I think the PR tries to resolve the issue described in the ticket, because they are using controlled input to ensure the user input is uppercase and that causes letter duplication.

Because this PR was made targeting 0.67, there have been quite a few changes to this logic over time. So it would be helpful to understand:

1. Does duplicated text repro for controlled inputs which make modifications?
2. Does this resolve or interact with the text duplication from autoCapitalize

1. Yes.

I created a new project via npx react-native@latest init AwesomeProject ([email protected]).
Added the sample from the linked issue (modified to hooks):

const [text, setText] = useState('');

return <TextInput
          value={text}
          autoCapitalize="characters"
          onChangeText={t => setText(t.toUpperCase())}
  />

And letter duplication still happens if we try to write lowercase letter into this input, and a character after that.

2. As I understand the duplication wasn't caused by autoCapitalize, but by the toUpperCase used in the reported sample.

Some comments regarding this on the linked issue, seems to point to that too:
#11068 (comment)
#11068 (comment)

[NickGerleman]

Thank you for the clarification. I must admit I read the issue title, and not the full description. I think your solution here makes sense with the context.

[fknives]

^ rebased, because a lot has changed on main since the PR was opened

[facebook-github-bot]

@lunaleaps has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[lunaleaps]

Hi @fknives, I'm trying to reproduce the issue on React Native 72 and verify the fix but I'm not able to reproduce the issue.

See #11068 (comment)

I've also tried copying and pasting the Rewrite Example like so:

class RewriteExample extends React.Component<{}, any> {
  constructor(props: {}) {
    super(props);
    this.state = {text: ''};
  }
  render() {
    const limit = 20;
    return (
      <View>
        <TextInput
          testID="rewrite_sp_underscore_input"
          autoCorrect={false}
          multiline={false}
          maxLength={limit}
          onChangeText={text => {
            text = text.toLowerCase();
            this.setState({text});
          }}
          value={this.state.text}
        />
      </View>
    );
  }
}

Have you been reproducing on 0.72? I've also tried with the New Architecture enabled and disabled. Both don't reproduce for me

[fknives]

@lunaleaps Hello, thank you for looking into this.
Yes, I am able to reproduce the issue:

I've created a new react-native project with npx react-native@latest init AwesomeProject ([email protected])

Changed the App.tsx function to:

import React from 'react';
import {TextInput, View} from 'react-native';

function App(): JSX.Element {
  return <RewriteExample />;
}
export default App;

class RewriteExample extends React.Component<{}, any> {
  constructor(props: {}) {
    super(props);
    this.state = {text: ''};
  }
  render() {
    const limit = 20;
    return (
      <View>
        <TextInput
          testID="rewrite_sp_underscore_input"
          autoCorrect={false}
          multiline={false}
          maxLength={limit}
          onChangeText={text => {
            text = text.toLowerCase();
            this.setState({text});
          }}
          value={this.state.text}
        />
      </View>
    );
  }
}

I try to type "Almost"
And as soon as the "l" comes after uppercase "A", the "a" becomes duplicated
Here is a short recording, I hope this helps:

Screen.Recording.2023-07-13.at.06.03.48.mov

Edit:
Additional note for testing with 0.72:
On real device (Samsung A52s, One UI 5.1, Android 13) with Gboard, the issue does not seem to reproduce, while with Samsung Keyboard it does.

[lunaleaps]

Ah I see, I am able to reproduce on the emulator! Okay let me verify the fix

[github-actions]

This pull request was successfully merged by @fknives in ab3c00d.

^{When will my fix make it into a release? | Upcoming Releases}

[lunaleaps]

Hi @fknives,

After merging this, we're getting an exception

java.lang.IndexOutOfBoundsException: setSpan (0 ... 3) ends beyond length 0

which is thrown from here:
https://cs.android.com/android/platform/superproject/main/+/refs/heads/main:frameworks/base/core/java/android/text/SpannableStringBuilder.java;l=1324;drc=3d4951806618b5119ec60a12d80c026bf7ae4153;bpv=0;bpt=1

Here is the relevant stack trace:

- android.text.SpannableStringBuilder.checkRange [SpannableStringBuilder.java:1326]
- android.text.SpannableStringBuilder.setSpan [SpannableStringBuilder.java:685]
- android.text.SpannableStringBuilder.setSpan [SpannableStringBuilder.java:677]
- com.facebook.react.views.textinput.ReactEditText.manageSpans [ReactEditText.java:688]
 [inlined]

I'm wondering if somehow the spannableStringBuilder state gets out of sync. Do you have any ideas? cc @NickGerleman as well. Here is the internal task: T159263581

[NickGerleman]

Looking at how this is called, I think this may be applying a span at the location of a previous string, which may not exist in the new one.

The other usage of these span indices is when the text is the same as previous. Maybe we should use the heuristic here that we use elsewhere in the PR of applying the span when the new text length is the same as before?

[fknives]

@lunaleaps Ohh, wow, I am so sorry, I should have caught that.

I agree with @NickGerleman, I think what I messed up, is that the length check should have been sooner.
Since the Composing spans are only "restored" if the new and old texts are the same length, it does make sense to only "keep" them on the same condition.

It should have been something like:

private void manageSpans(SpannableStringBuilder spannableStringBuilder) {
    boolean shouldKeepComposing = length() == spannableStringBuilder.length();   // new line
    Object[] spans = getText().getSpans(0, length(), Object.class);
    for (int spanIdx = 0; spanIdx < spans.length; spanIdx++) {
      // ...
      // We keep a copy of Composing spans
      if (shouldKeepComposing && isComposing) {    // changed line
        spannableStringBuilder.setSpan(span, spanStart, spanEnd, spanFlags);
        continue;
      }
    // ...

I think that should resolve the IndexOutOfBoundsException or similar issue, because with this check the "new" text has the space to host the Composing Span.
Again, I am sorry I haven't caught this, is there any way I could help?

[lunaleaps]

Got it, let me try to make these changes. I'm not sure I totally get the source of the issue. Let me read up on ComposingSpans

[sunzhaoyuan]

Hi @fknives and @lunaleaps, thanks for investigating this issue. I wonder can we prioritize to fix this issue today as it has a fairly large crash rate and it is blocking our app release for this week? If no action taken, how about we revert this PR to unblock and re-merge it later?

[fknives]

@lunaleaps I see you have reverted the changes, I think that was a good decision considering the issues it caused, again I am sorry I have not caught that.
I know that's not the best introduction of me, however I adjusted the solution and opened a new pull request here. I hope you still have the patience to check it out.
This checks the length before setSpan is called so it won't cause the same issues as this PR did.