Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Greedy context before in LB20a #988

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Conversation

eggrobin
Copy link
Member

@eggrobin eggrobin commented Dec 13, 2024

Alternatives are left-to-right, not greedy: In the string
<U+2E09 LEFT TRANSPOSITION BRACKET, U+2E0D RIGHT RAISED OMISSION BRACKET, U+1F3FC EMOJI MODIFIER FITZPATRICK TYPE-3>,
Searching for ( ^ | [^[\p{ea=F}\p{ea=W}\p{ea=H}]] ) \p{Line_Break=Quotation} will find a match consisting of the first character, whereas searching for ( [^[\p{ea=F}\p{ea=W}\p{ea=H}]] | ^ ) \p{Line_Break=Quotation} will find a match consisting of the first two characters.

ICU depends on a maximal match in its generated old monkeys: https://github.com/unicode-org/icu/blob/02951053b45f270c04ffeac6e0787dd2313bbd23/icu4c/source/test/intltest/rbbitst.cpp#L1779. This is messy to solve in general, but our alternatives here are simple enough that we can make them greedy by moving sot at the end.
Specifically, this works because except for sot, these alternatives are really unions of character classes. We could prohibit alternatives altogether and use unions of character classes, splitting the sot case into its own sub-rule, but that would be a divergence from the UAX14 formulation, which seems worse.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant