Checking Token is already consumed or not while parsing

[Siva-Harish98]

Is it possible to check wheather the token is consumed more than one time while parsing

[msujew]

Is it possible to check wheather the token is consumed more than one time while parsing

Can you elaborate a bit? Chevrotain will consume a token at most one time. In case of parser errors, a token might be skipped, but the parser will never attempt to parse a token a second time.

[Siva-Harish98]

The grammar should always starts on token (or) NumberLiteral token

User should allow to write the language in editor:

• 1+1+on
• on+1

But but one case gets failed, 1+1+on+1+on I don't want to allow on token multiple times.

[msujew]

I see, that's what you mean. I've seen in the other issue that you're using an OR inside the MANY. You can use a GATE property for the ALT that consumes on to prevent it from being parsed again. Just flip a boolean after consuming the token once.

T.RULE('additionExpression', () => {
  T.SUBRULE(T.multiplicationExpression, { LABEL: 'lhs' });
  let on = false;
  T.MANY(() => {
    T.CONSUME(ArithmeticOperator);
    T.OR([
      { 
        ALT: () => { 
          T.SUBRULE3(T.onKey); 
          on = true; 
        } 
        GATE: () => !on
      },
      { ALT: () => T.SUBRULE2(T.multiplicationExpression, { LABEL: 'rhs' }) }
    ])
  });
});

Note that I would recommend to perform a validation step as post processing of the parser. That way, you can use a better error message that explains why parsing failed, instead of a complicated parser error message (which might not help the user).

[Siva-Harish98]

Great, thanks a lot @msujew ❤️

[swami-sanapathi]

I would recommend to perform a validation step as post processing of the parser.

you mean visitor? @msujew

[msujew]

@swami-sanapathi I guess it depends? As part of my work, I usually use Chevrotain to build ASTs. For validation purposes in IDEs, I traverse the ASTs and check for any semantic errors which are displayed at the appropriate position in the editor. I don't use the chevrotain visitor API at all. See also the Langium project.