Get rid of parser hacks related to typedefs and identifiers

[ForNeVeR]

In the C17 standard, a common pair of AST nodes which are frequently used together are declaration_specifiers and something involving an identifier next.

Some examples are (Yoakke syntax, not direct EBNF from the standard):

• declaration: declaration_specifiers init_declarator_list? ';'
• struct_declaration: specifier_qualifier_list struct_declarator_list? ';' (where specifier_qualifier_list includes some of the declaration_specifiers)

The problem is that declaration_specifiers (or, say, specifier_qualifier_list) include an item called typedef_name, which it is: a name of a typedef item.

This causes Cesium to choke on simple definitions such as size_t foo: was it typedef_name: size_t and typedef_name: foo, or typedef_name: foo and identifier: foo?

Ideally, a parser should be able to choose in most contexts, since the former isn't even valid. But due to Yoakke issue LanguageDev/Yoakke#138, currently we have to support a great number of workarounds to be able to parse even the simplest of cases.

Current plan is to wait for the Yoakke issue to get resolved, and then remove all the hacks from the code.

To find all of these hacks, look for marks like TODO[#107].

Blocked on:

• A case of parser backtracking not being supported LanguageDev/Yoakke#138

[kant2002]

There additional cases where Cesium chokes in completely different ways.

typedef int int32_t;

void foo(int32_t[16]);
void foo2(const int32_t[16]);

First one have this AST

{
  "$type": "Cesium.Ast.Declaration, Cesium.Ast",
  "Specifiers": [
    {
      "$type": "Cesium.Ast.SimpleTypeSpecifier, Cesium.Ast",
      "TypeName": "void"
    },
    {
      "$type": "Cesium.Ast.NamedTypeSpecifier, Cesium.Ast",
      "TypeDefName": "foo"
    }
  ],
  "InitDeclarators": [
    {
      "$type": "Cesium.Ast.InitDeclarator, Cesium.Ast",
      "Declarator": {
        "$type": "Cesium.Ast.Declarator, Cesium.Ast",
        "Pointer": null,
        "DirectDeclarator": {
          "$type": "Cesium.Ast.DeclaratorDirectDeclarator, Cesium.Ast",
          "Declarator": {
            "$type": "Cesium.Ast.Declarator, Cesium.Ast",
            "Pointer": null,
            "DirectDeclarator": {
              "$type": "Cesium.Ast.ArrayDirectDeclarator, Cesium.Ast",
              "Base": {
                "$type": "Cesium.Ast.IdentifierDirectDeclarator, Cesium.Ast",
                "Identifier": "int32_t",
                "Base": null
              },
              "TypeQualifiers": null,
              "Size": {
                "$type": "Cesium.Ast.ConstantLiteralExpression, Cesium.Ast",
                "Constant": {
                  "Kind": "IntLiteral",
                  "Text": "16"
                }
              }
            }
          },
          "Base": null
        }
      },
      "Initializer": null
    }
  ]
}

Second one is this

{
  "$type": "Cesium.Ast.Declaration, Cesium.Ast",
  "Specifiers": [
    {
      "$type": "Cesium.Ast.SimpleTypeSpecifier, Cesium.Ast",
      "TypeName": "void"
    }
  ],
  "InitDeclarators": [
    {
      "$type": "Cesium.Ast.InitDeclarator, Cesium.Ast",
      "Declarator": {
        "$type": "Cesium.Ast.Declarator, Cesium.Ast",
        "Pointer": null,
        "DirectDeclarator": {
          "$type": "Cesium.Ast.ParameterListDirectDeclarator, Cesium.Ast",
          "Base": {
            "$type": "Cesium.Ast.IdentifierDirectDeclarator, Cesium.Ast",
            "Identifier": "foo",
            "Base": null
          },
          "Parameters": {
            "$type": "Cesium.Ast.ParameterTypeList, Cesium.Ast",
            "Parameters": [
              {
                "$type": "Cesium.Ast.ParameterDeclaration, Cesium.Ast",
                "Specifiers": [
                  {
                    "$type": "Cesium.Ast.TypeQualifier, Cesium.Ast",
                    "Name": "const"
                  }
                ],
                "Declarator": {
                  "$type": "Cesium.Ast.Declarator, Cesium.Ast",
                  "Pointer": null,
                  "DirectDeclarator": {
                    "$type": "Cesium.Ast.ArrayDirectDeclarator, Cesium.Ast",
                    "Base": {
                      "$type": "Cesium.Ast.IdentifierDirectDeclarator, Cesium.Ast",
                      "Identifier": "int32_t",
                      "Base": null
                    },
                    "TypeQualifiers": null,
                    "Size": {
                      "$type": "Cesium.Ast.ConstantLiteralExpression, Cesium.Ast",
                      "Constant": {
                        "Kind": "IntLiteral",
                        "Text": "16"
                      }
                    }
                  }
                },
                "AbstractDeclarator": null
              }
            ],
            "HasEllipsis": false
          }
        }
      },
      "Initializer": null
    }
  ]
}

[kant2002]

This issue is what currently blocking #61 from progressing.