Defining process for adding new syntax

[nzakas]

At the moment, the process for adding new syntax is a bit haphazard. There are some implicit agreements, like experimental syntax goes in experimental and we don't debate much until a feature reaches stage 3, but it would be good to get organized and make the entire process explicit.

As an example, here's what I envision as a starting point:

1. Create a GitHub project that tracks the stage of each ES proposal (so, stages 1-4). This can be updated whenever proposals progress to another stage.
2. When a new proposal reaches stage 1, an issue is created that contains:
3. A brief description of the proposal
4. Some code examples of the proposal
5. A link to the full proposal
6. At stage 1, anyone is free to submit an experimental proposal referencing the issue associated with the proposal
7. At stage 3, anyone is free to submit a formal proposal referencing the issue associated with the proposal
8. When the formal proposal is accepted, the experimental proposal is removed

My primary desire is to see that there's some traceability between the proposal, the experimental syntax proposal, and the final syntax proposal.

Again, this is just a suggestion for discussion, as having any well-defined process will make me happy. :)

[JLHwung]

I think the step 1, 3, 4 is not necessary.

Create a GitHub project that tracks the stage of each ES proposal (so, stages 1-4). This can be updated whenever proposals progress to another stage.

https://github.com/tc39/proposals serves this purpose well.

A brief description of the proposal
Some code examples of the proposal

They can be implied from step 5 "A link to the full proposal" given that the proposal is the single source of truth.

I am good with other steps.

[RReverser]

Yeah I think only steps 6-8 are necessary.

[nzakas]

My concern with leaving off the first few steps is that it's harder for us (ESTree) to track where we are relative to each proposal's stage. The TC39 proposals list is at least one click away, and doesn't tell us if we've already merged an experimental or final representation of a new syntax.

If you don't want to go with a GitHub project, I'd at least suggest we have one issue for each proposal that we can merge experimental and final PRs against. Another benefit of having an issue per proposal is that it signifies to onlookers where we are with regard to developing ESTree representation for it.

[RReverser]

My concern with tracking proposals separately is in increased maintenance burden cost with low / unknown value.

There's usually way more proposals in general than those that make it beyond experimental, and even for experimental ones it's not like we want to rush and define structures for everything right away, without knowing if anyone actually needs it / how it plays out with other proposals / etc..

As for stage 3 proposals minus the API changes - the one that we are explicitly interested in - they don't appear very frequently, and have already been easy enough to track IMO.

[bradzacher]

I think there is value in having at least a tracking issue.

As an example from last night.

The TS4.0 beta has released with support for Short-Circuiting Assignment Operators. In planning out what work is required for typescript-eslint, I wanted to see if there was an AST already defined for it.
I didn't have a link to the TC39 proposal handy, so I didn't know that it was stage 3.

This is what I did:

• opened this repo
• attempt to use Github repo search to search for ||=
  • GH's search sucks hard so it found no results
• attempt to use issue search to find an issue for the feature
  • found no issues (opened or closed).
• attempt to use PR search to find an issue for the feature
  • found a merged PR.
• figured out that the spec is stage 3, so it's not in the main spec, but is instead in the experimental folder.

If there was a tracking issue, I'd have been able to see the status in a much clearer form with links to any PRs, the underlying TC39 spec, and from there it would have been easy to see that it was experimental and find the experimental AST.

Issue tags would also make it easy to transparently track the status of the proposal for both TC39 stages, as well as ESTree stages.

[JLHwung]

see the status in a much clearer form with links to any PRs, the underlying TC39 spec

The https://github.com/estree/estree/tree/master/experimental serves well, the underlying TC39 spec is linked from each proposal title. I don't think another tracking issue can offer more values than maintaining cost.

figured out that the spec is stage 3, so it's not in the main spec, but is instead in the experimental folder

We can briefly introduce on README about the folder structure: e.g. what extensions and experimental are for.

Issue tags would also make it easy to transparently track the status of the proposal for both TC39 stages

In Babel we use issue tags to issue/PR related to specific proposal. However I don't think it is of great value for a document-only project like ESTree. IMO the GitHub history link is sufficient: https://github.com/JLHwung/estree/commits/add-class-features/experimental/private-fields-in-in.md -- because the artifact of each proposal is a single markdown file.

[bradzacher]

We can briefly introduce on README about the folder structure: e.g. what extensions and experimental are for.

If I know what stage a proposal is in, then sure, I would have known to go straight to the experimental folder.

The problem is that I had no idea what the TC39 stage is because I hadn't seen the spec. Without knowing the stage, I didn't know to look in the experimental folder.

As another example, if I had missed a memo and didn't hear that a feature had moved to stage 4 and merged into the main spec, then I'd likely be confused that it isn't in the experimental folder, then I'd have to use the commit history to figure out why.

For someone like yourself - you work with babel and bleeding edge specs, so you're well versed in the TC39 process, the features that are being worked on, and can likely quickly and easily find those specs, or even know offhand the stage of a given spec.

For someone like me that isn't - I would have to go googling to find the spec link to figure out the stage, or go clicking around this repo to attempt to find the spec for the feature.

If this is catalogued as an issue, then there wouldn't be a question for anyone at all. Anyone can use the issue search here as a centralised place to locate the status and spec of the feature within ESTree's pipeline.

[RReverser]

As another example, if I had missed a memo and didn't hear that a feature had moved to stage 4 and merged into the main spec, then I'd likely be confused that it isn't in the experimental folder, then I'd have to use the commit history to figure out why.

I'd say you should always start at https://github.com/tc39/proposals as source of truth, which quickly gives you a stage. I wouldn't say you need to be "well versed in the TC39 process" to find the proposal in that structured list :)

Maintaining a copy or even a derived data of that table creates more maintenance churn and still has a risk of going out of sync from time to time. As a result, you will still end up having to check the original source to figure out what the actual status of proposal is, which diminishes value of such copy as well.

I'd much rather have a single source of truth that anyone can refer to.

[bradzacher]

I'd say you should always start at tc39/proposals as source of truth

I didn't even know that page existed!!! (I doubt a lot of people do)
Whenever I'm told about a proposal itself, there is usually a link directly to the proposal repo.
Similarly when googling for a proposal - the tc39/proposals page doesn't show up in the results.

I guess you're kind of proposing that the workflow be something like - open up estree, click a link to proposals, see that "logical assignment operators" is stage 3, go back to estree, open up the experimental folder, hope to find it in there (in this case, I would).

Hypothetically, say that there wasn't an experimental spec that had been merged - where do I go next?
Straight to the PRs to see if there's an open PR? What if there's no PR - does that mean there's no AST changes necessary, or does that just mean nobody's raised one yet?
Do I go to the issues to see if there's a discussion about the proposal? If there's no issues does that mean that there's been no discussion? Maybe the ESTree members have had discussions offline about it? Do I raise an issue to ask a question?

As a consumer of ESTree, IMO having an issue to track the lifecycle of a proposal's implementation into this spec would be so much cleaner and remove all ambiguity from the process.

I could easily use the issues as the record of what state each proposal is in its ESTree implementation.

This would also make it trivial for a consumer to track if there are required AST changes for a proposal.
For example, if I was wondering if there were any required changes for this proposal proposal-regexp-match-indices, I would easily be able to find a closed issue cataloguing that nothing needs to be actioned for it.

[nzakas]

Sorry for the late reply, been having some health-related challenges lately.

So I think we're having a disconnect about the purpose of what I'm proposing (a tracking issue). The goal of having a tracking issue is not to simply mirror what is available at https://github.com/tc39/proposals, because I agree, that would just be maintenance overhead without any clear value. The goal is so we, as a project, can track our progress on making these changes.

I do not find it easy right now to figure out if we've already addressed experimental changes or whether there has been a final node format submitted that would override the experimental format that has already been merged. Having a tracking issue is a signal to everyone (not just us) about where we are in the process of defining the AST for new features.

If the primary concern is maintenance overhead, I believe the front-end of what I've described can be automated. I could foresee writing a script that runs each night against https://github.com/tc39/proposals and pulls in the relevant data. It could probably create issues, move them along in a GitHub project as their stage changes, and maybe leave a comment on the issue once a proposal reaches stage 3 to say "hey, it's time to define this officially!". I'm happy to work on this.