RFC: upstream this into rules_nodejs repo

[alexeagle]

Heya @Dig-Doug

I appreciate so much how you stepped up here to maintain a proto/grpc solution for Bazel/JS users. I know we dropped the ball on supporting this in rules_nodejs and the @bazel/labs package has been incomplete and fragmented.

I'd like to fix this for real by upstreaming. Maybe this is actually history-preserving: slurping in all your commits to a packages/protoc folder (like we did for rules_typescript itself). Or maybe we just do a clean-room implementation and borrow bits from here as needed.

A couple things I'd like to see while we port:

• distribute the protoc rule(s) via an npm package so the dependencies/peerDependencies just work, with no extra lines in WORKSPACE (those cause eager fetches)
• don't rely on the user having an @npm external repository
• don't rely on compiling any TS code in the users build
• give choices where needed:
  • between the TS codegen options (three that I know of are proto-loader: Add TypeScript generator grpc/grpc-node#1474 https://www.npmjs.com/package/grpc_tools_node_protoc_ts and http://npmjs.com/package/ts-protoc-gen )
  • between JS emit choices (reflective like https://github.com/protobufjs/protobuf.js vs Google-style jspb, and node runtime vs browser runtime)

I think the benefit is that a bunch of infra and release engineering will come for free - for example #5 would be easily solved since we have executable examples running on CI already.

WDYT? I guess my main questions are whether you're okay with this plan, and if so whether you'd like to be involved in doing it :)

[Dig-Doug]

Hi @alexeagle,

I'm 100% onboard with this and happy to help upstream it. Let me know what the next steps should be.

[alexeagle]

I'm considering a couple other places rather than rules_nodejs - maybe we should contribute to rules_proto_grpc (I'm trying that out in rules-proto-grpc/rules_proto_grpc#103 ) or maybe even work with Bazel team to have the bazelbuild/rules_proto give some of the language generation features (though they don't want to have grpc there, since from their perspective that's just one of several transport layers atop proto Services)

So I think my next step is to understand what users can do with this ruleset that they can't do with rules_proto_grpc

[alexeagle]

Just discussed with @pcj and @mattem - we think it probably does make more sense for the proto/grpc rules to live in the language repo, with sth like bazelbuild/rules_proto giving utilities like Providers.

[Dig-Doug]

So the plan is still to upstream these into rules_nodejs?

[alexeagle]

yeah I think that's correct, though perhaps we want to base on top of the proto_plugin model that @pcj wrote in rules-proto-grpc. Still thinking it through a bit.
How much time do you have for OSS stuff? Maybe we can find a way to get paid for this work...

[Dig-Doug]

I see, so would you put the proto_plugin within rules_proto and then have each language repo use it as needed?

Work keeps me pretty busy during the week, but I can devote some time on the weekends. No need to get paid for this.

[alexeagle]

hey @Dig-Doug sorry for the long silence. We've decided to reduce scope in rules_nodejs rather than increase it. So I'm planning to remove our labs package in the next major (5.x this winter). I'll point users here as one alternative. The Bazel ecosystem still really needs a proto+grpc champion because the options are so confusing.

Also if you'd rather work on this during the week, we are hiring!

[Dig-Doug]

No problem, that's fine. Thanks for closing the loop.

[alexeagle]

Could we talk a bit about proto and Bazel in general? If you have a few minutes - [email protected]