Replace deprecated --load-extension with Extensions.loadUnpacked CDP to continue "web-ext run" support in Chrome

[Rob--W]

Chrome is looking into removing the --load-extension flag. web-ext run currently relies on --load-extension to load/run an extension and to load a helper extension (whose purpose is to reload the extension on demand). When --load-extension is removed, web-ext run will be unable to load extensions in Chrome (and Chromium based browsers that follow the same implementation).

A supported alternative to --load-extension is the Extensions.loadUnpacked Chrome DevTools Protocol (CDP) command:
https://chromedevtools.github.io/devtools-protocol/tot/Extensions/#method-loadUnpacked

To use this, Chrome's DevTools server has to be set up with --remote-debugging-pipe instead of --remote-debugging-port. But the chrome-launcher library that web-ext relies on right now hardcodes a dependency on --remote-debugging-pipe.

For reference, Puppeteer supports pipes (and port):

• https://github.com/puppeteer/puppeteer/blob/5b777616198ae31625b9b2c1eba82c0968b82340/packages/puppeteer-core/src/node/ChromeLauncher.ts#L86-L100
• https://github.com/puppeteer/puppeteer/blob/main/packages/puppeteer-core/src/node/BrowserLauncher.ts#L112-L140
• https://github.com/puppeteer/puppeteer/blob/5b777616198ae31625b9b2c1eba82c0968b82340/packages/puppeteer-core/src/node/PipeTransport.ts

Ideally, chrome-launcher would also accept a mechanism to switch to a pipe-based communication.

For visibility, here is an issue about switching to WebDriver BiDi in Firefox: #3351. If the mechanisms to use WebDriver BiDi are similar (without too much overhead/dependencies), it would be nice to use WebDriver BiDi for both. However, since the --load-extension deprecation is landing soon, we should not block on that for this issue.

[oliverdunk]

Thanks for filing this Rob! We've just confirmed that we are moving forward with this change: https://groups.google.com/a/chromium.org/g/chromium-extensions/c/1-g8EFx2BBY/m/S0ET5wPjCAAJ

[Rob--W]

Chrome 137 is scheduled for beta next week (30 april) and targeting release 20 May 2025, per https://chromiumdash.appspot.com/schedule

I'll try to find a weekend or so to look into fixing this.