add example for pooling browsers #14

rahulkumar66 · 2018-08-20T12:15:12Z

this is an example for pooling browsers resulting in faster load times of pages since we not don't have to open new browser for each request .
This results in improve load times especially in case of SPAs since cached resources can be reused.

googlebot · 2018-08-20T12:15:16Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here (e.g. I signed it!) and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

googlebot · 2018-08-20T12:18:53Z

CLAs look good, thanks!

rahulkumar66 · 2018-08-20T12:28:50Z

I signed it.

rahulkumar66 · 2018-08-22T05:02:51Z

This fixes #14

ebidel

I think it might be interesting to have a flag that controls using a pool and not using a pool. And then throw in some console.time to show the difference in perf.

ebidel · 2018-08-22T18:38:33Z

puppeteer-browser-pool.js

+const genericPool = require('generic-pool');
+const puppeteer = require('puppeteer');
+
+const VIEWPORT = { width: 1028, height: 800, deviceScaleFactor: 2 };


Sorry, we need linting.

Can you remove the surrounding whitespaces after { and the first key name and the space before }.

Use 2-space indentation everywhere.

ebidel · 2018-08-22T18:38:39Z

puppeteer-browser-pool.js

+
+const factory = {
+ create: async () => {
+ const browser = await puppeteer.launch(puppeteerArgs);


where does puppeteerArgs come from?

ebidel · 2018-08-22T18:38:52Z

puppeteer-browser-pool.js

+const HACKER_NEWS_QUERY_SPA_URL = 'https://hn.algolia.com/?query=angular';
+
+
+const factory = {


can we use a better name?

ebidel · 2018-08-22T18:39:30Z

puppeteer-browser-pool.js

+
+const VIEWPORT = { width: 1028, height: 800, deviceScaleFactor: 2 };
+const HACKER_NEWS_SPA_URL = 'https://hn.algolia.com/';
+const HACKER_NEWS_QUERY_SPA_URL = 'https://hn.algolia.com/?query=angular';


let's use https://hn.algolia.com/?query=chrome

This could also build off the base url.

ebidel · 2018-08-22T18:41:09Z

puppeteer-browser-pool.js

+ return page;
+ },
+ destroy: (browser) => {
+ browser.close();


return browser.close();

ebidel · 2018-08-22T18:42:01Z

puppeteer-browser-pool.js

+ // this request will use the instance from the current browser pool resulting in faster response
+ const responseHackerNewsWithQuery = await getContent(HACKER_NEWS_QUERY_SPA_URL);
+ console.log(responseHackerNewsWithQuery);
+})();


ebidel · 2018-08-23T14:44:32Z

Is it possible to add a few more examples to the demo that helps show why a pool is necessary in some cases. Otherwise it's easy just to use the same browser instance to open 2 tabs for the URLs you're using.

rahulkumar66 · 2018-08-24T05:40:31Z

The reason we are parallelizing work through browsers and not pages is because one page crash might bring down the whole browser and also each page isn't guaranteed to be totally clean (cookies and storage might bleed-through as seen here).

ebidel · 2018-08-24T13:34:35Z

But that is unlikely :) The other one is a big in chrome that hopefully gets fixed soon. I'd prefer to encourage folks to use separate browser instances only when it makes sense. It comes with much more overhead than launching a single instance. Having a demo that shows when/why you'd need to go with many instances would be more helpful IMO.

…

On Thu, Aug 23, 2018, 10:40 PM Rahul Kumar ***@***.***> wrote: The reason we are parallelizing work through browsers and not pages is because one page crash might bring down the whole browser and also each page isn't guaranteed to be totally clean (cookies and storage might bleed-through as seen here <https://bugs.chromium.org/p/chromium/issues/detail?id=754576>). — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#14 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAOigNgNRi0CBOgBxvKXxhW1e54Lpzaoks5uT5HQgaJpZM4WD4pV> .

rahulkumar66 · 2018-08-27T09:36:45Z

i have used single browser in production and it was not stable over long periods of time maybe because i was using --single process flag.
Even browserless.io recommends running multiple browsers opposed to running multiple pages

ebidel · 2018-08-27T15:17:59Z

I'm not arguing the benefits for certain use cases. I'm saying we should have a demo that shows a case where you would want to do pooling.

On Mon, Aug 27, 2018, 2:36 AM Rahul Kumar ***@***.***> wrote: i have used single browser in production and it was not stable over long periods of time maybe because i was using --single process flag. Even browserless.io <https://docs.browserless.io/blog/2018/06/04/puppeteer-best-practices.html> recommends running multiple browsers opposed to running multiple pages

I'm confused because this is an argument against keeping a pool of long running browsers open....

…

— You are receiving this because you commented. Reply to this email directly, view it on GitHub <#14 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAOigJeh0Q54JCtldEvyBbZbs6FyvoDpks5uU72xgaJpZM4WD4pV> .

rahulkumar66 · 2018-09-04T05:48:16Z

Ok i will add it

SebastiyanKasurov · 2019-11-04T14:14:00Z

puppeteer-browser-pool.js

+ return page;
+ },
+ destroy: (browser) => {
+ return browser.close();


You receive page from (create), in destroy you receive bro page, so page will be closed not browser

add example for pooling browsers

1364357

Remove redundant comment

f2c9240

ebidel suggested changes Aug 22, 2018

View reviewed changes

FIx several code quality issues

86f913c

SebastiyanKasurov reviewed Nov 4, 2019

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add example for pooling browsers #14

add example for pooling browsers #14

rahulkumar66 commented Aug 20, 2018

googlebot commented Aug 20, 2018

googlebot commented Aug 20, 2018

rahulkumar66 commented Aug 20, 2018

rahulkumar66 commented Aug 22, 2018

ebidel left a comment

ebidel Aug 22, 2018

ebidel Aug 22, 2018

ebidel Aug 22, 2018

rahulkumar666 Jun 14, 2022

ebidel Aug 22, 2018

ebidel Aug 22, 2018

ebidel Aug 22, 2018

ebidel Aug 22, 2018

ebidel commented Aug 23, 2018

rahulkumar66 commented Aug 24, 2018

ebidel commented Aug 24, 2018 via email

rahulkumar66 commented Aug 27, 2018

ebidel commented Aug 27, 2018 via email

rahulkumar66 commented Sep 4, 2018

SebastiyanKasurov Nov 4, 2019 •

edited

		const HACKER_NEWS_QUERY_SPA_URL = 'https://hn.algolia.com/?query=angular';


		const factory = {

add example for pooling browsers #14

Are you sure you want to change the base?

add example for pooling browsers #14

Conversation

rahulkumar66 commented Aug 20, 2018

googlebot commented Aug 20, 2018

What to do if you already signed the CLA

Individual signers

Corporate signers

googlebot commented Aug 20, 2018

rahulkumar66 commented Aug 20, 2018

rahulkumar66 commented Aug 22, 2018

ebidel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ebidel commented Aug 23, 2018

rahulkumar66 commented Aug 24, 2018

ebidel commented Aug 24, 2018 via email

rahulkumar66 commented Aug 27, 2018

ebidel commented Aug 27, 2018 via email

rahulkumar66 commented Sep 4, 2018

SebastiyanKasurov Nov 4, 2019 • edited

Choose a reason for hiding this comment

SebastiyanKasurov Nov 4, 2019 •

edited