Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws lambda support? #434

Open
johnHastings772 opened this issue Jan 28, 2022 · 5 comments
Open

aws lambda support? #434

johnHastings772 opened this issue Jan 28, 2022 · 5 comments

Comments

@johnHastings772
Copy link

I guess it's unsupported right now but... would it be possible to run secret-agent on aws lambda with or without any limitations?

I'm trying to get it running but I keep getting PipeTransport errors on the real aws enviroment while on a local invocation works just fine.

This is an excerpt of the log:

`2022-01-28T11:42:40.802Z ERROR [/var/task/node_modules/@secret-agent/puppet/lib/PipeTransport] PipeTransport.WriteError { context: {}, sessionId: �[1mnull�[22m, sessionName: �[90mundefined�[39m } Error: read ECONNRESET

�[90m at Pipe.onStreamRead (internal/stream_base_commons.js:209:20)�[39m {

errno: �[33m-104�[39m,

code: �[32m'ECONNRESET'�[39m,

syscall: �[32m'read'�[39m

}

2022-01-28T11:42:40.803Z ERROR [/var/task/node_modules/@secret-agent/puppet/lib/PipeTransport] PipeTransport.WriteError { context: {}, sessionId: �[1mnull�[22m, sessionName: �[90mundefined�[39m } Error: read ECONNRESET

�[90m at Pipe.onStreamRead (internal/stream_base_commons.js:209:20)�[39m {

errno: �[33m-104�[39m,

code: �[32m'ECONNRESET'�[39m,

syscall: �[32m'read'�[39m

}

2022-01-28T11:42:40.804Z INFO [/var/task/node_modules/@secret-agent/puppet/lib/PipeTransport] PipeTransport.Closed { context: {} }

2022-01-28T11:42:40.820Z STATS [/var/task/node_modules/@secret-agent/puppet/lib/BrowserProcess] chrome.ProcessExited { context: {} }

2022-01-28T11:42:40.821Z INFO [/var/task/node_modules/@secret-agent/core/lib/Session] Session.Closing { context: {} }`

Looking at the stacktrace it seems that there's some problem while launching chrome and thus the connection is closed.

@blakebyrnes
Copy link
Contributor

I definitely want to support this, but this is going to take some effort to troubleshoot. I think we need to figure out what isn't working on Lambda. I'm guessing Lambda is balking either at the size of Chrome, or the architecture on the machine is somehow incompatible with the Chrome builds we have.

I think the starting place is to look at how Playwright and Puppeteer are getting running on AWS. You can change out the "Chrome" that SecretAgent is using as a temporary measure by using the environment variable CHROME_88_BIN to point at your executable.

@johnHastings772
Copy link
Author

johnHastings772 commented Jan 31, 2022

Hi! I tried with latest chomium 88 binary from https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=Linux/ and as before it works just fine on local but on lambda I got an error, a different one this time:

On the lambda call where the exception error is logged:
{
"message": "CoreServer needs further setup to launch the browserEmulator. See server logs.",
"name": "Error",
"stack": "Error: CoreServer needs further setup to launch the browserEmulator. See server logs.\n at ConnectionToClient.serializeError (/var/task/node_modules/core/server/ConnectionToClient.ts:372:14)\n at ConnectionToClient.handleRequest (/var/task/node_modules/core/server/ConnectionToClient.ts:94:19)\n------REMOTE CORE---------------------------------\n at Function.reviver (/var/task/node_modules/commons/TypeSerializer.ts:208:26)\n at JSON.parse ()\n at Function.parse (/var/task/node_modules/commons/TypeSerializer.ts:24:17)\n at WebSocket. (/var/task/node_modules/client/connections/RemoteConnectionToCore.ts:67:42)\n at WebSocket.emit (events.js:400:28)\n at WebSocket.emit (domain.js:475:12)\n at Receiver.receiverOnMessage (/var/task/node_modules/ws/lib/websocket.js:1008:20)\n at Receiver.emit (events.js:400:28)\n at Receiver.emit (domain.js:475:12)\n at Receiver.dataMessage (/var/task/node_modules/ws/lib/receiver.js:517:14)\n------CONNECTION----------------------------------\n at new Resolvable (/var/task/node_modules/commons/Resolvable.ts:16:18)\n at Object.createPromise (/var/task/node_modules/commons/utils.ts:68:10)\n at RemoteConnectionToCore.createPendingResult (/var/task/node_modules/client/connections/ConnectionToCore.ts:357:31)\n at RemoteConnectionToCore.internalSendRequestAndWait (/var/task/node_modules/client/connections/ConnectionToCore.ts:263:43)\n at RemoteConnectionToCore.sendRequest (/var/task/node_modules/client/connections/ConnectionToCore.ts:164:17)\n at processTicksAndRejections (internal/process/task_queues.js:95:5)\n at CoreCommandQueue.sendRequest (/var/task/node_modules/client/lib/CoreCommandQueue.ts:150:22)\n at Object.cb (/var/task/node_modules/client/lib/CoreCommandQueue.ts:117:16)\n at Queue.next (/var/task/node_modules/commons/Queue.ts:82:19)\n------CORE COMMANDS-------------------------------\n at Queue.run (/var/task/node_modules/commons/Queue.ts:35:19)\n at CoreCommandQueue.run (/var/task/node_modules/client/lib/CoreCommandQueue.ts:114:8)\n at RemoteConnectionToCore.createSession (/var/task/node_modules/client/connections/ConnectionToCore.ts:205:51)\n at SessionConnection.getCoreSessionOrReject (/var/task/node_modules/client/lib/Agent.ts:559:36)\n at SessionConnection.get activeTab [as activeTab] (/var/task/node_modules/client/lib/Agent.ts:480:10)\n at Agent.get activeTab [as activeTab] (/var/task/node_modules/client/lib/Agent.ts:152:37)\n at Agent.goto (/var/task/node_modules/client/lib/Agent.ts:365:17)\n at Runtime.exports.handler (/var/task/app.js:41:42)\n at Runtime.handleOnce (/var/runtime/Runtime.js:66:25)"
}

On the cloundfront log
2022-01-31T08:46:02.986Z STATS [/var/task/node_modules/@secret-agent/puppet/index] Puppet.LaunchError { context: {} } [PuppetLaunchError: EROFS: read-only file system, open '/opt/chrome/.validated' Error: EROFS: read-only file system, open '/opt/chrome/.validated'] { isSandboxError: false } 2022-01-31T08:46:02.987Z ERROR [/var/task/node_modules/@secret-agent/core/server/ConnectionToClient] ConnectionToClient.HandleRequestError { context: {}, sessionId: undefined, sessionName: undefined } [PuppetLaunchError: EROFS: read-only file system, open '/opt/chrome/.validated' Error: EROFS: read-only file system, open '/opt/chrome/.validated'] { isSandboxError: false

This two flags are being used to launch chrome:

'--homedir=/tmp',
'--user-data-dir=/tmp/chrome-user-data',

I have no idea where that .validated dir comes from, is it something custom from the browser emulator?

Edit So I noticed that .validated "dir" is just an empty file on a default secret-agent install I touched it and now it loads but the launch args are being ignored, which seems quite strange.

Edit2 I see the .validated file flags if the current browser dependencies are met.

I tried to load a different chromium 88 binary, this time the build number was closest to the default one in secret agent, the build was so "old" that it used gtk2 and thus different dependencies were needed I had to resort to symlinking the bzip2 .so because the build was expecting libbzip2.1.0 instead onf libbzip2.1 or libbzip2.1.06 which was what amazon linux bzip2 package provides. This time the browser loaded but failed with the error: Gtk-WARNING **: 14:39:47.950: cannot open display: ' this was in the local enviroment I also used the --disable-gpu and --headless launch args for chrome

Edit 3 Tried with the binary dev-headless-chromium-88.0.4298.4-amazonlinux-2017-03.zip from:

https://github.com/adieuadieu/serverless-chrome/releases

And got this error on aws lambda, works just fine in the docker container locally:

(/var/task/node_modules/core/server/ConnectionToClient.ts:372:14)\n at ConnectionToClient.handleRequest (/var/task/node_modules/core/server/ConnectionToClient.ts:94:19)\n at processTicksAndRejections (internal/process/task_queues.js:95:5)\n------REMOTE CORE---------------------------------\n at Function.reviver (/var/task/node_modules/commons/TypeSerializer.ts:208:26)\n at JSON.parse (<anonymous>)\n at Function.parse (/var/task/node_modules/commons/TypeSerializer.ts:24:17)\n at WebSocket.<anonymous> (/var/task/node_modules/client/connections/RemoteConnectionToCore.ts:67:42)\n at WebSocket.emit (events.js:400:28)\n at WebSocket.emit (domain.js:475:12)\n at Receiver.receiverOnMessage (/var/task/node_modules/ws/lib/websocket.js:1008:20)\n at Receiver.emit (events.js:400:28)\n at Receiver.emit (domain.js:475:12)\n at Receiver.dataMessage (/var/task/node_modules/ws/lib/receiver.js:517:14)\n------CONNECTION----------------------------------\n at new Resolvable (/var/task/node_modules/commons/Resolvable.ts:16:18)\n at Object.createPromise (/var/task/node_modules/commons/utils.ts:68:10)\n at RemoteConnectionToCore.createPendingResult (/var/task/node_modules/client/connections/ConnectionToCore.ts:357:31)\n at RemoteConnectionToCore.internalSendRequestAndWait (/var/task/node_modules/client/connections/ConnectionToCore.ts:263:43)\n at RemoteConnectionToCore.sendRequest (/var/task/node_modules/client/connections/ConnectionToCore.ts:164:17)\n at processTicksAndRejections (internal/process/task_queues.js:95:5)\n at CoreCommandQueue.sendRequest (/var/task/node_modules/client/lib/CoreCommandQueue.ts:150:22)\n at Object.cb (/var/task/node_modules/client/lib/CoreCommandQueue.ts:117:16)\n at Queue.next (/var/task/node_modules/commons/Queue.ts:82:19)\n------CORE COMMANDS-------------------------------\n at Queue.run (/var/task/node_modules/commons/Queue.ts:35:19)\n at CoreCommandQueue.run (/var/task/node_modules/client/lib/CoreCommandQueue.ts:114:8)\n at RemoteConnectionToCore.createSession (/var/task/node_modules/client/connections/ConnectionToCore.ts:205:51)\n at SessionConnection.getCoreSessionOrReject (/var/task/node_modules/client/lib/Agent.ts:559:36)\n at SessionConnection.get activeTab [as activeTab] (/var/task/node_modules/client/lib/Agent.ts:480:10)\n at Agent.get activeTab [as activeTab] (/var/task/node_modules/client/lib/Agent.ts:152:37)\n at Agent.goto (/var/task/node_modules/client/lib/Agent.ts:365:17)\n at Runtime.exports.handler (/var/task/app.js:41:42)\n at Runtime.handleOnce (/var/runtime/Runtime.js:66:25)"}

There seems to be something wrong with the communication with the core, could it be some mismatch on chrome dev tools/debug protocols or whatever secret agent is using to communicate with chrome? Mind you I haven't got yet full understanding on how secret agent operates in this regard.

@blakebyrnes
Copy link
Contributor

Thanks for running these experiments! It's been awhile since I've used Lambda. You used to have to pre-compile native modules and package them. Is that still true?

You're correct - the .validated folder is whether we've checked linux dependencies. On debian based Operating systems, we have a .deb file that will install any missing deps so that Chrome has everything it needs. I'm not sure if the binaries you're finding are coming with everything or not.

This is an example of one of the projects I'd seen in the past that precompiled some binaries: https://github.com/alixaxel/chrome-aws-lambda/tree/master/bin

Regarding Core->Client, SecretAgent is forking a process and then (if you are using the default full-client project), it communicates over a socket to that process. I'm not sure what restrictions Lambda puts on launching processes. I'm guessing there are probably some other logs that are happening that show something else breaking prior to this?

@johnHastings772
Copy link
Author

Well for deploying to aws I'm using docker images which you can build taking base on other images, there's one with node14 provided by amazon, so it's just a matter of installing any node modules with npm install and the required dependencies of those modules (in this case not only the modules but the chrome binary too) on the base system ex: gtk, alsa, etc....

Using the binary from chrome-aws-lambda I think I have it working but with a lot of kludges, still need to do more testing code/cleanup.

@johnHastings772
Copy link
Author

johnHastings772 commented Feb 7, 2022

Hi, although I got it to run under aws-lambda I couldn't get clodflare checks to pass, and cloudflare support is the main feature I need to get working for my specific use case.

Here's a detalied description on how I got it to run:

I'm using aws-lambda docker images built with this dockerfile:

FROM public.ecr.aws/lambda/nodejs:14
###### Alternatively, you can pull the base image from Docker Hub: amazon/aws-lambda-nodejs:14

###### Assumes your function is named "app.js", and there is a package.json file in the app directory 
COPY app.js package.json /var/task/

###### Enviroment variables persistent on the built image and needed during install
ENV XDG_CACHE_HOME=/var/task/

###### install Chrome dependencies for default binary
###### only aws-chrome-binary
###### RUN yum install tar gzip -y -q
###### both
RUN yum install libXrandr gtk3 alsa-lib tar gzip -y -q

###### Enviroment variables used during install 
ARG SA_REPLAY_SKIP_BINARY_DOWNLOAD=true
ARG SA_REBUILD_MITM_SOCKET=true
ARG GO_URL="https://golang.org/dl/go1.17.6.linux-amd64.tar.gz"

###### Install Go for Mitm sockets
ARG OLD_PATH=$PATH
RUN set -eux; \
	curl -# -o go.tgz -L "https://golang.org/dl/go1.17.6.linux-amd64.tar.gz"; \
	tar -C /usr/local -xzf go.tgz; \
	rm go.tgz; \
	export PATH="/usr/local/go/bin:$PATH"; \
	go version

ENV GOPATH /go
ENV PATH $GOPATH/bin:/usr/local/go/bin:$PATH

###### Install NPM dependencies for function
RUN npm install

###### Restore old PATH and remove go
ENV PATH $OLD_PATH
RUN unset GOPATH
RUN rm -rf /usr/local/go

###### needed for custom chrome launch args
COPY configureBrowserLaunchArgs.js /var/task/node_modules/@secret-agent/default-browser-emulator/lib/helpers/

###### No need to show replay as secret-agent runs headless
ENV SA_SHOW_REPLAY=0

###### Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
CMD [ "app.handler" ]

I had to resort to hardcoded launch args replacing the whole configureBrowserLaunchArgs.js file as I couldn't get the launchargs plugin to work, the plugin worked just fine on my local machine inside the docker container and out of it, it didn't worked in the real lambda enviroment, it was giving some errors related to the connection to Core.

The flags I add are:

'--force-webrtc-ip-handling-policy=default_public_interface_only', '--no-startup-window',
    // Custom args
    '--homedir=/tmp',
    '--user-data-dir=/tmp/chrome-user-data',
    '--disable-gpu',
    '--no-sandbox',
    '--in-process-gpu',
    '--single-process'
    if (options.showBrowser) {

Package.json:

{
    "name": "app",
    "version": "1.0.0",
    "description": "secret-agent-test",
    "main": "app.js",
    "scripts": {
    "test": "test"
    },
    "author": "none",
    "license": "PRIVATE",
    "private": true,
    "dependencies": {
    "secret-agent": "^1.6.4",
    "aws-sdk": "^2.1063.0",
    "chrome-aws-lambda": "^10.1.0"
    }
}

app.js:

const process = require('process');
const Agent = require('secret-agent');
const chromium = require('chrome-aws-lambda');

exports.handler = async function(event, context) {
    const url = 'https://nowsecure.nl';
    const baseWaitMS = 5000;
    const timeoutMS = 10*baseWaitMS;
    var cloudflareDetected = false;
    var cloudflareBypasseed = false;
    var cloudflareRetries = 10;
    //process.env.DEBUG = true;
    //Uncomment to use alternate binary
    process.env.CHROME_88_BIN = await chromium.executablePath;    
    try{
        const agent = new Agent.Agent({ upstreamProxyUrl: 'http://user:pass@proxy:port' });

        agent.activeTab.on('resource', async (event) => {
            var cfHeaders = await event.response.headers;
            var responseUrl = await event.response.url;

            if ( responseUrl.replace(/\/$/, "") === url.replace(/\/$/, "") && 'cf-ray' in cfHeaders) {
                
                if(!cloudflareDetected){
                    cloudflareDetected = true;
                    console.log("Cloudflare detected, ray-id:" + cfHeaders['cf-ray']);
                }
                
                var repsonseCode = await event.response.statusCode;
                console.log("Response code: " + repsonseCode + ' ray-id: ' + cfHeaders['cf-ray']);

                if (repsonseCode >=200 && repsonseCode <=299){
                    cloudflareBypasseed = true
                    console.log("Cloudflare bypassed, ray-id:" + cfHeaders['cf-ray']);
                    cloudflareRetries = 0;
                }

                if (repsonseCode == 403){
                    console.log("Cloudflare forbidden response");
                    cloudflareRetries = 0;
                }
                
                if(cloudflareRetries > 0){
                    //await agent.activeTab.waitForMillis(baseWaitMS-minWaitMS);
                    cloudflareRetries--;
                }
                
            }
        });

        var responseObject = await agent.goto(url);
        await agent.activeTab.waitForResource({url: url}, {timeoutMs: timeoutMS})
        await agent.activeTab.waitForMillis(baseWaitMS);
        var responseStatusCode = await responseObject.response.statusCode;

        if(!cloudflareDetected){
            if (responseStatusCode >=400){
                console.log('SCRAPE ERROR: HTTP ERROR CODE: ' + responseStatusCode + ' ' + url + ': ' + pageSource)
                result = 1;
            }

            else {
                var pageSource = await agent.activeTab.document.documentElement.outerHTML;
                var screenShot = await agent.activeTab.takeScreenshot({'format':'png'});
                console.log('SCRAPE: ' + url + ': Scraped Success.')
                result = 0;
            }
        }

        else{

            while(!cloudflareBypasseed && cloudflareRetries>0){
                await agent.activeTab.waitForMillis(baseWaitMS);
            }

            if (cloudflareBypasseed){
                var pageSource = await agent.activeTab.document.documentElement.outerHTML;
                var screenShot = await agent.activeTab.takeScreenshot({'format':'png'});
                console.log('SCRAPE: ' + url + ': Scraped Success.')
                result = 0;
            }

            else{
                throw "Cloudflare bypass error";
            }
        }

    } catch(error){
        console.log('SCRAPE ERROR: ' + url + ':' + error);
        result = 2;
        
    }
    console.log(result);
    return result;
}

I guess the differences between the binary provided by chrome-aws-lambda and the default binary in secret agent are causing it to fail in passing the cloudflare checks. I tried with other versions of the chrome-aws-lambda package (one with chrome version 88.xx) and found no differences in the outcome.

Will a different build of the secret agent binary be needed to support aws-lambda and pass cloudflare checks?
Is there something else I could try to get it running?

Here are some logs:

With default binary, the lambda fails with the aforementioned PipeTransport error:

START RequestId: f21741ac-cfa0-400e-aba7-7edf7e501e4e Version: $LATEST

2022-02-07T08:54:18.489Z ERROR [/var/task/node_modules/@secret-agent/puppet/lib/PipeTransport] PipeTransport.WriteError { context: {}, sessionId: null, sessionName: undefined } Error: read ECONNRESET

    at Pipe.onStreamRead (internal/stream_base_commons.js:209:20) {

errno: -104,

code: 'ECONNRESET',

syscall: 'read'

}

2022-02-07T08:54:18.493Z ERROR [/var/task/node_modules/@secret-agent/puppet/lib/PipeTransport] PipeTransport.WriteError { context: {}, sessionId: null, sessionName: undefined } Error: read ECONNRESET

    at Pipe.onStreamRead (internal/stream_base_commons.js:209:20) {

errno: -104,

code: 'ECONNRESET',

syscall: 'read'

}

2022-02-07T08:54:18.507Z ERROR [/var/task/node_modules/@secret-agent/core/index] UnhandledError(fatal) { context: {}, sessionId: null, sessionName: undefined } TypeError: Cannot read property 'isClosed' of undefined

at onClosed (/var/task/node_modules/puppet-chrome/lib/Connection.ts:84:14)

at PipeTransport.onReadClosed (/var/task/node_modules/puppet/lib/PipeTransport.ts:70:42)

    at Socket.emit (events.js:412:35)

    at Pipe.<anonymous> (net.js:686:12)

/var/task/node_modules/@secret-agent/puppet-chrome/lib/Connection.js:75

if (this.isClosed)

^

TypeError: Cannot read property 'isClosed' of undefined

at onClosed (/var/task/node_modules/puppet-chrome/lib/Connection.ts:84:14)

at PipeTransport.onReadClosed (/var/task/node_modules/puppet/lib/PipeTransport.ts:70:42)

at Socket.emit (events.js:412:35)

at Pipe.<anonymous> (net.js:686:12)

2022-02-07T08:54:18.529Z        f21741ac-cfa0-400e-aba7-7edf7e501e4e    INFO    SCRAPE ERROR: https://nowsecure.nl:TypeError: Cannot read property 'goto' of null

]     at Runtime.exports.handler (/var/task/app.js:69:30)ts:160:36)e    INFO    [

2022-02-07T08:54:18.547Z        f21741ac-cfa0-400e-aba7-7edf7e501e4e    ERROR   Unhandled Promise Rejection     {"errorType":"Runtime.UnhandledPromiseRejection","errorMessage":"TypeError: Cannot read property 'addEventListener' of null","reason":{"errorType":"TypeError","errorMessage":"Cannot read property 'addEventListener' of null","stack":["TypeError: Cannot read property 'addEventListener' of null","    at Tab.addEventListener (/var/task/node_modules/client/lib/AwaitedEventTarget.ts:16:27)"]},"promise":{},"stack":["Runtime.UnhandledPromiseRejection: TypeError: Cannot read property 'addEventListener' of null","    at process.<anonymous> (/var/runtime/index.js:35:15)","    at process.emit (events.js:400:28)","    at process.emit (domain.js:475:12)","    at processPromiseRejections (internal/process/promises.js:245:33)","    at processTicksAndRejections (internal/process/task_queues.js:96:32)"]}

[ERROR] [1644224058549] LAMBDA_RUNTIME Failed to post handler success response. Http response code: 403.

END RequestId: f21741ac-cfa0-400e-aba7-7edf7e501e4e

REPORT RequestId: f21741ac-cfa0-400e-aba7-7edf7e501e4e  Duration: 13608.83 ms   Billed Duration: 21865 ms       Memory Size: 1024 MB                                       Max Memory Used: 241 MB  Init Duration: 8255.26 ms

RequestId: f21741ac-cfa0-400e-aba7-7edf7e501e4e Error: Runtime exited with error: exit status 128
Runtime.ExitError

With chrome-aws-lambda binary, the lambda runs fine but cloudflare checks are not passed and ends up in a timeout error, on local (inside container or outside container) it usually passes the test on the first 5 retries:

START RequestId: 20bcf4ee-7f6b-4737-92af-20829c326e92 Version: $LATEST

2022-02-07T09:24:51.769Z	20bcf4ee-7f6b-4737-92af-20829c326e92	INFO	Cloudflare detected, ray-id:6d9b914ea97d83e1-BRU

2022-02-07T09:24:51.771Z	20bcf4ee-7f6b-4737-92af-20829c326e92	INFO	Response code: 503 ray-id: 6d9b914ea97d83e1-BRU

2022-02-07T09:24:56.911Z	20bcf4ee-7f6b-4737-92af-20829c326e92	INFO	Response code: 503 ray-id: 6d9b916efbaf83e1-BRU

2022-02-07T09:25:04.491Z	20bcf4ee-7f6b-4737-92af-20829c326e92	INFO	Response code: 503 ray-id: 6d9b919e485d83e1-BRU

2022-02-07T09:25:16.194Z	20bcf4ee-7f6b-4737-92af-20829c326e92	INFO	Response code: 503 ray-id: 6d9b91e76f5683e1-BRU

2022-02-07T09:25:35.612Z	20bcf4ee-7f6b-4737-92af-20829c326e92	INFO	Response code: 503 ray-id: 6d9b9260ed3383e1-BRU

2022-02-07T09:26:11.130Z	20bcf4ee-7f6b-4737-92af-20829c326e92	INFO	Response code: 503 ray-id: 6d9b933eadc483e1-BRU

2022-02-07T09:26:46.612Z	20bcf4ee-7f6b-4737-92af-20829c326e92	INFO	Response code: 503 ray-id: 6d9b941c8a9983e1-BRU

2022-02-07T09:27:06.672Z	20bcf4ee-7f6b-4737-92af-20829c326e92	INFO	Response code: 503 ray-id: 6d9b949a0e8a83e1-BRU

2022-02-07T09:27:24.890Z	20bcf4ee-7f6b-4737-92af-20829c326e92	INFO	Response code: 503 ray-id: 6d9b950be9dd83e1-BRU

2022-02-07T09:27:42.813Z	20bcf4ee-7f6b-4737-92af-20829c326e92	INFO	Response code: 503 ray-id: 6d9b957bef7183e1-BRU

END RequestId: 20bcf4ee-7f6b-4737-92af-20829c326e92

REPORT RequestId: 20bcf4ee-7f6b-4737-92af-20829c326e92	Duration: 180072.95 ms	Billed Duration: 187658 ms	Memory Size: 1024 MB	Max Memory Used: 574 MB	Init Duration: 7584.09 ms	

2022-02-07T09:27:43.309Z 20bcf4ee-7f6b-4737-92af-20829c326e92 Task timed out after 180.07 seconds

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants