Skip to content
This repository has been archived by the owner on Mar 15, 2022. It is now read-only.

Google bot-guard protection bypass #14

Open
axel578 opened this issue Feb 20, 2019 · 25 comments
Open

Google bot-guard protection bypass #14

axel578 opened this issue Feb 20, 2019 · 25 comments
Labels
enhancement New feature or request help wanted Extra attention is needed published temporary fix

Comments

@axel578
Copy link

axel578 commented Feb 20, 2019

Your bot should integrate the bg.request wich is the most important of all , if google sees that you login without it , he will wait that yoyu connect like this with all your account and will all shut them down ( i happened to me for 3000 of them ) , bg.request is part of the botguard process developped by google , it 's an enormous javascript script that generates a sort of sentence encrypted in Xtea with the key integrated in the bg request , the bg request contains your canvas identity webgl ....

@BitTheByte
Copy link
Owner

Hi @axel578

Could you please rephrase your comment? as i'm not able to understand it properly.

@axel578
Copy link
Author

axel578 commented Feb 23, 2019

botguard , the request you send should contains in data bg.request= ... , the ... should contains the bg data , those datas contains everything about your computer , if the bg.request is not here , all the accounts who send request without this bg data will be traced down and terminated .( closed )

@axel578
Copy link
Author

axel578 commented Feb 23, 2019

the idea know is to reverse engineer the bg.request wich is process with this script : https://textup.fr/313569za , to see the bg. request, just go to youtube/upload with the network inspector of chrome or firefox open (developper tool ) and upload a video and catch the request named ...rupio?authuser=1 , there are sometimes two of them , one of them contains : X-Goog-BGR: !4eKl4sNCyy... the goog-bgr is the botguard and after it is the encrypted text containing all your computer information , this encrypted text is generated by this script: https://textup.fr/313569za , just know have to reverse engenire it .

@BitTheByte
Copy link
Owner

Hey @axel578

Thanks for the clarification I appreciate it.

First of all i do understand your request but unfortunately I will not fix this issue.
I created this repository as a side fun project so me attempting to reverse/crack google's anti-bot protection is over do, instead of i could provide a another slow but reliable solution by using Chrome web driver For those who want a reliable results But using this mode will cost alot at the side of system resources since running ~20/30 threaded chrome instances simultaneously running Youtube will eat up at least 4/5GB of your RAM and probably 30~50% of the CPU if it's a high-end model

Of course any better ideas are welcomed so i'll wait a 5 to 8 weeks before implementing the Optional (ChromeDriver mode) just in-case anyone has came up with a better idea

@BitTheByte BitTheByte changed the title Becarefull , you missed one important thing Google bot-guard protection bypass Feb 24, 2019
@BitTheByte BitTheByte added enhancement New feature or request help wanted Extra attention is needed In progress Working on solution labels Feb 24, 2019
@BitTheByte
Copy link
Owner

Sorry for the delay. After further reviewing this i decided not to implement the suggested fix.

@ikp4success
Copy link

if you go to any youtube video link like https://www.youtube.com/watch?v=AV8GNQTOyBU, open the page source, there is "botguardData" in the youtube source, if you ctrl + f. If you can manage to regex out that data you can use that botguardData to login. All you have to do is make the request to random youtube video and regex that botguarddata before your google account login request. I am sure this will be blocked soon..lol

Another way is to try and sign in with a fake account while inspect element is opened and pre-save bgrequest in file or list, dict(do this process again till you have enough), and randomly get bg request from list/dict/file for acoount sign in. Bg request is persistent, but expire after some days.

Another way reverse engineer BGrequest.
In python
https://github.com/ikp4success/bypass_google_bot_guard

@BitTheByte
Copy link
Owner

Hello @ikp4success,

Interesting approach, I'll be taking this with the recent open issues in consideration in the next release thanks for helping out on this

@BitTheByte BitTheByte added published temporary fix and removed In progress Working on solution labels Dec 15, 2019
@BitTheByte
Copy link
Owner

Hello @ikp4success

I've been unable to bypass botgaurd using your python code. based on my knowledge creating a successful botgaurd token require some form of binary which get executed at custom javascript VM and return the token I've taken a shortcut by grabbing the binary from the page and let chrome do the heavy lifting. I also thought of getting as many tokens as possible and store them for later use but if tokens are verified based on timestamp or has a life span this will fail. I'll be researching this more. let me know if you have any ideas

@mewtlu
Copy link

mewtlu commented Apr 27, 2020

Hi @BitTheByte, sorry to necro a fairly old issue but have you gotten anywhere with this issue? I'd be really interested to hear whether you've found any methods of getting around Botguard!

@BitTheByte
Copy link
Owner

BitTheByte commented Apr 27, 2020

Hello @mewtlu

The bot-guard is actually very complex and many factors can effect the token generation process also google now uses more parameters than before which requires many requests to different endpoints. the login process has gotten exponentially complex don't forget it's powered by AI to detect any unusual behavior so even if I found a bypass to the botgaurd and a way to login I still have to deal with the AI. I will continue not working on this project until I found stable way retrieve cookies by email-password login

@mewtlu
Copy link

mewtlu commented May 8, 2020

Ah okay, thanks for the info! Will keep checking on this repo to see whether you come up with anything, interesting stuff!

@nilaydzya
Copy link

Getting this error as soon as I run the yt.py
[CORE]: Handing off botguard.js execution to chrome
[45060:12812:0529/223315.113:ERROR:configuration_policy_handler_list.cc(90)] Unknown policy: Quicdisabled
[45060:12812:0529/223315.286:ERROR:configuration_policy_handler_list.cc(90)] Unknown policy: Quicdisabled

DevTools listening on ws://127.0.0.1:56628/devtools/browser/7e9c6680-fd4a-43e7-aaad-542fb337b9c7
[45060:12812:0529/223315.441:ERROR:browser_switcher_service.cc(238)] XXX Init()

@sodevrom
Copy link

sodevrom commented Jun 11, 2020

Curious to see if anybody knows how to transform the "programdata" variable that we get from YouTube to the final request.
If you try to upload a video for example, you get a script that looks like this:
{"program":"DLB2YySmiIdSeak5DNP4p7n24iJVanhQwCYuZ179EihOWz9...","interpreterUrl":"\/\/www.google.com\/js\/bg\/imtBY8ACr81PWsnimbKUgHx1eKHLlspVp9xQGu-eDVE.js"}

We get programdata and interpreterUrl
Any idea on how to use interpreterUrl to parse programdata and get final string?

@BitTheByte
Copy link
Owner

Hello @sodevrom

Sorry for editing the command since it was too long that's being said, I used a trick like this when I published the temporary fix however now it's required to compute some other tokens besides the bot-guard one which is not an easy task.

@BitTheByte
Copy link
Owner

Just to let everyone know, if there's an app including web apps, Android or IOS that could interface google's API and could get valid cookies not API tokens from google I'm ready to reverse engineer it myself just make sure it's not just an iframe or webview and hosing google's login page inside it

@sodevrom
Copy link

Hey,
No problem for editing.
I am using GeckoFX c# to simulate logins and login to YouTube and then upload videos.
The idea is that I have been using the old YT interface to do the uploads, but now, YouTube is switching exclusively to the new YouTube interface.
So what I want to do is use geckofx to login to YouTube only, and then, upload the video via http requests.
I have the upload page that contains programdata for botguard, I have the link for the script, but I can't manage to execute the script with the programdata to get the encrypted botguarddata for the http request.

@sodevrom
Copy link

Wait... I think you saved me on this one. So since I am actually logged in a "real" browser, I see I can call "botguard.bg(PROGRAM DATA STRING).invoke()" and it gets me the code I need. I think it will work!

@BitTheByte
Copy link
Owner

BitTheByte commented Jun 11, 2020

if you're going to implement the same idea please note that bot-guard tokens are detecting (time, clicks, mouse movements, events) and feeding it to AI (which will detect your bot eventually) and are computed based on the browser itself make sure to add a bit of randomness to your program and use chrome-based webdriver

@sodevrom
Copy link

Thank you for the info!

@shahin1849
Copy link

DevTools listening on ws://127.0.0.1:64055/devtools/browser/79bc8f8a-e31d-4b61-95af-7e275e2e2547
Traceback (most recent call last):
File "C:\Users\MTIL-PC\AppData\Roaming\Python\Python39\site-packages\selenium\webdriver\common\service.py", line 72, in start
self.process = subprocess.Popen(cmd, env=self.env,
File "C:\Program Files\Python39\lib\subprocess.py", line 947, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Program Files\Python39\lib\subprocess.py", line 1416, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "F:\YouTubeShop-master\YouTubeShop-master\yt.py", line 100, in
driver = webdriver.Chrome(options=chrome_options);
File "C:\Users\MTIL-PC\AppData\Roaming\Python\Python39\site-packages\selenium\webdriver\chrome\webdriver.py", line 73, in init
self.service.start()
File "C:\Users\MTIL-PC\AppData\Roaming\Python\Python39\site-packages\selenium\webdriver\common\service.py", line 81, in start
raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home

@SeanWoo
Copy link

SeanWoo commented Dec 20, 2020

Hello, do you have any recent bot guard news?

@axel578
Copy link
Author

axel578 commented Dec 20, 2020

Dont want to seems defeated, but We should consider this issue hopeless since botguard is changing every day ( some sort of procedural javascript code generation on google side maybe ).Since 1st deobfuscating the code is just so much of hard work, but reverse engineering the patternt they use for their sort of procedural code generation is way harder to identify.

@thangtm0212
Copy link

Wait... I think you saved me on this one. So since I am actually logged in a "real" browser, I see I can call "botguard.bg(PROGRAM DATA STRING).invoke()" and it gets me the code I need. I think it will work!

hello, can you help me this issue?

@sabriabri88
Copy link

Wait... I think you saved me on this one. So since I am actually logged in a "real" browser, I see I can call "botguard.bg(PROGRAM DATA STRING).invoke()" and it gets me the code I need. I think it will work!

hello, can you help me this issue?

can you help me how can i contact you

@sabriabri88
Copy link

Hey @axel578

Thanks for the clarification I appreciate it.

First of all i do understand your request but unfortunately I will not fix this issue.
I created this repository as a side fun project so me attempting to reverse/crack google's anti-bot protection is over do, instead of i could provide a another slow but reliable solution by using Chrome web driver For those who want a reliable results But using this mode will cost alot at the side of system resources since running 20/30 threaded chrome instances simultaneously running Youtube will eat up at least 4/5GB of your RAM and probably 3050% of the CPU if it's a high-end model

Of course any better ideas are welcomed so i'll wait a 5 to 8 weeks before implementing the Optional (ChromeDriver mode) just in-case anyone has came up with a better idea

oki i need to test it how can i test it , i try to have the bgrequest for the signup bage not you tube

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request help wanted Extra attention is needed published temporary fix
Projects
None yet
Development

No branches or pull requests

10 participants