invalid_grant error when fetching Google Search Console domains -- whenever you setup several websites

[ameshkov]

Precheck

This might be important that I am testing the self-hosted version, but it looks like this bug should happen with the SAAS version as well.

Prerequisites

• I have searched open and closed issues to make sure that the bug has not yet been reported.

Bug report

1. Add two different websites to Plausible
2. Setup Google Search Console integration for the first of them - everything works great
3. Setup integration for the second one - everything seems to be okay as well
4. Check the first one again - you'll see the "invalid_grant" error there

Expected behavior
Integration works for both websites

Screenshots (If applicable)

[metmarkosaric]

thanks for reporting this @ameshkov! i tested it on the cloud but there it seems to work all fine. we'll check if we can see this on the self-hosted version.

by the way as i see you work for AdGuard: any chance we can get you to whitelist Plausible? you use Plausible yourself as I assume you believe that this is the better way of doing web analytics. not only as third-party but we've had couple of reports from people using your DNS that you're blocking our website too when it's first party so they for instance cannot read our blog without disabling your DNS. would be good if you can help out with this. thanks!

[ameshkov]

hi @metmarkosaric!

Note that I am using the same Google account to create integration with two different domains. Have you also used the same acc?

Regarding AdGuard, as I recall, in our tracking protection list we use a more granular approach and block just the JS file (so it's blocked even on our own website) and don't block the whole domain. Probably the rule comes from EasyPrivacy, I'll see if we can avoid blocking your website in AG DNS.

However, if you want a really viable way to avoid content blockers doing their job, please consider extending the API to allow importing website access logs. You see, the root cause for blocking is that there's an arbitrary JS loaded from a third-party. I understand that you're only sending a limited amount of data, but this can be changed any time, this is a good enough reason for people to be concerned about it.

Using a logs-based analytics system completely eliminates that possibility, there're no external trackers, site owners have their stats, everyone is happy. At first glance, it'd be enough simply to extend create_event and add an optional timestamp parameter. screen_width can be made optional and replaced with User-Agent parsing. I thought about pull requesting it myself, but I am not quite sure about how it aligns with your plans.

[metmarkosaric]

Thanks for the update! I tried with the same account and it works fine. I'm only testing the cloud version so not sure if there's something else on the self-hosted.

we have no current plans to import server logs as they're not that useful with mostly bot traffic so we would need to find a way to make those stats accurate and insightful first. we keep the option open for the future and there's a thread already.

we're an open source and transparent project so you'll easily see if we make some evil moves and can block us any time we do something you think is not ethical. but obviously it's your decision. just wanted to mention it as it's not the best user experience for your users to have normal websites blocked and then they have to reach out to us asking why our server is down.

[ameshkov]

Thanks for the update! I tried with the same account and it works fine. I'm only testing the cloud version so not sure if there's something else on the self-hosted.

Are there any logs I could look into and try to figure it out myself?

we have no current plans to import server logs as they're not that useful with mostly bot traffic so we would need to find a way to make those stats accurate and insightful first. we keep the option open for the future and there's a thread already.

Thank you, subscribed there! So the problem is just filtering out the bot traffic? We could experiment on our side with different ways of doing it and share the result.

just wanted to mention it as it's not the best user experience for your users to have normal websites blocked

Don't worry, we'll definitely address this in AG DNS. DNS is supposed to be casual-user friendly so we avoid blocking whole websites.

[metmarkosaric]

Are there any logs I could look into and try to figure it out myself?

I'm not aware of logs for self-hosting so someone else might be able to help with that part.

Thank you, subscribed there! So the problem is just filtering out the bot traffic? We could experiment on our side with different ways of doing it and share the result.

Some of the more interesting stats won't be able to be counted using server logs but the main issue really is bot traffic. You can read a bit more here on the comparison we did between Plausible and server logs accuracy: https://plausible.io/blog/server-log-analysis

[ukutaht]

Replying to @ameshkov

So the problem is just filtering out the bot traffic?

That's the biggest one, yes. We haven't found a server log tool that filters out bots anywhere close to as well as JS-based analytics tools do. I'm very curious about potential solutions in this space.

The second question is getting server logs into our system. I understand and agree that we could easily augment our /api/event endpoint to accept server logs. But how is a non-developer wordpress user supposed to integrate with this? Or even a developer running on Netlify? To make it truly usable we need countless numbers of different integrations for all potential tech/ops stacks.

A javascript-based tracker is much easier to integrate with since the client-side web is standardized. Even then, getting people to install the script is absolutely the most difficult part in getting them to use the tool.

I fear that even if we support some amazing ways to direct your server logs to Plausible, it will be a very niche feature used by <5% of our userbase. Our goal is to remove Google Analytics from as many sites as possible so it makes sense for us to focus on a solution that's convenient for the majority of our audience who is non-technical.

I'm not against log analysis or anything. Technically speaking it's preferable because the end-user doesn't need to load anything and you still get basic stats. If the bot issue can be solved I would happily build some integrations with backend tooling.

But I also believe that the JS-based tracker is more effective in getting people off GA which is our mission.

[ameshkov]

@ukutaht oh, I am not implying that you should choose logs over JS, this is definitely just a niche feature.

Also, from reading the thread that Marko linked it actually seems that this is already possible, we'll just have the timestamp a little bit out of sync, this is not that problematic. Anyways, thank you for the answers!

[ukutaht]

Sure, thanks for yout input @ameshkov.

BTW you can check logs with docker logs <name-of-your-container. Usually the container name is hosting_plausible_1 or something like that, you can find it with docker ps.

[danielpost]

I'm getting the same error, and mine is not self-hosted. Is there anything I can provide to give more info or context? It happens when trying to see which search terms are used in Google.

[metmarkosaric]

thanks @danielpost! you can email us with your details or post them here if you don't mind them being public and we can then take a manual look at your account to see what's happening.

[danielpost]

@metmarkosaric Thanks. The URL is https://plausible.io/socialimagegenerator.com.

[metmarkosaric]

thanks! can you please send a screenshot of how your Search Console report looks like in Plausible and in Search Console?

[danielpost]

@metmarkosaric I've been messing around with my G Suite account and I think I actually didn't have the property activated in Search Console anymore. So, I think in my case the error actually makes perfect sense 🤦🏼‍♂️. Sorry!

[metmarkosaric]

ahh ok i see! thanks for the update!

[JokerQyou]

The second website is not required to trigger this bug. You can just add a single site, verify that it works, and wait for 2 weeks, check Plausible dashboard again and it will break exactly the same way.

[hiepxanh]

I can confirm that, It work verysmooth at first time

[asifbacchus]

Same issue here. Tracking multiple domains all associated with one Google Search account. Plausible reports "invalid_grant" on the GA Search integration after a few days. This is the only thing preventing me from switching fully over to Plausible and becoming a sponsor. Everything else is great! I'm assuming this is also why "Top Sources" doesn't update, right?

Please let me know what I can provide to help with resolving this issue. Thanks!

[mhsabbagh]

I am facing the same issue here, self-hosted instance and only one website.

I tried regenerating new Google OAuth 2.0 keys and using them instead of the old ones, but still no luck.

[fheidenreich]

I'm also getting the invalid_grant error on my self-hosted instance for all tracked sites. It works initially, but after a few days clicking on the Google top source only shows the spinner and the Search Console setting shows invalid_grant. I then have to unlink the Google account and link it again, for each of the tracked sites.

None of the reasons listed at https://plausible.io/docs/google-search-console-integration#i-get-the-invalid-grant-error applies.

If there is anything I can do to prevent this from happening, or any further details I can provide, please let me know.

Edit 2022-04-22: It happened again.
Edit 2022-05-15: Haven't checked in a while, it happened again.

[gbissland]

We are getting this on the Cloud version (non-self hosted).
Lots of sites added.
Same as above, have to go through and re-link.

[metmarkosaric]

Thanks for the feedback @gbissland! How many websites did you add? Google has a limit of 50 live tokens. When you try to add the 51st site, the site you added first will be disconnected from your Plausible account and so on. This is a limitation on Google's side so if you have more than 50 different sites in your Search Console, you will need to split them between different Google accounts in order to get them into Plausible at the same time. We have listed some other reasons why Google sends the invalid_grant error here: https://plausible.io/docs/google-search-console-integration#i-get-the-invalid-grant-error

[mil-ad]

Having the same issue.

My container logs:

10:36:22.898 [error] Clickhousex.Protocol (#PID<0.2550.0>) disconnected: ** (Clickhousex.Error) closed
10:38:09.903 [error] Clickhousex.Protocol (#PID<0.2550.0>) disconnected: ** (Clickhousex.Error) closed
10:38:09.903 [error] Clickhousex.Protocol (#PID<0.2555.0>) disconnected: ** (Clickhousex.Error) closed
12:05:58.402 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
17:45:54.944 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
23:30:48.974 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:00:01.212 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:00:18.950 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:00:39.074 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:01:04.228 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:01:37.432 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:02:24.737 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:03:46.255 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:06:14.202 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:10:25.835 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:18:42.033 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:37:12.957 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
01:13:55.567 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
02:21:48.752 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
04:49:55.351 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
09:28:32.554 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
13:32:13.683 [error] Clickhousex.Protocol (#PID<0.2550.0>) disconnected: ** (Clickhousex.Error) closed
13:35:43.048 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
13:47:49.568 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
18:39:08.157 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
19:27:11.246 [error] Clickhousex.Protocol (#PID<0.2552.0>) disconnected: ** (Clickhousex.Error) closed
19:44:36.281 [error] Clickhousex.Protocol (#PID<0.2547.0>) disconnected: ** (Clickhousex.Error) closed
23:55:35.373 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:00:00.652 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:00:19.121 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:00:39.293 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:01:03.398 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:01:35.602 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:02:25.922 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:03:44.429 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:05:38.160 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:06:00.284 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:10:47.109 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:19:29.358 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
00:37:03.919 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
01:12:24.048 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
02:24:49.188 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
03:48:48.698 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
04:39:01.516 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
09:36:53.341 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
09:47:00.527 [error] Clickhousex.Protocol (#PID<0.2554.0>) disconnected: ** (Clickhousex.Error) closed
12:58:20.771 [warn] Failed to send Sentry event. Cannot send Sentry event because of invalid DSN
14:34:34.202 [error] Clickhousex.Protocol (#PID<0.2555.0>) disconnected: ** (Clickhousex.Error) closed

[asifbacchus]

Is this being looked into? The problem is still happening. If it is not a priority or cannot be fixed, it would be nice to be told that so I can evaluate my options going forward. If you need more information, I'm happy to provide it or even provide unlimited full access for any devs to an instance on my server if that would help. I would switch to the cloud service but it seems the issue occurs there too. Thanks in advance for any updates :-)

[fheidenreich]

Hi @metmarkosaric I think no issues is a bit of a stretch :) I'm also encountering this issue and made sure that none of the reasons from the linked document apply. Maybe there is a reason not listed in the document.

I updated my reply above a few times when I had to relink the Google account, but gave up on it eventually (it's again not working atm):

#690 (comment)

[metmarkosaric]

i see. you seem to be on a self-hosted instance. @asifbacchus mentioned the cloud which is why i responded. there's no issues with the Search Console integration on our managed hosting service in the cloud i should have written in full. self-hosted depends on many things in your setup and i'm not sure where it could go wrong. perhaps someone else who has had the same issue can help. sorry

[fheidenreich]

OK, thanks for clarifying — I misunderstood your reply! And yes, I'm on a self-hosted instance. Do you know of any place where potential errors might be logged and where I can start to poke around?

[metmarkosaric]

no problem! i'm not a developer so have no idea where to look. sorry i cannot help more! i do know that someone recently updated our self-hosted instructions for how to integrate with google as google often changes things so perhaps you can take an extra look and see if everything was the way it should be. this part specifically: https://plausible.io/docs/self-hosting-configuration#google-search-integration

[asifbacchus]

Yes, I should have been more clear too: I only mentioned the cloud because other users have said they had the same issue with the cloud hosted instance (see earlier in this thread). If the issue does not happen on the cloud hosted version, I may have to look more seriously into that option. It just sucks since I have many sites but very very very low traffic so self-hosted is ideal for me. I don't mind paying (sponsoring) I just don't want to pay monthly fees for sites that get like 20 combined visitors in total a month, haha :-P

[mhsabbagh]

I had this issue before like everyone else here, and simply linking and unlinking the website with Google Search Console fixed the problem for me:

The problem goes and come back every few weeks, and I have to link and unlink again... But eventually it works.

[asifbacchus]

Agreed @mhsabbagh, that is what I'm doing too. However, having to re-link all my sites every few days is tedious and leads to inaccurate stats during the 'unlinked' period. Seems pretty clear there is an issue with the grant/token expiring and that should be looked into, IMHO. Anyways, appreciate the feedback from everyone thus far, looking forward to maybe finding someone that comes up with a fix!

[metmarkosaric]

just a note that Search Console doesn't impact the number of visitors from Google that we display in the dashboard so there should be no inaccurate stats without Search Console. number of visitors from Google we get directly from our tracking script. Search Console integration helps us list the keywords people find your site with only (what you see when you click on Google in the Top Sources chart)

[asifbacchus]

@metmarkosaric Yes, correct! Thanks for clarifying, I meant that the keywords and that entire section is not accurate while un-linked since it obviously cannot be updated. To clarify for anyone else reading this thread: stats are UN-affected, in fact, everything else works amazingly well :-)

[hiepxanh]

Look like google do somthing. You should add your key again, I am using self-host version, google still sometime do something on it side. I have to add the key again

[mheland]

Bastard tokens expiring after 7 days if the App is in Testing. Could we kindly have a weekly refresh of the API token for Docker self-hosted, @metmarkosaric ?

A Google Cloud Platform project with an OAuth consent screen configured for an external user type and a publishing status of "Testing" is issued a refresh token expiring in 7 days.

Source: https://developers.google.com/identity/protocols/oauth2

[wheeleran]

I'm looking for a solution to this too, self-hosted, and it does seem to be the 7 day token that's the problem. But Points 4 and 5 in the support documentation are unclear, too, regarding" finally, return to APIs & Services and select Domain Verification..." with a few hints on where to find it, but i don't find it anywhere and am unsure whether i've even set this up properly. Are the Plausible docs up to date?

[bobsoap]

We're encountering this very issue on a selfhosted instance as well. Nearly 4 years later, it's still about GCP's expiring OAuth tokens.

Via @mheland's comment on Dec 10, 2022, this seems to be the cause:

A Google Cloud Platform project with an OAuth consent screen configured for an external user type and a publishing status of "Testing" is issued a refresh token expiring in 7 days.
Source: https://developers.google.com/identity/protocols/oauth2

How many sites are set up is not a factor.

Since this issue was converted to a discussion before this revelation and the problem persists, I'm proposing chaning it back to an issue. I apologize if this is against protocol.

Thanks.

[bobsoap]

Created issue #4784