Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spotlight hour calendar not using correct local time #6

Open
ShahrumAmiri opened this issue Aug 9, 2023 · 31 comments
Open

Spotlight hour calendar not using correct local time #6

ShahrumAmiri opened this issue Aug 9, 2023 · 31 comments
Labels
bug Something isn't working

Comments

@ShahrumAmiri
Copy link

Description

Hello,

I’m currently subscribed to the .ics feed (via Google Calendar) for the spotlight hour calendar and the events are coming through at 2PM EDT instead of 6PM EDT, the expected time for my timezone. Is there a configuration I’m missing?

Version

Latest

Steps to Reproduce

Add calendar feed to Google Calendar
Events loaded
Look at calendar and see events at wrong local time (expected 6pm EDT, seen at 2pm EDT)

Screenshots

No response

Device

All

Operating System

Latest

Additional Context

No response

@ShahrumAmiri ShahrumAmiri added the bug Something isn't working label Aug 9, 2023
@PokeIMGs
Copy link

Having similar issues.

What I found is just importing the ICS file directly into another calender works fine, but subscribing sets the times out of whack.

I am GMT + 1 (BST)

Spotlight Hours are an hour ahead
Raid Hours are 7 hours ahead
Go fest global is an hour ahead
Community day is 5 hours ahead
City Sefari Seoul and Barcelona are an hour ahead, but Mexico is the right time (funilly enough this is after daylight savings ends)

@ShahrumAmiri
Copy link
Author

ShahrumAmiri commented Aug 11, 2023 via email

@PokeIMGs
Copy link

PokeIMGs commented Aug 11, 2023 via email

@kiararey
Copy link

kiararey commented Aug 21, 2023

I'm having a similar issue. I'm on GMT-5, and the following are at different times for me:

Events calendar synced fine I think, though Noxious Swamp should end on Tuesday and not Monday
Spotlight hours: show up 5 hours early
Raid hours: show up 1 hour early
GO fest: show up 5 hours early
I also subscribed to the community day calendar but there's currently no event for me to check if it works correctly

@othyn
Copy link
Owner

othyn commented Aug 24, 2023

This is the same issue as #4, it appears to be a long standing issue specifically with Google Calendar's processing of the feed. For whatever reason, it seems to imply a TZ from some arbitrary place and use that for the subscribed feed. All other calendar services/clients do what they are supposed to and just use the local users TZ.

I have no idea why it does it and all the fixes I've put in appear to have made no difference. I purposefully do not embed a TZ within the generated files so that the consuming client will use its own and correctly translate it into the local TZ.

#5 aimed to resolve this issue, but I haven't had time yet to look into it properly as its a large amount of infra work, basically to generate versions for all timezones on the fly, which appears to be the only way to have it work with Google Calendar. Which is what I wanted to avoid at all costs as it adds a ton of complexity. A dynamic URL that holds all the necessary query parameters (what calendars, TZ's, etc.), so you can still subscribe to it, but each time its visited a service worker can spit out the ICS file dynamically.

If someone knows a concrete fix for this, or how the files can be correctly imported to Google Calendar using the users local TZ, I'd love to hear it as at this point I'm stumped.

@ShahrumAmiri
Copy link
Author

ShahrumAmiri commented Aug 24, 2023 via email

@othyn
Copy link
Owner

othyn commented Aug 24, 2023

No specific recommendations, I just use the default Apple ones too.

Google Calendar appears to be the only calendar affected, based on user reports of this issue.

@gaylie
Copy link

gaylie commented Oct 9, 2023

Hello. Poking in to say that the issue still seems to exist. Also using Google Calendar, the times for Spotlight hours show up at 8PM-9PM for me

@othyn
Copy link
Owner

othyn commented Oct 9, 2023

Not really sure what else I can do on my side, I'd love someone's input with more knowledge or experience with Google Calendar event generation from a dev perspective.

Someone submitted an amazing PR for dynamic generation of calendars for each timezone, however its a bit of a heavy handed solution and it feels like there should be an easier solve that I'm just not seeing.

It appears to only be a Google Calendar issue with a really weird and non-industry standard interpretation (from what I can see, would like to be corrected on that!) of the generated events. What is extra weird is that it only appears to be a problem for subscriptions, if you manually import the calendar as a one-off into Google Calendar then its fine and imports it with the local timezone interpreted as intended. All events are generated in a way in which forces there to be no timezone and its left to the local client to interpret the timezone.

When subscribing to it, Google just forces the timezone to be (GMT+00:00) Coordinated Universal Time instead of interpreting it into the users timezone like all other clients do, and you are not given the option in the calendar settings to change it.

@othyn
Copy link
Owner

othyn commented Oct 9, 2023

On a hunch, I've tried something and it appears to have worked in my testing. I've subscribed to the production one and this new one side by side, and the new one imports into the correct timezone (client) whereas the old one appears an hour after it should.

Can people subscribe to the temporary calendar I've created with a potential fix in here and report back?

https://raw.githubusercontent.com/othyn/go-calendar/gcal-sub-fix/dist/gocal.ics

Please let me know if that subscribes correctly?

If it does, I'll have to publish a set of 'If you're using gcal, use this link instead' files specifically for Google Calendar subscriptions.

@gaylie
Copy link

gaylie commented Oct 9, 2023

On a hunch, I've tried something and it appears to have worked in my testing. I've subscribed to the production one and this new one side by side, and the new one imports into the correct timezone (client) whereas the old one appears an hour after it should.

Can people subscribe to the temporary calendar I've created with a potential fix in here and report back?

https://raw.githubusercontent.com/othyn/go-calendar/gcal-sub-fix/dist/gocal.ics

Please let me know if that subscribes correctly?

If it does, I'll have to publish a set of 'If you're using gcal, use this link instead' files specifically for Google Calendar subscriptions.

Didn't expect such a quick response!

I subscribed to the link, though sadly it still seems slightly off (differently so than before, though).
For some examples: Spotlight Hours show up at 7PM-8PM for me, Pokestop Showcases from 11AM-9PM, Raid Hours from 7PM-8PM... Generally they seem an hour off for me.

That said, I am in Central European Summer Time (CEST) as of right now, and the times WOULD be correct for CET (Winter time). Perhaps Summer/Winter timezones are messing this up?

@PokeIMGs
Copy link

PokeIMGs commented Oct 9, 2023 via email

@othyn
Copy link
Owner

othyn commented Oct 9, 2023

That said, I am in Central European Summer Time (CEST) as of right now, and the times WOULD be correct for CET (Winter time). Perhaps Summer/Winter timezones are messing this up?

That's like a timezone in a timezone problem haha although I'm currently in BST (UTC+1) and it is working, which is the equivalent summer offset.

I need some people in more timezones to test it. I want to make sure its a local timezone fix and not something else, such as just a +1 offset that Google Calendar is applying internally (as CEST being UTC+2 is still leaving only an hour offset).

Tested using Google cal URL subscription on British Timezone (BST/GMT+1)

Awesome, that aligns with my testing too in BST.

@kiararey
Copy link

kiararey commented Oct 9, 2023

On a hunch, I've tried something and it appears to have worked in my testing. I've subscribed to the production one and this new one side by side, and the new one imports into the correct timezone (client) whereas the old one appears an hour after it should.

Can people subscribe to the temporary calendar I've created with a potential fix in here and report back?

https://raw.githubusercontent.com/othyn/go-calendar/gcal-sub-fix/dist/gocal.ics

Please let me know if that subscribes correctly?

If it does, I'll have to publish a set of 'If you're using gcal, use this link instead' files specifically for Google Calendar subscriptions.

this new calendar still shows the spotlight hours at 1pm on Google Calendar :(

@othyn
Copy link
Owner

othyn commented Oct 9, 2023

Damn, okay. Thank you for trying it and reporting back, that was a valuable insight into if this was a valid fix or not.

I don't think there is much more I can do then, it appears to be an issue specifically with Google Calendar subscription feeds and whatever they are doing to force/ignore timezones from what I can gather.

All other clients respect the standards and use the clients local timezone when one isn't specified, appears Google Calendar is the only one to not adhere to that rule, and also only on subscriptions - manual imports on Google Calendar work fine.

If anyone has any other ideas or fixes, I'm all ears. The complete re-write approach (PR) is awesome, but I'd rather not go that route if at all possible just due to whats involved. Especially as I don't play the game anymore either.

@KemptonM
Copy link

KemptonM commented Jan 9, 2024

For what it's worth, Google Calendar used my default time zone (where I live) but it was incorrect for where I currently am, living abroad.

@Boby360
Copy link

Boby360 commented Feb 4, 2024

To add to this, this is what is shown in Outlook.
The timezone is forced to UTC and I cannot change it when subscribing as it is read only.

My phone however shows it has my timezone, but still 6-9AM.
As this subscription is synced across devices I don't know if one thing is fighting the other, but they both show incorrect times.
It sounds like it was suggested to only be a google calendar issue, so figured id mention my observations.
These are synced through outlook on my hotmail email through an exchange server.

image
image
image
You can see what I see in my calendar on the last image.

@othyn
Copy link
Owner

othyn commented Feb 4, 2024

This issue is the worst, and a prime example of XKCD 927:

image

Every. Single. Client. appears to handle the 'standard' timezone signature differently.

Some of them interpret the lack of a timezone correctly as 'use the local users timezone'. Some of them read it in as fixed UTC. Some of them read it in as an undefined timezone. Some of them read it in as whatever they feel like that day.

It varies per device, per client and per calendar permutation. A thorn in my side on what should have been a very simple 'please just use the users local timezone' implementation.

I may run this problem through ChatGPT - as that wasn't an option when this was created, as I've exhausted just about every single blog, docs and SO post imaginable on this problem to try and resolve it. Give the AI a run at it lmao

@Boby360
Copy link

Boby360 commented Feb 4, 2024

I forked it and ran some versions through chatgpt but didn't have any improvement.

@othyn
Copy link
Owner

othyn commented Feb 4, 2024

Yeah I had a rather lengthy back and forth with ChatGPT over this, feeding it different examples and trying to get it to generate its own approach and compare. It just confirms the information that I've already found at length on the topic - this shouldn't be an issue and I still cannot understand why it is. The examples that it generates are identical to the ones that this repo does.

The goal: Given a hypothetical event, let's say starts at 6:00 PM, users in London and New York will see the event at their respective local times. If a user in London downloads the calendar, they will see the event at 6:00 PM local time in London. If a user in New York downloads the calendar, they will see the event at 6:00 PM local time in New York.

The recommended fix, as is already implemented, is to not define a timezone (e.g. TZID=Europe/London) within the DTSTART and DTEND components, at which point the calendar client should use floating time (without a specific timezone), adjusting to the user's local timezone.

That is how the current live implementation of any calendar generated by this tool is generated, so I'm still at at a loss to why any of this is an issue.

Google Calendar, Outlook/O365, iOS/iPadOS/macOS (Apple Calendar), etc. all support 'floating time' as defined by the absence of a timezone (e.g. TZID=Europe/London) definition within the DTSTART and DTEND components, which is exactly what this repo does.


If someone can find a working example on the web for this sort of problem, then I can reverse engineer a solution. I did think I found one in the Formula 1 calendar project, but those events are fixed in time to each timezone, as an F1 race occurs at a fixed point in time within a specified timezone and therefor will need to be timezone adjusted depending on the embedded timezone of which the race takes place in (and therefor translated by the client). Pokemon GO events however occur at the local time for each user, so its not a comparison that can be made.


Using a tool like https://icalendar.org/validator.html shows that the file is indeed valid with no errors. There is a warning about headers, but I can't change those as I'm just hosting via GitHub and have no control over the web server. Although that shouldn't have any affect as its just the content type header.


If all calendar clients adopted or adhered to RFC 5545 (as they probably should be), then we shouldn't have an issue:

The use of local time in a DATE-TIME or TIME value without the
"TZID" property parameter is to be interpreted as floating time,
regardless of the existence of "VTIMEZONE" calendar components in
the iCalendar object.

Which in practice translates to:

"floating time" is a commonly used term in the context of iCalendar (ICS) files, which are used to exchange calendar and scheduling information between users and systems. In the iCalendar specification (RFC 5545), a "floating time" refers to a time that is not associated with a specific time zone.

In the context of iCalendar events, a floating time is expressed without a time zone identifier. It is assumed to be in the local time zone of the calendar client or the observer. This allows the event to be displayed consistently across different time zones, adjusting to the local time zone of the user viewing the calendar.

So, when you specify an event with a time but without an explicit time zone (e.g., DTSTART:20231201T180000), it's considered a floating time. The interpretation of floating times depends on the local time zone settings of the calendar client or application displaying the event.

If an event in an ICS file is specified in floating time (without a specific time zone) and occurs at 6:00 PM, the event will be interpreted in the local time zone of each user's calendar client. Here's how it would appear for a user in London and another in New York:

  1. User in London:
    • The event will be displayed at 6:00 PM in the local time zone of London (GMT or GMT+1 depending on daylight saving time).
    • London time will be used as the reference, and the event will be shown at 6:00 PM on the user's calendar.
  2. User in New York:
    • The event will be displayed at 6:00 PM in the local time zone of New York (Eastern Time).
    • New York time will be used as the reference, and the event will be shown at 6:00 PM on the user's calendar.

In summary, when using floating time, the event time will adjust to the local time zone of each user, ensuring that users in different locations see the event at the appropriate time in their local time zone.

And thus we find ourselves back at square one. This problem simply shouldn't exist, if everyone is playing by the standards - but as with the modern web, that is rarely the case 🤷

@othyn
Copy link
Owner

othyn commented Feb 4, 2024

I've just done another test. No matter what, Google Calendar will always set the timezone of the calendar subscription to (GMT+00:00) Coordinated Universal Time with the field to change it being non-editable. Whereas, Apple Calendar correctly adheres to RFC 5545 and doesn't impose a timezone on the subscribed calendar.

Screenshot 2024-02-04 at 23 48 35

The events correctly display, and even when changing local timezone, the events remain at the same time respective to the local current timezone and not translated from UTC (they remain at the same time regardless of timezone).

Whereas in Google Calendar, the timezone is set to UTC by Google and you cannot change it, going against RFC 5545:

Screenshot 2024-02-04 at 23 55 24

Meaning if your timezone deviates from UTC, then Google will try to be 'helpful' and translate it into your local timezone, going against RFC 5545's defined handling of floating time.

The only solution to this would be something as built in #4 and #5, which is a complete re-write of the project to dynamically generate ICS files in each and every timezone. Urgh.

@othyn
Copy link
Owner

othyn commented Feb 5, 2024

There's a few comments on Hacker News that point to this being the case since at least 2019:


nfriedly on March 27, 2019

Google Calendar does let you set the time zone in the 'More options' view, FWIW. I use it occasionally.

maxxxxx on March 27, 2019

I know. But I want no time zone. When I plan my Germany trip next week I want to see the 10 am meeting there at 10 regardless of where I am.
It should operate like a good old paper calendar.

rlpb on March 27, 2019

The iCalendar standard does actually support that, but perhaps Google Calendar's UI does not. I suspect it does work internally, because it interoperates with iCalendar fairly well.

The standard calls it a "floating" DATE-TIME:

"The recipient of an iCalendar object with a property value consisting of a local time, without any relative time zone information, SHOULD interpret the value as being fixed to whatever time zone the "ATTENDEE" is in at any given moment."

https://news.ycombinator.com/item?id=19501955


funcDropShadow on March 27, 2019

For that you would like to have an explicit "local time zone" option. I had recently similar in application I am > developing, with datetime of which we are not sure in which time zone they are. That information might arrive later.

maxxxxx on March 27, 2019

On my Palm Pilot there was a “floating” time zone that behaved like this. But I can’t find this feature in any modernapp. Outlook doesn’t have it either.

kalleboo on March 27, 2019

Apple's calendar has it on macOS, but not on iOS... (it's annoying how mobile everything always has to be so neutered)

Nemo157 on March 27, 2019

And yet iOS calendar is also more powerful in that you can set different time zones on the start and end of a single > event, which macOS calendar will show but not let you edit.

https://news.ycombinator.com/item?id=19501832


I can attest to that last point. There are so many features that iOS Calendar can do that macOS Calendar cannot. Such as changing the starting location when adjusting the travel time... always have to swap to my iPhone to update calendar entries because the macOS version doesn't have feature parity.

@othyn
Copy link
Owner

othyn commented Feb 5, 2024

There are also many support threads such as this one on the Google support forums in which have been asking for this 'feature' for years.

@Boby360
Copy link

Boby360 commented Feb 5, 2024

I assume this is not up to date with your most recent commits, but just wanted to show that my S23 Samsung calendar shows/previously showed my timezone and the GMT time window.
Screenshot_20240204_170124_Calendar

@Boby360
Copy link

Boby360 commented Feb 5, 2024

I've just done another test. No matter what, Google Calendar will always set the timezone of the calendar subscription to (GMT+00:00) Coordinated Universal Time with the field to change it being non-editable. Whereas, Apple Calendar correctly adheres to RFC 5545 and doesn't impose a timezone on the subscribed calendar.

Screenshot 2024-02-04 at 23 48 35 The events correctly display, and even when changing local timezone, the events remain at the same time respective to the _local current_ timezone and _not_ translated from UTC (they remain at the same time regardless of timezone).

Whereas in Google Calendar, the timezone is set to UTC by Google and you cannot change it, going against RFC 5545:

Screenshot 2024-02-04 at 23 55 24 Meaning if your timezone deviates from UTC, then Google will try to be 'helpful' and translate it into your local timezone, going against RFC 5545's defined handling of floating time.

The only solution to this would be something as built in #4 and #5, which is a complete re-write of the project to dynamically generate ICS files in each and every timezone. Urgh.

Instead of generating multiple ICS files, the ICS file should be able to have time per each timezone within the ICS.
This still seems REAL stupid, although I would think a handful of timezones would probably cover the 99%?

Something like, -10 (Hawaii) then -8 to -5(Most of North/South America and then -1 to +2(EU and whatnot) then some japan, India, Australia and Newzealand?

I recall reading an article about how timezones (without all of this calendar stuff) is crazy complicated to have every timezone perfect.

@Boby360
Copy link

Boby360 commented Feb 5, 2024

This is supposed to auto-update right?

I manually deleted and added it, and it shows the correct time and the abbreviated title!!!
Timezone is not forced/show defined as it did previously.
image

Something I should of also mentioned./reminded... I think CD day was the only one broken.
( I checked the other issue, and indeed, it was the only one showing 6AM while the rest showed 6PM........)

I forgot to point to that being a localized issue and not an overall ICS issue... ooops.

@othyn
Copy link
Owner

othyn commented Feb 6, 2024

Instead of generating multiple ICS files, the ICS file should be able to have time per each timezone within the ICS.
This still seems REAL stupid, although I would think a handful of timezones would probably cover the 99%?

I didn't realise this! That could be a nice interim fix, I may have a play with doing that. Seems like covering the 9 below may gather the most coverage?

  1. Floating Time: Leave it in as the first option, so if the calendar client supports it, use it.
  2. UTC (Coordinated Universal Time): This serves as the reference time zone and is commonly used in international contexts.
  3. America/New_York: Represents Eastern Time in the United States.
  4. America/Los_Angeles: Represents Pacific Time in the United States.
  5. Europe/London: Represents Greenwich Mean Time (GMT) in the United Kingdom.
  6. Europe/Berlin: Represents Central European Time (CET) in mainland Europe.
  7. Asia/Tokyo: Represents Japan Standard Time (JST) in Japan.
  8. Asia/Shanghai: Represents China Standard Time (CST) in China.
  9. Australia/Sydney: Represents Australian Eastern Standard Time (AEST) in Australia.

I manually deleted and added it, and it shows the correct time and the abbreviated title!!!

Futurama good news everyone

I forgot to point to that being a localized issue and not an overall ICS issue... ooops.

Haha no worries. Odd that it would only happen in the Community Day ICS file... such a bizarre problem all round...

@Boby360
Copy link

Boby360 commented Feb 6, 2024

Instead of generating multiple ICS files, the ICS file should be able to have time per each timezone within the ICS.
This still seems REAL stupid, although I would think a handful of timezones would probably cover the 99%?

I didn't realise this! That could be a nice interim fix, I may have a play with doing that. Seems like covering the 9 below may gather the most coverage?

  1. Floating Time: Leave it in as the first option, so if the calendar client supports it, use it.
  2. UTC (Coordinated Universal Time): This serves as the reference time zone and is commonly used in international contexts.
  3. America/New_York: Represents Eastern Time in the United States.
  4. America/Los_Angeles: Represents Pacific Time in the United States.
  5. Europe/London: Represents Greenwich Mean Time (GMT) in the United Kingdom.
  6. Europe/Berlin: Represents Central European Time (CET) in mainland Europe.
  7. Asia/Tokyo: Represents Japan Standard Time (JST) in Japan.
  8. Asia/Shanghai: Represents China Standard Time (CST) in China.
  9. Australia/Sydney: Represents Australian Eastern Standard Time (AEST) in Australia.

I manually deleted and added it, and it shows the correct time and the abbreviated title!!!

Futurama good news everyone

I forgot to point to that being a localized issue and not an overall ICS issue... ooops.

Haha no worries. Odd that it would only happen in the Community Day ICS file... such a bizarre problem all round...

This is purely based on whatever chatgpt mentioned a couple times. Didn't validate and would take with a grain of salt. (the multi timezone data per ICS file)

Happy to see it working though. Thanks a lot for all your work :)

@othyn
Copy link
Owner

othyn commented Feb 6, 2024

This is purely based on whatever chatgpt mentioned a couple times. Didn't validate and would take with a grain of salt. (the multi timezone data per ICS file)

Ahhhh okay. Yeah having a look at the RFC and some SO posts on this, it doesn't appear to be actually possible in practice, its one DTSTART and DTEND per VEVENT block. Oh well, back to the drawing board!

@WobblySausage
Copy link

Has this ever been resolved? Ive tried to use it today via Google Calendar, and manually importing the iCal into a Discord bot. The time is always +1 extra.

Raids are always 18:00 local time, but in Google its still saying 19:00, in the Discord Bot to organise events, also says 19:00

@othyn
Copy link
Owner

othyn commented Jul 12, 2024

Has this ever been resolved? Ive tried to use it today via Google Calendar, and manually importing the iCal into a Discord bot. The time is always +1 extra.

Raids are always 18:00 local time, but in Google its still saying 19:00, in the Discord Bot to organise events, also says 19:00

Unfortunately not... I've tried just about everything at this point and nothing will get Google calendar (and now apparently Discord too) to interpret the ICS file properly, specifically when subscribed to. And at this point, I'm convinced it's a GCal bug.

The ICS spec says if you don't define a time zone, it should be interpreted in the users local calendar time. Apple (iOS, macOS, etc.) Calendar does this correctly, along with Microsoft too (Outlook, etc.) They both correctly adhere to the spec, even for subscribed calendars.

However, Google doesn't. They do if you import the calendar statically/manually from a download of the ICS file, but if you subscribe to it, for some reason it behaves differently. It assumes a time zone and removes the option to specify a time zone override, like you can when manually imported. But ironically, you then dont need the override, as it's interpreted correctly.

The only option is a complete rewrite of the entire repo to fix GCal, something similar to the serverless PR, to allow you to select a time zone and what event types you want and store that as a sequence of query parameters for the calendar URL. Basically move from static generation and relying on the client behaviour to offload compute locally, to dynamic generation for each client specific calendar server side, with aggressive daily caching. This way instead of publishing static calendars with no TZ definition, which should work according to the spec, they'd always be rendered server side/dynamically for the TZ you select.

My only concern on going that route and utilising something like CF's free tier for it, is that the repo gets a huge amount of hits, in the hundreds of thousands to millions a month, as the calendar clients are instructed to check daily for updates, as aligns with the cadence of updates from LeekDuck. It's an unavoidable requirement. And as we've seen more recently with CF on hacker news, I'm not sure they'd want it on the free tier for long. At the moment, it's all statically generated so that I can utilise GH pages/releases to host the static assets for free. I'm not even sure a cheap £5/mo DO droplet would do it, although I don't mind taking on that sort of cost or equivalent. I just love the simplicity of the deployment/hosting pipeline at the moment haha

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

8 participants