Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support time points as numeric indexes #240

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from
Draft

Conversation

moznuy
Copy link
Contributor

@moznuy moznuy commented May 7, 2022

This is a rough draft

What's is implemented:

  • datetime.datetime works as numeric index.
  • datetime.date works as numeric index.
  • datetime.time works as numeric index. (very sketchy, see below)
  • Add test for all above for:
    • Saving to Redis, checking if is valid.
    • Finding by less operator(any comparison should work) in sorted fashion.
    • Saving no-UTC and checking that restored version(UTC) is equal to pre-saving no-UTC value.

What can be added / improved:

  • datetime.time custom parsing implementation
  • Our type for datetime.time because user will have to write parsing(validation) themselves otherwise(see post_model_time in tests).
  • Support for datetime.timedelta (I skipped it for now).
  • Support for Model.find(Model.field < "Thu Sep 25 10:36:28 2003") or ISO format, or etc...
  • Documentation.
  • More tests.
  • MORE TESTS(Seriously, I don't think I understand time zones 100%).

Questions:

  • How to store timezone information as numeric field?
    Right now I just convert everything to UTC before saving. (This needs to be checked)
  • What to do if user did not provide timezone?
  • Do we want to support Model.find(Model.field < "Thu Sep 25 10:36:28 2003") or "2003-09-25T10:49:41.5-03:00" or any other string in expressions. (Because pydantic supports only ISO format(I think)). We could use dateutil. Or make it optional dependency.
  • Precision. To what precision should we save time-points. I implemented milliseconds because:
  • Cross type expression(Model.datetime < date/time). Should we support this? Or catch this and raise exception?
  • Will Redis at some point add "Timepoint" index field type(like NUMERIC)? (with timezone support). Because that would simplify things.
  • I probably forgot thing or 10, will add later

Problems:

  • datetime.time: pydantic support parsing only as:
    time, existing time object
    str, following formats work:
       HH:MM[:SS[.ffffff]][Z or [±]HH[:]MM]]]
    
    so no int parsing , very custom logic(probably wrong)

Sketchy things:

Fixes #199

@codecov-commenter
Copy link

codecov-commenter commented May 7, 2022

Codecov Report

Merging #240 (97c617c) into main (c2339b3) will increase coverage by 0.36%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #240      +/-   ##
==========================================
+ Coverage   77.25%   77.61%   +0.36%     
==========================================
  Files          12       12              
  Lines        1165     1184      +19     
==========================================
+ Hits          900      919      +19     
  Misses        265      265              
Flag Coverage Δ
unit 77.61% <100.00%> (+0.36%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
aredis_om/model/encoders.py 59.77% <100.00%> (+8.38%) ⬆️
aredis_om/model/model.py 86.33% <100.00%> (+0.06%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c2339b3...97c617c. Read the comment docs.

@moznuy
Copy link
Contributor Author

moznuy commented May 12, 2022

Any input will be appreciated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Find operation on datetime field raises error --> argument of type 'datetime.datetime' is not iterable
2 participants