Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX: Tableau writer schema mismatch with Bus schema #486

Merged
merged 9 commits into from
Mar 10, 2025
Merged

Conversation

huangh
Copy link
Contributor

@huangh huangh commented Mar 7, 2025

The bus_schema defined in the Tableau writer needs to be in the same order as the polars schema from the Bus_Performance_Manager outputs in bus_vehicle_events.

Changes:

  • Re-aligns the order of the schema
  • Add a select statement to ensure the read in Table is in the correct schema order (2nd line of defense)
  • Refactor conversions for tableau analysis out into a new method for easy testing and verification
  • Add new analysis scripts that help check for issues in Tableau/parquet writing

@huangh huangh requested a review from rymarczy March 7, 2025 18:19
Copy link

github-actions bot commented Mar 7, 2025

LCOV of commit 64b8783 during Continuous Integration (Python) #942

Summary coverage rate:
  lines......: 70.5% (2697 of 3824 lines)
  functions..: no data found
  branches...: no data found

Files changed coverage rate:
                                                                                     |Lines       |Functions  |Branches    
  Filename                                                                           |Rate     Num|Rate    Num|Rate     Num
  =========================================================================================================================
  src/lamp_py/tableau/conversions/convert_bus_performance_data.py                    | 100%      7|    -     0|    -      0
  src/lamp_py/tableau/hyper.py                                                       |23.6%    127|    -     0|    -      0
  src/lamp_py/tableau/jobs/bus_performance.py                                        |55.2%     58|    -     0|    -      0

rymarczy
rymarczy previously approved these changes Mar 10, 2025
Copy link
Collaborator

@rymarczy rymarczy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Couple notes, 1 question.

🍰

Copy link

LCOV of commit 58171f0 during Continuous Integration (Python) #943

Summary coverage rate:
  lines......: 70.6% (2697 of 3822 lines)
  functions..: no data found
  branches...: no data found

Files changed coverage rate:
                                                                                     |Lines       |Functions  |Branches    
  Filename                                                                           |Rate     Num|Rate    Num|Rate     Num
  =========================================================================================================================
  src/lamp_py/tableau/conversions/convert_bus_performance_data.py                    | 100%      7|    -     0|    -      0
  src/lamp_py/tableau/hyper.py                                                       |24.0%    125|    -     0|    -      0
  src/lamp_py/tableau/jobs/bus_performance.py                                        |55.2%     58|    -     0|    -      0

Copy link
Collaborator

@rymarczy rymarczy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🍰

@huangh huangh merged commit 0c76a7d into main Mar 10, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants