Skip to content

Conversation

@tdcmeehan
Copy link
Contributor

@tdcmeehan tdcmeehan commented Oct 28, 2025

Description

Fix timestamp handling in MySQL and PostgreSQL JDBC connectors when legacy_timestamp=false. Timestamps are now correctly stored and read as wall-clock values without timezone conversion.

Motivation and Context

When legacy_timestamp=false (the default), the JDBC connectors were incorrectly applying JVM timezone conversion on both reads and writes. This caused:

  • Values inserted directly into the database were read with incorrect timezone conversion
  • Values inserted via Presto were written with incorrect timezone conversion

The fix adds UTC calendar parameter for non-legacy timestamp mappings so they use wall-clock semantics.

Impact

Fixes data corruption for users with non-UTC JVM timezone.

Test Plan

Included unit tests.

Contributor checklist

  • Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.
  • If adding new dependencies, verified they have an OpenSSF Scorecard score of 5.0 or higher (or obtained explicit TSC approval for lower scores).

Release Notes

== RELEASE NOTES ==

MySQL Connector Changes
* Fix timestamp handling when ``legacy_timestamp`` is disabled. Timestamp values are now correctly stored and retrieved as wall-clock times without timezone conversion. Previously, values were incorrectly converted using the JVM timezone, causing data corruption.

PostgreSQL Connector Changes
* Fix timestamp handling when ``legacy_timestamp`` is disabled. Timestamp values are now correctly stored and retrieved as wall-clock times without timezone conversion. Previously, values were incorrectly converted using the JVM timezone, causing data corruption.

@prestodb-ci prestodb-ci added the from:IBM PR from IBM label Oct 28, 2025
@sourcery-ai
Copy link
Contributor

sourcery-ai bot commented Oct 28, 2025

Reviewer's Guide

This PR enforces UTC-based timestamp handling in JDBC connectors by introducing an explicit UTC calendar for read/write operations, preserves legacy behavior via new mappings toggled by the legacy_timestamp session property, propagates this flag through connector APIs, and adds rigorous tests validating timestamp/datetime storage and retrieval in both modes for PostgreSQL and MySQL.

Sequence diagram for timestamp read/write with legacy flag propagation

sequenceDiagram
    participant "User Query (Presto)"
    participant "ConnectorSession"
    participant "JdbcClient"
    participant "StandardColumnMappings"
    participant "Database (JDBC)"
    "User Query (Presto)"->>"ConnectorSession": Execute query with session
    "ConnectorSession"->>"JdbcClient": Request toReadMapping / toWriteMapping with session
    "JdbcClient"->>"StandardColumnMappings": Pass legacyTimestamp flag from session
    alt legacyTimestamp = true
        "StandardColumnMappings"->>"Database (JDBC)": Use legacy mapping (no UTC calendar)
    else legacyTimestamp = false
        "StandardColumnMappings"->>"Database (JDBC)": Use UTC calendar for timestamp
    end
Loading

ER diagram for timestamp mapping modes in session

erDiagram
    CONNECTOR_SESSION {
        bool legacyTimestamp
    }
    STANDARD_COLUMN_MAPPINGS {
        method timestampReadMapping()
        method timestampReadMappingLegacy()
        method timestampWriteMapping(TimestampType)
        method timestampWriteMappingLegacy(TimestampType)
    }
    CONNECTOR_SESSION ||--o| STANDARD_COLUMN_MAPPINGS : "uses legacyTimestamp to select mapping"
Loading

Class diagram for updated timestamp mapping in JDBC connectors

classDiagram
    class StandardColumnMappings {
        +booleanReadMapping()
        +timeWriteMapping()
        +timestampReadMapping()
        +timestampReadMappingLegacy() <<deprecated>>
        +timestampWriteMapping(TimestampType)
        +timestampWriteMappingLegacy(TimestampType) <<deprecated>>
        +uuidWriteMapping()
        +jdbcTypeToReadMapping(JdbcTypeHandle)
        +prestoTypeToWriteMapping(ConnectorSession, Type)
    }
    class BaseJdbcClient {
        +getColumns(ConnectorSession, JdbcTableHandle)
        +toPrestoType(ConnectorSession, JdbcTypeHandle)
        +toWriteMapping(ConnectorSession, Type)
    }
    class JdbcClient {
        +toPrestoType(ConnectorSession, JdbcTypeHandle)
        +toWriteMapping(ConnectorSession, Type)
    }
    class JdbcPageSink {
        +JdbcPageSink(ConnectorSession, JdbcOutputTableHandle, JdbcClient)
    }
    StandardColumnMappings <|-- BaseJdbcClient
    JdbcClient <|-- BaseJdbcClient
    JdbcClient <|-- JdbcPageSink
    BaseJdbcClient o-- StandardColumnMappings
    JdbcPageSink o-- JdbcClient
Loading

File-Level Changes

Change Details Files
Introduce UTC calendar and legacy mappings in StandardColumnMappings to remove timezone drift
  • Add UTC_CALENDAR constant
  • Update timestampReadMapping to use resultSet.getTimestamp with UTC_CALENDAR
  • Deprecate old timestampReadMapping as timestampReadMappingLegacy
  • Update timestampWriteMapping to setTimestamp with UTC_CALENDAR
  • Add deprecated timestampWriteMappingLegacy
presto-base-jdbc/src/main/java/com/facebook/presto/plugin/jdbc/mapping/StandardColumnMappings.java
Propagate legacy_timestamp flag through write mapping selection in JDBC client APIs
  • Change prestoTypeToWriteMapping signature to accept ConnectorSession and branch on legacy flag
  • Update BaseJdbcClient.toPrestoType to choose legacy or UTC read mapping for TIMESTAMP
  • Change BaseJdbcClient.toWriteMapping to accept session and select mapping accordingly
  • Modify JdbcClient interface to pass ConnectorSession into toWriteMapping
  • Update JdbcPageSink to supply session when resolving write functions
presto-base-jdbc/src/main/java/com/facebook/presto/plugin/jdbc/BaseJdbcClient.java
presto-base-jdbc/src/main/java/com/facebook/presto/plugin/jdbc/JdbcClient.java
presto-base-jdbc/src/main/java/com/facebook/presto/plugin/jdbc/JdbcPageSink.java
Update TestJdbcQueryBuilder to use UTC calendar and Instant for timestamp tests
  • Add UTC_CALENDAR constant in test class
  • Use resultSet.getTimestamp with UTC_CALENDAR instead of getObject
  • Revise toTimestamp helper to generate UTC-based Timestamp via Instant
presto-base-jdbc/src/test/java/com/facebook/presto/plugin/jdbc/TestJdbcQueryBuilder.java
Add detailed storage verification tests for timestamps/datetimes in Postgres and MySQL connectors
  • Rename and expand timestamp test in TestPostgreSqlTypeMapping to two methods covering legacy and non-legacy modes
  • Add JDBC-based assertions verifying wall-clock storage and Presto retrieval
  • Mirror similar changes in TestMySqlTypeMapping for DATETIME with legacy and UTC sessions
presto-postgresql/src/test/java/com/facebook/presto/plugin/postgresql/TestPostgreSqlTypeMapping.java
presto-mysql/src/test/java/com/facebook/presto/plugin/mysql/TestMySqlTypeMapping.java

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@tdcmeehan tdcmeehan force-pushed the jtz branch 2 times, most recently from 7636f52 to 968bb6f Compare October 29, 2025 17:51
@tdcmeehan tdcmeehan changed the title Fix Timestamp values in JDBC connectors fix(plugin-postgres): Fix Timestamp values in JDBC connectors Oct 29, 2025
@tdcmeehan tdcmeehan marked this pull request as ready for review October 29, 2025 21:09
@tdcmeehan tdcmeehan requested a review from a team as a code owner October 29, 2025 21:09
@prestodb-ci prestodb-ci requested review from a team, bibith4 and wanglinsong and removed request for a team October 29, 2025 21:09
Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes and they look great!

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location> `presto-base-jdbc/src/main/java/com/facebook/presto/plugin/jdbc/mapping/StandardColumnMappings.java:97` </location>
<code_context>
     private StandardColumnMappings() {}

     private static final ISOChronology UTC_CHRONOLOGY = ISOChronology.getInstanceUTC();
+    private static final Calendar UTC_CALENDAR = Calendar.getInstance(TimeZone.getTimeZone("UTC"));

     public static ReadMapping booleanReadMapping()
</code_context>

<issue_to_address>
**issue (bug_risk):** Consider using a thread-safe alternative to Calendar for UTC handling.

Since Calendar is mutable, sharing a static instance can cause concurrency issues. Use java.time APIs, instantiate Calendar per use, or clearly document thread-safety assumptions.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

assertEquals(dbValue1, "1970-01-01 00:00:00.000000", "JDBC insert should store wall clock time 1970-01-01 00:00:00 in DB");
}

// In legacy mode, DB value 1970-01-01 00:00:00 is interpreted as if it's in JVM timezone (America/Bahia_Banderas UTC-7)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if the user session and server are in different zone . what happens if we set session timezone key as "Asia/kolkata" for same scenario?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good . Thank you !

@tdcmeehan
Copy link
Contributor Author

@sourcery-ai summary

@tdcmeehan tdcmeehan requested a review from imjalpreet October 31, 2025 20:29
Copy link
Contributor

@BryanCutler BryanCutler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, very thorough tests. Thanks @tdcmeehan !

@tdcmeehan tdcmeehan merged commit 5efa463 into prestodb:master Nov 1, 2025
118 of 136 checks passed
shangm2 added a commit to shangm2/presto that referenced this pull request Nov 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

from:IBM PR from IBM

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JDBC connectors incorrectly handle timestamps when legacy_timestamp=false

5 participants