Skip to content

bug(query): Incorrect pruning when timestamp strings use different date/time separators #20442

Description

@TCeason

Version

  Databend Query v1.2.916-nightly-5074971eec
  (rust-1.94.0-nightly-2026-06-02T09:26:00.023295899Z)

The VARCHAR → TIMESTAMP domain inference assumes that converting the string minimum and maximum produces valid bounds for all converted timestamps. This assumption is incorrect even without mixed timezone offsets.

However, the middle value converts to 23:00:00, which lies outside that domain. Using the inferred range for pruning can therefore discard matching rows.

Reproduction

  SET timezone = 'UTC';

  CREATE OR REPLACE TABLE timestamp_domain_separator (
      a VARCHAR NOT NULL STATS_TRUNCATE_LEN 64
  );

  INSERT INTO timestamp_domain_separator VALUES
      ('2024-01-01 00:00:00'),
      ('2024-01-01 23:00:00'),
      ('2024-01-01T01:00:00');

  SELECT
      a AS raw_value,
      CAST(a AS TIMESTAMP) AS ts_value
  FROM timestamp_domain_separator
  ORDER BY raw_value;

  SELECT column_name, statistics
  FROM fuse_block_statistics('default', 'timestamp_domain_separator');
╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ column_name │                                                    statistics                                                    │
│    String   │                                                 Nullable(Variant)                                                │
├─────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ a           │ {"distinct_count":3,"in_memory_size":105,"max":"2024-01-01T01:00:00","min":"2024-01-01 00:00:00","null_count":0} │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

  SELECT a
  FROM timestamp_domain_separator
  WHERE CAST(a AS TIMESTAMP) >
        '2024-01-01 12:00:00'::TIMESTAMP;
0 row read in 0.030 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

  EXPLAIN SELECT a
  FROM timestamp_domain_separator
  WHERE CAST(a AS TIMESTAMP) >
        '2024-01-01 12:00:00'::TIMESTAMP;
-[ EXPLAIN ]-----------------------------------
TableScan
├── table: default.default.timestamp_domain_separator
├── scan id: 0
├── output columns: [a (#0)]
├── read rows: 0
├── read size: 0
├── partitions total: 1
├── partitions scanned: 0
├── pruning stats: [segments: <range pruning: 1 to 0 cost: 1 ms>]
├── push downs: [filters: [CAST(timestamp_domain_separator.a (#0) AS Timestamp) > '2024-01-01 12:00:00.000000'], limit: NONE]
└── estimated rows: 0.00

STATS_TRUNCATE_LEN 64 preserves the complete string endpoints, preventing truncated statistics from masking the issue.

Expected result

  2024-01-01 23:00:00

Predicted behavior with the old domain inference

The inferred maximum timestamp is 01:00:00, so the predicate > 12:00:00 may be incorrectly folded to false during range pruning, causing the query to return no rows.

This counterexample is derived from the old implementation and has not yet been executed. Unlike the mixed-offset example, it requires no timezone differences—using two
accepted date/time separators is sufficient to break the ordering assumption.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-bugCategory: something isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions