Unable to tokenize hex string with Rust tokenizer using Snowflake dialect

[whummer]

Follow-up from #4559, filing the reproducible issue here.

Fully reproducible code snippet

Note: this issue seems to be related to the Rust tokenizer.

The following piece of code (tested against Snowflake) ...:

from sqlglot import parse_one

query = "INSERT INTO test VALUES (X'48FAF43B0AFCEF9B63EE3A93EE2AC2')"
parse_one(query, dialect="snowflake")

... fails with:

E           sqlglot.errors.TokenError: Error tokenizing 'NTO test VALUES (X'48FAF43B0AFCEF9B63EE3A93EE2AC2'': Numeric string contains invalid characters from 1:25