[SPARK-47681][FOLLOWUP] Fix schema_of_variant(decimal)

### What changes were proposed in this pull request?

The PR #46338 found `schema_of_variant` sometimes could not correctly handle variant decimals and had a fix. However, I found that the fix is incomplete and `schema_of_variant` can still fail on some inputs. The reason is that `VariantUtil.getDecimal` calls `stripTrailingZeros`. For an input decimal `10.00`,  the resulting scale is -1 and the unscaled value is 1. However, negative decimal scale is not allowed by Spark. The correct approach is to use the `BigDecimal` to construct a `Decimal` and read its precision and scale, as what we did in `VariantGet`.

This PR also includes a minor change for `VariantGet`, where a duplicated expression is computed twice.

### Why are the changes needed?

They are bug fixes and are required to process decimals correctly.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

More unit tests. Some of them would fail without the change in this PR (e.g., `check("10.00", "DECIMAL(2,0)")`). Others wouldn't fail, but can still enhance test coverage.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #46549 from chenhao-db/fix_decimal_schema.

Authored-by: Chenhao Li <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/variant/variantExpressions.scala

@@ -341,7 +341,7 @@ case object VariantGet {
  case Type.DOUBLE => Literal(v.getDouble, DoubleType)
  case Type.DECIMAL =>
  val d = Decimal(v.getDecimal)
- Literal(Decimal(v.getDecimal), DecimalType(d.precision, d.scale))
+ Literal(d, DecimalType(d.precision, d.scale))
  case Type.DATE => Literal(v.getLong.toInt, DateType)
  case Type.TIMESTAMP => Literal(v.getLong, TimestampType)
  case Type.TIMESTAMP_NTZ => Literal(v.getLong, TimestampNTZType)
@@ -682,9 +682,8 @@ object SchemaOfVariant {
  case Type.STRING => SQLConf.get.defaultStringType
  case Type.DOUBLE => DoubleType
  case Type.DECIMAL =>
- val d = v.getDecimal
- // Spark doesn't allow `DecimalType` to have `precision < scale`.
- DecimalType(d.precision().max(d.scale()), d.scale())
+ val d = Decimal(v.getDecimal)
+ DecimalType(d.precision, d.scale)
  case Type.DATE => DateType
  case Type.TIMESTAMP => TimestampType
  case Type.TIMESTAMP_NTZ => TimestampNTZType

sql/core/src/test/scala/org/apache/spark/sql/VariantEndToEndSuite.scala

@@ -160,6 +160,16 @@ class VariantEndToEndSuite extends QueryTest with SharedSparkSession {
  check("1", "BIGINT")
  check("1.0", "DECIMAL(1,0)")
  check("0.01", "DECIMAL(2,2)")
+ check("1.00", "DECIMAL(1,0)")
+ check("10.00", "DECIMAL(2,0)")
+ check("10.10", "DECIMAL(3,1)")
+ check("0.0", "DECIMAL(1,0)")
+ check("-0.0", "DECIMAL(1,0)")
+ check("2147483647.999", "DECIMAL(13,3)")
+ check("9223372036854775808", "DECIMAL(19,0)")
+ check("-9223372036854775808.0", "DECIMAL(19,0)")
+ check("9999999999999999999.9999999999999999999", "DECIMAL(38,19)")
+ check("9999999999999999999.99999999999999999999", "DOUBLE")
  check("1E0", "DOUBLE")
  check("true", "BOOLEAN")
  check("\"2000-01-01\"", "STRING")