[FLINK-37862][table] Support inline structured types in SQL

[twalthr]

What is the purpose of the change

This adds support for inline structured types in Flink SQL. Inline structured types are not a completely new type. They have been added as part of FLIP-37 when reworking the Flink table type system. However, mostly Table API users were able to use them properly. For SQL, there was no type DDL to cast or declare the type.

This PR improves handling inline structured types by supporting the type declaration:

STRUCTURED<'c', n0 t0 'd0', n1 t1 'd1', ...>

where c is the class name, n is the unique name of a field, t is the logical type of a field, d is the description of a field.

Brief change log

• Support STRUCTURED<> in SQL parser and logical type parser
• Decouple structured types from the hard requirement of a class being present in the classpath
• Support equality between types for testing structured types with different class name
• Improve various JavaDocs and docs to highlight the benefits and shortcomings of structured types

Verifying this change

This change is already covered by existing tests such as LogicalTypeParserTest, LogicalTypeTest and others.

It adds casting and equality tests as well as a new StructuredFunctionITCase for testing functions dealing with structured types in the future.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): yes
• The serializers: yes
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? yes
• If yes, how is the feature documented? docs

[flinkbot]

CI report:

• 94c8474 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[raminqaf]

Overall excellent PR. I have some questions and small comments

[raminqaf]

Shouldn't this be?

Suggested change

	DataTypes.STRUCTURED(String, DataTypes.FIELD(n0, t0), DataTypes.FIELD(n1, t1), ...);
	DataTypes.STRUCTURED(String.class, DataTypes.FIELD(n0, t0), DataTypes.FIELD(n1, t1), ...);

And what is the difference between this line and the one before it?

[twalthr]

I just wanted to highlight that there are two methods taking either Class or String. I reworked this part.

[raminqaf]

Ah now I get it! I just got confused with the String class in Java, and not an actual "String"

[raminqaf]

Question: Why is the comment/description field not present here? I think this because we don't really need them, and they are just for clarity, written by the user?

Suggested change

	/**
	* Creates a STRUCTURED type such as {@code STRUCTURED('org.my.Class', name STRING, age INT)}.
	*/
	RelDataType createStructuredType(
	String className, List<RelDataType> typeList, List<String> fieldNameList);
	/**
	* Creates a STRUCTURED type such as {@code STRUCTURED('org.my.Class', name STRING 'full name of the person', age INT)}.
	*/
	RelDataType createStructuredType(
	String className, List<RelDataType> typeList, List<String> fieldNameList, List<String> fieldCommentList);

[twalthr]

Very thorough review! Yes, the comments don't end up in the planner. We do the same for ROW. This is historical legacy that bites us until today. These comments make comparisons and other operations unnecessary complicated. For now, we leave them out. However, a catalog can use them already for DESCRIBE t. From a type definition point of view the type is defined. Everything else we need to solve in a future PR.

[raminqaf]

Do you think it would be a good idea to fail fast here with a simple validation like className.isBlank() || className.isEmpty()?

Or is it a valid statement to have

STRUCTURED<``, f0 t0>
STRUCTURED<` `, f0 t0>

[twalthr]

The builder is currently only used internally, via DataTypes.STRUCTURED and other locations. When instantiating the StructuredType we run checkClassName which will do the full validation. Additional checks just spam the code.

[raminqaf]

Question: Changing the type to int on both classes will cause the tests to fail. Why is that?

Suggested change

	public Integer a;
	public int a;

[twalthr]

yes because there is a mismatch in extracted and explicit type. The function will return the extracted one. But in general it is up to the user to ensure that data type matches class. Usually, users should not declare the structured types manually but use reflective extraction by just using the POJO in a function signature.

[raminqaf]

LGTM!