Skip to content

test(native): Add AbstractTestQueries to native tests #23671

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

test(native): Add AbstractTestQueries to native tests #23671

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5834550
refactor(testing): Refactor testcases for native-tests
pramodsatya Sep 5, 2025
e8b547d
test(native): Add AbstractTestQueries to native-tests
pramodsatya Oct 31, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion presto-docs/src/main/sphinx/presto_cpp/limitations.rst
View file
Open in desktop
Copy link
Contributor

@aditi-pandit aditi-pandit Jul 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could move the changes in limitations.rst and TestNonIterativeDistributedQueries and TestTpchDistributedQueries into a separate PR as that will be much easier to review and submit than this PR.

Copy link
Contributor Author

@pramodsatya pramodsatya Sep 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes in TestNonIterativeDistributedQueries and TestTpchDistributedQueries depend on AbstractTestQueriesNative, and the comment in limitations.rst explains the difference in approx_set results for which the test is added here. Is it fine to retain these changes here?

Original file line number Diff line number Diff line change
Expand Up @@ -47,4 +47,12 @@ reduce_agg
In C++ based Presto, ``reduce_agg`` is not permitted to return ``null`` in either the
``inputFunction`` or the ``combineFunction``. In Presto (Java), this is permitted
but undefined behavior. For more information about ``reduce_agg`` in Presto,
see `reduce_agg <../functions/aggregate.html#reduce_agg>`_.
see `reduce_agg <../functions/aggregate.html#reduce_agg>`_.

approx_set
----------

Cardinality estimation with ``approx_set`` has a maximum standard error of ``e``
(default value of ``0.01625``). ``approx_set`` returns different values in Presto C++
and Presto (Java). For more information about ``approx_set`` in Presto, see
`approx_set <../functions/aggregate.html#approx_set>`_.
2 changes: 1 addition & 1 deletion ...cution/src/test/java/com/facebook/presto/nativeworker/AbstractTestNativeCteExecution.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ protected void createTables()
// This call avoids casting date fields to VARCHAR. DWRF requires the conversion, but this test uses PARQUET
// which doesn't have this restriction. The change is needed because several CTE tests use
// EXTRACT functions from date columns.
NativeQueryRunnerUtils.createAllTables(queryRunner, false);
NativeQueryRunnerUtils.createAllTables(queryRunner, false, false);
}

@Override
Expand Down
2 changes: 1 addition & 1 deletion ...java/com/facebook/presto/nativeworker/AbstractTestNativeHiveExternalTableTpchQueries.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ protected void createTables()
createSchemaIfNotExist(javaQueryRunner, TPCH_SCHEMA);
Session session = Session.builder(super.getSession()).setCatalog(HIVE).setSchema(TPCH_SCHEMA).build();
createOrders(session, javaQueryRunner, true);
createLineitemStandard(session, javaQueryRunner);
createLineitemStandard(session, javaQueryRunner, true);
createNationWithFormat(session, javaQueryRunner, "PARQUET");
createCustomer(session, javaQueryRunner);
createPart(session, javaQueryRunner);
Expand Down
2 changes: 1 addition & 1 deletion .../src/test/java/com/facebook/presto/nativeworker/AbstractTestNativeIcebergTpchQueries.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ public abstract class AbstractTestNativeIcebergTpchQueries
protected void createTables()
{
QueryRunner queryRunner = (QueryRunner) getExpectedQueryRunner();
createLineitemStandard(queryRunner);
createLineitemStandard(queryRunner, true);
createOrders(queryRunner);
createNationWithFormat(queryRunner, storageFormat);
createCustomer(queryRunner);
Expand Down
43 changes: 34 additions & 9 deletions ...tive-execution/src/test/java/com/facebook/presto/nativeworker/NativeQueryRunnerUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,12 +78,22 @@ public static Map<String, String> getNativeWorkerTpcdsProperties()
*/
public static void createAllTables(QueryRunner queryRunner)
{
createAllTables(queryRunner, true);
createAllTables(queryRunner, true, false);
}

public static void createAllTables(QueryRunner queryRunner, boolean castDateToVarchar)
/**
* When castDateToVarchar is true, date columns are cast to Varchar. This is useful for testing with DWRF file
* format, since DWRF does not support DATE type.
* When useTpchStandardSchema is true, lineitem table is created with 16 columns as specified in the TPC-H standard,
* otherwise the lineitem table is created with additional columns for testing purposes.
*
* @param queryRunner
* @param castDateToVarchar
* @param useTpchStandardSchema
*/
public static void createAllTables(QueryRunner queryRunner, boolean castDateToVarchar, boolean useTpchStandardSchema)
{
createLineitem(queryRunner, castDateToVarchar);
createLineitem(queryRunner, castDateToVarchar, useTpchStandardSchema);
createOrders(queryRunner, castDateToVarchar);
createOrdersEx(queryRunner);
createOrdersHll(queryRunner);
Expand All @@ -107,7 +117,7 @@ public static void createAllTables(QueryRunner queryRunner, boolean castDateToVa
*/
public static void createAllIcebergTables(QueryRunner queryRunner)
{
createLineitemStandard(queryRunner);
createLineitemStandard(queryRunner, true);
createOrders(queryRunner);
createNationWithFormat(queryRunner, ICEBERG_DEFAULT_STORAGE_FORMAT);
createCustomer(queryRunner);
Expand All @@ -122,6 +132,16 @@ public static void createLineitem(QueryRunner queryRunner)
createLineitem(queryRunner, true);
}

public static void createLineitem(QueryRunner queryRunner, boolean castDateToVarchar, boolean useTpchStandardSchema)
{
if (useTpchStandardSchema) {
createLineitemStandard(queryRunner, castDateToVarchar);
}
else {
createLineitem(queryRunner, castDateToVarchar);
}
}

public static void createLineitem(QueryRunner queryRunner, boolean castDateToVarchar)
{
queryRunner.execute("DROP TABLE IF EXISTS lineitem");
Expand All @@ -139,18 +159,23 @@ public static void createLineitem(QueryRunner queryRunner, boolean castDateToVar
"FROM tpch.tiny.lineitem");
}

public static void createLineitemStandard(QueryRunner queryRunner)
public static void createLineitemStandard(QueryRunner queryRunner, boolean castDateToVarchar)
{
createLineitemStandard(queryRunner.getDefaultSession(), queryRunner);
createLineitemStandard(queryRunner.getDefaultSession(), queryRunner, castDateToVarchar);
}

public static void createLineitemStandard(Session session, QueryRunner queryRunner)
public static void createLineitemStandard(Session session, QueryRunner queryRunner, boolean castDateToVarchar)
{
if (!queryRunner.tableExists(session, "lineitem")) {
String shipDate = castDateToVarchar ? "cast(shipdate as varchar) as shipdate" : "shipdate";
String commitDate = castDateToVarchar ? "cast(commitdate as varchar) as commitdate" : "commitdate";
String receiptDate = castDateToVarchar ? "cast(receiptdate as varchar) as receiptdate" : "receiptdate";

queryRunner.execute("DROP TABLE IF EXISTS lineitem");
queryRunner.execute(session, "CREATE TABLE lineitem AS " +
"SELECT orderkey, partkey, suppkey, linenumber, quantity, extendedprice, discount, tax, " +
" returnflag, linestatus, cast(shipdate as varchar) as shipdate, cast(commitdate as varchar) as commitdate, " +
" cast(receiptdate as varchar) as receiptdate, shipinstruct, shipmode, comment " +
" returnflag, linestatus, " + shipDate + ", " + commitDate + ", " + receiptDate + ", " +
" shipinstruct, shipmode, comment " +
"FROM tpch.tiny.lineitem");
}
}
Expand Down
6 changes: 6 additions & 0 deletions presto-native-tests/presto_cpp/tests/custom_functions/CustomFunctions.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,12 @@ extern "C" {
int64_t,
int64_t>("dynamic_custom_add");

facebook::presto::registerPrestoFunction<
custom::functionRegistry::CustomAdd,
int64_t,
int64_t,
int64_t>("custom_add");

facebook::presto::registerPrestoFunction<
custom::functionRegistry::SumArray,
int64_t,
Expand Down
Loading
Loading