Java/jOOQ: Add code generation add-on for Gradle, with reflection from the database

[amotl]

Hi again,

as a followup to GH-9, this patch adds the Gradle-based jOOQ code generation add-on. It can be invoked like:

./gradlew generateJooq

The style of code generation exercised here aims to work by directly connecting to a compatible JDBC database, and reflect its schema. 🚧 This currently does not work with CrateDB yet. 🚧

With kind regards,
Andreas.

[amotl]

Currently, being configured using the org.jooq.meta.postgres.PostgresDatabase generator, it will croak when hitting a WITH RECURSIVE CTE clause [1], and delete the generated classes within the src/generated/java directory.

Exception while executing meta query: ERROR: line 1:6: extraneous input 'recursive' expecting [...]

The SQL statement used for schema reflection is:

with recursive
  domains(domain_id, base_id, typbasetype, src) as (
    select
      d.oid,
      d.oid,
      d.typbasetype,
      case
        when c.oid is not null then array[pg_get_constraintdef(c.oid)]
      end
    from pg_catalog.pg_type as d
      join pg_catalog.pg_namespace as n
        on n.oid = d.typnamespace
      left outer join pg_catalog.pg_constraint as c
        on d.oid = c.contypid
    where (
      d.typtype = 'd'
      and n.nspname in ('testdrive')
    )
    union all
    select
      domains.domain_id,
      d.oid,
      d.typbasetype,
      case
        when c.conbin is null then domains.src
        else array_append(
          domains.src,
          pg_get_constraintdef(c.oid)
        )
      end
    from domains
      join pg_catalog.pg_type as d
        on domains.typbasetype = d.oid
      left outer join pg_catalog.pg_constraint as c
        on d.oid = c.contypid
  )
select
  n.nspname,
  d.typname,
  d.typnotnull,
  d.typdefault,
  b.typname,
  information_schema._pg_char_max_length(d.typbasetype, d.typtypmod) as character_maximum_length,
  information_schema._pg_numeric_precision(d.typbasetype, d.typtypmod) as numeric_precision,
  information_schema._pg_numeric_scale(d.typbasetype, d.typtypmod) as numeric_scale,
  domains.src
from pg_catalog.pg_type as d
  join domains
    on (
      domains.typbasetype = 0
      and domains.domain_id = d.oid
    )
  join pg_catalog.pg_type as b
    on domains.base_id = b.oid
  join pg_catalog.pg_namespace as n
    on n.oid = d.typnamespace
where (
  d.typtype = 'd'
  and n.nspname in ('testdrive')
)
order by n.nspname, d.typname

[1] This is already being tracked at the CrateDB repository.

• Add RECURSIVE support to common table expressions (WITH) crate#12544

[amotl]

Also wondering if code generation would work when using the org.jooq.meta.jdbc.JDBCDatabase generator?

diff --git a/by-language/java-jooq/jooq.gradle b/by-language/java-jooq/jooq.gradle
index 6a0caf1..5345b63 100644
--- a/by-language/java-jooq/jooq.gradle
+++ b/by-language/java-jooq/jooq.gradle
@@ -68,7 +68,7 @@ jooq {
                 generator {
                     name = 'org.jooq.codegen.DefaultGenerator'
                     database {
-                        name = 'org.jooq.meta.postgres.PostgresDatabase'
+                        name = 'org.jooq.meta.jdbc.JDBCDatabase'
                         inputSchema = 'testdrive'
                     }
                     generate {

@proddata: It also croaks, but the error output is different. The offending SQL statement is:

select 
  k.table_catalog, 
  k.table_schema, 
  k.table_name, 
  k.constraint_name, 
  k.column_name, 
  k.ordinal_position 
from 
  (
    pg_catalog.pg_constraint as c 
    join (
      pg_catalog.pg_class as alias_89552482 
      join pg_catalog.pg_namespace as alias_126892781 on alias_89552482.relnamespace = alias_126892781.oid
    ) on c.conrelid = alias_89552482.oid 
    join pg_catalog.pg_namespace as alias_87501535 on c.connamespace = alias_87501535.oid
  ) 
  join information_schema.key_column_usage as k on (
    k.table_schema = alias_126892781.nspname 
    and k.table_name = alias_89552482.relname 
    and k.constraint_schema = alias_87501535.nspname 
    and k.constraint_name = c.conname
  ) 
where 
  (
    c.contype = 'u' 
    and alias_87501535.nspname in (?)
  ) 
order by 
  k.table_schema asc, 
  k.table_name asc, 
  k.constraint_name asc, 
  k.ordinal_position asc;

This error message is probably relevant?

ERROR: joinPairs contains duplicate: Join{INNER alias_89552482 ⇔ alias_126892781} matches Join{INNER alias_89552482 ⇔ alias_126892781}

Finally, the preface of the stacktrace, for the sake of completeness.

17:37:56 SEVERE Error in file: /Users/amo/dev/crate/docs/cratedb-examples/by-language/java-jooq/build/tmp/generateJooq/config.xml. Error : Error generating code for catalog
org.jooq.codegen.GeneratorException: Error generating code for catalog
        at org.jooq.codegen.JavaGenerator.generate0(JavaGenerator.java:524)
        at org.jooq.codegen.AbstractGenerator.generate(AbstractGenerator.java:187)
        at org.jooq.codegen.JavaGenerator.generate(JavaGenerator.java:227)
        at org.jooq.codegen.GenerationTool.run0(GenerationTool.java:942)
        at org.jooq.codegen.GenerationTool.run(GenerationTool.java:240)
        at org.jooq.codegen.GenerationTool.generate(GenerationTool.java:235)
        at org.jooq.codegen.GenerationTool.main(GenerationTool.java:207)
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: org.jooq.exception.DataAccessException: SQL [...]

[proddata]

@proddata: It also croaks, but the error output is different. The offending SQL statement is:
Thanks for testing 🙇

Don't know if this a bug, or just not supported ...

cr> CREATE TABLE j1 (x INT);                                                    
cr> CREATE TABLE j2 (x INT);                                                    
cr> CREATE TABLE j3 (x INT);      
                                              
cr> SELECT * 
    FROM j1 
    JOIN (j2 JOIN j3 ON j2.x = j3.x) 
    ON j1.x = j2.x;                                                             
--IllegalStateException[joinPairs contains duplicate: Join{INNER doc.j2 ⇔ doc.j3} matches Join{INNER doc.j2 ⇔ doc.j3}]

[amotl]

@mkleen: Did you refer to this option at #9 (comment)?

[amotl]

@proddata: Thanks for analyzing the SQL reflection statement issued by the org.jooq.meta.jdbc.JDBCDatabase generator and for converging the outcome into crate/crate#13503 right away. 👍