RFD: Template Extensions

[yrashk]

Writing this while traveling in the back of a car to run the Khao Yai marathon this weekend!

This is a Request For Discussion on the extension layout approach that I've recently developed (and hoping to speak about at some of the conferences this year).

As we know, extension harnessing has some properties that can be seen as deficiencies:

• Multiple versions of the same extension can't be installed in the same database
• Extension configurability is low – yes, you can set GUC to certain values before creating an extension and let that affect the installation logic – but that's about it
• Extension tables are not saved in dumps by default. I understand where this is coming from – but I still think this is rather an unfortunate default ("delete data" by default always strikes me as such)

It has occurred to me at some point that there could be a solution to all this in one package. The core of the idea was that maybe the effects of extensions should be provisioned into the installing user's ownership, in the schema of their choosing?

And if we can specify the schema the extension should be installed into, perhaps we can also specify other parameters?

Taking an analogy from C++ templates (please bear with me, I don't mean this in a bad way 😃 ) I thought what if extension simply provided a template instantiation function that provisions everything that extension needs, with all the parameters (optional and non-optional) being function parameters?

Here's an example: https://github.com/omnigres/omnigres/blob/master/extensions/omni_credentials/src/instantiate.sql#L1-L2

create function instantiate(schema regnamespace default 'omni_credentials',
                            env_var text default 'OMNI_CREDENTIALS_MASTER_PASSWORD') returns void

The function then creates database objects it needs and puts them into the designated schema.

Turns out this seems to work pretty well:

1. We can now install different versions – at least into different schemas/namespaces.
2. We can definitely enjoy explicit configurability
3. All provisioned tables are dumped by default

Upgrades of these instantiations can be performed by newer versions of the extension by analyzing whatever information is stored in the instantiation – the shape of it, the metadata.

There are some interesting aspects like finding a set of shared patterns to be able to list all instantiated templated extensions. I am working on these, but one of the simplest ways is to create tables or functions with a certain naming and/or shape that act as providers of such information. Happy to elaborate here as I am going deeper on this concept (I've already started making new Omnigres extensions templated)

And yes, technically this means that extensions as we know them become mere envelopes of templates and can be shipped by other means (other than .control/.sql)

What do you think? Do you see any holes?

[danolivo]

The general idea seems interesting; however, I find the rationale weak for several reasons:

1. Different copies of the same extension in a database. I understand why people can want this, but I think it is intentionally designed this way: install the extension into the public schema and teach it to work with schemas separately if needed.
2. The argument for default dumping also does not seem compelling.

Providing a clear example of a simple model extension could reinforce this concept and help people better understand its key benefits and realise drawbacks.

[yrashk]

Thank you for your feedback. I look at this from a database-centric developer perspective. Just because component A requires a certain version of extension E, should not preclude B from using a different version of E.

This also raises some interesting questions that even transcend upgrades and are firmly in the view-based data migration territory (should we want to have shared data between versions)

An additional example: one of the Omnigres extensions builds a ledger. However, this means that there is only one instance of the ledgering in a single database, and this is rather limiting, especially if different ledger systems need to have different features and shapes.