Plugin feature

[nkinesis]

Description of the problem
The colouring-core repository contains the main features of Colouring Cities, and each group forks the core to add their own features on top of it, adapted to their needs. While this allows flexibility for each group to go on with their business, this also causes each fork to drift away from the core architecture over time. For example:

• Each group may use different file, folder, component and function naming conventions.
• Each group uses different text strings (for different languages).
• Implementations may be created using different paradigms (MVC, MVVM, MVW).

As a result, over time it becomes harder and harder to integrate changes from the fork into the core repository, because merge conflict increase in frequency, and sometimes these conflicts are unresolvable due to divergent implementation strategies.

Proposed solution
To mitigate this problem, we propose the development of a plugin feature, through which we could keep a stable core and let groups add, change and remove functionality "around" the core at ease. However, implementing such a plugin feature into an already implemented application presents several challenges, such as:

• We do not know which features can (or should) become plugins
• It is not immediately clear which group of files and folders represents one feature
• The very existence of a plugin feature implies several subsystems which need to be developed, such as:
  • Plugin detection, installation and un-installation
  • Plugin version control and dependency resolution (i.e., which plugins do my plugin need to work?)
  • The creation of a Core API through which plugins can "talk" to the core

Describe any alternatives
Alternatives to the implementation of a plugin system would be:

• Moving on with the current architecture and accepting that some features in some forks may not be mergeable upstream due to the large number of divergences (downside: each group repository is siloed, low code sharing and reusability)
• Asking groups to select features from their repositories which they think should be integrated into the core repository, and put time and effort to resolve conflicts and adapt the code to comply with the core architecture and standards (may be extremely time consuming)

Additional context
More details about the architecture and implementation of the proposed solution and currently unanswered research questions can be found in the attached PDF.

[matkoniecz]

I want to answer that I agree in general with various comments/conclusion, though I am also familiar with projects where plugin functionality was added/expanded/formalized late within program lifecycle.

I think that easier/more successful/easier to use plugin systems included functionality limited to more specific functionality, rather than able to change anything at all.

For example some computer games extended/added official modding API (say, computer-controlled players in OpenTTD)

[KoaCWells]

After the monthly engineering meeting held last week, I think this is the best place to continue the discussion that occurred. During the analysis of the codebase and with some explanations of the codebase by @matkoniecz , we realized that 'pluganizing' the different data categories may be a bit excessive. As such, we have come up with an alternative specifically for the data categories.

Understanding of the Data Categories
Before describing the suggestion, here is an explanation of my understanding of the website:

1. We start with the 12 Main Categories

2. We have the Subcategories under each Main Category (such as "General Land use" and "Specific land use(s)" under the "Land Use" category.

3. There are the Fields under the Subcategories

4. There are the Visualization Subcategories ("Land Use" and "Residential building"

5. Lastly, we have the different options under the Visualization Subcategories

This may not be the exact naming convention and I must apologize if this is so, I thought it would important to clarify my understanding first.

Proposed Alternative to Plugin
1. Keep the 12 Main Categories hardcoded and this should always stay consistent across all platforms.

2. Move the Subcategories to a database table

3. Move the Fields under the Subcategories to a database table

4. Move the Visualization Subcategories to a database table

5. Move the Values under the Visualization Subcategories to a database table

Potential Risks/Downsides
1. Developers will more easily be able to remove/modify the mandatory data. (Note that this is a risk that will always be present no matter what decision is made)

2. If the database tables are not architected in a flexible enough manner, it could make it difficult or impossible to add certain types of data.

3. Scripts will need to be developed to help each group migrate their databases to the new database formats (assuming the structure changes).

4. The age old problem: this only works if each group is willing to upgrade their core.

Potential Benefits
1. By moving the majority of the data to a database, it allows each team to add their own extra categories without needing to touch the code reducing the differences between each Colouring City codebase.

2. If there is an update to any of the mandatory Categories, Subcategories, etc. each team will be able to update their

3. It will no longer be required to restart the Colouring Core application allowing for on-the-fly updates to data as long as it conforms to the format of the frontend code.

4. It can simplify the codebase by removing 100s-1000s of lines of hardcoded values.

Conclusion
I believe this addresses all of the major points. I look forward to hearing feedback from others in the group on this.

[KoaCWells]

Last thing to note here. If we do decide to move forward with this suggestion, it opens a couple questions regarding the plugin idea:

1. What things have different groups historically modified in their Colouring Cities fork?

2. What things are not included in 1 that would be useful to add-on as plugins?

[matkoniecz]

Thanks for exploring and continuing to explore potential solutions and improvements to this customization challenges!

Database-defined structures may also have problem of being harder to understand and modify.