SOLR-14414: Introduce new UI (SIP-7) (#2605)

* Add basic Compose integration example for webapp

This commit creates a new module in Solr that sets up a
frontend written with Compose and targeting browser (WASM)
and desktop (JVM).

The webapp is modified so that it opens the WASM Compose app
when accessing /solr/compose.

IMPORTANT: The jetty configuration is updated to include
script-src: 'wasm-unsafe-eval' to allow WASM code execution
which may be considered a security issue.

* Add dev-docs for UI development with Kotlin/Compose

* Improve build times via development flag

This commit adds a development flag to our
gradle.properties that allows the selection of
the build variant for the new AdminUI. When
development enabled (default), Gradle will build
a development instance and will have less secure
configuration for the AdminUI to be able to attach
debugging tools.

When disabled, Gradle will optimize build output
for the new Admin UI, but will also take longer to
complete.

Default is set to true to always build development
locally and in CI/CD to avoid longer building times.

Additionally, user is able to disable the new AdminUI
via SOLR_ADMIN_UI_EXPERIMENTAL_DISABLED or by disabling
the AdminUI.

IMPORTANT: From this commit on, during releases, the
development flag needs to be set explicitly to false,
otherwise it will not generate an optimized Admin UI
with improved CSP directives.

.github/labeler.yml

@@ -96,6 +96,7 @@ admin-ui:
   - changed-files:
       - any-glob-to-any-file:
           - solr/webapp/**
+          - solr/ui/**
 
 # Add 'prometheus-exporter' label
 prometheus-exporter:

.github/renovate.json

@@ -6,7 +6,11 @@
   "enabledManagers": ["gradle", "github-actions"],
   "includePaths": ["gradle/libs.versions.toml", "versions.*", "build.gradle", ".github/workflows/*"],
   "postUpgradeTasks": {
-    "commands": ["./gradlew resolveAndLockAll --write-locks", "./gradlew updateLicenses"],
+    "commands": [
+      "./gradlew resolveAndLockAll --write-locks",
+      "./gradlew kotlinUpgradeYarnLock",
+      "./gradlew updateLicenses"
+    ],
     "fileFilters": ["solr/licenses/*.sha1"],
     "executionMode": "branch"
   },

.gitignore

@@ -36,6 +36,9 @@ __pycache__
 gradle/wrapper/gradle-wrapper.jar
 .gradletasknamecache
 
+# Kotlin
+.kotlin/
+
 # WANT TO ADD MORE?  You can tell Git without adding to this file:
 # See https://git-scm.com/docs/gitignore
 # In particular, if you have tools you use, add to $GIT_DIR/info/exclude or use core.excludesFile

build.gradle

@@ -25,6 +25,7 @@ plugins {
   alias(libs.plugins.owasp.dependencycheck)
   alias(libs.plugins.cutterslade.analyze)
   alias(libs.plugins.benmanes.versions)
+  alias(libs.plugins.kotlin.multiplatform) apply false
   alias(libs.plugins.littlerobots.versioncatalogupdate) apply false
   alias(libs.plugins.thetaphi.forbiddenapis) apply false
   alias(libs.plugins.undercouch.download) apply false
@@ -38,6 +39,9 @@ plugins {
 rootProject.ext.minJavaVersionDefault = JavaVersion.toVersion(libs.versions.java.min.get())
 rootProject.ext.minJavaVersionSolrJ = JavaVersion.toVersion(libs.versions.java.solrj.get())
 
+// Check development mode for entire project (defaults to true if 'production' not provided and set to true)
+rootProject.ext.development = !project.hasProperty('production') || project.findProperty('production') != 'true'
+
 apply from: file('gradle/globals.gradle')
 
 // General metadata.

dev-docs/dependency-upgrades.adoc

@@ -26,16 +26,18 @@ Read the https://github.com/apache/solr/blob/main/help/dependencies.txt[help/dep
 explanation of how dependencies are managed.
 
 == Manual dependency upgrades
-In order to upgrade a dependency, you need to run through a number of steps:
+To upgrade a dependency, you need to run through a number of steps:
 
 1. Identify the available versions from e.g. https://search.maven.org[Maven Central]
 2. Update the version in `gradle/libs.versions.toml` file
-3. Run `./gradlew resolveAndLockAll` to re-generate lockfiles. Note that this may cause a cascading effect where
-   the locked version of other dependencies also change.
-4. In case of a conflict, resolve the conflict according to `help/dependencies.txt`
-5. Update the license and notice files of the changed dependencies. See `help/dependencies.txt` for details.
-6. Run `./gradlew updateLicenses` to re-generate SHA1 checksums of the new jar files.
-7. Once in a while, a new version of a dependency will transitively bring in brand-new dependencies.
+3. Run `./gradlew resolveAndLockAll --write-locks` to re-generate lockfiles. Note that this may cause a cascading effect
+   where the locked version of other dependencies also changes.
+4. Run `./gradlew kotlinUpgradeYarnLock` to update the kotlin-js-store lockfile used for the new UI.
+   Most of the cases it will not have any changes.
+5. In case of a conflict, resolve the conflict according to `help/dependencies.txt`
+6. Update the license and notice files of the changed dependencies. See `help/dependencies.txt` for details.
+7. Run `./gradlew updateLicenses` to re-generate SHA1 checksums of the new jar files.
+8. Once in a while, a new version of a dependency will transitively bring in brand-new dependencies.
    You'll need to decide whether to keep or exclude them. See `help/dependencies.txt` for details.
 
 === Constraints and Version Alignment

dev-docs/ui/component-development.adoc

@@ -0,0 +1,117 @@
+= Component Development
+:toc: left
+
+== Overview
+
+The following list contains a possible approach for implementing a new UI component:
+
+1. Create a new design or start with an existing design, see for example Figma
+2. Validate the use case and analyze the components that may be used for the implementation
+3. Create Composables that represent the UI component(s) and use placeholders for data population
+4. Create a component interface and implementation with the UI state and UI component interactions
+5. Create previews with preview component implementations to check the UI implementation
+6. Create a store and store provider for fetching resources and interacting with Solr backend
+7. Implement the client used by the store provider
+8. Write tests and test the new component
+9. If not already done, integrate the component in the existing application
+10. If not already done, extract resources like texts to allow internationalization and localization
+
+It is recommended to take a look at existing components, so that you get a better understanding
+of how things are implemented, what they have in common, and how each technology is utilized.
+
+== Component's Logic
+
+=== Components (Decompose)
+
+The component integration interacts with the UI composables and the state store.
+
+The implementation of the component interface "catches" user inputs like clicks and passes them
+to the store as ``Intent``s. The intents are then handled by the store implementation and
+may send a request to the backend and / or update the store state. The component is consuming
+and mapping the store state to the UI state. So once the store state is updated, it will
+reflect the changes in the UI.
+
+=== State Stores and Store Providers
+
+The state stores manage the state of the application, but independent of the state that is
+represented in the UI. Instances are created by store providers that hold the logic of the
+store.
+
+Store providers consist of three elements:
+
+- an executor implementation that consumes actions and intents and creates messages and labels
+- a reducer that updates the store state with the messages produced by the executor
+- a function for retrieving an instance of the store
+
+The store provider does also define the interface for the client that has to be provided in
+order for the executor to make API calls and interact with the Solr backend.
+
+== Component's Visuals
+
+=== Composables
+
+Composables are the UI elements that are defined and styled. They can be seen as boxes, rows and
+columns that are nested and change their style and structure based on conditions, state and input.
+
+There are many ways to get started, but the easiest way probably is to get familiar with the basics
+and try things out. The Figma designs make use of almost the same elements for designing,
+so the structure and configurations there may be mapped almost one-by-one in Compose code.
+
+=== Styling
+
+The styling in Compose is done via ``Modifier``s. Each composable should normally accept a modifier
+as a parameter, so that the user can customize specific visual parameters of the composable like
+width, height and alignment in the parent's composable.
+
+Since we are using Material 3, you do not have to care much about colors, typography and shapes.
+These are configured for the entire app, and you only have to make use of the right properties
+that are provided by the theme.
+
+=== Accessibility
+
+Compose comes with many accessibility features that can be used to improve the user experience.
+
+The simplest form of accessibility in a UI is probably the responsiveness of the UI. This is
+realized with `WindowSizeClass`. Some composables may use a wrapper (usually suffixed with
+`Content`) that checks the window size and loads different UI based on the dimensions of the
+current window.
+
+Another accessibility feature is the resource loading based on the system's locale or the user's
+preference. This allows the UI to be displayed in the user's native language. For that, you have
+to simply provide translations in the Compose resources.
+
+Another accessibility feature often underestimated is coloring. Some people with color vision
+deficiency may need a different theme, so that elements with problematic contrasts may be
+better visible again.
+
+Additional accessibility features like font scaling, semantics for screen readers may also
+be considered. Jetpack Compose provides a https://developer.android.com/develop/ui/compose/accessibility[simplified overview]
+and https://developer.android.com/codelabs/jetpack-compose-accessibility#0[Codelabs] for getting started.
+
+=== Navigation and Child Components
+
+Some components may have navigation elements and will load other components inside a frame layout.
+Since components hold a hierarchical context that needs to be managed somehow, child components
+(also used in navigation) are instantiated in a slightly different manner.
+
+Decompose provides https://arkivanov.github.io/Decompose/navigation/overview/[a few examples]
+and details of the process behind the navigation and child components.
+
+== Additional Notes
+
+=== Dependency Locking
+
+When adding or changing dependencies, you typically run `./gradlew resolveAndLockAll --write-locks`.
+Since we are building a web application from kotlin sources, we also have to update the JS lockfile
+with `./gradlew kotlinUpgradeYarnLock`. This will update the lockfile found at `kotlin-js-store/yarn.lock`.
+
+Some multiplatform libraries have platform-specific dependency resolution that will result in different
+lockfiles being generated, based on the environment the lock task is executed. It is important to exclude
+these platform-specific libraries from the lockfile to ensure a consistent lockfile generation across
+different operating systems.
+
+Platform-specific libraries come with a module name suffix that includes the platform name, like
+in `org.jetbrains.compose.desktop:desktop-jvm-windows-x64`. To identify those, look into the
+changes after updating the lockfile and add the necessary ignore-clause if such libraries
+exist. These ignore-clauses should be added in `gradle/validation/dependencies.gradle` inside the
+`allprojects.dependencyLocking` block.

dev-docs/ui/introduction.adoc

@@ -0,0 +1,21 @@
+= New UI
+
+== Introduction
+
+The new UI that is introduced is a standalone frontend client that makes use of Solr's API.
+It is written in Kotlin and uses Compose Multiplatform as the UI framework.
+
+== Overview
+
+Since UI development mostly relies on different technologies and frameworks than backends use,
+the documentation is covering the following topics for new and experienced developers:
+
+- Technology Overview
+- Module Structure and Elements
+- Component Development
+- Testing and Deployment
+
+== Notes
+
+All the references to files and directories in the UI documentation are from within
+the `solr/ui` module, if not otherwise stated.

dev-docs/ui/module-structure.adoc

@@ -0,0 +1,84 @@
+= Module Structure and Elements
+
+== Module Structure
+
+The `ui` module follows a quite simple structure. It is split in
+
+- *components*, which covers the logic part of the frontend and therefore contains
+the interfaces and implementations of them,
+- the *UI* part, which covers the visual elements, styling and everything related to UI, and
+- the *utils*, which contains various utilities used in both UI and logic.
+
+In general, both UI and logic of the frontend are separated in "components" that follow similar,
+if not the same, structure and files.
+
+== Components (Logic)
+
+The logical part of a simple component (`org/apache/solr/ui/components`) usually consists of:
+
+- **The component interface** (`[ComponentName]Component.kt`), that defines the UI state and the
+interaction options,
+
+- **The store interface** (`store/[ComponentName]Store.kt`), that defines the (Solr) API state,
+intents for interacting with the state and optional labels (fire-and-forget events),
+
+- **The store provider** (`store/[ComponentName]StoreProvider.kt`), that defines an API client
+interface that is used for making requests against the Solr API by consuming intents,
+updates the state and publishes labels,
+
+- **The implementations** (`integration/*`), including component-specific mappings
+(`integration/Mappers.kt`) and interface implementations (like
+`integration/[Variant][ComponentName]Component.kt`),
+
+- **Component-specific data classes** (`domain/*`), that are used only by the UI module
+
+- **Component-specific API classes** (`data/*`), that are used for representing API requests /
+responses.
+
+Some components may use multiple stores to consume different API endpoints or no store at all
+if no API interaction is necessary. Multiple components may also be used to simplify the complexity
+or separate the responsibilities into smaller UI elements.
+
+Component data classes for API and internal use may also be merged and used interchangeably to
+reduce overall complexity. However, this affects the separation of concerns and may require
+in a later state the separation and mapping again.
+
+This structure is strongly inspired by Decompose's https://arkivanov.github.io/Decompose/samples/[samples].
+
+== User Interface (UI)
+
+Similar to the logical part, the UI classes are also separated in components under
+`org.apache.solr.ui.views`.
+
+Components may consist of one or multiple composables that make up a screen, section or
+element. The composables may also be reused, which is why they may be moved at some point
+during development to `.ui.components`.
+
+Some vector assets like logos may be migrated to `ImageVector` and placed in `.ui.icons`
+to later be used in the UI.
+
+Theme-related classes, functions and variables are all placed inside `.ui.theme`. The UI
+is making strong use of and customizes Material 3. You can find more information about
+Compose and Material 3 at https://m3.material.io/develop/android/jetpack-compose[Material Design - Jetpack Compose].
+
+Composables may accept a component interface and be stateful, or simply hoist the entire
+state via parameters and be stateless. For more information about state hoisting,
+you can have a look at https://developer.android.com/develop/ui/compose/state[Managing state].
+
+=== Compose Resources
+
+Compose Resources are similar to assets in a web app and can be found at
+`commonMain/composeResources`. They contain files like fonts, translations, images,
+raw files and more.
+
+The resource directories can use "qualifiers" that allow files to have variants.
+The most common use case is having translations for any text loaded. The directory
+`values/strings.xml` is the default / fallback strings resource file, that holds all values
+in english. If we would like to add translations for german now, we could use
+`values-de/strings.xml` and translate individual strings to german. Any string that is used
+and has no translation will automatically fall back to the value stored in `values/strings.xml`.
+
+At the moment of writing, the language, theme and density qualifier are supported.
+
+For more information about Compose Resources, see
+https://www.jetbrains.com/help/kotlin-multiplatform-dev/compose-images-resources.html[Multiplatform Resources].

dev-docs/ui/technology-overview.adoc

@@ -0,0 +1,83 @@
+= Technology Overview
+:toc: left
+
+This document lists information about the technologies used in the UI module.
+
+== Kotlin
+
+Kotlin is a modern, statically typed programming language that runs on the Java Virtual
+Machine (JVM) and can also be compiled to JavaScript or native code. It is designed to be
+fully interoperable with Java, making it an excellent choice for UI and
+server-side development.
+
+If you are new to Kotlin, it is strongly recommended to browse and go through the
+https://kotlinlang.org/[language documentation]. Since the UI module makes use of Kotlin's
+Multiplatform capabilities, you should also have a look at https://kotlinlang.org/docs/multiplatform.html[Kotlin Multiplatform].
+
+=== Coroutines
+
+Kotlin Coroutines provide a powerful and flexible way to manage asynchronous programming.
+They allow developers to write code that is sequential in nature but non-blocking under the hood,
+enabling efficient and straightforward concurrent programming. Coroutines simplify tasks such as
+making network requests, processing large datasets, or handling multiple user interactions
+simultaneously. They are integrated into the Kotlin language through libraries like
+kotlinx.coroutines.
+
+To get started with Kotlin Coroutines, you can have a look at https://kotlinlang.org/docs/coroutines-overview.html[Coroutines Overview].
+
+=== Ktor
+
+https://ktor.io/[Ktor] is a Kotlin framework designed for building asynchronous servers and clients.
+It provides a robust toolkit for making HTTP requests and handling responses, making it an
+excellent choice for frontend applications that need to communicate with a backend server.
+Ktor's client library is highly customizable and supports various features essential for
+frontend development.
+
+== Compose Multiplatform
+
+Compose Multiplatform is a UI framework that enables developers to create user interfaces
+for Android, iOS, desktop, and web applications using a single codebase. It is built on
+the principles of Jetpack Compose, providing a declarative way to build UIs.
+
+To get started with Compose Multiplatform, see the https://www.jetbrains.com/lp/compose-multiplatform/[Compose Multiplatform page].
+Many resources from https://developer.android.com/compose[Jetpack Compose] may be used for learning
+and as reference, but keep in mind that some of the information may not have a multiplatform
+integration yet.
+
+=== Material 3
+
+Material 3, also known as Material You, is the latest version of Google's Material Design system.
+It offers new components, dynamic theming, and updated guidelines to create a more personalized
+and adaptive user experience.
+
+The https://m3.material.io/[Material 3 website] gives a good introduction into user interface
+design and Material 3.
+
+=== Decompose
+
+Decompose is a library for managing the lifecycle of components in Compose Multiplatform
+applications. It provides tools for navigating between screens and managing state across
+different parts of an application.
+
+The main features of Decompose include navigation, state management and component lifecycle.
+You can find out more and get started via the https://arkivanov.github.io/Decompose/[Decompose documentation].
+
+=== MVIKotlin
+
+MVIKotlin is a library for implementing the Model-View-Intent (MVI) architecture pattern in
+Kotlin applications. It is particularly useful for managing complex state and side effects in a
+predictable manner. In combination with Decompose it provides a great foundation and
+unified structure for the UI components.
+
+You can find out more at https://arkivanov.github.io/MVIKotlin/.
+
+=== Essenty
+
+Essenty is a collection of utility libraries that complement the Kotlin ecosystem,
+particularly useful in Compose Multiplatform projects. It includes tools for handling events,
+lifecycles, and other common tasks in a type-safe and idiomatic way.
+
+Similar to MVIKotlin, it adds the foundation and structure of the UI module and plays an
+essential part in Decompose.
+
+For more information, you can have a look at https://github.com/arkivanov/Essenty[GitHub - Essenty].