Fb/unmanaged values 2 (#72)

* test for getRemoteFeaturesInfos

* changeFeatureValue with clearSubScopes

* better setup for changeFeatureValue with option clearSubScopes test

* changeFeatureValue with option remoteOnly

* test changeFeatureValue with option remoteOnly for failing case

* feature key consistency fix

* better test for remoteOnly cleanup

* disable special markup markup for .http calls

* wip service docs 1

* wip service docs 2

* make redisRead a read-privilege endpoint

* add an example for redisUpdate with remoteOnly option

* docs: add changeFeatureValue option remoteOnly

* more wording on remoteOnly option

* update webrick

* more wording on remoteOnly option 2

* more information for read /state

* more information for read /redis

* more polish

Author: rlindner81

docs/Gemfile.lock

@@ -289,7 +289,7 @@ GEM
       concurrent-ruby (~> 1.0)
     unicode-display_width (1.8.0)
     uri (0.13.1)
-    webrick (1.8.1)
+    webrick (1.8.2)
 
 PLATFORMS
   aarch64-linux

docs/concepts/index.md

@@ -83,7 +83,7 @@ And we could change the behavior again with for the more specific user `john` an
 
 - `changeFeatureValue(key, "new value just for john within t1", { user: "john", tenant: "t1" })`
 
-{: .warn}
+{: .warn }
 As we can see in the precedence check order, if we had just set `changeFeatureValue(key, "new value for john", { user: "john" })`,
 then it depends on the order used in the `getFeatureValue` call, whether the `user` scope is evaluated before
 the `tenant` scope.

docs/plugin/index.md

@@ -44,7 +44,7 @@ It will usually be sufficient to set the `serviceAccessRoles` configuration, whi
 endpoints, but not the admin endpoints. If more discriminating access control is required, the `readAccessRoles` and
 `writeAccessRoles` can be set separately. For debugging purposes, you can also set the `adminAccessRoles`.
 
-{: .warn}
+{: .warn }
 As the name suggests, the `adminAccessRoles` should be considered sensitive. It allows direct root access to the
 underlying redis.
 
@@ -97,7 +97,7 @@ automatically detect it and configure it as follows:
   fallbackValue: false
 ```
 
-{: .info}
+{: .info }
 This automatic configuration can be _overwritten_, by using a configuration file and adding a dedicated configuration
 with the same key `/fts/my-feature`.
 
@@ -109,14 +109,17 @@ for the related requests. For an example check out the [Example CAP Server](http
 This service endpoint will enable operations teams to understand toggle states. For practical requests, check the
 [http file](https://github.com/cap-js-community/feature-toggle-library/blob/main/example-cap-server/http/feature-service.http) in our example CAP Server.
 
-### Read Feature Toggles State
+### Read Server State
 
-Get all information about the current in-memory state of all toggles.
+Get information about the current in-memory state of all configured toggles. The response will give you transparency
+about maintained values and the underlying configuration of the toggles. In the following example, the
+`/check/priority` toggle has a maintained root value, and two scoped values. All other toggles have no maintained
+values, so they will use their fallback values.
 
 <b>Example Request/Response</b>
 
 - Request
-  ```http
+  ```
   GET /rest/feature/state
   Authorization: ...
   ```
@@ -129,6 +132,11 @@ Get all information about the current in-memory state of all toggles.
   {
     "/check/priority": {
       "fallbackValue": 0,
+      "rootValue": 1,
+      "scopedValues": {
+        "tenant::people": 10,
+        "user::[email protected]": 100
+      },
       "config": {
         "SOURCE": "FILE",
         "TYPE": "number",
@@ -168,19 +176,82 @@ Get all information about the current in-memory state of all toggles.
   }
   ```
 
+### Read Redis State
+
+Get information about the remote redis state of all toggles with maintained values, even ones that are not configured.
+This endpoint will show the state within redis. Only toggles with maintained values will be shown here. In the example,
+we can see `/check/priority` with the maintained values. We can also see a `/legacy-key` toggle, which has maintained
+values that are not associated with a configuration `{ "SOURCE": "NONE" }`.
+
+{: .info }
+Note that reading the redis state can reveal legacy key values that used to be configured and maintained but are no
+longer in the configuration. These values can be cleaned up by using the [redisUpdate](#update-feature-toggle) endpoint
+with the `remoteOnly` option.
+
+<b>Example Request/Responses</b>
+
+- Request
+  ```
+  POST /rest/feature/redisRead
+  Authorization: ...
+  ```
+- Response
+  ```
+  HTTP/1.1 200 OK
+  ...
+  ```
+  ```json
+  {
+    "/check/priority": {
+      "fallbackValue": 0,
+      "rootValue": 1,
+      "scopedValues": {
+        "tenant::people": 10,
+        "user::[email protected]": 100
+      },
+      "config": {
+        "SOURCE": "FILE",
+        "TYPE": "number",
+        "VALIDATIONS": [
+          {
+            "scopes": ["user", "tenant"]
+          },
+          {
+            "regex": "^\\d+$"
+          },
+          {
+            "module": "$CONFIG_DIR/validators",
+            "call": "validateTenantScope"
+          }
+        ]
+      }
+    },
+    "/legacy-key": {
+      "rootValue": 10,
+      "scopedValues": {
+        "tenant::a": 100,
+        "tenant::b": 1000
+      },
+      "config": {
+        "SOURCE": "NONE"
+      }
+    }
+  }
+  ```
+
 ## Service Endpoints for Write Privilege
 
 Similar to the read privilege endpoints, these endpoints are meant to modify toggle state. For practical requests,
 check the [http file](https://github.com/cap-js-community/feature-toggle-library/blob/main/example-cap-server/http/feature-service.http) in our example CAP Server.
 
 ### Update Feature Toggle
 
-Update the toggle state on Redis, which in turn is published to all server instances.
+Maintain a particular toggle value on Redis, which is automatically propagated to all server instances.
 
 <b>Example Request/Responses</b>
 
 - Valid Request
-  ```http
+  ```
   POST /rest/feature/redisUpdate
   Authorization: ...
   Content-Type: application/json
@@ -200,7 +271,7 @@ Update the toggle state on Redis, which in turn is published to all server insta
   ```
 
 - Valid Request with [clearSubScopes]({{ site.baseurl }}/usage/#updating-feature-value)
-  ```http
+  ```
   POST /rest/feature/redisUpdate
   Authorization: ...
   Content-Type: application/json
@@ -221,8 +292,31 @@ Update the toggle state on Redis, which in turn is published to all server insta
   ...
   ```
 
+- Valid Request with [remoteOnly]({{ site.baseurl }}/usage/#updating-feature-value)
+  ```
+  POST /rest/feature/redisUpdate
+  Authorization: ...
+  Content-Type: application/json
+  ```
+  ```json
+  {
+    "key": "/legacy-key",
+    "value": null,
+    "options": {
+      "clearSubScopes": true,
+      "remoteOnly": true
+    }
+  }
+  ```
+- Response
+
+  ```
+  HTTP/1.1 204 No Content
+  ...
+  ```
+
 - Invalid Request
-  ```http
+  ```
   POST /rest/feature/redisUpdate
   Authorization: ...
   Content-Type: application/json
@@ -248,21 +342,6 @@ Update the toggle state on Redis, which in turn is published to all server insta
   }
   ```
 
-### Re-Sync Server with Redis
-
-Force server to re-sync with Redis, this should never be necessary. It returns the same JSON structure as
-`/state`, after re-syncing.
-
-<b>Example Request/Response</b>
-
-- Request
-  ```http
-  POST /rest/feature/redisRead
-  Authorization: ...
-  ```
-- Response<br>
-  Same as [Read Feature Toggles State](#read-feature-toggles-state).
-
 ## Service Endpoints for Admin Privilege
 
 The service also offers an additional endpoint for deep problem analysis.
@@ -274,7 +353,7 @@ Send an arbitrary command to Redis. [https://redis.io/commands/](https://redis.i
 <b>Example Request/Responses</b>
 
 - Request INFO
-  ```http
+  ```
   POST /rest/feature/redisSendCommand
   Authorization: ...
   Content-Type: application/json
@@ -298,7 +377,7 @@ Send an arbitrary command to Redis. [https://redis.io/commands/](https://redis.i
   ...
   ```
 - Request KEYS
-  ```http
+  ```
   POST /rest/feature/redisSendCommand
   Authorization: ...
   Content-Type: application/json

docs/usage/index.md

@@ -302,8 +302,10 @@ processing delay until the change is picked up by all subscribers.
 Setting a feature value to `null` will delete the associated remote state and effectively reset it to its fallback
 value.
 
+_Option: clearSubScopes_
+
 Since setting values for scope-combinations happens additively, it can become hard to keep track of which combinations
-have dedicated values attached to them. If you want to set a value _and_ make sure that there isn't a more specific
+have maintained values attached to them. If you want to set a value _and_ make sure that there isn't a more specific
 scope-combination, which overrides that value, then you can use the option `{ clearSubScopes: true }` as a third
 argument. For example
 
@@ -314,6 +316,24 @@ await toggles.changeFeatureValue("/srv/util/logger/logLevel", "error", {}, { cle
 will set the root-scope value to `"error"` and remove all sub-scopes. See
 [scoping]({{ site.baseurl }}/concepts/#scoping) for context.
 
+_Option: remoteOnly_
+
+When you find toggle values in Redis that are not configured, marked with `{ "SOURCE": "NONE" }`, it usually makes
+sense to remove them. In this situation, we want to change _just Redis_ and bypass the local server state update, the
+usual validation, change handlers, and server instance change propagation. To achieve this, you can use the
+`{ remoteOnly: true }` option. For example
+
+```javascript
+await toggles.changeFeatureValue("/legacy-key", null, {}, { clearSubScopes: true, remoteOnly: true });
+```
+
+will remove all maintained values associated with the `/legacy-key` key in Redis.
+
+{: .info }
+Changes with the `{ remoteOnly: true }` option will be blocked for _configured_ toggles. This happens to avoid
+situations where the remote state of these toggles is accidentally changed in a way that bypasses validations and
+server state updates.
+
 ### Resetting Feature Value
 
 There is a convenience reset API just to reset a feature toggle and remove all associated persisted values. Reading

src/plugin.js

@@ -25,8 +25,8 @@ const ACCESS = Object.freeze({
   ADMIN: "ADMIN",
 });
 const SERVICE_ENDPOINTS = Object.freeze({
-  [ACCESS.READ]: [`${SERVICE_NAME}.state`],
-  [ACCESS.WRITE]: [`${SERVICE_NAME}.redisRead`, `${SERVICE_NAME}.redisUpdate`],
+  [ACCESS.READ]: [`${SERVICE_NAME}.state`, `${SERVICE_NAME}.redisRead`],
+  [ACCESS.WRITE]: [`${SERVICE_NAME}.redisUpdate`],
   [ACCESS.ADMIN]: [`${SERVICE_NAME}.redisSendCommand`],
 });
 

test/__mocks__/redisWrapper.js

@@ -10,13 +10,23 @@ const getObject = jest.fn(async (key) => {
   return mockRedisState.values[key];
 });
 
+const hashGetAllObjects = jest.fn(async () => {
+  mockRedisState.values = mockRedisState.values ? mockRedisState.values : {};
+  return mockRedisState.values[redisKey];
+});
+
 const type = jest.fn(async () => "hash");
 
 const watchedHashGetSetObject = jest.fn(async (key, field, newValueCallback) => {
   mockRedisState.values = mockRedisState.values ? mockRedisState.values : {};
   mockRedisState.values[key] = mockRedisState.values[key] ? mockRedisState.values[key] : {};
-  mockRedisState.values[key][field] = await newValueCallback(mockRedisState.values[key][field]);
-  return mockRedisState.values[key][field];
+  const newValue = await newValueCallback(mockRedisState.values[key][field]);
+  if (newValue === null) {
+    Reflect.deleteProperty(mockRedisState.values[key], field);
+  } else {
+    mockRedisState.values[key][field] = newValue;
+  }
+  return newValue;
 });
 
 const registerMessageHandler = jest.fn((channel, handler) => {
@@ -56,6 +66,7 @@ module.exports = {
   publishMessage,
   type,
   getObject,
+  hashGetAllObjects,
   watchedHashGetSetObject,
   subscribe: jest.fn(),
   getIntegrationMode: jest.fn(() => REDIS_INTEGRATION_MODE.CF_REDIS),

test/__snapshots__/featureToggles.test.js.snap

@@ -123,6 +123,31 @@ exports[`feature toggles test basic apis getFeaturesInfos 1`] = `
 }
 `;
 
+exports[`feature toggles test basic apis getRemoteFeaturesInfos 1`] = `
+{
+  "legacy-key": {
+    "config": {
+      "SOURCE": "NONE",
+    },
+    "rootValue": "legacy-root",
+    "scopedValues": {
+      "tenant::a": "legacy-scoped-value",
+    },
+  },
+  "test/feature_b": {
+    "config": {
+      "SOURCE": "RUNTIME",
+      "TYPE": "number",
+    },
+    "fallbackValue": 1,
+    "rootValue": 1,
+    "scopedValues": {
+      "tenant::a": 10,
+    },
+  },
+}
+`;
+
 exports[`feature toggles test basic apis initializeFeatureToggles 1`] = `
 {
   "test/feature_a": {