Skip to content

Commit 2e55c67

Browse files
authored
Update DATABASE_MANAGEMENT.md
Removed BaseX Signed-off-by: Sander Jansen <[email protected]>
1 parent 28ca258 commit 2e55c67

File tree

1 file changed

+2
-103
lines changed

1 file changed

+2
-103
lines changed

DATABASE_MANAGEMENT.md

Lines changed: 2 additions & 103 deletions
Original file line numberDiff line numberDiff line change
@@ -6,31 +6,9 @@ SPDX-License-Identifier: CC-BY-4.0
66

77
## Database Management
88

9-
## Versioning Overview
10-
![Versioning overview](./images/database/BaseX_Versioning.png)
11-
12-
To achieve versioning (which is not available out-of-the-box), we need to add something smart to BaseX. This smart thing is [RESTXQ](http://exquery.github.io/exquery/exquery-restxq-specification/restxq-1.0-specification.html) in our case.
13-
14-
With RESTXQ, functions can be created using xQuery and some added intelligence like variables and for-loops for example.
9+
CoMPAS uses a PostgreSQL database.
1510

16-
Example RESTXQ function:
17-
18-
```
19-
declare
20-
%rest:path("/search")
21-
%rest:query-param("term", "{$term}")
22-
%rest:single
23-
function page:search($term as xs:string) {
24-
<ul>{
25-
for $result in db:open('large-db')//*[text() = $term]
26-
return <li>{ $result }</li>
27-
}</ul>
28-
};
29-
```
30-
31-
By using RESTXQ, a versioning mechanism can be created. So for example, in a edit (PUT) function we can do something like: When editing a already stored configuration, save it by incrementing the version and store as a separate configuration. The old configuration is stored in the archive database, the current version is replaced in the current database.
32-
33-
In a get (GET) function, we can make distinction between newer and older versions using RESTXQ. By using xQuery syntax (scl[@version="1"] for example), we can get specific versions of a configuration.
11+
## Versioning Overview
3412

3513
### Versioning type
3614
For type of versioning, we prefer [Semantic Versioning](https://semver.org/). This to keep versioning simple. For every changeset CoMPAS is going to ask if it's a major, minor or a patch. This way the version will be adjusted according to the user's needs. An example of distinction can be:
@@ -52,88 +30,9 @@ This creates provenance, and version is one of them. The version attribute will
5230
Another solution could be [Branch Based Versioning](https://simon-maxen.medium.com/branch-based-versioning-5ebf6ca2bccb). This way, a configuration file can be 'branched', and can be 'merged' when the user think it's fine. When merging, a newer version number can be added (can be done in combination with semantic versioning).
5331
This in indeed a fancy way of versioning, but it's too complex for our use cases. We don't see users branching a configuration file and saving it for a couple of days, before merging it. Besides, this kind of versioning isn't supported in BaseX out of the box so we have to create it ourselves. When comparing added value to effort, this isn't what we want.
5432

55-
## Tech Talk
56-
57-
### Points to remember
58-
- home of BaseX = /srv/basex
59-
- RESTXQ file extension = .xqm
60-
- RESTXQPATH variable (in {home}/webapp/WEB-INF/web.xml) points to directory containing the RESTXQ modules (.xqm files)
61-
- Default is '.', which is relative to the WEBPATH variable (which is {home}/webapp)
62-
63-
### Example using RESTXQ
64-
65-
- Run a BaseX container
66-
- Use shell inside container (docker exec -it <container id> bash)
67-
- create a RESTXQ module: vi /srv/basex/webapp/test.xqm for example
68-
- copy paste the following code:
69-
70-
```
71-
module namespace page = 'http://basex.org/examples/web-page';
72-
73-
declare %rest:path("hello/{$who}") %rest:GET function page:hello($who) {
74-
<response>
75-
<title>!Hello { $who }!</title>
76-
</response>
77-
};
78-
```
79-
80-
- You don't have to restart the container, when doing a REST request it seaches on the fly for functions.
81-
- Do a GET request like http://localhost:8984/hello/World
82-
- You will get a XML containing a title !Hello World!
83-
84-
### Restrictions
85-
A single database is restricted to 2 billion nodes (also, see [BaseX Statistics](https://docs.basex.org/wiki/Statistics))
86-
A node in this case is an XML node like an element, attribute, text, etc.
87-
88-
### Sources
89-
http://www.adamretter.org.uk/presentations/restxq_mugl_20120308.pdf
90-
9133
## Database Rights
9234
In a microservice architecture, a microservice's database should be part of the implementation of that service and cannot be accessed directly by other services. This way, the service is loosely coupled and can be developed/scaled/deployed independently.
9335

94-
There are some patterns to keep persistent data private:
95-
- private-tables-per-service
96-
- schema-per-service
97-
- database-server-per-service
98-
As seen, 2 options are not available for BaseX because it's not a relational database. It doesn't have tables or schemas.
99-
A Database-server-per-service pattern helps ensure that the services are lossely coupled.
100-
101-
The CIM - IEC 61850 service for example get's their own database. If another service wants to get SCD files from this service, use the API of that particular service.
102-
103-
### Where do we set the user privileges of Basex?
104-
Basex has it's own [User Management](https://docs.basex.org/wiki/User_Management).
105-
106-
It's pretty straight forward: Basex has Users that can be created. These users can have so-called permissions that can be applied to the user:
107-
![BaseX permissions overview](./images/database/basex_permissions.png)
108-
109-
In this overview, we see 'Global' permissions and 'Local' permissions.
110-
In both permission groups, a higher permission includes all lower permissions. So a user with the 'Create' permission also has the 'Read' permission.
111-
112-
All permissions are stored in a file called users.xml (which can be edited manually) inside the database directory, and is being parsed once BaseX is started.
113-
114-
### How do we connect BaseX with a central identity repository/application?
115-
BaseX doesn't have compatibility with a central identity repository (like Keycloak) out of the box available, but after discussing it with the BaseX community it's pretty comfortable to achieve this with RESTXQ or xQuery. There are examples available for making use of Keycloak:
116-
117-
[Example with xQuery](https://code-repo.d4science.org/gCubeSystem/d4science-keycloak-themes/src/branch/master/src/utils/xquery)
118-
119-
[Example with RESTXQ](./blob-files/code_examples/auth_sk.xqm)
120-
Author: Marco Lettere. Origin: [BaseX Mailing List](https://mailman.uni-konstanz.de/pipermail/basex-talk/2021-May/016157.html)
121-
122-
Full attached description about this example:
123-
124-
>I attach here an example of an OIDC code grant flow implemented with RestXQ, BaseX permission and error handler.
125-
The file includes a sort of library for performing the steps of the OIDC flow plus a minimal application that is registered as public client inside keycloak and which is what you should access from your browser by calling http://localhost:8984/authtest or http://localhost:8984/authtest/internal.
126-
I've put into it also the logout procedure for performing the back-channel logout which closes the SSO session.
127-
This is only a resume of a more generic and complex module but it should be useful as a howto and it should be as simple to install as copying the file to your BaseX' webapp folder. Use it as you like.
128-
129-
### Is direct database access allowed within the microservices architecture?
130-
For maintenance for example, it's of course allowed to have direct database access. There is no best practice available for this. For some things, you just need direct database access.
131-
132-
If other microservices need access to the data of an other microservice, the only way (best practice) to do this is by API calls.
133-
134-
Source:
135-
https://microservices.io/patterns/data/database-per-service.html
136-
13736
## Provenance Overview
13837
If the generation of a substation fails for example, we would like to know the provenance of the file.
13938
This way it's easier to get the cause.

0 commit comments

Comments
 (0)