开源之夏 2023｜讨论｜为 Apollo 增加项⽬、集群、Namespace、管理员授权等操作的审计⽇志

[BlackBear2003]

项目详情
issue
项目分析文档

大致需求分析，详细分析和实施方案详见上面的分析文档：

1. 审计⽇志记录要素：需要记录⽤户的关键操作，包括修改的对象、操作发⽣的时间、操作的具体内容以及执⾏
   操作的⽤户标识和IP地址。
2. ⾃动记录⽤户关键操作：系统需要能够⾃动捕获⽤户进⾏的关键操作，并将其记录为审计⽇志。关键操作包括对app、
   cluster、namespace、item等关键数据的修改，例如新增、更新和删除操作，作为操作⽇志。还有管理员授
   权，⽤户登录登出的记录作为安全⽇志作为补充。
3. 查询审计⽇志：系统需要提供查询审计⽇志的功能，以便⽤户能够根据需要检索和查看特定时间段、特定⽤户
   或特定操作类型的审计⽇志。⽽且需要能够提供操作事件的溯源，以便对错误进⾏追踪，⽀撑强⼒的审计功
   能。提供⼀个前端⻚⾯以供⽤户使⽤。查询功能应该具备⾼效性和良好的⽤户体验。
4. 设计新的数据库表结构：为存储审计⽇志⽽设计新的数据库表结构，以便有效地存储审计⽇志的要素信息。表
   结构应该能够⽀持⾼效的⽇志记录和查询操作。

后续开发时相关进度情况、出现的问题及后续讨论将于此 discussion 贴出和讨论。

[Anilople]

Design in https://excalidraw.com/#room=18012fafab93167ccba5,tlgN3msGmhPiu4_l65Y7Hg

[BlackBear2003]

经过和导师三个星期的交流，目前已经初步定下这个功能的方案。
首先对Apollo进行了关于审计日志功能的分析，梳理用户在操作Apollo时，不同进程之间的相互关系和可能发生数据变动的关系线并使用红色线标识

导师作出了如下的功能模型

针对Apollo多环境、多机集群的特点和我们需要的功能模型，决定借鉴分布式链路追踪的设计，将Apollo中所有引起或可能引起数据变动的行为(如数据库增删改或RPC操作)定义为一个审计跨度AuditSpan。在用户的一个操作中通过一个唯一的AuditTraceID来将所有分布式服务中的AuditSpan结合到一起，并且通过维护AuditSpan之间的相互关系(parent_span_id ,follows_from_span_id)，可以将Span建立成一张有向无环图。

在这个审计系统中，每个审计跨度AuditSpan都可能会产生数据变动，将其合理的记录是审计功能的核心目的。基于4个W(Who, Where, When, What)因素，目前已建立数据模型如下：

[Anilople]

For user's side

• annotation: add annotation for audit log collection
• config: use apollo.audit.log.enabled=true to enable the audit log feature for compability. All config start with apollo.audit.log.

For audit log internal

• context: trace, span, rpc
• the way of collection: aop

[BlackBear2003]

Common part

App、AppNamespace operation's audit operation logs and data influences on both parts.

contains root-operation:

• App.create

• App.update

• App.delete

• App.create.forEnv

• AppNamespace.create

• AppNamespace.delete

Portal part

Auth relates

User - UserRole - Role - RolePermission - Permission

I think we should mainly start from the user's perspective, who has granted what permissions. If the operation is generated by the system itself, then it can be ignored. Only the impact caused by the user should be considered.

In this case, only the UserRole and Role need to be recorded. I don't think it is a good way to record the operation further, and the user cannot actually understand the logic beneath, and the feature's effect is not very good.

The main focus is on permission changes. The table designed to change permissions directly points to UserRole. But there is a disadvantage in that this table only contains UserId, which is easier to understand. RoleId is just a number, so I added Role too, so that when viewing the audit log, I can see what kind of Role was created and related.

I only intercepted the two methods assignRoleToUsers and removeRoleFromUsers. The remove operation is directly written to the database, so I annotated the input parameters. The result is as follows:

• Auth.removeNamespaceEnvRoleFromUser

• Auth.assignNamespaceEnvRoleToUser

Namespace(Branch), Cluster and AccessKey

All these are persisted in AdminService, so there is no data influences in portal, so only record the operation logs.

contains root-operation:

• Namespace.create

• Namespace.deleteLinkedNamespace

• Namespace.createMissingNamespaces

• NamespaceBranch.create

• NamespaceBranch.delete

• NamespaceBranch.merge

• NamespaceBranch.updateBranchRules

• Cluster.create

• Cluster.update

• AccessKey.create

• AccessKey.delete

• AccessKey.enable

• AccessKey.disable

ServerConfig

There are two ServerConfig, one is in PortalDB and the other one is in ConfigDB. Both will audit operation logs, when operating 'portal' one, data influences would be audited.

contains root-operation:

• ServerConfig.createOrUpdatePortalDBConfig
• ServerConfig.createOrUpdateConfigDBConfig

[Anilople]

Nice design.

And need to test the origin feature didn't be destroy too.