Skip to content

[Contrib] Agent-OS Governance Extension: Kernel-Level Policy Enforcement#7212

Open
imran-siddique wants to merge 6 commits intomicrosoft:mainfrom
imran-siddique:contrib/agent-os
Open

[Contrib] Agent-OS Governance Extension: Kernel-Level Policy Enforcement#7212
imran-siddique wants to merge 6 commits intomicrosoft:mainfrom
imran-siddique:contrib/agent-os

Conversation

@imran-siddique
Copy link
Member

Summary

Adds kernel-level governance for AutoGen multi-agent conversations using Agent-OS.

Why This Matters

AutoGen enables powerful multi-agent conversations, but lacks built-in policy enforcement. This extension provides:

  • Content Filtering: Block dangerous patterns (SQL injection, shell commands)
  • Tool Control: Limit which tools agents can use
  • Rate Limiting: Cap messages and tool calls per session
  • Audit Trail: Full logging for compliance and debugging

Changes

  • Added \python/packages/autogen-ext/src/autogen_ext/governance/\
    • _governance.py\ - GovernedAgent, GovernedTeam, GovernancePolicy classes
    • _init_.py\ - Public exports
    • \README.md\ - Documentation and examples

Example Usage

\\python
from autogen_ext.governance import GovernedTeam, GovernancePolicy
from autogen_agentchat.agents import AssistantAgent

Define policy

policy = GovernancePolicy(
max_tool_calls=10,
blocked_patterns=["DROP TABLE", "rm -rf"],
blocked_tools=["shell_execute"],
)

Create governed team

team = GovernedTeam(
agents=[analyst, reviewer],
policy=policy,
)

Run with governance

result = await team.run("Analyze Q4 sales")
audit = team.get_audit_log()
\\

Value for AutoGen Users

Feature Without Extension With Agent-OS
Content Filtering Manual Automatic
Tool Limits None Configurable
Audit Trail DIY Built-in
Policy Violations Runtime errors Controlled handling

Integration Path

This extension works standalone, but can also integrate with the full Agent-OS kernel for:

  • GDPR/HIPAA compliance policies
  • Cost control limits
  • Human-in-the-loop approval flows
  • Cross-framework governance

References

Adds kernel-level governance for AutoGen multi-agent conversations.

Features:
- GovernancePolicy: Define rules for agent behavior
- GovernedAgent: Wrap individual agents with policy enforcement
- GovernedTeam: Govern entire agent teams
- Content filtering with blocked patterns
- Tool call limits and filtering
- Full audit trail

Integration with Agent-OS kernel for enterprise governance.

See: https://github.com/imran-siddique/agent-os
@codecov
Copy link

codecov bot commented Feb 5, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.20%. Comparing base (13e144e) to head (9f1cc61).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7212      +/-   ##
==========================================
- Coverage   81.22%   81.20%   -0.02%     
==========================================
  Files         244        2     -242     
  Lines       18512      149   -18363     
==========================================
- Hits        15036      121   -14915     
+ Misses       3476       28    -3448     
Flag Coverage Δ
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@imran-siddique
Copy link
Member Author

Ready for Final Review 🙏

This PR has been open for a while. The AgentMesh trust layer integration is complete and tested.

Could a maintainer please provide a final review? Happy to address any remaining concerns.

Thank you!

Covers all 5 classes:
- GovernancePolicy: defaults, custom values, dataclass structure
- ExecutionContext: creation, event recording, multiple events
- PolicyViolationError: attributes, severity, exception hierarchy
- GovernedAgent: content checks, tool checks, on_messages, streaming,
  message limits, pattern blocking, attribute forwarding
- GovernedTeam: agent wrapping, content checks, run/run_stream,
  audit log, stats, violation handlers

Integration tests: full workflow + multi-pattern enforcement

All 46 tests pass locally (pytest 9.0.2, Python 3.13).
@imran-siddique
Copy link
Member Author

Codecov Report

❌ Patch coverage is 0% with 162 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.47%. Comparing base (13e144e) to head (c338201).

Files with missing lines Patch % Lines
...ogen-ext/src/autogen_ext/governance/_governance.py 0.00% 160 Missing ⚠️
...autogen-ext/src/autogen_ext/governance/__init__.py 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7212      +/-   ##
==========================================
- Coverage   81.22%   80.47%   -0.76%     
==========================================
  Files         244      246       +2     
  Lines       18512    18674     +162     
==========================================
- Hits        15036    15027       -9     
- Misses       3476     3647     +171     
Flag Coverage Δ
unittests 80.47% <0.00%> (-0.76%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Try again

@imran-siddique
Copy link
Member Author

Hi maintainers! This PR adds Agent-OS governance as an autogen-ext package. All review threads are addressed and the integration follows the existing autogen-ext package patterns. Would a maintainer be able to review? Happy to make any changes needed. Thank you!

@imran-siddique
Copy link
Member Author

Friendly nudge -- just wanted to share that Agent OS was merged into microsoft/agent-lightning (14k stars) yesterday: microsoft/agent-lightning#478 -- The Microsoft team validated the kernel-level governance approach for RL training. Happy to address any feedback here, including the patch coverage question!

- Use datetime.now(timezone.utc) instead of datetime.utcnow()
- Eliminates DeprecationWarning in Python 3.12+
- Tests: 46 passed, 91% coverage, 0 warnings
@imran-siddique
Copy link
Member Author

Update: Our AgentMesh trust layer was just merged into LlamaIndex (47k stars): run-llama/llama_index#20644. This is our second major integration merge this week after Microsoft's agent-lightning (14k stars). Would love to get this PR reviewed as well!

@imran-siddique
Copy link
Member Author

Friendly follow-up! This PR has been open for ~11 days. In the meantime, our governance layer has been merged into three major frameworks:

This shows real demand for governance in agent frameworks. Happy to address any feedback to get this merged for AutoGen as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant