-
Notifications
You must be signed in to change notification settings - Fork 771
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(meta): implement distributed semaphore based on meta-service #17651
Conversation
The CI may still need to be fixed, but the PR implementation is now complete. I invite you to review the API layer specifically to ensure it aligns with your requirements and expectations. And if you got time, I'd also appreciate your feedback on the other components of this implementation. Any additional review would be valuable. |
1c54ea8
to
1cc9394
Compare
02a4ec7
to
2cec4e0
Compare
This commit introduces a distributed semaphore implementation that provides reliable resource management across distributed systems. The semaphore is built on top of meta-service and offers: - Fair queueing mechanism using sequence numbers - Automatic lease management with TTL - Atomic operations for consistency - Real-time updates via meta-service watch API Usage: ```rust let client = MetaGrpcClient::try_create(/*..*/); let acquired_guard = Semaphore::new_acquired( client, "your/semaphore/name/in/meta/service", 2, // capacity: 2 acquired at most "id11", // ID of this acquirer Duration::from_secs(3) // lease time ).await?; acquired_guard.await; // Released ``` The implementation uses a queue-based structure with sequence numbers to ensure fair ordering of resource requests. It handles distributed coordination through meta-service, providing automatic lease extension and cleanup of stale entries. Key components: - Semaphore: Main entry point for semaphore operations - Acquirer: Handles the acquisition process - AcquiredGuard: Manages semaphore lifecycle - Queue-based storage with sequence numbers Note: Current version sets capacity per client. Future versions will store capacity in meta key for global consistency.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 1 of 1 files at r1, 17 of 34 files at r2, 3 of 10 files at r3, 12 of 17 files at r4, 4 of 4 files at r5, 5 of 5 files at r6, 10 of 10 files at r7, 6 of 6 files at r8, 4 of 4 files at r9, 5 of 5 files at r10, all commit messages.
Reviewable status:complete! all files reviewed, all discussions resolved (waiting on @drmingdrmer)
I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/
Summary
feat(meta): implement distributed semaphore based on meta-service
This commit introduces a distributed semaphore implementation that provides
reliable resource management across distributed systems. The semaphore is
built on top of meta-service and offers:
Usage:
The implementation uses a queue-based structure with sequence numbers to
ensure fair ordering of resource requests. It handles distributed
coordination through meta-service, providing automatic lease extension
and cleanup of stale entries.
Key components:
Note: Current version sets capacity per client. Future versions will
store capacity in meta key for global consistency.
docs: update meta-service change-log
Tests
Type of change
Related Issues
This change is