Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(meta): implement distributed semaphore based on meta-service #17651

Merged
merged 17 commits into from
Mar 28, 2025

Conversation

drmingdrmer
Copy link
Member

@drmingdrmer drmingdrmer commented Mar 25, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

feat(meta): implement distributed semaphore based on meta-service

This commit introduces a distributed semaphore implementation that provides
reliable resource management across distributed systems. The semaphore is
built on top of meta-service and offers:

  • Fair queueing mechanism using sequence numbers
  • Automatic lease management with TTL
  • Atomic operations for consistency
  • Real-time updates via meta-service watch API

Usage:

let client = MetaGrpcClient::try_create(/*..*/);
let acquired_guard = Semaphore::new_acquired(
        client,
        "your/semaphore/name/in/meta/service",
        2,                          // capacity: 2 acquired at most
        "id11",                     // ID of this acquirer
        Duration::from_secs(3)      // lease time
).await?;

acquired_guard.await;
// Released

The implementation uses a queue-based structure with sequence numbers to
ensure fair ordering of resource requests. It handles distributed
coordination through meta-service, providing automatic lease extension
and cleanup of stale entries.

Key components:

  • Semaphore: Main entry point for semaphore operations
  • Acquirer: Handles the acquisition process
  • AcquiredGuard: Manages semaphore lifecycle
  • Queue-based storage with sequence numbers

Note: Current version sets capacity per client. Future versions will
store capacity in meta key for global consistency.

docs: update meta-service change-log

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Other

Related Issues


This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Mar 25, 2025
@drmingdrmer drmingdrmer requested a review from zhang2014 March 25, 2025 15:01
@drmingdrmer
Copy link
Member Author

@zhang2014

The CI may still need to be fixed, but the PR implementation is now complete. I invite you to review the API layer specifically to ensure it aligns with your requirements and expectations.

And if you got time, I'd also appreciate your feedback on the other components of this implementation. Any additional review would be valuable.

@drmingdrmer drmingdrmer force-pushed the 203-sem branch 2 times, most recently from 1c54ea8 to 1cc9394 Compare March 25, 2025 15:13
@drmingdrmer drmingdrmer force-pushed the 203-sem branch 3 times, most recently from 02a4ec7 to 2cec4e0 Compare March 27, 2025 15:40
@drmingdrmer drmingdrmer marked this pull request as ready for review March 27, 2025 15:41
This commit introduces a distributed semaphore implementation that provides
reliable resource management across distributed systems. The semaphore is
built on top of meta-service and offers:

- Fair queueing mechanism using sequence numbers
- Automatic lease management with TTL
- Atomic operations for consistency
- Real-time updates via meta-service watch API

Usage:

```rust
let client = MetaGrpcClient::try_create(/*..*/);
let acquired_guard = Semaphore::new_acquired(
        client,
        "your/semaphore/name/in/meta/service",
        2,                          // capacity: 2 acquired at most
        "id11",                     // ID of this acquirer
        Duration::from_secs(3)      // lease time
).await?;

acquired_guard.await;
// Released
```

The implementation uses a queue-based structure with sequence numbers to
ensure fair ordering of resource requests. It handles distributed
coordination through meta-service, providing automatic lease extension
and cleanup of stale entries.

Key components:
- Semaphore: Main entry point for semaphore operations
- Acquirer: Handles the acquisition process
- AcquiredGuard: Manages semaphore lifecycle
- Queue-based storage with sequence numbers

Note: Current version sets capacity per client. Future versions will
store capacity in meta key for global consistency.
Copy link
Member Author

@drmingdrmer drmingdrmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r1, 17 of 34 files at r2, 3 of 10 files at r3, 12 of 17 files at r4, 4 of 4 files at r5, 5 of 5 files at r6, 10 of 10 files at r7, 6 of 6 files at r8, 4 of 4 files at r9, 5 of 5 files at r10, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @drmingdrmer)

@drmingdrmer drmingdrmer merged commit c3d5d79 into databendlabs:main Mar 28, 2025
94 of 98 checks passed
@drmingdrmer drmingdrmer deleted the 203-sem branch March 28, 2025 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants