Skip to content

Red transaction process #4424

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: dag-master
Choose a base branch
from
Open

Conversation

jackzhhuang
Copy link
Collaborator

@jackzhhuang jackzhhuang commented Mar 11, 2025

Pull request type

Please check the type of change your PR introduces:

  • Bugfix
  • Feature
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • Documentation content changes
  • Other (please describe):

What is the current behavior?

Issue Number: N/A

What is the new behavior?

Other information

Summary by CodeRabbit

  • New Features

    • Added support for tracking red blocks throughout block creation, validation, and mining.
    • Enhanced block template generation to process red block transactions.
    • Introduced a dedicated interface for retrieving detailed block depth information.
    • Updated miner responses to separately include blue and red block hashes.
  • Tests

    • Expanded test coverage for ancestor relationships and for validating block pruning and merge depth computations.
  • Refactor

    • Improved error handling and data encapsulation for better system reliability.
    • Simplified code by removing unused logic from the reset method.

Copy link

coderabbitai bot commented Mar 11, 2025

Walkthrough

This pull request integrates the handling of red blocks across several modules without altering the primary functionality. It adds a new red_blocks field to various structs and updates method signatures to account for the additional data, including changes in block template creation and block depth management. A new public method in the block depth manager centralizes depth information retrieval. Several test functions have been updated to reflect this new data flow, and minor refactoring such as renaming variables and cleaning up commented code has been applied.

Changes

File(s) Change Summary
chain/mock/src/mock_chain.rs
chain/src/chain.rs
Added a red_blocks field to the block production path: in produce_block_for_pruning and in the MineNewDagBlockInfo struct, with no change in overall functionality.
flexidag/src/block_depth/block_depth_info.rs
flexidag/src/blockdag.rs
Introduced a new public get_block_depth_info method; refactored depth retrieval, including renaming of merge depth methods and updating logic to handle block depth info along with insertion into the depth store.
flexidag/src/consensusdb/consensus_block_depth.rs
flexidag/src/consensusdb/db.rs
Refined error handling in block depth info retrieval (distinguishing key-not-found errors) and updated field visibility of db from public to crate-level in FlexiDagStorage.
flexidag/tests/tests.rs Added a new test (test_check_ancestor_of) and updated existing tests to include red_blocks in their validations, ensuring the new red block fields are handled correctly in block and ancestor verifications.
miner/src/create_block_template/mod.rs Updated the Inner struct to require TemplateTxProvider + TxPoolSyncService and renamed variables in the block template creation method; introduced logic to retrieve and process red block transactions alongside blue blocks.
sync/src/block_connector/block_connector_service.rs
sync/src/block_connector/mod.rs
Modified the MineNewDagBlockInfo and MinerResponse structures to include a new red_blocks (and red_blocks_hash) field, and updated variable renaming for clarity in handling both blue and red block information during block connection.
sync/src/block_connector/write_block_chain.rs Removed a large block of commented-out code from the reset method in the WriteBlockChainService, simplifying the code without affecting functionality.

Sequence Diagram(s)

Block Template Creation with Red Block Processing

sequenceDiagram
    participant M as Miner
    participant CT as CreateBlockTemplate
    participant GD as GhostData Provider
    participant TP as TxPoolSyncService

    M->>CT: Request block template
    CT->>GD: Retrieve ghostdata (blue & red blocks)
    GD-->>CT: Return ghostdata (mergeset blues & reds)
    CT->>TP: Process transactions from red blocks
    TP-->>CT: Return processed red block transactions
    CT-->>M: Return block template with blue and red block info
Loading

Block Depth Information Retrieval

sequenceDiagram
    participant Client as Caller
    participant BDM as BlockDepthManagerT
    participant DS as DepthStore

    Client->>BDM: get_block_depth_info(hash)
    BDM->>DS: Retrieve block depth info for hash
    DS-->>BDM: Return Option<BlockDepthInfo> or error
    BDM-->>Client: Return block depth information (or handle error)
Loading

Possibly related PRs

Suggested reviewers

  • nkysg
  • sanlee42

Poem

I'm a rabbit, hopping by,
In a code garden where changes lie.
Red blocks bloom in fields anew,
Blue blocks dance with a vibrant hue.
Lines of code, like carrots, sweet,
Making our blockchain garden complete.
🥕🐇 Happy coding, my friend, so neat!


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5bcc84c and a4b3dff.

📒 Files selected for processing (5)
  • flexidag/src/blockdag.rs (4 hunks)
  • chain/src/chain.rs (1 hunks)
  • flexidag/src/blockdag.rs (2 hunks)
  • chain/src/chain.rs (1 hunks)
  • flexidag/src/blockdag.rs (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (3)
  • chain/src/chain.rs
  • chain/src/chain.rs
  • flexidag/src/blockdag.rs
🔇 Additional comments (1)
flexidag/src/blockdag.rs (1)

59-59: Good enhancement to the struct for red block handling.

Adding red_blocks field to the MineNewDagBlockInfo struct is a logical extension to include information about red blocks, mirroring the existing pattern for blue blocks.

✨ Finishing Touches
  • 📝 Generate Docstrings

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@jackzhhuang jackzhhuang requested review from welbon and simonjiao March 11, 2025 03:43
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (6)
flexidag/tests/tests.rs (2)

1452-1452: Variable name typo in method call.

There's a typo in the variable name (merge_dapth_info instead of merge_depth_info).

-let merge_dapth_info = dag.generate_the_block_depth(pruning_point, &ghostdata, merge_depth)?;
+let merge_depth_info = dag.generate_the_block_depth(pruning_point, &ghostdata, merge_depth)?;

1457-1457: Same typo in variable reference.

The same typo appears here when accessing the field.

-assert_eq!(fork, merge_dapth_info.merge_depth_root);
+assert_eq!(fork, merge_depth_info.merge_depth_root);
miner/src/create_block_template/mod.rs (2)

226-235: Consider using more specific error messages for red blocks.

The error message "uncle block header not found" is the same for both blue and red blocks. Consider using more specific error messages to differentiate between them for easier debugging.

-                op_block_header.ok_or_else(|| format_err!("uncle block header not found."))
+                op_block_header.ok_or_else(|| format_err!("red block header not found."))

289-289: Consider updating log messages to include red blocks information.

The existing log message on line 241 only mentions uncles (blue blocks) but doesn't include information about red blocks being processed. Consider updating the log message to include the count of red blocks and transactions from them.

info!(
-            "[CreateBlockTemplate] previous_header: {:?}, block_gas_limit: {}, max_txns: {}, txn len: {}, uncles len: {}, timestamp: {}",
+            "[CreateBlockTemplate] previous_header: {:?}, block_gas_limit: {}, max_txns: {}, txn len: {}, uncles len: {}, red blocks len: {}, red txns len: {}, timestamp: {}",
            previous_header,
            block_gas_limit,
            max_txns,
            txns.len(),
            uncles.len(),
+           red_blocks.len(),
+           red_transactions.len(),
            now_millis,
        );
flexidag/src/blockdag.rs (2)

26-26: Rename or remove the Ok import.

Importing Ok from anyhow may shadow the standard usage of the Ok enum variant. Consider removing it to avoid confusion, unless it’s explicitly required.

- use anyhow::{bail, ensure, format_err, Ok};
+ use anyhow::{bail, ensure, format_err};

811-846: Review generate_the_block_depth control flow.

• The early return at lines 820-824 neatly handles the merge_depth_root == zero case, but leads to a double-return structure.
• After the insertion into block_depth_info_store, the function logs root and finality, then returns the same data.
Consider storing the BlockDepthInfo in a variable once and returning it at the end to simplify.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 053d376 and 5bcc84c.

📒 Files selected for processing (11)
  • chain/mock/src/mock_chain.rs (1 hunks)
  • chain/src/chain.rs (1 hunks)
  • flexidag/src/block_depth/block_depth_info.rs (3 hunks)
  • flexidag/src/blockdag.rs (6 hunks)
  • flexidag/src/consensusdb/consensus_block_depth.rs (1 hunks)
  • flexidag/src/consensusdb/db.rs (1 hunks)
  • flexidag/tests/tests.rs (8 hunks)
  • miner/src/create_block_template/mod.rs (5 hunks)
  • sync/src/block_connector/block_connector_service.rs (3 hunks)
  • sync/src/block_connector/mod.rs (1 hunks)
  • sync/src/block_connector/write_block_chain.rs (0 hunks)
💤 Files with no reviewable changes (1)
  • sync/src/block_connector/write_block_chain.rs
🔇 Additional comments (26)
flexidag/src/consensusdb/db.rs (1)

26-26: Improved encapsulation by restricting access to the db field

The visibility of the db field has been changed from pub to pub(crate), restricting access to only within the same crate. This is a good practice that better encapsulates the internal implementation details.

chain/src/chain.rs (3)

317-318: LGTM: Added placeholder for new red_blocks field

Adding the placeholder for the new red_blocks field in the destructured assignment properly aligns with the changes in the MineNewDagBlockInfo struct.


320-321: LGTM: Correctly extracting red blocks from ghostdata

The code properly extracts the red blocks from ghostdata.mergeset_reds similar to how blue blocks are handled, ensuring consistency in the implementation.


325-326: LGTM: Adding the red_blocks field to MineNewDagBlockInfo construction

The red_blocks field is correctly integrated into the MineNewDagBlockInfo struct construction, completing the implementation of red blocks tracking alongside blue blocks.

flexidag/src/consensusdb/consensus_block_depth.rs (1)

84-91: Improved error handling for KeyNotFound scenario

The error handling has been refined to distinguish between KeyNotFound errors and other types of errors. This is a good improvement as it prevents unnecessary warning logs for expected scenarios where a key is simply not found.

chain/mock/src/mock_chain.rs (1)

287-288: LGTM: Added red_blocks field support in produce_block_for_pruning

The red_blocks field has been added to the destructured assignment from calc_mergeset_and_tips with a wildcard pattern _ since its value is not needed in this context. This change properly aligns with the updated MineNewDagBlockInfo struct definition.

sync/src/block_connector/mod.rs (2)

59-59: The field name change improves clarity.

Renaming blues_hash to blue_blocks_hash makes the field's purpose more explicit and maintains consistency with the new red_blocks_hash field.


65-65: New field added to support red blocks processing.

The addition of red_blocks_hash field complements the blue blocks tracking and enables the system to process both types of blocks. This field appears to be part of a broader implementation of red block handling across the codebase.

flexidag/src/block_depth/block_depth_info.rs (3)

7-7: Updated import to include BlockDepthInfoReader.

This import change supports the implementation of the new get_block_depth_info method in the BlockDepthManagerT struct.


60-62: Good encapsulation with new public method.

Adding this public method centralizes access to block depth information through the BlockDepthManagerT interface rather than requiring direct access to the depth_store. This improves the API design and makes the code more maintainable.


88-88: Code refactored to use the new method.

The implementation now uses the newly added get_block_depth_info method instead of directly accessing the depth store, which is more consistent and better encapsulated.

flexidag/tests/tests.rs (4)

1028-1028: Updated to handle new red_blocks field.

The test has been updated to accommodate the new red_blocks field from calc_mergeset_and_tips, ensuring tests remain compatible with the implementation changes.


1059-1059: Consistent handling of red_blocks in another test case.

This change maintains consistency with the previous test update, ensuring all test code properly handles the new red_blocks field.


1405-1407: Method call updated to reflect API changes.

The code now uses generate_the_block_depth instead of what appears to have been renamed from check_bounded_merge_depth, and appropriately accesses the merge_depth_root field from the returned BlockDepthInfo struct.


1509-1569: Good addition of test for ancestor relationship verification.

This new test thoroughly validates the check_ancestor_of and check_ancestor_of_chain functionality by creating a DAG with a clear ancestor hierarchy and verifying the expected relationships. This improves test coverage for an important component of the system.

miner/src/create_block_template/mod.rs (4)

27-27: Added HashSet import for transaction deduplication.

This import supports the new functionality to deduplicate transactions from red blocks.


160-160: Expanded trait constraint for better functionality.

Adding the TxPoolSyncService constraint enables the use of methods like add_txns on the transaction provider, supporting the integration of red block transactions.


187-187: Variable naming improved for clarity and new field handled.

Renamed blues_hash to blue_blocks_hash for clarity and added destructuring of the new red_blocks_hash field to support red blocks processing.

Also applies to: 193-193


282-289: Good implementation of transaction deduplication from red blocks.

Using a HashSet to collect and deduplicate transactions from red blocks before adding them to the transaction pool is an efficient approach. This ensures that identical transactions aren't processed multiple times.

sync/src/block_connector/block_connector_service.rs (4)

429-429: Add new field to track red blocks.

This extends the MineNewDagBlockInfo destructuring to include red_blocks. Ensure all internal uses of MineNewDagBlockInfo handle the new field and that it’s correctly populated from upstream.

Would you like a search script to ensure the red_blocks property is fully utilized across the codebase?


469-470: Ensure consistency of ghostdata mergeset references.

Assigning blue_blocks from mergeset_blues and red_blocks from mergeset_reds aligns with the new red/blue block distinction. Confirm that any logic handling these vectors matches this intended purpose (e.g., verifying that red blocks are truly red rather than leftover references).


507-507: Expose red blocks in MinerResponse.

The red_blocks_hash field in MinerResponse helps external consumers handle or track red blocks. Confirm that downstream logic uses this field correctly, for example, in the miner or block verification steps.


466-466:

❓ Verification inconclusive

Check dag.ghostdata error handling.

Be sure that tips is never empty and can reliably provide valid ghost data. If tips is empty, this call may produce undesired behavior or errors.


🏁 Script executed:

#!/bin/bash
# Verify safe usage of dag.ghostdata calls
rg -A 3 "dag\.ghostdata\(.*\)"

Length of output: 5176


Ensure Non-Empty tips for Reliable ghostdata Execution

Please verify that the tips vector passed to dag.ghostdata(&tips)? in sync/src/block_connector/block_connector_service.rs is always non-empty. Our grep output shows that in other parts of the codebase (and notably in tests), ghostdata is consistently called with non-empty identifiers. However, be sure that upstream validations guarantee that an empty tips list never reaches this call, or introduce an explicit check/assertion to handle that case gracefully.

flexidag/src/blockdag.rs (3)

59-59: Add red_blocks to MineNewDagBlockInfo.

This public field tracks “red blocks” for DAG-based consensus. Confirm that any associated logic, such as mergeset computations, properly initiates and updates the red_blocks vector.


769-772: Invoke block depth generation conditionally.

When pruning_point is zero, the code sets it to genesis_id; otherwise, it calls generate_the_block_depth. Ensure skipping depth generation for the genesis scenario is intended, or consider clarifying with an inline comment.


848-880: Validate bounded merge depth on red blocks.

Ensuring each red block is either under the merge depth root or “kosherized” by a blue block prevents protocol violations. This approach of lazily loading kosherizing blues is sound, though note performance overhead if many red blocks exist.

@jackzhhuang jackzhhuang marked this pull request as draft March 11, 2025 07:18
@jackzhhuang jackzhhuang marked this pull request as ready for review March 11, 2025 08:35
Copy link

Benchmark for 0e931dd

Click to view benchmark
Test Base PR %
accumulator_append 828.2±135.47µs 846.4±140.20µs +2.20%
block_apply/block_apply_10 406.8±66.30ms 377.4±11.34ms -7.23%
block_apply/block_apply_1000 41.2±1.26s 41.6±1.54s +0.97%
get_with_proof/db_store 44.6±1.99µs 42.3±1.50µs -5.16%
get_with_proof/mem_store 65.2±4.28µs 38.1±3.26µs -41.56%
put_and_commit/db_store/1 123.6±14.16µs 121.6±12.16µs -1.62%
put_and_commit/db_store/10 1111.2±100.16µs 1108.1±102.57µs -0.28%
put_and_commit/db_store/100 9.9±1.10ms 10.2±1.21ms +3.03%
put_and_commit/db_store/5 568.0±53.20µs 563.5±39.23µs -0.79%
put_and_commit/db_store/50 5.1±0.39ms 5.3±0.34ms +3.92%
put_and_commit/mem_store/1 74.4±8.94µs 70.4±6.77µs -5.38%
put_and_commit/mem_store/10 650.7±55.30µs 702.9±63.42µs +8.02%
put_and_commit/mem_store/100 6.5±0.28ms 6.5±0.60ms 0.00%
put_and_commit/mem_store/5 334.4±35.69µs 336.4±28.02µs +0.60%
put_and_commit/mem_store/50 3.3±0.42ms 3.2±0.20ms -3.03%
query_block/query_block_in(10)_times(100) 8.6±0.61ms 8.1±0.40ms -5.81%
query_block/query_block_in(10)_times(1000) 86.2±3.31ms 82.6±3.00ms -4.18%
query_block/query_block_in(10)_times(10000) 881.8±47.91ms 849.9±26.31ms -3.62%
query_block/query_block_in(1000)_times(100) 1228.2±67.44µs 1197.6±39.91µs -2.49%
query_block/query_block_in(1000)_times(1000) 12.3±0.60ms 12.2±0.52ms -0.81%
query_block/query_block_in(1000)_times(10000) 127.0±6.79ms 118.9±3.46ms -6.38%
storage_transaction 1120.6±438.37µs 1084.9±433.19µs -3.19%
vm/transaction_execution/1 451.9±64.64ms 429.0±22.81ms -5.07%
vm/transaction_execution/10 134.0±6.05ms 132.0±3.75ms -1.49%
vm/transaction_execution/20 132.8±10.66ms 134.1±9.73ms +0.98%
vm/transaction_execution/5 167.7±12.44ms 167.7±6.06ms 0.00%
vm/transaction_execution/50 142.5±8.23ms 146.6±7.37ms +2.88%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant