Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split ebpf_domain_t into domain, transformer, checker #787

Merged
merged 3 commits into from
Nov 9, 2024

Conversation

elazarg
Copy link
Collaborator

@elazarg elazarg commented Nov 9, 2024

The checks and transforms occur on different places and have different dependencies. They may also be moved in the future to different cpp files.

There were mistakes on jump.yaml, but I'm not sure what's the reason for the different results.

Summary by CodeRabbit

Release Notes

  • New Features

    • Introduced new classes ebpf_checker and ebpf_transformer for enhanced handling of eBPF operations.
    • Added methods for managing various eBPF domain functionalities, including memory operations and constraints.
  • Improvements

    • Refined command-line interface for better user experience, including enhanced error messages and flexible argument handling.
    • Enhanced test cases with clearer post-conditions and improved logical consistency.
  • Bug Fixes

    • Updated handling of assertions and checks in the verification process to improve accuracy.
  • Refactor

    • Significant restructuring of the ebpf_domain_t class to improve organization and functionality.

Signed-off-by: Elazar Gershuni <[email protected]>
Copy link

coderabbitai bot commented Nov 9, 2024

Walkthrough

This pull request includes several updates, primarily focusing on restructuring the ebpf_domain_t class into two new classes: ebpf_checker and ebpf_transformer. It also updates various method signatures and introduces new methods to enhance functionality related to eBPF operations. Additionally, there are changes to subproject commit references across multiple directories and improvements to test cases in test-data/jump.yaml to clarify conditions and expected outcomes.

Changes

File Change Summary
ebpf-samples Updated subproject commit reference from 33f01fafa414520ad56ab219db343422bb655b32 to 325cce1bc528a8b70e02ea914d407b4e4f89731d.
external/bpf_conformance Updated subproject commit reference from b544539840db58bcae75b153d3c0368820a15828 to 1ce286105edf3577ef380bd8a498cfa2314d6dfa.
external/libbtf Updated subproject commit reference from 2de28d74884f051a1b129a4535d46841fbf6d6de to 69e02688e89352d49c72707c2819448515d856df.
src/crab/ebpf_domain.cpp Refactored ebpf_domain_t to ebpf_transformer, updated method signatures, and moved set_require_check to ebpf_checker.
src/crab/ebpf_domain.hpp Added ebpf_checker and ebpf_transformer classes, updated ebpf_domain_t with new methods and friend declarations.
src/crab/fwd_analyzer.cpp Updated transform_to_post and run_forward_analyzer to utilize ebpf_transformer.
src/crab_verifier.cpp Modified generate_report and get_ebpf_report for improved invariant checking and reporting.
src/main/check.cpp Changed _get_conformance_group_by_name parameter type from std::string to const std::string&.
test-data/jump.yaml Enhanced test cases with post-conditions and adjusted conditional logic for clarity and correctness.

Possibly related PRs

Suggested reviewers

  • dthaler

🐰 Hopping through the code, with changes so bright,
ebpf transformed, a marvelous sight!
With checks and balances, we refine and renew,
In the world of functions, there's much we can do.
From jumps to conditions, our tests now align,
A leap for the future, in code so divine! 🌟


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 12

🧹 Outside diff range comments (4)
test-data/jump.yaml (1)

Line range hint 763-1359: Architecture suggestion: Consider test case consolidation.

The test cases follow a highly repetitive pattern with similar structures. Consider:

  1. Using a test case generator to reduce duplication
  2. Creating shared fixtures for common pre-conditions and post-conditions
  3. Implementing parameterized tests for different operators
src/crab/ebpf_domain.cpp (3)

Line range hint 952-964: Fix use of undeclared variable 'dom' in forget_packet_pointers

The variable dom used in initialize_packet(dom); is not declared within the scope of the forget_packet_pointers function. This will cause a compilation error due to the use of an undeclared identifier.

Apply this diff to correct the issue:

 void ebpf_transformer::forget_packet_pointers() {
     using namespace crab::dsl_syntax;

     for (const variable_t type_variable : variable_t::get_type_variables()) {
         if (type_inv.has_type(m_inv, type_variable, T_PACKET)) {
             havoc(variable_t::kind_var(data_kind_t::types, type_variable));
             havoc(variable_t::kind_var(data_kind_t::packet_offsets, type_variable));
             havoc(variable_t::kind_var(data_kind_t::svalues, type_variable));
             havoc(variable_t::kind_var(data_kind_t::uvalues, type_variable));
         }
     }
-    initialize_packet(dom);
+    initialize_packet(m_inv);
 }

Line range hint 2519-2527: Handle unsupported operation in operator() for Bin::Op::MOVSX8, MOVSX16, and MOVSX32

In the operator() method for Bin operations, the case for Bin::Op::MOVSX8, MOVSX16, and MOVSX32 results in CRAB_ERROR("Unsupported operation");, which could terminate the program unexpectedly.

Modify the code to handle these cases appropriately or provide a clear error message without terminating the program:

 case Bin::Op::MOVSX8:
 case Bin::Op::MOVSX16:
 case Bin::Op::MOVSX32:
-    CRAB_ERROR("Unsupported operation");
+    // Handle sign-extension operations
+    sign_extend(bin.dst, imm, finite_width, bin.op);
+    break;

Line range hint 2895-2908: Correct invocation of initialize_packet in setup_entry

In the setup_entry method, initialize_packet is called without an instance, which may lead to incorrect behavior if initialize_packet is not a static method.

Apply this diff to ensure correct invocation:

     initialize_packet(inv);
     return inv;
 }
 
-void ebpf_transformer::initialize_packet(ebpf_domain_t& inv) {
+void ebpf_domain_t::initialize_packet(ebpf_domain_t& inv) {
     using namespace crab::dsl_syntax;

     inv -= variable_t::packet_size();
     inv -= variable_t::meta_offset();
📜 Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE

📥 Commits

Reviewing files that changed from the base of the PR and between 41fbd5a and cd560a4.

📒 Files selected for processing (9)
  • ebpf-samples (1 hunks)
  • external/bpf_conformance (1 hunks)
  • external/libbtf (1 hunks)
  • src/crab/ebpf_domain.cpp (54 hunks)
  • src/crab/ebpf_domain.hpp (3 hunks)
  • src/crab/fwd_analyzer.cpp (2 hunks)
  • src/crab_verifier.cpp (3 hunks)
  • src/main/check.cpp (1 hunks)
  • test-data/jump.yaml (21 hunks)
🧰 Additional context used
🪛 cppcheck
src/crab/ebpf_domain.cpp

[style] 1286-1286: The function 'truncate' is never used.

(unusedFunction)

🔇 Additional comments (19)
ebpf-samples (1)

1-1: Verify submodule compatibility with new architecture.

The submodule update appears intentional, but given the significant architectural changes in the main project (splitting ebpf_domain_t), please ensure this specific version of ebpf-samples is compatible with the new domain/transformer/checker structure.

Let's verify the submodule changes:

✅ Verification successful

Let me try a different approach to verify the submodule changes.


Let me try one final verification to check if the submodule is properly initialized and to examine its contents.


Submodule update appears safe and properly structured

The submodule has been successfully initialized and checked out to the new commit 325cce1. The repository structure shows a well-organized collection of eBPF samples across different categories (linux, cilium, falco, etc.) with no obvious breaking changes. Since this is a test samples repository, the update is unlikely to affect the main project's architectural changes, as these samples serve as test cases rather than dependent code.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Analyze the submodule changes between the old and new commits
# to ensure compatibility with the new architecture.

# Test 1: Check commit message and changes
echo "Analyzing commit changes in ebpf-samples submodule..."
git -C ebpf-samples log --oneline --patch 33f01fafa414520ad56ab219db343422bb655b32..325cce1bc528a8b70e02ea914d407b4e4f89731d

# Test 2: Look for any related configuration or interface files that might need updates
echo "Checking for potential interface files..."
cd ebpf-samples && fd -e yaml -e h -e hpp

Length of output: 551


Script:

#!/bin/bash
# Check the current state of the submodule
echo "Current submodule status:"
git submodule status

# Try to get information about the submodule URL and branch
echo -e "\nSubmodule configuration:"
git config --file .gitmodules --get-regexp '^submodule\..*\.(path|url)$'

# List files in the submodule directory
echo -e "\nFiles in ebpf-samples directory:"
ls -la ebpf-samples/

# Check if there's a jump.yaml file mentioned in the PR context
echo -e "\nSearching for jump.yaml:"
find ebpf-samples -name "jump.yaml" -type f

Length of output: 1300


Script:

#!/bin/bash
# Initialize and update the submodule
echo "Initializing and updating submodule:"
git submodule update --init ebpf-samples

echo -e "\nListing contents after initialization:"
ls -la ebpf-samples/

echo -e "\nChecking git status of submodule:"
cd ebpf-samples && git status

Length of output: 1908

external/bpf_conformance (1)

1-1: Verify compatibility with main PR changes.

The subproject commit reference update appears to be part of a coordinated effort to synchronize dependencies. However, since this PR involves significant restructuring of ebpf_domain_t, please ensure this specific version of bpf_conformance is compatible with the new domain/transformer/checker separation.

Let's verify the commit message and changes in the referenced commit:

external/libbtf (1)

1-1: Verify compatibility with the new libbtf version.

The subproject update looks good. However, since this is a dependency update, please ensure that:

  1. The new version is compatible with the refactored ebpf_domain_t structure
  2. All existing functionality depending on libbtf continues to work as expected

Let's check for any major changes in the libbtf subproject:

src/main/check.cpp (1)

41-41: LGTM: Performance optimization for string parameter.

The change from std::string to const std::string& is a good optimization that avoids unnecessary string copies when looking up conformance groups.

test-data/jump.yaml (7)

763-766: LGTM: Consistent post-conditions for pointer comparison tests.

The post-conditions for JNE, JSET, and JNSET tests with pointers are consistent, correctly verifying:

  • The context offset remains unchanged
  • The value range is preserved
  • The type remains ctx

Also applies to: 788-791, 812-815


885-885: LGTM: Consistent error handling for greater-than comparisons.

The error messages for JGT, JGE, JSGT, and JSGE tests correctly handle:

  • Unreachable code detection
  • Invalid type checking for pointer comparisons
  • Proper sequencing of error messages

Also applies to: 897-899, 911-911, 923-925, 993-993, 1006-1008, 1020-1020, 1033-1035


942-948: LGTM: Consistent post-conditions for signed comparisons.

The post-conditions for JSLT, JSLE, and JSGT tests properly verify:

  • Register value preservation
  • Type checking
  • Value range constraints

Also applies to: 952-952, 970-976, 981-981, 998-1001, 1006-1008


1052-1058: LGTM: Consistent 32-bit comparison handling.

The post-conditions and error messages for JEQ32, JNE32, and JSET32 tests correctly:

  • Maintain type consistency
  • Preserve value ranges
  • Handle unreachable code cases

Also applies to: 1063-1063, 1080-1083, 1088-1090, 1107-1110, 1115-1116


1133-1136: LGTM: Consistent error handling for 32-bit bitwise operations.

The error messages for JNSET32, JLT32, and JLE32 tests properly handle:

  • Invalid type checks
  • Unreachable code detection
  • Proper error sequencing

Also applies to: 1141-1142, 1159-1165, 1170-1170, 1187-1190, 1195-1196


1208-1208: LGTM: Consistent 32-bit greater-than comparison handling.

The post-conditions and error messages for JGT32 and JGE32 tests correctly:

  • Maintain type consistency
  • Handle unreachable code
  • Verify invalid type cases

Also applies to: 1213-1216, 1221-1222, 1234-1234, 1239-1242, 1247-1249


1266-1272: LGTM: Consistent 32-bit signed comparison handling.

The post-conditions and error messages for JSLT32, JSLE32, JSGT32, and JSGE32 tests properly:

  • Verify register states
  • Handle type checking
  • Manage unreachable code cases

Also applies to: 1277-1277, 1294-1300, 1305-1305, 1317-1317, 1322-1325, 1330-1332, 1344-1344, 1349-1352, 1357-1359

src/crab/fwd_analyzer.cpp (2)

73-75: Transformation now processes each instruction individually

The code change correctly updates the transform_to_post method to apply the ebpf_transformer to each GuardedInstruction within the basic block. This enhances granularity in how transformations are applied.


135-136: Loop counter initialization encapsulated in ebpf_transformer

The modification to use ebpf_transformer{entry_inv}.initialize_loop_counter(label); appropriately encapsulates loop counter initialization within the transformer class, improving modularity and code organization.

src/crab_verifier.cpp (4)

65-67: Initialization and Setup of pre_bot and checker Are Correct

The introduction of pre_bot and the initialization of crab::ebpf_checker with from_inv correctly reflect the separation of concerns after refactoring. This aligns with the PR objective to split ebpf_domain_t into domain, transformer, and checker.


90-95: Proper Handling of Assertions and Instructions

The nested loops efficiently process each GuardedInstruction and its associated Assertions. The use of checker(assertion); ensures that all preconditions are appropriately checked before applying the transformer. The refactored code enhances readability and maintainability.


95-95: 🧹 Nitpick (assertive)

Remove Unnecessary Semicolon After Closing Brace

There is an unnecessary semicolon after the closing brace of the for-loop at line 95. Removing it will improve code clarity and adhere to standard coding practices.

Apply this diff to remove the unnecessary semicolon:

-     };
+     }

Likely invalid or redundant comment.


180-180: Ensure All References to setup_entry Are Updated

With the change from ebpf_domain_t::setup_entry to crab::ebpf_transformer::setup_entry, please verify that all references to setup_entry throughout the codebase have been updated accordingly to prevent any inconsistencies or runtime errors.

Run the following script to check for any remaining outdated references:

✅ Verification successful

Based on the search results, I can now provide a conclusive response:

No Action Required - Change is Consistent

The change from ebpf_domain_t::setup_entry to crab::ebpf_transformer::setup_entry is correct and consistent with the codebase. The search results show that:

  1. setup_entry is correctly defined in the ebpf_transformer class in src/crab/ebpf_domain.hpp
  2. All usages of setup_entry in the codebase are already using the correct ebpf_transformer::setup_entry form:
    • In src/crab_verifier.cpp: crab::ebpf_transformer::setup_entry(true)
    • In src/crab/ebpf_domain.cpp: ebpf_transformer::setup_entry(false)
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify there are no remaining references to `ebpf_domain_t::setup_entry`

# Test: Search for old references. Expect: No matches found.
rg 'ebpf_domain_t::setup_entry'

Length of output: 31


Script:

#!/bin/bash
# Let's search for any usage of setup_entry to understand the context better
rg "setup_entry" -A 2

# Also search for ebpf_domain_t to understand its usage
rg "ebpf_domain_t" -A 2

# Look for transformer class definition
ast-grep --pattern 'class ebpf_transformer {
  $$$
}'

Length of output: 17068

src/crab/ebpf_domain.hpp (2)

49-49: New method calculate_constant_limits is appropriately introduced

The static method calculate_constant_limits enhances ebpf_domain_t by providing functionality to compute constant limits, which is beneficial for the domain's operations.


60-69: Accessor methods correctly utilize [[nodiscard]] and return types

The new accessor methods for map properties are well-defined, with proper use of [[nodiscard]] to prevent ignored return values and appropriate return types (std::optional<uint32_t> and interval_t).

src/crab/ebpf_domain.hpp Show resolved Hide resolved
src/crab/ebpf_domain.hpp Show resolved Hide resolved
src/crab/ebpf_domain.hpp Show resolved Hide resolved
src/crab/ebpf_domain.hpp Show resolved Hide resolved
src/crab/ebpf_domain.hpp Show resolved Hide resolved
src/crab/ebpf_domain.hpp Show resolved Hide resolved
src/crab/ebpf_domain.cpp Show resolved Hide resolved
src/crab/ebpf_domain.cpp Show resolved Hide resolved
src/crab/ebpf_domain.cpp Show resolved Hide resolved
src/crab/ebpf_domain.cpp Show resolved Hide resolved
Comment on lines +763 to +766
post:
- r1.ctx_offset=0
- r1.svalue=[1, 2147418112]
- r1.type=ctx
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems correct to me, but I'm not 100% sure, and I don't know why this was not already the case. This PR is supposed to only be a refactoring, so I may have introduced a bug.
@Alan-Jowett what do you think?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm merging the PR despite this question. If needed it will be fixed later on.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Turns out I broke assume-assert. Will fix soon.

@coveralls
Copy link

coveralls commented Nov 9, 2024

Coverage Status

coverage: 90.559% (+0.03%) from 90.532%
when pulling 13b216c on split-transformers
into 41fbd5a on main.

Signed-off-by: Elazar Gershuni <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants