Skip to content

Commit

Permalink
refactor: remove legacy code suggestions feature from review tool
Browse files Browse the repository at this point in the history
  • Loading branch information
mrT23 committed Dec 25, 2024
1 parent ad71de8 commit 495c1eb
Show file tree
Hide file tree
Showing 8 changed files with 9 additions and 149 deletions.
22 changes: 5 additions & 17 deletions docs/docs/tools/review.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,14 +93,6 @@ extra_instructions = "..."
!!! example "General options"

<table>
<tr>
<td><b>num_code_suggestions</b></td>
<td>Number of code suggestions provided by the 'review' tool. Default is 0, meaning no code suggestions will be provided by the `review` tool. Note that this is a legacy feature, that will be removed in future releases. Use the `improve` tool instead for code suggestions</td>
</tr>
<tr>
<td><b>inline_code_comments</b></td>
<td>If set to true, the tool will publish the code suggestions as comments on the code diff. Default is false. Note that you need to set `num_code_suggestions`>0 to get code suggestions </td>
</tr>
<tr>
<td><b>persistent_comment</b></td>
<td>If set to true, the review comment will be persistent, meaning that every new review request will edit the previous one. Default is true.</td>
Expand Down Expand Up @@ -189,9 +181,9 @@ If enabled, the `review` tool can approve a PR when a specific comment, `/review
!!! tip "Automation"
When you first install Qodo Merge app, the [default mode](../usage-guide/automations_and_usage.md#github-app-automatic-tools-when-a-new-pr-is-opened) for the `review` tool is:
```
pr_commands = ["/review --pr_reviewer.num_code_suggestions=0", ...]
pr_commands = ["/review", ...]
```
Meaning the `review` tool will run automatically on every PR, without providing code suggestions.
Meaning the `review` tool will run automatically on every PR, without any additional configurations.
Edit this field to enable/disable the tool, or to change the configurations used.

!!! tip "Possible labels from the review tool"
Expand Down Expand Up @@ -249,12 +241,8 @@ If enabled, the `review` tool can approve a PR when a specific comment, `/review
maximal_review_effort = 5
```

[//]: # (!!! tip "Code suggestions")

[//]: # ()
[//]: # ( If you set `num_code_suggestions`>0 , the `review` tool will also provide code suggestions.)
!!! tip "Code suggestions"

[//]: # ( )
[//]: # ( Notice If you are interested **only** in the code suggestions, it is recommended to use the [`improve`]&#40;./improve.md&#41; feature instead, since it is a dedicated only to code suggestions, and usually gives better results.)
The `review` tool previously included a legacy feature for providing code suggestions (controlled by `--pr_reviewer.num_code_suggestion`). This functionality has been deprecated and replaced by the [`improve`](./improve.md) tool, which offers higher quality and more actionable code suggestions.

[//]: # ( Use the `review` tool if you want to get more comprehensive feedback, which includes code suggestions as well.)
2 changes: 1 addition & 1 deletion docs/docs/usage-guide/automations_and_usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -285,7 +285,7 @@ To control which commands will run automatically when a new PR is opened, you ca
[azure_devops_server]
pr_commands = [
"/describe",
"/review --pr_reviewer.num_code_suggestions=0",
"/review",
"/improve",
]
```
2 changes: 0 additions & 2 deletions pr_agent/agent/pr_agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,6 @@ async def handle_request(self, pr_url, request, notify=None) -> bool:
return False
with get_logger().contextualize(command=action, pr_url=pr_url):
get_logger().info("PR-Agent request handler started", analytics=True)
if action == "reflect_and_review":
get_settings().pr_reviewer.ask_and_reflect = True
if action == "answer":
if notify:
notify()
Expand Down
16 changes: 0 additions & 16 deletions pr_agent/algo/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -270,22 +270,6 @@ def convert_to_markdown_v2(output_data: dict,
if gfm_supported:
markdown_text += "</table>\n"

if 'code_feedback' in output_data:
if gfm_supported:
markdown_text += f"\n\n"
markdown_text += f"<details><summary> <strong>Code feedback:</strong></summary>\n\n"
markdown_text += "<hr>"
else:
markdown_text += f"\n\n### Code feedback:\n\n"
for i, value in enumerate(output_data['code_feedback']):
if value is None or value == '' or value == {} or value == []:
continue
markdown_text += parse_code_suggestion(value, i, gfm_supported)+"\n\n"
if markdown_text.endswith('<hr>'):
markdown_text= markdown_text[:-4]
if gfm_supported:
markdown_text += f"</details>"

return markdown_text

def extract_relevant_lines_str(end_line, files, relevant_file, start_line, dedent=False):
Expand Down
3 changes: 0 additions & 3 deletions pr_agent/settings/configuration.toml
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,6 @@ require_can_be_split_review=false
require_security_review=true
require_ticket_analysis_review=true
# general options
num_code_suggestions=0 # legacy mode. use the `improve` command instead
inline_code_comments = false
ask_and_reflect=false
persistent_comment=true
extra_instructions = ""
final_update_message = true
Expand Down
42 changes: 0 additions & 42 deletions pr_agent/settings/pr_reviewer_prompts.toml
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
[pr_review_prompt]
system="""You are PR-Reviewer, a language model designed to review a Git Pull Request (PR).
{%- if num_code_suggestions > 0 %}
Your task is to provide constructive and concise feedback for the PR, and also provide meaningful code suggestions.
{%- else %}
Your task is to provide constructive and concise feedback for the PR.
{%- endif %}
The review should focus on new code added in the PR code diff (lines starting with '+')
Expand Down Expand Up @@ -49,16 +45,6 @@ __new hunk__
{%- endif %}
- When quoting variables or names from the code, use backticks (`) instead of single quote (').
{%- if num_code_suggestions > 0 %}
Code suggestions guidelines:
- Provide up to {{ num_code_suggestions }} code suggestions. Try to provide diverse and insightful suggestions.
- Focus on important suggestions like fixing code problems, issues and bugs. As a second priority, provide suggestions for meaningful code improvements, like performance, vulnerability, modularity, and best practices.
- Avoid making suggestions that have already been implemented in the PR code. For example, if you want to add logs, or change a variable to const, or anything else, make sure it isn't already in the PR code.
- Don't suggest to add docstring, type hints, or comments.
- Suggestions should address the new code added in the PR diff (lines starting with '+')
{%- endif %}
{%- if extra_instructions %}
Expand Down Expand Up @@ -118,25 +104,9 @@ class Review(BaseModel):
{%- if require_can_be_split_review %}
can_be_split: List[SubPR] = Field(min_items=0, max_items=3, description="Can this PR, which contains {{ num_pr_files }} changed files in total, be divided into smaller sub-PRs with distinct tasks that can be reviewed and merged independently, regardless of the order ? Make sure that the sub-PRs are indeed independent, with no code dependencies between them, and that each sub-PR represent a meaningful independent task. Output an empty list if the PR code does not need to be split.")
{%- endif %}
{%- if num_code_suggestions > 0 %}
class CodeSuggestion(BaseModel):
relevant_file: str = Field(description="The full file path of the relevant file")
language: str = Field(description="The programming language of the relevant file")
suggestion: str = Field(description="a concrete suggestion for meaningfully improving the new PR code. Also describe how, specifically, the suggestion can be applied to new PR code. Add tags with importance measure that matches each suggestion ('important' or 'medium'). Do not make suggestions for updating or adding docstrings, renaming PR title and description, or linter like.")
relevant_line: str = Field(description="a single code line taken from the relevant file, to which the suggestion applies. The code line should start with a '+'. Make sure to output the line exactly as it appears in the relevant file")
{%- endif %}
{%- if num_code_suggestions > 0 %}
class PRReview(BaseModel):
review: Review
code_feedback: List[CodeSuggestion]
{%- else %}
class PRReview(BaseModel):
review: Review
{%- endif %}
=====
Expand Down Expand Up @@ -185,18 +155,6 @@ review:
title: ...
- ...
{%- endif %}
{%- if num_code_suggestions > 0 %}
code_feedback:
- relevant_file: |
directory/xxx.py
language: |
python
suggestion: |
xxx [important]
relevant_line: |
xxx
{%- endif %}
```
Answer should be a valid YAML, and nothing else. Each YAML output MUST be after a newline, with proper indent, and block scalar indicator ('|')
Expand Down
62 changes: 0 additions & 62 deletions pr_agent/tools/pr_reviewer.py
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,6 @@ def __init__(self, pr_url: str, is_answer: bool = False, is_auto: bool = False,
"require_estimate_effort_to_review": get_settings().pr_reviewer.require_estimate_effort_to_review,
'require_can_be_split_review': get_settings().pr_reviewer.require_can_be_split_review,
'require_security_review': get_settings().pr_reviewer.require_security_review,
'num_code_suggestions': get_settings().pr_reviewer.num_code_suggestions,
'question_str': question_str,
'answer_str': answer_str,
"extra_instructions": get_settings().pr_reviewer.extra_instructions,
Expand Down Expand Up @@ -168,8 +167,6 @@ async def run(self) -> None:
self.git_provider.publish_comment(pr_review)

self.git_provider.remove_initial_comment()
if get_settings().pr_reviewer.inline_code_comments:
self._publish_inline_code_comments()
else:
get_logger().info("Review output is not published")
get_settings().data = {"artifact": pr_review}
Expand Down Expand Up @@ -235,33 +232,6 @@ def _prepare_pr_review(self) -> str:
key_issues_to_review = data['review'].pop('key_issues_to_review')
data['review']['key_issues_to_review'] = key_issues_to_review

if 'code_feedback' in data:
code_feedback = data['code_feedback']

# Filter out code suggestions that can be submitted as inline comments
if get_settings().pr_reviewer.inline_code_comments:
del data['code_feedback']
else:
for suggestion in code_feedback:
if ('relevant_file' in suggestion) and (not suggestion['relevant_file'].startswith('``')):
suggestion['relevant_file'] = f"``{suggestion['relevant_file']}``"

if 'relevant_line' not in suggestion:
suggestion['relevant_line'] = ''

relevant_line_str = suggestion['relevant_line'].split('\n')[0]

# removing '+'
suggestion['relevant_line'] = relevant_line_str.lstrip('+').strip()

# try to add line numbers link to code suggestions
if hasattr(self.git_provider, 'generate_link_to_relevant_line_number'):
link = self.git_provider.generate_link_to_relevant_line_number(suggestion)
if link:
suggestion['relevant_line'] = f"[{suggestion['relevant_line']}]({link})"
else:
pass

incremental_review_markdown_text = None
# Add incremental review section
if self.incremental.is_incremental:
Expand Down Expand Up @@ -292,38 +262,6 @@ def _prepare_pr_review(self) -> str:

return markdown_text

def _publish_inline_code_comments(self) -> None:
"""
Publishes inline comments on a pull request with code suggestions generated by the AI model.
"""
if get_settings().pr_reviewer.num_code_suggestions == 0:
return

first_key = 'review'
last_key = 'security_concerns'
data = load_yaml(self.prediction.strip(),
keys_fix_yaml=["ticket_compliance_check", "estimated_effort_to_review_[1-5]:", "security_concerns:", "key_issues_to_review:",
"relevant_file:", "relevant_line:", "suggestion:"],
first_key=first_key, last_key=last_key)
comments: List[str] = []
for suggestion in data.get('code_feedback', []):
relevant_file = suggestion.get('relevant_file', '').strip()
relevant_line_in_file = suggestion.get('relevant_line', '').strip()
content = suggestion.get('suggestion', '')
if not relevant_file or not relevant_line_in_file or not content:
get_logger().info("Skipping inline comment with missing file/line/content")
continue

if self.git_provider.is_supported("create_inline_comment"):
comment = self.git_provider.create_inline_comment(content, relevant_file, relevant_line_in_file)
if comment:
comments.append(comment)
else:
self.git_provider.publish_inline_comment(content, relevant_file, relevant_line_in_file, suggestion)

if comments:
self.git_provider.publish_inline_comments(comments)

def _get_user_answers(self) -> Tuple[str, str]:
"""
Retrieves the question and answer strings from the discussion messages related to a pull request.
Expand Down
9 changes: 3 additions & 6 deletions tests/unittest/test_convert_to_markdown.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,13 +47,10 @@ class TestConvertToMarkdown:
def test_simple_dictionary_input(self):
input_data = {'review': {
'estimated_effort_to_review_[1-5]': '1, because the changes are minimal and straightforward, focusing on a single functionality addition.\n',
'relevant_tests': 'No\n', 'possible_issues': 'No\n', 'security_concerns': 'No\n'}, 'code_feedback': [
{'relevant_file': '``pr_agent/git_providers/git_provider.py\n``', 'language': 'python\n',
'suggestion': "Consider raising an exception or logging a warning when 'pr_url' attribute is not found. This can help in debugging issues related to the absence of 'pr_url' in instances where it's expected. [important]\n",
'relevant_line': '[return ""](https://github.com/Codium-ai/pr-agent-pro/pull/102/files#diff-52d45f12b836f77ed1aef86e972e65404634ea4e2a6083fb71a9b0f9bb9e062fR199)'}]}
'relevant_tests': 'No\n', 'possible_issues': 'No\n', 'security_concerns': 'No\n'}}


expected_output = f'{PRReviewHeader.REGULAR.value} 🔍\n\nHere are some key observations to aid the review process:\n\n<table>\n<tr><td>⏱️&nbsp;<strong>Estimated effort to review</strong>: 1 🔵⚪⚪⚪⚪</td></tr>\n<tr><td>🧪&nbsp;<strong>No relevant tests</strong></td></tr>\n<tr><td>&nbsp;<strong>Possible issues</strong>: No\n</td></tr>\n<tr><td>🔒&nbsp;<strong>No security concerns identified</strong></td></tr>\n</table>\n\n\n<details><summary> <strong>Code feedback:</strong></summary>\n\n<hr><table><tr><td>relevant file</td><td>pr_agent/git_providers/git_provider.py\n</td></tr><tr><td>suggestion &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</td><td>\n\n<strong>\n\nConsider raising an exception or logging a warning when \'pr_url\' attribute is not found. This can help in debugging issues related to the absence of \'pr_url\' in instances where it\'s expected. [important]\n\n</strong>\n</td></tr><tr><td>relevant line</td><td><a href=\'https://github.com/Codium-ai/pr-agent-pro/pull/102/files#diff-52d45f12b836f77ed1aef86e972e65404634ea4e2a6083fb71a9b0f9bb9e062fR199\'>return ""</a></td></tr></table><hr>\n\n</details>'
expected_output = f'{PRReviewHeader.REGULAR.value} 🔍\n\nHere are some key observations to aid the review process:\n\n<table>\n<tr><td>⏱️&nbsp;<strong>Estimated effort to review</strong>: 1 🔵⚪⚪⚪⚪</td></tr>\n<tr><td>🧪&nbsp;<strong>No relevant tests</strong></td></tr>\n<tr><td>&nbsp;<strong>Possible issues</strong>: No\n</td></tr>\n<tr><td>🔒&nbsp;<strong>No security concerns identified</strong></td></tr>\n</table>'

assert convert_to_markdown_v2(input_data).strip() == expected_output.strip()

Expand All @@ -67,7 +64,7 @@ def test_empty_dictionary_input(self):
assert convert_to_markdown_v2(input_data).strip() == expected_output.strip()

def test_dictionary_with_empty_dictionaries(self):
input_data = {'review': {}, 'code_feedback': [{}]}
input_data = {'review': {}}

expected_output = ''

Expand Down

0 comments on commit 495c1eb

Please sign in to comment.