Copilot won't summarize a pull request

[ken-swyfft]

Select Topic Area

Product Feedback

Body

One of the reasons why I enabled Copilot for our account is to have it summarize complex pull requests. The ability to do that is specifically advertised as one of its capabilities. But that functionality seems to be completely broken - or at least, completely unintuitive as to how to use or access.

Below is a conversation I had with Copilot. It repeatedly tells me that it can summarize a PR if I just tell it which repo and PR number (or by specifying a URL) - but when I repeatedly attempt to supply that information, it consistently says, "I can't do that."

ken-swyfft 7 minutes ago
What does this PR do?

Copilot 7 minutes ago
I'm sorry for the confusion, but I need the specific pull request number or link to provide accurate information about it. Could you please provide the pull request number or link?

ken-swyfft 6 minutes ago
11819

Copilot 6 minutes ago
Search results for "11819" in "repo:swyfft-insurance/swyfft_web"
The number "11819" corresponds to a test case in the repository swyfft-insurance/swyfft_web. It is used in multiple test files as part of test addresses. Here are some examples:

1. It is used in the file Swyfft.TestUtilities/ConstantsAndExpects/Homeowner/GoodTestAddressesMdv4EAndSFL.cs as part of the test address "11819 Wax Berry Ln, Jacksonville FL 32218". You can find the file here.
2. It also appears in Swyfft.TestUtilities/ConstantsAndExpects/Homeowner/GoodTestAddressesMdv4AdmittedFL.cs as part of the test address "11819 Clubhouse Dr, Lakewood Ranch FL 34202". You can find the file here.
   Please note that "11819" is not a symbol in the repository, but rather a part of test data used in various test cases.
   6 references

ken-swyfft 6 minutes ago
What does PR 11819 do?

Copilot 6 minutes ago
Bing results for "swyfft-insurance/swyfft_web PR 11819 details"
I'm sorry, but I couldn't find specific information about PR 11819 in the swyfft-insurance/swyfft_web repository. It might be helpful to directly check the pull request on GitHub for details, including the description, changes, and any associated discussion.

ken-swyfft 5 minutes ago
What does this PR do? https://github.com/swyfft-insurance/swyfft_web/pull/11819/files

Copilot 5 minutes ago
Bing results for "swyfft-insurance/swyfft_web PR 11819 details"
I'm sorry, but I wasn't able to find specific details about PR 11819 in the swyfft-insurance/swyfft_web repository from the web search results. You may want to directly check the pull request on GitHub for its details, changes, and any associated discussion.

ken-swyfft 4 minutes ago
How do I ask you to summarize a specific PR?

Copilot4 minutes ago
To ask me to summarize a specific Pull Request (PR), you can provide the URL of the PR or the PR number and the repository it belongs to. Here are examples of how you can ask:
• "Can you summarize this PR? [insert PR URL here]"
• "What does PR #123 do in the owner/repo repository?"
Please replace the placeholders with actual data. For example, if you want to summarize PR #11819 from the swyfft-insurance/swyfft_web repository, you can ask, "What does PR #11819 do in the swyfft-insurance/swyfft_web repository?"

ken-swyfft 4 minutes ago
Can you summarize this PR? https://github.com/swyfft-insurance/swyfft_web/pull/11819/files

Copilot 4 minutes ago
I'm sorry for the misunderstanding, but as a GitHub Copilot chat assistant, I don't have the capability to directly summarize a specific PR from a URL. I suggest checking the PR directly on GitHub, looking at the description, changes, and any associated discussions for information about the PR.

ken-swyfft 2 minutes ago
Wait, you just said that if I want you to summarize a given PR, just give you the URL. But when I do, you say that you can't.

Copilot 2 minutes ago
I apologize for the confusion. Unfortunately, I currently don't have the capability to directly summarize a specific PR from a URL. I can assist with general GitHub-related questions, help understand code, and provide information about programming concepts. For details about a specific PR, it's best to check the PR directly on GitHub, including the description, changes, and any associated discussion. I'm here to help with any other questions you might have.

ken-swyfft 2 minutes ago
How do I ask you to summarize a given PR?

Copilot 2 minutes ago
To ask for a summary of a specific Pull Request (PR), you can simply provide the URL or the PR number along with the repository it belongs to. Here are some examples:

1. "Can you summarize this PR? [insert PR URL]"
2. "What does PR #[insert PR number] do in the [insert repository name] repository?"
   Just replace the placeholders with the actual information. For instance, if you want to know about PR #123 in the repository swyfft-insurance/swyfft_web, you can ask:
   "What does PR #123 do in the swyfft-insurance/swyfft_web repository?"

ken-swyfft 1 minute ago
What does PR 11819 do in the swyfft-insurance/swyfft_web repository?

Copilot 1 minute ago
Bing results for "swyfft-insurance/swyfft_web PR 11819 details"
I'm sorry, but I couldn't find specific information about PR 11819 in the swyfft-insurance/swyfft_web repository. It might be helpful to directly check the pull request on GitHub for details, including the description, changes, and any associated discussion.

[Akash1134]

Hi @ken-swyfft ,

Did you tried creating a pull request summary with Copilot? It will generate accurate results since it will have the PR/file context.

See the below documentation:

• Creating PR Summaries
• About PR Summaries

Thanks!

[ken-swyfft]

That does work, and is valuable. But:

(1) If I ask Copilot how to get a summary of a PR, shouldn't it tell me that, rather than telling me to do stuff that doesn't work?
(2) I might want a summary that I don't want memorialized in the PR proper - just for my own edification, for instance. It would be nice to have it available in that Chat interface rather than only as a more-or-less permanent comment or description in the PR.

[Akash1134]

For both of your questions above you need to provide the specific knowledge base of the repo where the PR exists. Here's more details about it: https://docs.github.com/en/enterprise-cloud@latest/copilot/github-copilot-enterprise/copilot-chat-in-github/using-github-copilot-chat-in-githubcom#asking-a-question-about-a-knowledge-base

Create a knowledge base of your repo and then you can ask questions related to specific files since Copilot will only respond keeping that context in mind.

Hope that helps.