Skip to content

Clarify review app security guidance#350

Open
justin808 wants to merge 1 commit into
mainfrom
jg-codex/review-app-security-docs
Open

Clarify review app security guidance#350
justin808 wants to merge 1 commit into
mainfrom
jg-codex/review-app-security-docs

Conversation

@justin808

@justin808 justin808 commented Jun 2, 2026

Copy link
Copy Markdown
Member

Summary

  • document that generated review-app workflows skip fork PR heads in public repositories
  • clarify that trusted +review-app-deploy comments still require a branch in the base repository
  • add review-app secret guidance for disposable credentials, scoped Control Plane tokens, SSH deploy keys, and cpln://secret runtime exposure

Verification

  • git diff --check
  • script/check_cpln_links
  • script/check_command_docs

Note

Low Risk
Documentation-only; no runtime, workflow, or secret configuration changes.

Overview
Documentation-only updates to docs/ci-automation.md and docs/secrets-and-env-values.md that spell out review-app security for public and open-source repos—no workflow or generator changes.

ci-automation.md now states upfront that review deploys only run for base-repo branches (fork PRs get help but not deploys because builds use repo secrets), and that maintainers must land fork work on a trusted branch for a review app. It expands CPLN_TOKEN_STAGING guidance (prefer a review/staging token that cannot touch production), DOCKER_BUILD_SSH_KEY cautions (minimal read-only deploy keys), and a new Review App Security For Public Repositories section covering trusted commenters, fork skips, disposable credentials, scoped tokens, separate data stores, rotation, and cleanup. Workflow descriptions and the demo rollout checklist add matching bullets (including verifying staging tokens cannot reach production).

secrets-and-env-values.md adds that review apps execute PR code, so cpln://secret/... does not protect values once mounted, and recommends dummy/disposable/revocable review credentials plus separate secret dictionaries from production or long-lived staging when orgs are shared.

Reviewed by Cursor Bugbot for commit 09f553d. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • Documentation
    • Updated CI/CD automation documentation with enhanced guidance for review-app deployments on public repositories, including security considerations and fork-PR handling.
    • Added comprehensive secrets management best practices for review apps, emphasizing the use of protected secret references and scoped credentials to prevent production access.

@coderabbitai

coderabbitai Bot commented Jun 2, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 904e5e4b-7752-4ad3-bc6b-c2e431dff80c

📥 Commits

Reviewing files that changed from the base of the PR and between 037b74f and 09f553d.

📒 Files selected for processing (2)
  • docs/ci-automation.md
  • docs/secrets-and-env-values.md

Walkthrough

This PR updates two documentation files to enhance guidance around review-app deployments in public repositories. It establishes security constraints for fork PRs, revises secret token requirements, adds a comprehensive review-app security section, and clarifies safe secret storage using Control Plane references and scoped credentials.

Changes

Review App Security for Public Repositories

Layer / File(s) Summary
Security Model and Token Requirements
docs/ci-automation.md
Public repositories restrict review-app deployments to base-repo branches; fork PRs skip deploy workflows but may receive help comments. CPLN_TOKEN_STAGING is documented as a dedicated review/staging token without production access. A new "Review App Security For Public Repositories" section details default behaviors, execution risks, credential disposal requirements, and secret-mounting limits.
Secure Implementation and Credential Handling
docs/ci-automation.md, docs/secrets-and-env-values.md
DOCKER_BUILD_SSH_KEY and tokens must be least-privilege and revocable. Generated workflows skip fork PR heads by design; production promotion remains manual with separate secrets. Review app secrets must be disposable. cpln://secret/... references keep sensitive values protected in Control Plane while allowing safe application mounting; review and staging secret dictionaries must be scoped separately.

Sequence Diagram(s)

(Diagrams are embedded in the hidden review stack artifact above.)

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • shakacode/control-plane-flow#285: Updates review-app command trigger semantics and fork-handling behavior in similar documentation sections.
  • shakacode/control-plane-flow#326: Overlapping updates to docs/ci-automation.md covering review-app and staging secret token requirements, with additional bootstrap instructions.
  • shakacode/control-plane-flow#323: Clarifies PR-open review-app help wrapper behavior for forks and clones, aligning with this PR's fork/public-repo deployment constraints.

Suggested labels

documentation, review needed

Poem

🐰 A guardian bunny hops through secure review lands,
Forks stay in comments, base branches in hands,
Disposable secrets, no prod-danger touch,
Control Plane holds tight—safety means much! 🔐

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Clarify review app security guidance' directly aligns with the main changes: adding security documentation for review apps, including fork PR handling, secret scoping, and credential best practices.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch jg-codex/review-app-security-docs
⚔️ Resolve merge conflicts
  • Resolve merge conflict in branch jg-codex/review-app-security-docs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot added documentation Improvements or additions to documentation review needed labels Jun 2, 2026
@greptile-apps

greptile-apps Bot commented Jun 2, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds security guidance for review apps in public repositories, explaining why fork PR heads are skipped during deploy, clarifying that trusted +review-app-deploy comments cannot bypass that restriction, and advising that secrets mounted into review-app workloads are readable by PR code regardless of cpln://secret indirection.

  • docs/ci-automation.md: Adds a new "Review App Security For Public Repositories" section with disposable-credential requirements, SSH deploy key scoping guidance, and a clarified description of fork-PR behavior in both the generated workflow summary and the rollout checklist.
  • docs/secrets-and-env-values.md: Adds two paragraphs reinforcing that cpln://secret indirection protects values only at rest, and advising against pointing review-app templates at shared production or persistent staging secret dictionaries.

Confidence Score: 4/5

Documentation-only changes with accurate security guidance; safe to merge.

All changes are documentation. The security claims are accurate and internally consistent, with one bullet point in the new security section using an "unless" clause that reads differently from the clearer prose elsewhere in the file — a reader could interpret it as an automatic workflow check rather than a manual workaround.

The one bullet at line 209 of docs/ci-automation.md that uses "unless the branch exists in the base repository" would benefit from a small reword to match the unambiguous phrasing used later in the same section.

Important Files Changed

Filename Overview
docs/ci-automation.md Adds fork-PR skip explanation at the top, expands secret-scoping guidance for CPLN_TOKEN_STAGING and DOCKER_BUILD_SSH_KEY, and adds a new "Review App Security For Public Repositories" section; one bullet in that section uses ambiguous "unless the branch exists in the base repository" phrasing that differs slightly from the clearer prose elsewhere in the same file.
docs/secrets-and-env-values.md Adds two new paragraphs clarifying that cpln://secret indirection does not protect values after workload mounting, and advising against shared production secret dictionaries for review-app templates; the added content is accurate and consistent with the ci-automation.md guidance.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[PR opened or push] --> B{Is PR head from\nbase repository?}
    B -- Yes\n(same-repo branch) --> C[Review app deploy eligible]
    B -- No\n(fork head) --> D[Skip deploy\nPost help comment only]
    C --> E{Trigger: push or\n+review-app-deploy comment?}
    E -- Push --> F{Is review app\nalready active?}
    F -- Yes --> G[Auto-redeploy]
    F -- No --> H[Skip until first comment]
    E -- Comment --> I{Is commenter\nOWNER / MEMBER /\nCOLLABORATOR?}
    I -- No --> J[Ignore comment]
    I -- Yes --> K[Create or update review app]
    K --> L{Are secrets disposable?\nCPLN_TOKEN_STAGING\nnot prod-scoped?}
    L -- Yes --> M[Safe review-app deploy]
    L -- No --> N[⚠️ Credential exposure risk\nRotate and scope secrets]
    D --> O[Maintainer workaround:\nmove change to base-repo branch\nthen open new PR]
    O --> C
Loading

Reviews (1): Last reviewed commit: "docs: clarify review app security" | Re-trigger Greptile

Comment thread docs/ci-automation.md
- same-repository pull requests can create or update review apps;
- fork pull requests are skipped for review-app deploys;
- `+review-app-deploy` and `+review-app-delete` are accepted only from trusted commenters;
- trusted comments still cannot deploy fork heads unless the branch exists in the base repository;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Ambiguous "unless" clause may mislead readers

The bullet reads "trusted comments still cannot deploy fork heads unless the branch exists in the base repository," which can be parsed as an automatic condition check — as if the workflow detects the branch name in the base repo and switches behavior. The clearer phrasing used a few lines later in the same file ("move the change to a branch in the base repository if it needs a review app") makes it explicit that this is a manual workaround. Aligning the bullet with the prose avoids the implication that the workflow itself checks whether the branch exists in both repos.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 09f553d9a9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/ci-automation.md
Comment on lines +187 to +188
- for open-source repositories, use a dedicated review-app org or a tightly scoped staging token so review-app builds
cannot reveal production or long-lived staging secrets

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Don't advertise a separate review org without workflow support

When a public repo follows this recommendation and provisions a dedicated review-app org/token, the generated workflows do not have separate review-app org settings: review deploys use vars.CPLN_ORG_STAGING/secrets.CPLN_TOKEN_STAGING (.github/workflows/cpflow-deploy-review-app.yml lines 29 and 201-202), and staging deploy/promotion require the same staging token/org (.github/workflows/cpflow-deploy-staging.yml lines 105-106; .github/workflows/cpflow-promote-staging-to-production.yml lines 72-82). A review-only token would therefore break staging/promotion or, if the staging token is kept, review apps still deploy into the staging org, so this doc should either say the generated flow shares the staging org or call out the required workflow customization.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation review needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant