feat: add review mode for automated PR code reviews (#374)

* feat: add review mode for PR code reviews

- Add 'review' as a new execution mode in action.yml
- Use default GitHub Action token (ACTIONS_TOKEN) for review mode
- Create review mode implementation with GitHub MCP tools included by default
- Move review-specific prompt to review mode's generatePrompt method
- Add comprehensive review workflow instructions for inline comments
- Fix type safety with proper mode validation
- Keep agent mode's simple inline prompt handling

* docs: add review mode example workflow

* update sample workflow

* fix: update review mode example to use @beta tag

* fix: enable automatic triggering for review mode on PR events

* fix: export allowed tools environment variables in review mode

The GitHub MCP tools were not being properly allowed because review mode
wasn't exporting the ALLOWED_TOOLS environment variable like agent mode does.
This caused all GitHub MCP tool calls to be blocked with permission errors.

* feat: add review mode workflow for testing

* fix: use INPUT_ prefix for allowed/disallowed tools environment variables

The base action expects INPUT_ALLOWED_TOOLS and INPUT_DISALLOWED_TOOLS
(following GitHub Actions input naming convention) but we were exporting
them without the INPUT_ prefix. This was causing the tools to not be
properly allowed in the base action.

* fix: add explicit review tool names and additional workflow permissions

- Add explicit tool names in case wildcards aren't working properly
- Add statuses and checks write permissions to workflow
- Include both github and github_comment MCP server tools

* refactor: consolidate review workflows and use review mode

- Update claude-review.yml to use review mode instead of direct_prompt
- Use km-anthropic fork action
- Remove duplicate claude-review-mode.yml workflow
- Add synchronize event to review PR updates
- Update permissions for review mode (remove id-token, add pull-requests/issues write)

* feat: enhance review mode to provide detailed tracking comment summary

- Update review mode prompt to explicitly request detailed summaries
- Include issue counts, key findings, and recommendations in tracking comment
- Ensure users can see complete review overview without checking each inline comment

* Revert "refactor: consolidate review workflows and use review mode"

This reverts commit 54ca948599.

* fix: address PR review feedback for review mode

- Make generatePrompt required in Mode interface
- Implement generatePrompt in all modes (tag, agent, review)
- Remove unnecessary git/branch operations from review mode
- Restrict review mode triggers to specific PR actions
- Fix type safety issues by removing any types
- Update tests to support new Mode interface

* test: update mode registry tests to include review mode

* chore: run prettier formatting

* fix: make mode parameter required in generatePrompt function

Remove optional mode parameter since the function throws an error when mode is not provided. This makes the type signature consistent with the actual behavior.

* fix: remove last any type and update README with review mode

- Remove any type cast in review mode by using isPullRequestEvent type guard
- Add review mode documentation to README execution modes section
- Update mode parameter description in README configuration table

* mandatory bun format

* fix: improve review mode GitHub suggestion format instructions

- Add clear guidance on GitHub's suggestion block format
- Emphasize that suggestions must only replace the specific commented lines
- Add examples of correct vs incorrect suggestion formatting
- Clarify when to use multi-line comments with startLine and line parameters
- Guide on handling complex changes that require multiple modifications

This should resolve issues where suggestions aren't directly committable.

* Add missing MCP tools for experimental-review mode based on test requirements

* chore: format code

* docs: add experimental-review mode documentation with clear warnings

* docs: remove emojis from experimental-review mode documentation

* docs: clarify experimental-review mode triggers - depends on workflow configuration

* minor format update

* test: fix registry tests for experimental-review mode name change

* refactor: clean up review mode implementation based on feedback

- Remove unused parameters from generatePrompt in agent and review modes
- Keep Claude comment requirement for review mode (tracking comment)
- Add overridePrompt support to review mode
- Remove non-existent MCP tools from review mode allowed list
- Fix unused import in agent mode

These changes address all review feedback while maintaining clean code
and proper functionality.

* fix: remove redundant update_claude_comment from review mode allowed tools

The github_comment server is always included automatically, so we don't
need to explicitly list mcp__github_comment__update_claude_comment in
the allowed tools.

* feat: review mode now uses review body instead of tracking comment

- Remove tracking comment creation from review mode
- Update prompt to instruct Claude to write comprehensive review in body
- Remove comment ID requirement for review mode
- The review submission body now serves as the main review content

This makes review mode cleaner with one less comment on the PR. The
review body contains all the information that would have been in the
tracking comment.

* add back id-token: write for example

* Add PR number for context + make it mandatory to have a PR associated

* add `mcp__github__add_issue_comment` tool

* rename token

* bun format

---------

Co-authored-by: km-anthropic <km-anthropic@users.noreply.github.com>
This commit is contained in:
km-anthropic
2025-08-01 15:04:23 -07:00
committed by GitHub
parent 0e5fbc0d44
commit 56179f5fc9
12 changed files with 604 additions and 91 deletions

View File

@@ -168,8 +168,8 @@ jobs:
## Inputs ## Inputs
| Input | Description | Required | Default | | Input | Description | Required | Default |
| ------------------------------ | ---------------------------------------------------------------------------------------------------------------------- | -------- | --------- | | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------- | -------- | --------- |
| `mode` | Execution mode: 'tag' (default - triggered by mentions/assignments), 'agent' (for automation with no trigger checking) | No | `tag` | | `mode` | Execution mode: 'tag' (default - triggered by mentions/assignments), 'agent' (for automation), 'experimental-review' (for PR reviews) | No | `tag` |
| `anthropic_api_key` | Anthropic API key (required for direct API, not needed for Bedrock/Vertex) | No\* | - | | `anthropic_api_key` | Anthropic API key (required for direct API, not needed for Bedrock/Vertex) | No\* | - |
| `claude_code_oauth_token` | Claude Code OAuth token (alternative to anthropic_api_key) | No\* | - | | `claude_code_oauth_token` | Claude Code OAuth token (alternative to anthropic_api_key) | No\* | - |
| `direct_prompt` | Direct prompt for Claude to execute automatically without needing a trigger (for automated workflows) | No | - | | `direct_prompt` | Direct prompt for Claude to execute automatically without needing a trigger (for automated workflows) | No | - |
@@ -204,7 +204,7 @@ jobs:
## Execution Modes ## Execution Modes
The action supports two execution modes, each optimized for different use cases: The action supports three execution modes, each optimized for different use cases:
### Tag Mode (Default) ### Tag Mode (Default)
@@ -238,7 +238,28 @@ For automation and scheduled tasks without trigger checking.
Check for outdated dependencies and create an issue if any are found. Check for outdated dependencies and create an issue if any are found.
``` ```
See [`examples/claude-modes.yml`](./examples/claude-modes.yml) for complete examples of each mode. ### Experimental Review Mode
> **EXPERIMENTAL**: This mode is under active development and may change significantly. Use with caution in production workflows.
Specialized mode for automated PR code reviews using GitHub's review API.
- **Triggers**: Automatically on PR events (opened, synchronize, reopened) when configured in workflow
- **Features**: Creates inline review comments with suggestions, batches feedback into a single review
- **Use case**: Automated code reviews, security scanning, best practices enforcement
```yaml
- uses: anthropics/claude-code-action@beta
with:
mode: experimental-review
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
custom_instructions: |
Focus on security vulnerabilities, performance issues, and code quality.
```
Review mode automatically includes GitHub MCP tools for creating pending reviews and inline comments. See [`examples/claude-experimental-review-mode.yml`](./examples/claude-experimental-review-mode.yml) for a complete example.
See [`examples/claude-modes.yml`](./examples/claude-modes.yml) for complete examples of available modes.
### Using Custom MCP Configuration ### Using Custom MCP Configuration

View File

@@ -26,7 +26,7 @@ inputs:
# Mode configuration # Mode configuration
mode: mode:
description: "Execution mode for the action. Valid modes: 'tag' (default - triggered by mentions/assignments), 'agent' (for automation with no trigger checking)" description: "Execution mode for the action. Valid modes: 'tag' (default - triggered by mentions/assignments), 'agent' (for automation with no trigger checking), 'experimental-review' (experimental mode for code reviews with inline comments and suggestions)"
required: false required: false
default: "tag" default: "tag"
@@ -158,7 +158,7 @@ runs:
OVERRIDE_GITHUB_TOKEN: ${{ inputs.github_token }} OVERRIDE_GITHUB_TOKEN: ${{ inputs.github_token }}
GITHUB_RUN_ID: ${{ github.run_id }} GITHUB_RUN_ID: ${{ github.run_id }}
USE_STICKY_COMMENT: ${{ inputs.use_sticky_comment }} USE_STICKY_COMMENT: ${{ inputs.use_sticky_comment }}
ACTIONS_TOKEN: ${{ github.token }} DEFAULT_WORKFLOW_TOKEN: ${{ github.token }}
ADDITIONAL_PERMISSIONS: ${{ inputs.additional_permissions }} ADDITIONAL_PERMISSIONS: ${{ inputs.additional_permissions }}
USE_COMMIT_SIGNING: ${{ inputs.use_commit_signing }} USE_COMMIT_SIGNING: ${{ inputs.use_commit_signing }}

View File

@@ -0,0 +1,45 @@
name: Claude Experimental Review Mode
on:
pull_request:
types: [opened, synchronize]
issue_comment:
types: [created]
jobs:
code-review:
# Run on PR events, or when someone comments "@claude review" on a PR
if: |
github.event_name == 'pull_request' ||
(github.event_name == 'issue_comment' &&
github.event.issue.pull_request &&
contains(github.event.comment.body, '@claude review'))
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
issues: write
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0 # Full history for better diff analysis
- name: Code Review with Claude
uses: anthropics/claude-code-action@beta
with:
mode: experimental-review
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
# github_token not needed - uses default GITHUB_TOKEN for GitHub operations
timeout_minutes: "30"
custom_instructions: |
Focus on:
- Code quality and maintainability
- Security vulnerabilities
- Performance issues
- Best practices and design patterns
- Test coverage gaps
Be constructive and provide specific suggestions for improvements.
Use GitHub's suggestion format when proposing code changes.

View File

@@ -530,6 +530,7 @@ export function generatePrompt(
context: PreparedContext, context: PreparedContext,
githubData: FetchDataResult, githubData: FetchDataResult,
useCommitSigning: boolean, useCommitSigning: boolean,
mode: Mode,
): string { ): string {
if (context.overridePrompt) { if (context.overridePrompt) {
return substitutePromptVariables( return substitutePromptVariables(
@@ -539,6 +540,19 @@ export function generatePrompt(
); );
} }
// Use the mode's prompt generator
return mode.generatePrompt(context, githubData, useCommitSigning);
}
/**
* Generates the default prompt for tag mode
* @internal
*/
export function generateDefaultPrompt(
context: PreparedContext,
githubData: FetchDataResult,
useCommitSigning: boolean = false,
): string {
const { const {
contextData, contextData,
comments, comments,
@@ -810,7 +824,9 @@ export async function createPrompt(
let claudeCommentId: string = ""; let claudeCommentId: string = "";
if (mode.name === "tag") { if (mode.name === "tag") {
if (!modeContext.commentId) { if (!modeContext.commentId) {
throw new Error("Tag mode requires a comment ID for prompt generation"); throw new Error(
`${mode.name} mode requires a comment ID for prompt generation`,
);
} }
claudeCommentId = modeContext.commentId.toString(); claudeCommentId = modeContext.commentId.toString();
} }
@@ -831,6 +847,7 @@ export async function createPrompt(
preparedContext, preparedContext,
githubData, githubData,
context.inputs.useCommitSigning, context.inputs.useCommitSigning,
mode,
); );
// Log the final prompt to console // Log the final prompt to console

View File

@@ -10,13 +10,37 @@ import { setupGitHubToken } from "../github/token";
import { checkWritePermissions } from "../github/validation/permissions"; import { checkWritePermissions } from "../github/validation/permissions";
import { createOctokit } from "../github/api/client"; import { createOctokit } from "../github/api/client";
import { parseGitHubContext, isEntityContext } from "../github/context"; import { parseGitHubContext, isEntityContext } from "../github/context";
import { getMode } from "../modes/registry"; import { getMode, isValidMode, DEFAULT_MODE } from "../modes/registry";
import type { ModeName } from "../modes/types";
import { prepare } from "../prepare"; import { prepare } from "../prepare";
async function run() { async function run() {
try { try {
// Step 1: Setup GitHub token // Step 1: Get mode first to determine authentication method
const githubToken = await setupGitHubToken(); const modeInput = process.env.MODE || DEFAULT_MODE;
// Validate mode input
if (!isValidMode(modeInput)) {
throw new Error(`Invalid mode: ${modeInput}`);
}
const validatedMode: ModeName = modeInput;
// Step 2: Setup GitHub token based on mode
let githubToken: string;
if (validatedMode === "experimental-review") {
// For experimental-review mode, use the default GitHub Action token
githubToken = process.env.DEFAULT_WORKFLOW_TOKEN || "";
if (!githubToken) {
throw new Error(
"DEFAULT_WORKFLOW_TOKEN not found for experimental-review mode",
);
}
console.log("Using default GitHub Action token for review mode");
core.setOutput("GITHUB_TOKEN", githubToken);
} else {
// For other modes, use the existing token exchange
githubToken = await setupGitHubToken();
}
const octokit = createOctokit(githubToken); const octokit = createOctokit(githubToken);
// Step 2: Parse GitHub context (once for all operations) // Step 2: Parse GitHub context (once for all operations)
@@ -36,7 +60,7 @@ async function run() {
} }
// Step 4: Get mode and check trigger conditions // Step 4: Get mode and check trigger conditions
const mode = getMode(context.inputs.mode, context); const mode = getMode(validatedMode, context);
const containsTrigger = mode.shouldTrigger(context); const containsTrigger = mode.shouldTrigger(context);
// Set output for action.yml to check // Set output for action.yml to check

View File

@@ -1,7 +1,7 @@
import * as core from "@actions/core"; import * as core from "@actions/core";
import { mkdir, writeFile } from "fs/promises";
import type { Mode, ModeOptions, ModeResult } from "../types"; import type { Mode, ModeOptions, ModeResult } from "../types";
import { isAutomationContext } from "../../github/context"; import { isAutomationContext } from "../../github/context";
import type { PreparedContext } from "../../create-prompt/types";
/** /**
* Agent mode implementation. * Agent mode implementation.
@@ -42,24 +42,7 @@ export const agentMode: Mode = {
async prepare({ context }: ModeOptions): Promise<ModeResult> { async prepare({ context }: ModeOptions): Promise<ModeResult> {
// Agent mode handles automation events (workflow_dispatch, schedule) only // Agent mode handles automation events (workflow_dispatch, schedule) only
// Create prompt directory // Agent mode doesn't need to create prompt files here - handled by createPrompt
await mkdir(`${process.env.RUNNER_TEMP}/claude-prompts`, {
recursive: true,
});
// Write the prompt file - the base action requires a prompt_file parameter,
// so we must create this file even though agent mode typically uses
// override_prompt or direct_prompt. If neither is provided, we write
// a minimal prompt with just the repository information.
const promptContent =
context.inputs.overridePrompt ||
context.inputs.directPrompt ||
`Repository: ${context.repository.owner}/${context.repository.repo}`;
await writeFile(
`${process.env.RUNNER_TEMP}/claude-prompts/claude-prompt.txt`,
promptContent,
);
// Export tool environment variables for agent mode // Export tool environment variables for agent mode
const baseTools = [ const baseTools = [
@@ -80,8 +63,9 @@ export const agentMode: Mode = {
...context.inputs.disallowedTools, ...context.inputs.disallowedTools,
]; ];
core.exportVariable("ALLOWED_TOOLS", allowedTools.join(",")); // Export as INPUT_ prefixed variables for the base action
core.exportVariable("DISALLOWED_TOOLS", disallowedTools.join(",")); core.exportVariable("INPUT_ALLOWED_TOOLS", allowedTools.join(","));
core.exportVariable("INPUT_DISALLOWED_TOOLS", disallowedTools.join(","));
// Agent mode uses a minimal MCP configuration // Agent mode uses a minimal MCP configuration
// We don't need comment servers or PR-specific tools for automation // We don't need comment servers or PR-specific tools for automation
@@ -114,4 +98,18 @@ export const agentMode: Mode = {
mcpConfig: JSON.stringify(mcpConfig), mcpConfig: JSON.stringify(mcpConfig),
}; };
}, },
generatePrompt(context: PreparedContext): string {
// Agent mode uses override or direct prompt, no GitHub data needed
if (context.overridePrompt) {
return context.overridePrompt;
}
if (context.directPrompt) {
return context.directPrompt;
}
// Minimal fallback - repository is a string in PreparedContext
return `Repository: ${context.repository}`;
},
}; };

View File

@@ -13,11 +13,12 @@
import type { Mode, ModeName } from "./types"; import type { Mode, ModeName } from "./types";
import { tagMode } from "./tag"; import { tagMode } from "./tag";
import { agentMode } from "./agent"; import { agentMode } from "./agent";
import { reviewMode } from "./review";
import type { GitHubContext } from "../github/context"; import type { GitHubContext } from "../github/context";
import { isAutomationContext } from "../github/context"; import { isAutomationContext } from "../github/context";
export const DEFAULT_MODE = "tag" as const; export const DEFAULT_MODE = "tag" as const;
export const VALID_MODES = ["tag", "agent"] as const; export const VALID_MODES = ["tag", "agent", "experimental-review"] as const;
/** /**
* All available modes. * All available modes.
@@ -26,6 +27,7 @@ export const VALID_MODES = ["tag", "agent"] as const;
const modes = { const modes = {
tag: tagMode, tag: tagMode,
agent: agentMode, agent: agentMode,
"experimental-review": reviewMode,
} as const satisfies Record<ModeName, Mode>; } as const satisfies Record<ModeName, Mode>;
/** /**

352
src/modes/review/index.ts Normal file
View File

@@ -0,0 +1,352 @@
import * as core from "@actions/core";
import type { Mode, ModeOptions, ModeResult } from "../types";
import { checkContainsTrigger } from "../../github/validation/trigger";
import { prepareMcpConfig } from "../../mcp/install-mcp-server";
import { fetchGitHubData } from "../../github/data/fetcher";
import type { FetchDataResult } from "../../github/data/fetcher";
import { createPrompt } from "../../create-prompt";
import type { PreparedContext } from "../../create-prompt";
import { isEntityContext, isPullRequestEvent } from "../../github/context";
import {
formatContext,
formatBody,
formatComments,
formatReviewComments,
formatChangedFilesWithSHA,
} from "../../github/data/formatter";
/**
* Review mode implementation.
*
* Code review mode that uses the default GitHub Action token
* and focuses on providing inline comments and suggestions.
* Automatically includes GitHub MCP tools for review operations.
*/
export const reviewMode: Mode = {
name: "experimental-review",
description:
"Experimental code review mode for inline comments and suggestions",
shouldTrigger(context) {
if (!isEntityContext(context)) {
return false;
}
// Review mode only works on PRs
if (!context.isPR) {
return false;
}
// For pull_request events, only trigger on specific actions
if (isPullRequestEvent(context)) {
const allowedActions = ["opened", "synchronize", "reopened"];
const action = context.payload.action;
return allowedActions.includes(action);
}
// For other events (comments), check for trigger phrase
return checkContainsTrigger(context);
},
prepareContext(context, data) {
return {
mode: "experimental-review",
githubContext: context,
commentId: data?.commentId,
baseBranch: data?.baseBranch,
claudeBranch: data?.claudeBranch,
};
},
getAllowedTools() {
return [
// Context tools - to know who the current user is
"mcp__github__get_me",
// Core review tools
"mcp__github__create_pending_pull_request_review",
"mcp__github__add_comment_to_pending_review",
"mcp__github__submit_pending_pull_request_review",
"mcp__github__delete_pending_pull_request_review",
"mcp__github__create_and_submit_pull_request_review",
// Comment tools
"mcp__github__add_issue_comment",
// PR information tools
"mcp__github__get_pull_request",
"mcp__github__get_pull_request_reviews",
"mcp__github__get_pull_request_status",
];
},
getDisallowedTools() {
return [];
},
shouldCreateTrackingComment() {
return false; // Review mode uses the review body instead of a tracking comment
},
generatePrompt(
context: PreparedContext,
githubData: FetchDataResult,
): string {
// Support overridePrompt
if (context.overridePrompt) {
return context.overridePrompt;
}
const {
contextData,
comments,
changedFilesWithSHA,
reviewData,
imageUrlMap,
} = githubData;
const { eventData } = context;
const formattedContext = formatContext(contextData, true); // Reviews are always for PRs
const formattedComments = formatComments(comments, imageUrlMap);
const formattedReviewComments = formatReviewComments(
reviewData,
imageUrlMap,
);
const formattedChangedFiles =
formatChangedFilesWithSHA(changedFilesWithSHA);
const formattedBody = contextData?.body
? formatBody(contextData.body, imageUrlMap)
: "No description provided";
return `You are Claude, an AI assistant specialized in code reviews for GitHub pull requests. You are operating in REVIEW MODE, which means you should focus on providing thorough code review feedback using GitHub MCP tools for inline comments and suggestions.
<formatted_context>
${formattedContext}
</formatted_context>
<repository>${context.repository}</repository>
${eventData.isPR && eventData.prNumber ? `<pr_number>${eventData.prNumber}</pr_number>` : ""}
<comments>
${formattedComments || "No comments yet"}
</comments>
<review_comments>
${formattedReviewComments || "No review comments"}
</review_comments>
<changed_files>
${formattedChangedFiles}
</changed_files>
<formatted_body>
${formattedBody}
</formatted_body>
${
(eventData.eventName === "issue_comment" ||
eventData.eventName === "pull_request_review_comment" ||
eventData.eventName === "pull_request_review") &&
eventData.commentBody
? `<trigger_comment>
User @${context.triggerUsername}: ${eventData.commentBody}
</trigger_comment>`
: ""
}
${
context.directPrompt
? `<direct_prompt>
${context.directPrompt}
</direct_prompt>`
: ""
}
REVIEW MODE WORKFLOW:
1. First, understand the PR context:
- You are reviewing PR #${eventData.isPR && eventData.prNumber ? eventData.prNumber : "[PR number]"} in ${context.repository}
- Use mcp__github__get_pull_request to get PR metadata
- Use the Read, Grep, and Glob tools to examine the modified files directly from disk
- This provides the full context and latest state of the code
- Look at the changed_files section above to see which files were modified
2. Create a pending review:
- Use mcp__github__create_pending_pull_request_review to start your review
- This allows you to batch comments before submitting
3. Add inline comments:
- Use mcp__github__add_comment_to_pending_review for each issue or suggestion
- Parameters:
* path: The file path (e.g., "src/index.js")
* line: Line number for single-line comments
* startLine & line: For multi-line comments (startLine is the first line, line is the last)
* side: "LEFT" (old code) or "RIGHT" (new code)
* subjectType: "line" for line-level comments
* body: Your comment text
- When to use multi-line comments:
* When replacing multiple consecutive lines
* When the fix requires changes across several lines
* Example: To replace lines 19-20, use startLine: 19, line: 20
- For code suggestions, use this EXACT format in the body:
\`\`\`suggestion
corrected code here
\`\`\`
CRITICAL: GitHub suggestion blocks must ONLY contain the replacement for the specific line(s) being commented on:
- For single-line comments: Replace ONLY that line
- For multi-line comments: Replace ONLY the lines in the range
- Do NOT include surrounding context or function signatures
- Do NOT suggest changes that span beyond the commented lines
Example for line 19 \`var name = user.name;\`:
WRONG:
\\\`\\\`\\\`suggestion
function processUser(user) {
if (!user) throw new Error('Invalid user');
const name = user.name;
\\\`\\\`\\\`
CORRECT:
\\\`\\\`\\\`suggestion
const name = user.name;
\\\`\\\`\\\`
For validation suggestions, comment on the function declaration line or create separate comments for each concern.
4. Submit your review:
- Use mcp__github__submit_pending_pull_request_review
- Parameters:
* event: "COMMENT" (general feedback), "REQUEST_CHANGES" (issues found), or "APPROVE" (if appropriate)
* body: Write a comprehensive review summary that includes:
- Overview of what was reviewed (files, scope, focus areas)
- Summary of all issues found (with counts by severity if applicable)
- Key recommendations and action items
- Highlights of good practices observed
- Overall assessment and recommendation
- The body should be detailed and informative since it's the main review content
- Structure the body with clear sections using markdown headers
REVIEW GUIDELINES:
- Focus on:
* Security vulnerabilities
* Bugs and logic errors
* Performance issues
* Code quality and maintainability
* Best practices and standards
* Edge cases and error handling
- Provide:
* Specific, actionable feedback
* Code suggestions when possible (following GitHub's format exactly)
* Clear explanations of issues
* Constructive criticism
* Recognition of good practices
* For complex changes that require multiple modifications:
- Create separate comments for each logical change
- Or explain the full solution in text without a suggestion block
- Communication:
* All feedback goes through GitHub's review system
* Be professional and respectful
* Your review body is the main communication channel
Before starting, analyze the PR inside <analysis> tags:
<analysis>
- PR title and description
- Number of files changed and scope
- Type of changes (feature, bug fix, refactor, etc.)
- Key areas to focus on
- Review strategy
</analysis>
Then proceed with the review workflow described above.
IMPORTANT: Your review body is the primary way users will understand your feedback. Make it comprehensive and well-structured with:
- Executive summary at the top
- Detailed findings organized by severity or category
- Clear action items and recommendations
- Recognition of good practices
This ensures users get value from the review even before checking individual inline comments.`;
},
async prepare({
context,
octokit,
githubToken,
}: ModeOptions): Promise<ModeResult> {
if (!isEntityContext(context)) {
throw new Error("Review mode requires entity context");
}
// Review mode doesn't create a tracking comment
const githubData = await fetchGitHubData({
octokits: octokit,
repository: `${context.repository.owner}/${context.repository.repo}`,
prNumber: context.entityNumber.toString(),
isPR: context.isPR,
triggerUsername: context.actor,
});
// Review mode doesn't need branch setup or git auth since it only creates comments
// Using minimal branch info since review mode doesn't create or modify branches
const branchInfo = {
baseBranch: "main",
currentBranch: "",
claudeBranch: undefined, // Review mode doesn't create branches
};
const modeContext = this.prepareContext(context, {
baseBranch: branchInfo.baseBranch,
claudeBranch: branchInfo.claudeBranch,
});
await createPrompt(reviewMode, modeContext, githubData, context);
// Export tool environment variables for review mode
const baseTools = [
"Edit",
"MultiEdit",
"Glob",
"Grep",
"LS",
"Read",
"Write",
];
// Add mode-specific and user-specified tools
const allowedTools = [
...baseTools,
...this.getAllowedTools(),
...context.inputs.allowedTools,
];
const disallowedTools = [
"WebSearch",
"WebFetch",
...context.inputs.disallowedTools,
];
// Export as INPUT_ prefixed variables for the base action
core.exportVariable("INPUT_ALLOWED_TOOLS", allowedTools.join(","));
core.exportVariable("INPUT_DISALLOWED_TOOLS", disallowedTools.join(","));
const additionalMcpConfig = process.env.MCP_CONFIG || "";
const mcpConfig = await prepareMcpConfig({
githubToken,
owner: context.repository.owner,
repo: context.repository.repo,
branch: branchInfo.claudeBranch || branchInfo.currentBranch,
baseBranch: branchInfo.baseBranch,
additionalMcpConfig,
allowedTools: [...this.getAllowedTools(), ...context.inputs.allowedTools],
context,
});
core.setOutput("mcp_config", mcpConfig);
return {
branchInfo,
mcpConfig,
};
},
};

View File

@@ -7,8 +7,10 @@ import { setupBranch } from "../../github/operations/branch";
import { configureGitAuth } from "../../github/operations/git-config"; import { configureGitAuth } from "../../github/operations/git-config";
import { prepareMcpConfig } from "../../mcp/install-mcp-server"; import { prepareMcpConfig } from "../../mcp/install-mcp-server";
import { fetchGitHubData } from "../../github/data/fetcher"; import { fetchGitHubData } from "../../github/data/fetcher";
import { createPrompt } from "../../create-prompt"; import { createPrompt, generateDefaultPrompt } from "../../create-prompt";
import { isEntityContext } from "../../github/context"; import { isEntityContext } from "../../github/context";
import type { PreparedContext } from "../../create-prompt/types";
import type { FetchDataResult } from "../../github/data/fetcher";
/** /**
* Tag mode implementation. * Tag mode implementation.
@@ -120,4 +122,12 @@ export const tagMode: Mode = {
mcpConfig, mcpConfig,
}; };
}, },
generatePrompt(
context: PreparedContext,
githubData: FetchDataResult,
useCommitSigning: boolean,
): string {
return generateDefaultPrompt(context, githubData, useCommitSigning);
},
}; };

View File

@@ -1,6 +1,9 @@
import type { GitHubContext } from "../github/context"; import type { GitHubContext } from "../github/context";
import type { PreparedContext } from "../create-prompt/types";
import type { FetchDataResult } from "../github/data/fetcher";
import type { Octokits } from "../github/api/client";
export type ModeName = "tag" | "agent"; export type ModeName = "tag" | "agent" | "experimental-review";
export type ModeContext = { export type ModeContext = {
mode: ModeName; mode: ModeName;
@@ -54,6 +57,16 @@ export type Mode = {
*/ */
shouldCreateTrackingComment(): boolean; shouldCreateTrackingComment(): boolean;
/**
* Generates the prompt for this mode.
* @returns The complete prompt string
*/
generatePrompt(
context: PreparedContext,
githubData: FetchDataResult,
useCommitSigning: boolean,
): string;
/** /**
* Prepares the GitHub environment for this mode. * Prepares the GitHub environment for this mode.
* Each mode decides how to handle different event types. * Each mode decides how to handle different event types.
@@ -62,10 +75,10 @@ export type Mode = {
prepare(options: ModeOptions): Promise<ModeResult>; prepare(options: ModeOptions): Promise<ModeResult>;
}; };
// Define types for mode prepare method to avoid circular dependencies // Define types for mode prepare method
export type ModeOptions = { export type ModeOptions = {
context: GitHubContext; context: GitHubContext;
octokit: any; // We'll use any to avoid circular dependency with Octokits octokit: Octokits;
githubToken: string; githubToken: string;
}; };

View File

@@ -3,13 +3,37 @@
import { describe, test, expect } from "bun:test"; import { describe, test, expect } from "bun:test";
import { import {
generatePrompt, generatePrompt,
generateDefaultPrompt,
getEventTypeAndContext, getEventTypeAndContext,
buildAllowedToolsString, buildAllowedToolsString,
buildDisallowedToolsString, buildDisallowedToolsString,
} from "../src/create-prompt"; } from "../src/create-prompt";
import type { PreparedContext } from "../src/create-prompt"; import type { PreparedContext } from "../src/create-prompt";
import type { Mode } from "../src/modes/types";
describe("generatePrompt", () => { describe("generatePrompt", () => {
// Create a mock tag mode that uses the default prompt
const mockTagMode: Mode = {
name: "tag",
description: "Tag mode",
shouldTrigger: () => true,
prepareContext: (context) => ({ mode: "tag", githubContext: context }),
getAllowedTools: () => [],
getDisallowedTools: () => [],
shouldCreateTrackingComment: () => true,
generatePrompt: (context, githubData, useCommitSigning) =>
generateDefaultPrompt(context, githubData, useCommitSigning),
prepare: async () => ({
commentId: 123,
branchInfo: {
baseBranch: "main",
currentBranch: "main",
claudeBranch: undefined,
},
mcpConfig: "{}",
}),
};
const mockGitHubData = { const mockGitHubData = {
contextData: { contextData: {
title: "Test PR", title: "Test PR",
@@ -133,7 +157,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
expect(prompt).toContain("You are Claude, an AI assistant"); expect(prompt).toContain("You are Claude, an AI assistant");
expect(prompt).toContain("<event_type>GENERAL_COMMENT</event_type>"); expect(prompt).toContain("<event_type>GENERAL_COMMENT</event_type>");
@@ -161,7 +185,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
expect(prompt).toContain("<event_type>PR_REVIEW</event_type>"); expect(prompt).toContain("<event_type>PR_REVIEW</event_type>");
expect(prompt).toContain("<is_pr>true</is_pr>"); expect(prompt).toContain("<is_pr>true</is_pr>");
@@ -187,7 +211,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
expect(prompt).toContain("<event_type>ISSUE_CREATED</event_type>"); expect(prompt).toContain("<event_type>ISSUE_CREATED</event_type>");
expect(prompt).toContain( expect(prompt).toContain(
@@ -215,7 +239,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
expect(prompt).toContain("<event_type>ISSUE_ASSIGNED</event_type>"); expect(prompt).toContain("<event_type>ISSUE_ASSIGNED</event_type>");
expect(prompt).toContain( expect(prompt).toContain(
@@ -242,7 +266,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
expect(prompt).toContain("<event_type>ISSUE_LABELED</event_type>"); expect(prompt).toContain("<event_type>ISSUE_LABELED</event_type>");
expect(prompt).toContain( expect(prompt).toContain(
@@ -269,7 +293,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
expect(prompt).toContain("<direct_prompt>"); expect(prompt).toContain("<direct_prompt>");
expect(prompt).toContain("Fix the bug in the login form"); expect(prompt).toContain("Fix the bug in the login form");
@@ -292,7 +316,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
expect(prompt).toContain("<event_type>PULL_REQUEST</event_type>"); expect(prompt).toContain("<event_type>PULL_REQUEST</event_type>");
expect(prompt).toContain("<is_pr>true</is_pr>"); expect(prompt).toContain("<is_pr>true</is_pr>");
@@ -317,7 +341,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
expect(prompt).toContain("CUSTOM INSTRUCTIONS:\nAlways use TypeScript"); expect(prompt).toContain("CUSTOM INSTRUCTIONS:\nAlways use TypeScript");
}); });
@@ -336,7 +360,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
expect(prompt).toBe("Simple prompt for owner/repo PR #123"); expect(prompt).toBe("Simple prompt for owner/repo PR #123");
expect(prompt).not.toContain("You are Claude, an AI assistant"); expect(prompt).not.toContain("You are Claude, an AI assistant");
@@ -371,7 +395,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
expect(prompt).toContain("Repository: test/repo"); expect(prompt).toContain("Repository: test/repo");
expect(prompt).toContain("PR: 456"); expect(prompt).toContain("PR: 456");
@@ -418,7 +442,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, issueGitHubData, false); const prompt = generatePrompt(envVars, issueGitHubData, false, mockTagMode);
expect(prompt).toBe("Issue #789: Bug: Login form broken in owner/repo"); expect(prompt).toBe("Issue #789: Bug: Login form broken in owner/repo");
}); });
@@ -438,7 +462,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
expect(prompt).toBe("PR: 123, Issue: , Comment: "); expect(prompt).toBe("PR: 123, Issue: , Comment: ");
}); });
@@ -458,7 +482,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
expect(prompt).toContain("You are Claude, an AI assistant"); expect(prompt).toContain("You are Claude, an AI assistant");
expect(prompt).toContain("<event_type>ISSUE_CREATED</event_type>"); expect(prompt).toContain("<event_type>ISSUE_CREATED</event_type>");
@@ -481,7 +505,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
expect(prompt).toContain("<trigger_username>johndoe</trigger_username>"); expect(prompt).toContain("<trigger_username>johndoe</trigger_username>");
// With commit signing disabled, co-author info appears in git commit instructions // With commit signing disabled, co-author info appears in git commit instructions
@@ -503,7 +527,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
// Should contain PR-specific instructions (git commands when not using signing) // Should contain PR-specific instructions (git commands when not using signing)
expect(prompt).toContain("git push"); expect(prompt).toContain("git push");
@@ -534,7 +558,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
// Should contain Issue-specific instructions // Should contain Issue-specific instructions
expect(prompt).toContain( expect(prompt).toContain(
@@ -573,7 +597,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
// Should contain the actual branch name with timestamp // Should contain the actual branch name with timestamp
expect(prompt).toContain( expect(prompt).toContain(
@@ -603,7 +627,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
// Should contain branch-specific instructions like issues // Should contain branch-specific instructions like issues
expect(prompt).toContain( expect(prompt).toContain(
@@ -641,7 +665,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
// Should contain open PR instructions (git commands when not using signing) // Should contain open PR instructions (git commands when not using signing)
expect(prompt).toContain("git push"); expect(prompt).toContain("git push");
@@ -672,7 +696,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
// Should contain new branch instructions // Should contain new branch instructions
expect(prompt).toContain( expect(prompt).toContain(
@@ -700,7 +724,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
// Should contain new branch instructions // Should contain new branch instructions
expect(prompt).toContain( expect(prompt).toContain(
@@ -728,7 +752,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
// Should contain new branch instructions // Should contain new branch instructions
expect(prompt).toContain( expect(prompt).toContain(
@@ -752,7 +776,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, false); const prompt = generatePrompt(envVars, mockGitHubData, false, mockTagMode);
// Should have git command instructions // Should have git command instructions
expect(prompt).toContain("Use git commands via the Bash tool"); expect(prompt).toContain("Use git commands via the Bash tool");
@@ -781,7 +805,7 @@ describe("generatePrompt", () => {
}, },
}; };
const prompt = generatePrompt(envVars, mockGitHubData, true); const prompt = generatePrompt(envVars, mockGitHubData, true, mockTagMode);
// Should have commit signing tool instructions // Should have commit signing tool instructions
expect(prompt).toContain("mcp__github_file_ops__commit_files"); expect(prompt).toContain("mcp__github_file_ops__commit_files");

View File

@@ -3,6 +3,7 @@ import { getMode, isValidMode } from "../../src/modes/registry";
import type { ModeName } from "../../src/modes/types"; import type { ModeName } from "../../src/modes/types";
import { tagMode } from "../../src/modes/tag"; import { tagMode } from "../../src/modes/tag";
import { agentMode } from "../../src/modes/agent"; import { agentMode } from "../../src/modes/agent";
import { reviewMode } from "../../src/modes/review";
import { createMockContext, createMockAutomationContext } from "../mockContext"; import { createMockContext, createMockAutomationContext } from "../mockContext";
describe("Mode Registry", () => { describe("Mode Registry", () => {
@@ -30,6 +31,12 @@ describe("Mode Registry", () => {
expect(mode.name).toBe("agent"); expect(mode.name).toBe("agent");
}); });
test("getMode returns experimental-review mode", () => {
const mode = getMode("experimental-review", mockContext);
expect(mode).toBe(reviewMode);
expect(mode.name).toBe("experimental-review");
});
test("getMode throws error for tag mode with workflow_dispatch event", () => { test("getMode throws error for tag mode with workflow_dispatch event", () => {
expect(() => getMode("tag", mockWorkflowDispatchContext)).toThrow( expect(() => getMode("tag", mockWorkflowDispatchContext)).toThrow(
"Tag mode cannot handle workflow_dispatch events. Use 'agent' mode for automation events.", "Tag mode cannot handle workflow_dispatch events. Use 'agent' mode for automation events.",
@@ -57,17 +64,17 @@ describe("Mode Registry", () => {
test("getMode throws error for invalid mode", () => { test("getMode throws error for invalid mode", () => {
const invalidMode = "invalid" as unknown as ModeName; const invalidMode = "invalid" as unknown as ModeName;
expect(() => getMode(invalidMode, mockContext)).toThrow( expect(() => getMode(invalidMode, mockContext)).toThrow(
"Invalid mode 'invalid'. Valid modes are: 'tag', 'agent'. Please check your workflow configuration.", "Invalid mode 'invalid'. Valid modes are: 'tag', 'agent', 'experimental-review'. Please check your workflow configuration.",
); );
}); });
test("isValidMode returns true for all valid modes", () => { test("isValidMode returns true for all valid modes", () => {
expect(isValidMode("tag")).toBe(true); expect(isValidMode("tag")).toBe(true);
expect(isValidMode("agent")).toBe(true); expect(isValidMode("agent")).toBe(true);
expect(isValidMode("experimental-review")).toBe(true);
}); });
test("isValidMode returns false for invalid mode", () => { test("isValidMode returns false for invalid mode", () => {
expect(isValidMode("invalid")).toBe(false); expect(isValidMode("invalid")).toBe(false);
expect(isValidMode("review")).toBe(false);
}); });
}); });