feat: add review mode for automated PR code reviews (#374)

* feat: add review mode for PR code reviews - Add 'review' as a new execution mode in action.yml - Use default GitHub Action token (ACTIONS_TOKEN) for review mode - Create review mode implementation with GitHub MCP tools included by default - Move review-specific prompt to review mode's generatePrompt method - Add comprehensive review workflow instructions for inline comments - Fix type safety with proper mode validation - Keep agent mode's simple inline prompt handling * docs: add review mode example workflow * update sample workflow * fix: update review mode example to use @beta tag * fix: enable automatic triggering for review mode on PR events * fix: export allowed tools environment variables in review mode The GitHub MCP tools were not being properly allowed because review mode wasn't exporting the ALLOWED_TOOLS environment variable like agent mode does. This caused all GitHub MCP tool calls to be blocked with permission errors. * feat: add review mode workflow for testing * fix: use INPUT_ prefix for allowed/disallowed tools environment variables The base action expects INPUT_ALLOWED_TOOLS and INPUT_DISALLOWED_TOOLS (following GitHub Actions input naming convention) but we were exporting them without the INPUT_ prefix. This was causing the tools to not be properly allowed in the base action. * fix: add explicit review tool names and additional workflow permissions - Add explicit tool names in case wildcards aren't working properly - Add statuses and checks write permissions to workflow - Include both github and github_comment MCP server tools * refactor: consolidate review workflows and use review mode - Update claude-review.yml to use review mode instead of direct_prompt - Use km-anthropic fork action - Remove duplicate claude-review-mode.yml workflow - Add synchronize event to review PR updates - Update permissions for review mode (remove id-token, add pull-requests/issues write) * feat: enhance review mode to provide detailed tracking comment summary - Update review mode prompt to explicitly request detailed summaries - Include issue counts, key findings, and recommendations in tracking comment - Ensure users can see complete review overview without checking each inline comment * Revert "refactor: consolidate review workflows and use review mode" This reverts commit 54ca948599. * fix: address PR review feedback for review mode - Make generatePrompt required in Mode interface - Implement generatePrompt in all modes (tag, agent, review) - Remove unnecessary git/branch operations from review mode - Restrict review mode triggers to specific PR actions - Fix type safety issues by removing any types - Update tests to support new Mode interface * test: update mode registry tests to include review mode * chore: run prettier formatting * fix: make mode parameter required in generatePrompt function Remove optional mode parameter since the function throws an error when mode is not provided. This makes the type signature consistent with the actual behavior. * fix: remove last any type and update README with review mode - Remove any type cast in review mode by using isPullRequestEvent type guard - Add review mode documentation to README execution modes section - Update mode parameter description in README configuration table * mandatory bun format * fix: improve review mode GitHub suggestion format instructions - Add clear guidance on GitHub's suggestion block format - Emphasize that suggestions must only replace the specific commented lines - Add examples of correct vs incorrect suggestion formatting - Clarify when to use multi-line comments with startLine and line parameters - Guide on handling complex changes that require multiple modifications This should resolve issues where suggestions aren't directly committable. * Add missing MCP tools for experimental-review mode based on test requirements * chore: format code * docs: add experimental-review mode documentation with clear warnings * docs: remove emojis from experimental-review mode documentation * docs: clarify experimental-review mode triggers - depends on workflow configuration * minor format update * test: fix registry tests for experimental-review mode name change * refactor: clean up review mode implementation based on feedback - Remove unused parameters from generatePrompt in agent and review modes - Keep Claude comment requirement for review mode (tracking comment) - Add overridePrompt support to review mode - Remove non-existent MCP tools from review mode allowed list - Fix unused import in agent mode These changes address all review feedback while maintaining clean code and proper functionality. * fix: remove redundant update_claude_comment from review mode allowed tools The github_comment server is always included automatically, so we don't need to explicitly list mcp__github_comment__update_claude_comment in the allowed tools. * feat: review mode now uses review body instead of tracking comment - Remove tracking comment creation from review mode - Update prompt to instruct Claude to write comprehensive review in body - Remove comment ID requirement for review mode - The review submission body now serves as the main review content This makes review mode cleaner with one less comment on the PR. The review body contains all the information that would have been in the tracking comment. * add back id-token: write for example * Add PR number for context + make it mandatory to have a PR associated * add `mcp__github__add_issue_comment` tool * rename token * bun format --------- Co-authored-by: km-anthropic <km-anthropic@users.noreply.github.com>
2026-01-23 06:54:13 +08:00 · 2025-08-01 15:04:23 -07:00
parent 0e5fbc0d44
commit 56179f5fc9
12 changed files with 604 additions and 91 deletions
--- a/src/modes/agent/index.ts
+++ b/src/modes/agent/index.ts
@@ -1,7 +1,7 @@
 import * as core from "@actions/core";
-import { mkdir, writeFile } from "fs/promises";
 import type { Mode, ModeOptions, ModeResult } from "../types";
 import { isAutomationContext } from "../../github/context";
+import type { PreparedContext } from "../../create-prompt/types";

 /**
 * Agent mode implementation.
@@ -42,24 +42,7 @@ export const agentMode: Mode = {
  async prepare({ context }: ModeOptions): Promise<ModeResult> {
    // Agent mode handles automation events (workflow_dispatch, schedule) only

-    // Create prompt directory
-    await mkdir(`${process.env.RUNNER_TEMP}/claude-prompts`, {
-      recursive: true,
-    });
-
-    // Write the prompt file - the base action requires a prompt_file parameter,
-    // so we must create this file even though agent mode typically uses
-    // override_prompt or direct_prompt. If neither is provided, we write
-    // a minimal prompt with just the repository information.
-    const promptContent =
-      context.inputs.overridePrompt ||
-      context.inputs.directPrompt ||
-      `Repository: ${context.repository.owner}/${context.repository.repo}`;
-
-    await writeFile(
-      `${process.env.RUNNER_TEMP}/claude-prompts/claude-prompt.txt`,
-      promptContent,
-    );
+    // Agent mode doesn't need to create prompt files here - handled by createPrompt

    // Export tool environment variables for agent mode
    const baseTools = [
@@ -80,8 +63,9 @@ export const agentMode: Mode = {
      ...context.inputs.disallowedTools,
    ];

-    core.exportVariable("ALLOWED_TOOLS", allowedTools.join(","));
-    core.exportVariable("DISALLOWED_TOOLS", disallowedTools.join(","));
+    // Export as INPUT_ prefixed variables for the base action
+    core.exportVariable("INPUT_ALLOWED_TOOLS", allowedTools.join(","));
+    core.exportVariable("INPUT_DISALLOWED_TOOLS", disallowedTools.join(","));

    // Agent mode uses a minimal MCP configuration
    // We don't need comment servers or PR-specific tools for automation
@@ -114,4 +98,18 @@ export const agentMode: Mode = {
      mcpConfig: JSON.stringify(mcpConfig),
    };
  },
+
+  generatePrompt(context: PreparedContext): string {
+    // Agent mode uses override or direct prompt, no GitHub data needed
+    if (context.overridePrompt) {
+      return context.overridePrompt;
+    }
+
+    if (context.directPrompt) {
+      return context.directPrompt;
+    }
+
+    // Minimal fallback - repository is a string in PreparedContext
+    return `Repository: ${context.repository}`;
+  },
 };
--- a/src/modes/registry.ts
+++ b/src/modes/registry.ts
@@ -13,11 +13,12 @@
 import type { Mode, ModeName } from "./types";
 import { tagMode } from "./tag";
 import { agentMode } from "./agent";
+import { reviewMode } from "./review";
 import type { GitHubContext } from "../github/context";
 import { isAutomationContext } from "../github/context";

 export const DEFAULT_MODE = "tag" as const;
-export const VALID_MODES = ["tag", "agent"] as const;
+export const VALID_MODES = ["tag", "agent", "experimental-review"] as const;

 /**
 * All available modes.
@@ -26,6 +27,7 @@ export const VALID_MODES = ["tag", "agent"] as const;
 const modes = {
  tag: tagMode,
  agent: agentMode,
+  "experimental-review": reviewMode,
 } as const satisfies Record<ModeName, Mode>;

 /**
--- a/src/modes/review/index.ts
+++ b/src/modes/review/index.ts
@@ -0,0 +1,352 @@
+import * as core from "@actions/core";
+import type { Mode, ModeOptions, ModeResult } from "../types";
+import { checkContainsTrigger } from "../../github/validation/trigger";
+import { prepareMcpConfig } from "../../mcp/install-mcp-server";
+import { fetchGitHubData } from "../../github/data/fetcher";
+import type { FetchDataResult } from "../../github/data/fetcher";
+import { createPrompt } from "../../create-prompt";
+import type { PreparedContext } from "../../create-prompt";
+import { isEntityContext, isPullRequestEvent } from "../../github/context";
+import {
+  formatContext,
+  formatBody,
+  formatComments,
+  formatReviewComments,
+  formatChangedFilesWithSHA,
+} from "../../github/data/formatter";
+
+/**
+ * Review mode implementation.
+ *
+ * Code review mode that uses the default GitHub Action token
+ * and focuses on providing inline comments and suggestions.
+ * Automatically includes GitHub MCP tools for review operations.
+ */
+export const reviewMode: Mode = {
+  name: "experimental-review",
+  description:
+    "Experimental code review mode for inline comments and suggestions",
+
+  shouldTrigger(context) {
+    if (!isEntityContext(context)) {
+      return false;
+    }
+
+    // Review mode only works on PRs
+    if (!context.isPR) {
+      return false;
+    }
+
+    // For pull_request events, only trigger on specific actions
+    if (isPullRequestEvent(context)) {
+      const allowedActions = ["opened", "synchronize", "reopened"];
+      const action = context.payload.action;
+      return allowedActions.includes(action);
+    }
+
+    // For other events (comments), check for trigger phrase
+    return checkContainsTrigger(context);
+  },
+
+  prepareContext(context, data) {
+    return {
+      mode: "experimental-review",
+      githubContext: context,
+      commentId: data?.commentId,
+      baseBranch: data?.baseBranch,
+      claudeBranch: data?.claudeBranch,
+    };
+  },
+
+  getAllowedTools() {
+    return [
+      // Context tools - to know who the current user is
+      "mcp__github__get_me",
+      // Core review tools
+      "mcp__github__create_pending_pull_request_review",
+      "mcp__github__add_comment_to_pending_review",
+      "mcp__github__submit_pending_pull_request_review",
+      "mcp__github__delete_pending_pull_request_review",
+      "mcp__github__create_and_submit_pull_request_review",
+      // Comment tools
+      "mcp__github__add_issue_comment",
+      // PR information tools
+      "mcp__github__get_pull_request",
+      "mcp__github__get_pull_request_reviews",
+      "mcp__github__get_pull_request_status",
+    ];
+  },
+
+  getDisallowedTools() {
+    return [];
+  },
+
+  shouldCreateTrackingComment() {
+    return false; // Review mode uses the review body instead of a tracking comment
+  },
+
+  generatePrompt(
+    context: PreparedContext,
+    githubData: FetchDataResult,
+  ): string {
+    // Support overridePrompt
+    if (context.overridePrompt) {
+      return context.overridePrompt;
+    }
+
+    const {
+      contextData,
+      comments,
+      changedFilesWithSHA,
+      reviewData,
+      imageUrlMap,
+    } = githubData;
+    const { eventData } = context;
+
+    const formattedContext = formatContext(contextData, true); // Reviews are always for PRs
+    const formattedComments = formatComments(comments, imageUrlMap);
+    const formattedReviewComments = formatReviewComments(
+      reviewData,
+      imageUrlMap,
+    );
+    const formattedChangedFiles =
+      formatChangedFilesWithSHA(changedFilesWithSHA);
+    const formattedBody = contextData?.body
+      ? formatBody(contextData.body, imageUrlMap)
+      : "No description provided";
+
+    return `You are Claude, an AI assistant specialized in code reviews for GitHub pull requests. You are operating in REVIEW MODE, which means you should focus on providing thorough code review feedback using GitHub MCP tools for inline comments and suggestions.
+
+<formatted_context>
+${formattedContext}
+</formatted_context>
+
+<repository>${context.repository}</repository>
+${eventData.isPR && eventData.prNumber ? `<pr_number>${eventData.prNumber}</pr_number>` : ""}
+
+<comments>
+${formattedComments || "No comments yet"}
+</comments>
+
+<review_comments>
+${formattedReviewComments || "No review comments"}
+</review_comments>
+
+<changed_files>
+${formattedChangedFiles}
+</changed_files>
+
+<formatted_body>
+${formattedBody}
+</formatted_body>
+
+${
+  (eventData.eventName === "issue_comment" ||
+    eventData.eventName === "pull_request_review_comment" ||
+    eventData.eventName === "pull_request_review") &&
+  eventData.commentBody
+    ? `<trigger_comment>
+User @${context.triggerUsername}: ${eventData.commentBody}
+</trigger_comment>`
+    : ""
+}
+
+${
+  context.directPrompt
+    ? `<direct_prompt>
+${context.directPrompt}
+</direct_prompt>`
+    : ""
+}
+
+REVIEW MODE WORKFLOW:
+
+1. First, understand the PR context:
+   - You are reviewing PR #${eventData.isPR && eventData.prNumber ? eventData.prNumber : "[PR number]"} in ${context.repository}
+   - Use mcp__github__get_pull_request to get PR metadata
+   - Use the Read, Grep, and Glob tools to examine the modified files directly from disk
+   - This provides the full context and latest state of the code
+   - Look at the changed_files section above to see which files were modified
+
+2. Create a pending review:
+   - Use mcp__github__create_pending_pull_request_review to start your review
+   - This allows you to batch comments before submitting
+
+3. Add inline comments:
+   - Use mcp__github__add_comment_to_pending_review for each issue or suggestion
+   - Parameters:
+     * path: The file path (e.g., "src/index.js")
+     * line: Line number for single-line comments
+     * startLine & line: For multi-line comments (startLine is the first line, line is the last)
+     * side: "LEFT" (old code) or "RIGHT" (new code)
+     * subjectType: "line" for line-level comments
+     * body: Your comment text
+   
+   - When to use multi-line comments:
+     * When replacing multiple consecutive lines
+     * When the fix requires changes across several lines
+     * Example: To replace lines 19-20, use startLine: 19, line: 20
+   
+   - For code suggestions, use this EXACT format in the body:
+     \`\`\`suggestion
+     corrected code here
+     \`\`\`
+   
+   CRITICAL: GitHub suggestion blocks must ONLY contain the replacement for the specific line(s) being commented on:
+   - For single-line comments: Replace ONLY that line
+   - For multi-line comments: Replace ONLY the lines in the range
+   - Do NOT include surrounding context or function signatures
+   - Do NOT suggest changes that span beyond the commented lines
+   
+   Example for line 19 \`var name = user.name;\`:
+   WRONG:
+   \\\`\\\`\\\`suggestion
+   function processUser(user) {
+       if (!user) throw new Error('Invalid user');
+       const name = user.name;
+   \\\`\\\`\\\`
+   
+   CORRECT:
+   \\\`\\\`\\\`suggestion
+   const name = user.name;
+   \\\`\\\`\\\`
+   
+   For validation suggestions, comment on the function declaration line or create separate comments for each concern.
+
+4. Submit your review:
+   - Use mcp__github__submit_pending_pull_request_review
+   - Parameters:
+     * event: "COMMENT" (general feedback), "REQUEST_CHANGES" (issues found), or "APPROVE" (if appropriate)
+     * body: Write a comprehensive review summary that includes:
+       - Overview of what was reviewed (files, scope, focus areas)
+       - Summary of all issues found (with counts by severity if applicable)
+       - Key recommendations and action items
+       - Highlights of good practices observed
+       - Overall assessment and recommendation
+   - The body should be detailed and informative since it's the main review content
+   - Structure the body with clear sections using markdown headers
+
+REVIEW GUIDELINES:
+
+- Focus on:
+  * Security vulnerabilities
+  * Bugs and logic errors
+  * Performance issues
+  * Code quality and maintainability
+  * Best practices and standards
+  * Edge cases and error handling
+
+- Provide:
+  * Specific, actionable feedback
+  * Code suggestions when possible (following GitHub's format exactly)
+  * Clear explanations of issues
+  * Constructive criticism
+  * Recognition of good practices
+  * For complex changes that require multiple modifications:
+    - Create separate comments for each logical change
+    - Or explain the full solution in text without a suggestion block
+
+- Communication:
+  * All feedback goes through GitHub's review system
+  * Be professional and respectful
+  * Your review body is the main communication channel
+
+Before starting, analyze the PR inside <analysis> tags:
+<analysis>
+- PR title and description
+- Number of files changed and scope
+- Type of changes (feature, bug fix, refactor, etc.)
+- Key areas to focus on
+- Review strategy
+</analysis>
+
+Then proceed with the review workflow described above.
+
+IMPORTANT: Your review body is the primary way users will understand your feedback. Make it comprehensive and well-structured with:
+- Executive summary at the top
+- Detailed findings organized by severity or category
+- Clear action items and recommendations
+- Recognition of good practices
+This ensures users get value from the review even before checking individual inline comments.`;
+  },
+
+  async prepare({
+    context,
+    octokit,
+    githubToken,
+  }: ModeOptions): Promise<ModeResult> {
+    if (!isEntityContext(context)) {
+      throw new Error("Review mode requires entity context");
+    }
+
+    // Review mode doesn't create a tracking comment
+    const githubData = await fetchGitHubData({
+      octokits: octokit,
+      repository: `${context.repository.owner}/${context.repository.repo}`,
+      prNumber: context.entityNumber.toString(),
+      isPR: context.isPR,
+      triggerUsername: context.actor,
+    });
+
+    // Review mode doesn't need branch setup or git auth since it only creates comments
+    // Using minimal branch info since review mode doesn't create or modify branches
+    const branchInfo = {
+      baseBranch: "main",
+      currentBranch: "",
+      claudeBranch: undefined, // Review mode doesn't create branches
+    };
+
+    const modeContext = this.prepareContext(context, {
+      baseBranch: branchInfo.baseBranch,
+      claudeBranch: branchInfo.claudeBranch,
+    });
+
+    await createPrompt(reviewMode, modeContext, githubData, context);
+
+    // Export tool environment variables for review mode
+    const baseTools = [
+      "Edit",
+      "MultiEdit",
+      "Glob",
+      "Grep",
+      "LS",
+      "Read",
+      "Write",
+    ];
+
+    // Add mode-specific and user-specified tools
+    const allowedTools = [
+      ...baseTools,
+      ...this.getAllowedTools(),
+      ...context.inputs.allowedTools,
+    ];
+    const disallowedTools = [
+      "WebSearch",
+      "WebFetch",
+      ...context.inputs.disallowedTools,
+    ];
+
+    // Export as INPUT_ prefixed variables for the base action
+    core.exportVariable("INPUT_ALLOWED_TOOLS", allowedTools.join(","));
+    core.exportVariable("INPUT_DISALLOWED_TOOLS", disallowedTools.join(","));
+
+    const additionalMcpConfig = process.env.MCP_CONFIG || "";
+    const mcpConfig = await prepareMcpConfig({
+      githubToken,
+      owner: context.repository.owner,
+      repo: context.repository.repo,
+      branch: branchInfo.claudeBranch || branchInfo.currentBranch,
+      baseBranch: branchInfo.baseBranch,
+      additionalMcpConfig,
+      allowedTools: [...this.getAllowedTools(), ...context.inputs.allowedTools],
+      context,
+    });
+
+    core.setOutput("mcp_config", mcpConfig);
+
+    return {
+      branchInfo,
+      mcpConfig,
+    };
+  },
+};
--- a/src/modes/tag/index.ts
+++ b/src/modes/tag/index.ts
@@ -7,8 +7,10 @@ import { setupBranch } from "../../github/operations/branch";
 import { configureGitAuth } from "../../github/operations/git-config";
 import { prepareMcpConfig } from "../../mcp/install-mcp-server";
 import { fetchGitHubData } from "../../github/data/fetcher";
-import { createPrompt } from "../../create-prompt";
+import { createPrompt, generateDefaultPrompt } from "../../create-prompt";
 import { isEntityContext } from "../../github/context";
+import type { PreparedContext } from "../../create-prompt/types";
+import type { FetchDataResult } from "../../github/data/fetcher";

 /**
 * Tag mode implementation.
@@ -120,4 +122,12 @@ export const tagMode: Mode = {
      mcpConfig,
    };
  },
+
+  generatePrompt(
+    context: PreparedContext,
+    githubData: FetchDataResult,
+    useCommitSigning: boolean,
+  ): string {
+    return generateDefaultPrompt(context, githubData, useCommitSigning);
+  },
 };
--- a/src/modes/types.ts
+++ b/src/modes/types.ts
@@ -1,6 +1,9 @@
 import type { GitHubContext } from "../github/context";
+import type { PreparedContext } from "../create-prompt/types";
+import type { FetchDataResult } from "../github/data/fetcher";
+import type { Octokits } from "../github/api/client";

-export type ModeName = "tag" | "agent";
+export type ModeName = "tag" | "agent" | "experimental-review";

 export type ModeContext = {
  mode: ModeName;
@@ -54,6 +57,16 @@ export type Mode = {
   */
  shouldCreateTrackingComment(): boolean;

+  /**
+   * Generates the prompt for this mode.
+   * @returns The complete prompt string
+   */
+  generatePrompt(
+    context: PreparedContext,
+    githubData: FetchDataResult,
+    useCommitSigning: boolean,
+  ): string;
+
  /**
   * Prepares the GitHub environment for this mode.
   * Each mode decides how to handle different event types.
@@ -62,10 +75,10 @@ export type Mode = {
  prepare(options: ModeOptions): Promise<ModeResult>;
 };

-// Define types for mode prepare method to avoid circular dependencies
+// Define types for mode prepare method
 export type ModeOptions = {
  context: GitHubContext;
-  octokit: any; // We'll use any to avoid circular dependency with Octokits
+  octokit: Octokits;
  githubToken: string;
 };