Devin 101：用 Devin API 自动完成 PR 审查

Update: We recommend Devin Review (https://app.devin.ai/review) as our new and improved code review experience. Learn more: https://docs.devin.ai/work-with-devin/devin-review.

Introduction

Code reviews are an essential component of the software development lifecycle, helping catch bugs, ensuring code quality, and maintaining standards.

However, manual reviews can be time-consuming, delaying projects and adding to workloads. And, many bugs are subtle and hard to catch.

What if you could automate pull request (PR) reviews, ensuring that every PR receives prompt, thorough feedback in minutes?

Check out this short video, which shows our automation in action, catching a common but tricky bug. You can also see the Devin session and the PR.

Developers can use Devin's External API to instantly spin-up Devin instances in response to PRs, making it easy to integrate the power of AI into existing CI/CD workflows.

In this article, we'll outline the problem, discuss the solution, and walk you through a quick three-step tutorial to integrate Devin with GitHub Actions.

Why you should care

PR reviews eat up engineers' time and cause friction. A study of over 700,000 PRs found that PRs wait an average of 4 days to get reviewed. The problem is even worse for large PRs (500+ lines changed): these take an average of 9 days to get merged. Meanwhile, another study found that as the number of changed files in a PR increases, the amount of time spent reviewing each file decreases drastically. This compromises code quality.

There are also other challenges to effectively reviewing PRs, like reviewer availability, poor prioritization of PRs, and back-and-forth friction.

Devin helps by delivering on:

Speed: PRs are reviewed in minutes, not hours/days; Scalability: Spin up as many Devins as needed. Devin scales with you; Consistency: Devin learns and follows your team's coding standards

Solving the Problem

You can think of Devin's External API as engineering on demand. Spin up Devins in response to events like PRs, Jira issues, Slack messages, etc. The sky's the limit! We actually use the API in our own development workflow for QA—that is, we use Devin to build Devin. Check out the open source code here.

For our current task (automating PR reviews) we can use the GitHub Actions platform. Our Actions workflow is triggered whenever a PR is opened, updated, or reopened. It fetches the code, retrieves the list of changed files, and sends the information to Devin, which reviews the changes and comments feedback. It usually takes five to ten minutes for Devin to finish reviewing the PR and begin posting comments.

Please note that Devin isn't guaranteed to catch every bug, and it's best to still have a human check your PRs before merging. We recommend using Devin as an extra set of eyes, not a complete replacement for human oversight.

Here's the steps to set it up:

1. Add a GitHub Actions workflow that triggers on PRs to your repository

2. Add your Devin API Key to GitHub

3. Create a PR to trigger workflow

4. Customize Devin's behavior to match your team's practices and conventions

Prerequisites

A GitHub repo to test on; A Devin API key (go to Settings -> API Keys); GitHub Actions enabled on your repo (usually enabled by default)

Step 1: Add the workflow

Create a new file in your repository: .github/workflows/pr-review.yml. This file will contain our workflow, which consists of two steps: retrieving a list of changed files using the GitHub API, and spinning up a Devin.

Get the complete pr-review.yml file from this gist. It might seem long but we'll break it down after this. For now, just paste it into your file.

After pasting in the script, commit and push your changes.

Step 2: Configure secrets

You'll need a Devin API key.

Go to Devin's API Key in Settings; Click View Key to view and copy your key

Then, add your key to your GitHub repository.

Go to your repository on GitHub; Click on Settings > Secrets and variables > Actions; Click New repository secret; Name the secret DEVIN_API_KEY; Paste your Devin API key into the Value field; Click Add secret

Step 3: Trigger PR review workflow

To make sure everything is set up correctly, edit some code in your repo and create a PR. After a few seconds, you should see that a Check called "Automated PR Review" has passed. This means that we've successfully spun up a Devin.

If you'd like, you can go to the Devin web interface and monitor the process. To view Devin sessions started via API, click X on the filter "Creator is {your name}".

In ten minutes or less, Devin's comments will start appearing on your PR.

Step 4: Customize Devin

We highly recommend customizing the workflow based on your preferences. For example, consider including your code conventions in the prompt. Or, include a file with your conventions in your repository and give Devin the path so it can find the file.

Other customizations:

Create a custom label in GitHub so that Devin only runs on specific PRs, not all; Adjust the number of comments Devin is allowed to make per PR; Allow Devin to create Suggested Changes; Allow Devin to mark the PR as approved

Breaking down the workflow

If you're interested, here's more detail on how the workflow itself works.

The first step is simple: fetch the list of files modified in the PR.

The next step is to spin up a Devin using the API with a detailed prompt. We recommend customizing the prompt to suit your needs, such as by adding your team's coding conventions.

The prompt instructs Devin to follow this series of steps:

Note that step 2 is to set up a pre-push hook. The hook intercepts any attempts by this specific instance of Devin to push changes to the repository. It provides an added layer of safety, preventing unwanted PRs. We also instruct Devin to "NEVER make any commits or pushes to the repository" under the Rules section of the prompt. Together, the hook and rule serve as guardrails. Consider adding more rules as you see fit to ensure that Devin reviews PRs as desired. In particular, you may want to adjust the comment limit—set to three by default.

Next, let's dive into how exactly Devin posts comments.

In order to support in-line comments (with the relevant code embedded in the comment), we teach Devin to use the GitHub REST API to "create a review comment for a pull request". It's a little more complicated than it seems, which is why there is a relatively large section of the prompt dedicated to it.

Finally, we convert our large prompt into a JSON-safe string and send a POST request to Devin's External API.

We're only scratching the surface of what can be achieved with the API. We recommend exploring the API for other event-driven and responsive use cases. For instance, you can configure Jira and Linear webhooks so that any time an issue/ticket is created, a Devin is triggered to address it.

Check out the docs to explore API features like idempotency (enables safe retries), file uploads, sending follow-up messages/prompts to a session, and more.

Conclusion

Devin's External API and GitHub Actions can be combined to create powerful, useful CI/CD workflows that simplify and speed up code review processes.

Let Devin handle the tedious parts of code reviews so you and your team can focus on building great software.