AI agents in GitHub Actions: how we started automating code review

An issue that gets solved on its own: the system opens the PR, resolves it, and audits it. That’s already how a real workflow runs inside a client’s GitHub organization.

DANIEL ALARCÓN
Software Architect

We’ve only had this running for a very short time —we’re literally starting to test it right now; it hasn’t been rolling for weeks— but it already hints at where engineering is heading over the next few years: AI agents in GitHub Actions reviewing pull requests and, in some cases, resolving issues without anyone writing a line of code by hand. We set it up across several of the client’s repositories, and we’re rolling it out to the rest; even though it’s a first version with plenty of rough edges, the mechanism already works end to end.

Reviewing pull requests with a model is nothing new: tools like GitHub Copilot have been doing it for a while, with the review running in the provider’s cloud and billing for every API call. What changes here is that the entire inference system lives inside the client’s own infrastructure: no variable cost from an external provider, and no review data leaving that environment.

There’s also a second change, more about process: a single pipeline takes the request to fix something, generates the change, opens it as a PR, and puts it through the same filter as any code written by a person — without anyone having to orchestrate each step by hand.

How we now review every pull request

The first thing we set up was a reusable GitHub Actions workflow that wraps PR-Agent and connects it to a client’s internal inference machine: it’s not a tool we built from scratch, but rather something already mature that we plugged into the client’s infrastructure. Any repository in the organization can add it with a few lines and starts receiving automatic review comments as soon as a pull request is opened.

The workflow supports configuration along three axes:

• The behavior of the review: what kind of problems it prioritizes and how demanding it is
• The prompt that defines the criteria it uses to review the code
• The model it runs inference on underneath

For now, we have everything on the default configuration —we haven’t touched the prompt or the model— and even so, the result already adds value: it flags real problems before a person gets to look at the PR. One thing we like about PR-Agent is that it labels the review effort of each PR, and with the volume of PRs a team handles, that label helps a lot to prioritize what to look at first.

Using the client’s inference machine for this isn’t just about reviewing code either: it also helps us measure the sizing and real capabilities of that machine, something hard to test without real load. It’s the same question we asked ourselves when writing about whether clean code still matters when it’s an agent writing the code: review with the same criteria, no matter who writes it.

AI agents in GitHub Actions: when the pipeline resolves the issue itself

The second piece goes a step further. It’s another workflow that listens to comments on issues and, when it detects the right signal, triggers an autonomous resolution:

1. Someone comments on an issue asking for it to be resolved
2. The workflow fires and passes the context to opencode, also backed by that client’s inference machine
3. Opencode generates the change and opens a pull request
4. The review action from the first point kicks in on that same PR, without anyone invoking it by hand

That last step is the one I like most about this first setup: the two pieces feed each other. The PR born from an automatic resolution goes through the same filter as any human PR. There’s no less-supervised path for the code the agent writes.

What still doesn’t work well

This is the very first version, and I’d rather say it with the same clarity I shared with the team: there’s plenty to fine-tune. The prompt isn’t just about reviewing code: it has to be tuned to make sure the code the agent generates is shaped the way we want, not just that it “passes review”.

The model also limits us quite a bit —we’re tied to the capacity of the client’s inference machine, with no access to frontier models— so we’re testing open alternatives like Kimi, GLM, or Qwen to see which ones give the best results with the tools we have. And there’s a deeper problem with the setup itself: running opencode inside a GitHub Action has a practical limit, because if resolving an issue drags on, the action drags on with it.

There’s also a process question we haven’t solved yet: we’re testing opencode through GitHub tickets, but if the real workflow runs through Jira, whatever happens in GitHub leaves no trace there. It’s transparent work the client doesn’t see, and the time spent reviewing and deploying that code —which is exactly where we’ll be spending more time in the future— has to be made visible somehow, not left invisible.

What’s next is fine-tuning the prompt and trying other models from the inference machine for the review, and rethinking where opencode’s execution lives so we don’t depend on the limits of a GitHub Action. We already have it running across several repositories in that client’s organization, and we’re extending it to the rest.

The underlying value isn’t just automating one step: it’s starting to model a different way of working, with the system reviewing pull requests proactively inside the client’s own infrastructure, not in a third party’s cloud. From there, the idea is to delegate work overnight and shift the percentage of time we spend coding, even if that means working inside a Claude session: moving from executors —even if we execute with Claude Code or similar tools— to fully delegated execution, reviewing the work at another moment.

It’s a major mindset shift, probably the one organizations will struggle with the most. And we’re already looking one step beyond: agents that spin up on demand to run tasks as soon as they detect them, with tools like Hermes or OpenClaw. But that’s step two; reaching step one already restructures the mindset so much that what comes next almost follows on its own.