The AI Review Assistant (AIRA) lets you use any supported LLM service to simplify and automate tasks related to pull request review.

In this page we demonstrate how to solve common use cases in code-review with the help of custom prompt templates for AIRA. Custom templates can be used to suggest pull request titles and descriptions, or to chat with AIRA through comments anywhere in a pull request page. (Results were obtained using a real LLM service).

Custom prompt templates for chat help reduce cognitive load by pre-packaging common functionality into the context that will be sent to the LLM alongside the user’s comment. This means a reviewer only has to type a few words to drive powerful functionality.

In your custom prompt templates, you can enhance the context send to the LLM with knowledge about the pull request, through the use of dynamic variables.

Prerequisites

If you want to put these examples into practice, first familiarise yourself with how to create custom prompt templates.

The following examples build upon a framework for constructing a prompt, which you can read below. Additionally you can read about the available prompt template variables.

Suggest a PR title and description

AIRA comes already equipped with default prompt templates for suggesting a title and description for a PR, but you can create your own with custom prompt templates. Here is the end result for a PR that modifies a route in an Express.js application:

To achieve this, create custom prompts with the Title or Description usage, and be sure to set them as the default for PR title and description suggestions in the repository admin settings panel (e.g. here we created custom prompt templates generate-title and generate-description):

...
# WORKFLOW

Please provide a Pull Request description. Use the following structure:

WHAT: Briefly state the purpose of the pull request and its overall effect.

✨ New features:
Feature 1 description
Feature 2 description
...

🐛 Bug fixes:
Bug fix 1 description
Bug fix 2 description
...

💄 Design:
Design 1 description
Design 2 description
...

⚡ Performance:
Performance 1 description
Performance 2 description
...

🔒 Security:
Performance 1 description
Performance 2 description
...

📝 Documentation:
Documentation 1 description
Documentation 2 description
...

➕ Other type:
Other 1 description
Other 2 description
...


WHY: Briefly summarize the motivation for this change and the specific business or engineering goal it achieves. Reference relevant Jira issues if applicable.

HOW: Provide a summary of the approach used to implement the change, highlighting significant design decisions. Avoid mentioning low-level file changes.

TESTING: Answer with YES or NO if there are automated tests. Mention automated tests in the diff, if applicable.

## Format
- Use markdown
- Please ensure that the description is easy to read and understand while maintaining the concise format.

Using Custom Chat Prompt Templates

If you create a custom prompt template for Chat. You can use it talk to AIRA by starting the comment with /aira <template-slug>, this can be automatically inserted by choosing a prompt template from the list when you click on the AI Prompt button in the comment editor.

Guide for a reviewer

While PR title and description suggestion is useful for the contributor opening the PR, we can create a useful tool for reviewers to familiarise themselves with potential problems. In this example, we created a prompt template generate-review with the Chat usage, which can be invoked in a conversation with AIRA from the pull request overview.

# ROLE
You are an expert software tech lead, in charge of reviewing a pull request from
source <source> into target <target>.
You must understand this pull request and provide necessary feedback.

To inform your knowledge as a pull request reviewer, you are provided with the
following information about the pull request:

## PR Diff
Below you find the diff information. A line starting with ++ has been added,
a line starting with -- has been removed and a line starting
with == was unchanged.
<diff>

## PR Commits
Here are the first <commit-count> commit titles in this pull request: <commits>

# WORKFLOW
- In the following messages, the user should ask you to advise
  the pull request reviewer (or provide no extra details)
- If the user asks you to do something else then respond with
  "I expected you to ask me to advise how to review the pull request"
- otherwise generate a guide for how to review this PR,
  following the below steps:

## REVIEW STEPS
Please provide a guide to provide the Pull Request reviewer with useful feedback and insights on this current Pull Request. It comes in the form of two tables, described as such:

### Insights Table
First comes a Table with two columns, No Header. Here are the rows:

"Estimated effort to review" numeric rating (from 1 to 5 inclusive) - effort required to review. Use emojis to represent the value out of 5, e.g. 1 would be 🟢⚪️⚪️⚪️⚪️, 2 is 🟢🟢⚪️⚪️⚪️, and so on.

"Relevant tests": Yes/No - does the diff include tests relevant to code changes?

"Security concerns": Yes/No - does the code make changes with security concerns?

### Issues Table
Second comes another table with two columns.
Column headers are "Emoji", and "Details"

Each row in this table enumerates a key issue for the reviewer to consider. In the first column, put an emoji that corresponds to the problem, and in the second column describe the issue.

In the above prompt we use a few prompt template variables. As the output focuses on code quality, we selected to use <diff> and <commits> to be able to ask relevant questions. We avoid using <title> and <description>to eliminate potential interference with false claims from the contributor about their PR.

Identify potential problems

In the following example we ask AIRA from the pull request overview to identify issues in the code, before a reviewer begins to take a look. Thanks to the identify-problems custom prompt template, we get precise output, pinpointing the location in the code to look at.

# WORKFLOW
- Identify and categorise all problems with the code mentioned in the diff
- report in a table with three columns
  - category
  - explanation
  - line number and file
- **ONLY OUTPUT A TABLE**
- DO NOT EXPLAIN OUTPUT
- ignore trivial problems

In the above snippet, we followed the same # ROLE section as for generating a reviewer guide, only adjusting the # WORKFLOW. Here is the diff of the PR for comparison:

Generate test cases

If a reviewers identifies suspicious or untested code in the pull request, they could use another Chat prompt template to generate test cases for a given snippet in the diff, for example here is displayed a user instructing AIRA to use the generate-tests custom prompt template:

In the above snippet from the prompt template, we use a few more prompt template variables. Importantly, as we are expecting the prompt to be applied directly to the diff, we use the anchor variables <anchor-line-number>, <anchor-line>, <anchor-path>, and <anchor-file>. These specifically pinpoint the code that user has commented on to provide useful context.

Suggest code improvements

At a glance a reviewer may spot code that needs improving. Here the repo admin created a suggest-improvements custom Chat prompt template, with a workflow that rewrites the code that the comment is attached to:

The above snippet reuses the same # ROLE section as for generating test cases, only the # WORKFLOW changes.

Generate code documentation

You can create a custom prompt template with the Chat usage with a workflow to add documentation comments to a given unit of code. For example here using the generate-documentation custom prompt:

The above snippet reuses the same # ROLE section as for generating test cases, only the # WORKFLOW changes.

Ask general questions with the default chat prompt template

Reviewers can also chat without selecting any prompt-template, in which case the default one configured for the repository is used. In this case you can still set a custom prompt template as the default in the repository admin settings panel. In this case the prompt should really only provide the expected role and knowledge of the AI Review assistant, and let the chat drive the workflow.

1. Ask about facts

AI Review Assistant will be good at scanning your pull requests for particular facts that a human can easily double-check. The following questions can be applied automatically to any pull request.

/aira Make a list of all files and line numbers where there is a password in plain-text in this Pull Request
/aira Make a list of all files and line numbers where PII information might leak from the code in this Pull Request
/aira Make a list of any inconsistencies between inline comments in the code, and the implementation, or the description of this Pull Request?

2. Ask about help understanding the pull request

AI Review Assistant can also help the reviewers to understand a particular piece of code that was changed. The following questions can be applied by a human directly on the source code file

/aira Please write the steps of the algorithm implementing this function <name of the function>
/aira Please list the places where this new class <name of the class> is used in this pull request

3. Ask about generating new content

AI Review Assistant can also generate code related to the pull request, which can then be added to it.

/aira Please write another test case that is relevant to the implementation of this function <name of the function>
/aira Please write a docstring (Java style) explaining the purpose of this function <name of the function>

Extra assistance

Prompt template variables

Dynamic variables help you create generic prompts. They are replaced in the text with their values when the request is sent to the configured AI provider.

For example if this was the prompt template:

Then the final message sent to the AI provider would become This is a PR from develop to main, for a pull request from the develop branch into main.

Below is a table of all the built-in prompt template variables:

variable	substituted by

variable	substituted by
`<source>`	the source branch of the pull request
`<target>`	the target branch of the pull request
`<diff>`	the diff of the pull request
`<title>`	the title of the pull request
`<description>`	the description of the pull request
`<commit-count>`	the number of commits in the pull request
`<commits>`	the first commit titles in the pull request (with the same length as `<commit-count>`)
`<jira>`	the Jira information of the pull request
`<template>`	the slug of the prompt template

Framework for prompt engineering

As any tool based on large language models, you can ask about anything and it will respond. However, it takes a bit of practice to know how to ask questions the right way. These questions are called prompts in the context of AI.

Each LLM model is different, so you will likely need to fine tune the advice here based on the specific model, but the framework used in all examples in this page is Role and Workflow. i.e. you give clear instructions of the purpose and knowledge of the system in the role section, and fine tune the expected functionality in the workflow section.

The structure of the prompt will be a markdown document, divided into sections with headers. At the top level we will use two top level titles:

In the Role section, you should instruct the system of who it acts as, for example:

In this section, you should also provide the “knowlege” that the system has access to, using the provided prompt template variables. You should only use what is necessary for the expected functionality. In general is it better to avoid overloading the context.

You can also subdivide each section of the prompt with nested markdown titles, which helps to structure the context.

Here is a more complete # ROLE section, including some usage of prompt variables:

In the Workflow section, outline the functionality of the prompt template. Think how you want the user to use this prompt. For example:

Code Review Assistant for Bitbucket

Example Prompts by Use Case

Analytics