AI Review Assistant

The AI Review Assistant lets you interact with AI from within the comments section of your pull requests, passing all the needed context of the pull request so that you get relevant and valuable replies.

Set up

Step 1: Enable the feature globally

The first thing you need to do is to enable AI Review Assistant. To do that, head over the Global Admin page of Code Review Assistant, and enable the toggle in front of “AI Review Assistant enabled”.

After doing that, you need to configure it on each repository for which you want to use it.

Step 2: Configure the API credentials at the repository level

As a repository administrator, follow the following steps:

1. Get your API key from your AI provider.

   1. For you get the API key OpenAI by going here.

2. Go to the Code Review Assistant settings page of your repository

3. Select the AI Provider: OpenAI or another AI Provider

   1. For other AI providers, like Azure, provide the API URL for the chat completion API.

   2. Contact our support for AI providers not yet supported.

4. Select a model supported by your API Key, and copy-paste your key in the input field.

5. Click on “Save”. You should see a confirmation flag that your data as been updated.

Usage

You can interact with the AI Review Assistant directly in your pull request. To do so, write a comment in the view of your pull request starting by /aira (short for AI Review Assistant) followed by your question for the AI. The AI service will receive the information related to your pull request (current title, description, commit titles, the diff and comments of the existing thread), followed by your question.

Once ChatGPT has answered, AI Review Assistant will post a reply to your comment.

Some examples to get you started (see also below for more examples):

• /aira What is the purpose of this change?

• /aira Are there any password or PII information that leaked in this pull request?

• /aira Can you write a description for this pull request?

You can then reply to AI Review Assistant’s comment, and it will again reply back, with the knowledge of the comment thread.

You can also write a comment directly in the view of a file affected by the pull request.

For example, you can ask

• /aira Can you explain the algorithm of this function?

Automate Pull Request reviews with AI Review Assistant

It is possible to have the AI Review Assistant perform reviews of your pull requests automatically. To achieve that, you need to configure your pull request to create an automatic comment. This comment should begin with the standard /aira prefix, followed by a question requesting a review. You can accomplish this in two ways: by utilizing Default tasks from Bitbucket, or using the Checklist Buddy plugin for Bitbucket.

Using Default tasks

In your repository settings, go to Default tasks, then click on Add a default task. Then, fill in the Default task description with a prompt starting by /aira. See figure below for an example.

When you create a Pull Request, Bitbucket will create the task for you, and AI Review Assistant will respond automatically.

Using Checklist Buddy for Bitbucket

Checklist Buddy for Bitbucket is a plugin that automatically generates checklists of tasks and comments based on the specific context of a pull request. To utilize Checklist Buddy, you define a PRCHECKLIST file in your repository. This file outlines the comments and tasks you want Checklist Buddy to create, taking into account criteria relevant to your pull requests.

The AI Review Assistant will address all tasks and comments that begin with /aira. Below is an example of a PRCHECKLIST file:

# PRCHECKLIST generated online at https://mibexsoftware.bitbucket.io/checklist-buddy-editor/

--target main +title+ AI Review
  --target main +comment+ /aira Please review the Code Style in this pull request
  --target main +comment+ /aira Please make a report about the presence of tests, and their relevance

Upon creation of a pull request to the main branch, these comments will be created, and AI Review Assistant will respond like in the image below.

Example prompts

As any tool based on ChatGPT (or similar technologies), you can ask about anything to it and it will respond. However, it takes a bit of practice to know how to ask questions the right way. These questions are called prompts in the context of AI. Below, we walk you through some examples that have brought some success in our internal usage.

1. Ask about facts

AI Review Assistant will be good at scanning your pull requests for particular facts that a human can easily double-check. The following questions can be applied automatically to any pull request.

1. /aira Make a list of all files and line numbers where there is a password in plain-text in this Pull Request

2. /aira Make a list of all files and line numbers where PII information might leak from the code in this Pull Request

3. /aira Make a list of any inconsistencies between inline comments in the codes, and the implementation, or the description of this Pull Request?

2. Ask about help understanding the pull request

AI Review Assistant can also help the reviewers to understand a particular piece of code that was changed. The following questions can be applied by a human directly on the source code file

1. /aira Please write the steps of the algorithm implementing this function <name of the function>

2. /aira Please list the places where this new class <name of the class> is used in this pull request

3. Ask about generating new content

AI Review Assistant can also generate code related to the pull request, which can then be added to it.

1. /aira Please write another test case that is relevant to the implementation of this function <name of the function>

2. /aira Please write a docstring (Java style) explaining the purpose of this function <name of the function>

FAQ

How can I disable AI Review Assistant for my Bitbucket Instance?

Global Bitbucket Admins can disable AI Review Assistant through the Code Review Assistant admin page. At the top of the page, you can disable the toggle. Then, even if a repository has configured AI Review Assistant, it will not respond (and data will not be shared).

Note that AI Review Assistant is disabled by default, and none of the repositories are configured.

How can I disable AI Review Assistant for a repository?

There is no toggle to disable AI Review Assistant for a repository. However, if you do not configure the assistant (by not filling the token information), then it will be effectively disabled, and no data will be shared to the AI API.

If you previously configured AI Review Assistant, you can remove the token by saving the configuration with an empty token.

Who can use AI Review Assistant on a pull request?

Anyone who is allowed to write a comment on a pull request will be allowed to interact with the AI Review Assistant.

I wrote a comment starting by /aira, but nothing happened

First, try to refresh the page. AI Review Assistant responds to Pull Request comments, and those are not refreshed automatically. If the response is still not there, please check the following things.

Sometimes the AI Review Assistant can take a bit of time to answer (up to 2 minutes). Past that time, you should see a message saying that it took too much time.

AI Review Assistant will also respond in case there is another issue with your request (for example, API Key is not properly configured). If, however, nothing happens after 2 minutes, this indicates that there is an unknown issue, and you should contact an Administrator.

Can I make the AI only look at a smaller part of the Pull Request?

By default, when writing a comment to the AI Review Assistant (or responding to a comment where it is involved), AI Review Assistant receives the exhaustive diff of the Pull Request.

It is however possible to narrow down the scope on demand. To do that, respond to an AI comment by a comment starting with Context:, then the list of files (with complete path from the repository root), with one file per line. Such a comment would look like the following:

Context:
path/to/file1.txt
path/to/file2.py

You can see an example in action below: