Why should I measure GitHub Copilot usage?

As more engineering teams incorporate GitHub Copilot into their daily workflows, the ability to measure the usage and adoption rates of GitHub CoPilot, along with its impact on development teams, becomes more important. Allstacks allows you to understand the ROI of this generative AI coding tool across your teams.

Learn how you can track, view adoption trends, and optimize your Copilot development cycle in our overview video below.

Prerequisites to get started

Copilot metrics will only appear under “Configure” → “Metrics” after you connect GitHub with your Allstacks environment. You will need to have an existing GitHub account.
Allstacks supports GitHub, GitHub App, and GitHub Enterprise versions. If you have a different version of GitHub, please reach out to your CSM to inquire about support.

Where to find GitHub Copilot metrics in Allstacks

Click “Configure” in the lefthand sidebar and select “Metrics” in the pop-out menu, like below:

On the “Metrics” page, scroll down until you see “Copilot Usage” at the bottom of the page. There, you’ll see the option to select either the “Copilot Adoption Rate” or “Copilot Suggestion Acceptance” metrics:

How metrics are generated

Usage metrics are generated by pulling data from Copilot API.
A “suggestion” by Copilot is defined by either a prompt or autocomplete.

Copilot Adoption Rate

The adoption rate metric measures the effectiveness of the roll out of Copilot at your organization. You can see what team members are using Copilot, who has not yet tried Copilot, and who does not yet have an invitation to use Copilot.

Active - A user who accepted or rejected a suggestion from Copilot.
Invite - A user who was invited to use Copilot but has not accepted or rejected a suggestion yet.
Not invited - A user that do not have access to Copilot

Copilot Suggestion Acceptance Rate

This metric measures the number of suggestions accepted and suggestions rejected over time. This data is pulled from users that are defined as “active.”

Additional Reports

With the copilot integration, your contributors will get an auto-generated tag that can help you take a new perspective on the metrics your team uses to measure productivity, code quality, and coding productivity. This will allow you to compare the output of your cohort that uses copilot against those who are not yet using copilot to speed their development.