Working with LLMs Behind GitHub Copilot

The engineers at GitHub who helped build the early versions of GitHub Copilot were excited and astonished by the capabilities of OpenAI's large language models (LLMs). They worked with different LLMs from OpenAI, such as a prototype and a multilingual model, to improve GitHub Copilot's accuracy and completion capabilities.

GitHub Copilot utilizes machine learning techniques to improve its pre-trained models, with one technique being fine-tuning for a specific task or domain. The team also looked for additional pieces of context inside the IDE to provide better completions, such as pulling similar texts from neighboring editor tabs.

As the models got stronger, GitHub Copilot gained new capabilities, including chat functionality, voice-assisted development, and more via GitHub Copilot X on the horizon. Improvements were also made to solve language ambiguity and to lift code from other files.

GitHub Copilot X is the team's vision for an AI-powered developer experience, set to launch in March 2023.

Tags: generative AI, GitHub Copilot, LLM