Revolution in programming:
How GitHub Copilot is transforming software development

Since the introduction of ChatGPT3 in November 2022, generative AI has entered the mainstream. Artificial intelligence excels in environments where patterns and regularities prevail, which makes it particularly suitable for the field of programming. This suitability results from the core competence of AI to recognize patterns and learn from them.

Meanwhile, AI programming support goes far beyond copying existing code into the chat window of Large Language Models (such as ChatGPT, Gemini, Claude3, …). There are tools that are directly integrated into the development environments and enable a workflow that does not require you to leave the development environment. In this blog post, we would like to introduce one of the key players on the market: GitHub Copilot.

S/4 HANA Speedboat

Tim Küllmey, Consultant Data Management

GitHub Copilot – an overview

GitHub Copilot [1] serves as an AI-driven search engine that translates natural language input into finished code. This tool, based on OpenAI’s Codex (a model specifically for programming), learns from billions of lines of code stored in GitHub repositories. It masters dozens of programming languages and uses probabilities to suggest the best code for the specific context.

GitHub Copilot shows its strengths especially in repetitive tasks by helping developers to focus on more complex problems. It adapts to the style and conventions of the current project, whereby developers can choose between different proposals and customize them. The integration is seamless in development environments such as Visual Studio and VS Code, which increases productivity and simplifies everyday development processes. Although Copilot is getting better and better over time and is widely used (27% of all GitHub developers use it, in some languages it is even over 40%), it remains an assistance system. It is currently not possible to have complete projects programmed automatically by GitHub Copilot, but you can get conceptual advice, have individual functionalities programmed, have existing code improved and you can use it for targeted debugging of errors. Another great strength is the ability to have existing program code explained. This makes it much easier to get started with a new codebase. With these explanations, the code can be understood quickly and without many questions from colleagues. GitHub Copilot is chargeable, with different subscriptions for private and professional use, although a free trial version is offered. [2]

For a brief overview of how GitHub Copilot is used in practice, we recommend the following Youtube video from the VSC code developers.

Advantages

  • Improved coding efficiency:
    GitHub Copilot suggests code blocks that match the context as you type, which significantly speeds up the coding process, especially for repetitive tasks.
  • Focus on conceptual level:
    By taking over routine coding tasks, developers can focus on more complex, cross-cutting and creative aspects of their projects.
  • Improved debugging efficiency:
    Copilot can help to understand error messages and localize the bug in the code and directly suggest a corresponding fix.
  • Educational tool:
    New developers can have existing code analyzed, which can make it easier to familiarize themselves with new languages and frameworks.

Disadvantages

  • Potential errors:
    The code generated by the AI is not infallible and can introduce errors or bugs that the developer must recognize and correct.
  • Reduced involvement with detailed code logic:
    A heavy reliance on AI for coding can lead to developers being less concerned with the underlying logic of the generated code.
  • Dependence on an Internet connection:
    Copilot requires an active internet connection to function, which can be a limitation in environments with poor connectivity.
  • Data security:
    You have to trust GitHub and OpenAI that the code is only analyzed, but not stored for their training data.

Studies on productivity and code quality

An exciting question that arises is to what extent GitHub Copilot increases productivity. A joint study from 2022 by GitHub and MIT (Massachusetts Institute of Technology) examined how much faster developers implement an HTTP server in JavaScript when they use GitHub Copilot. The result is that using GitHub Copilot reduced the time to solve the task by 56%. [3] Other interesting findings are that programming beginners and older programmers in particular were able to increase their productivity using GitHub Copilot.

Another study from 2023 examined the code quality of generated code. The HumanEval dataset was used for this purpose. The HumanEval Dataset is a collection of programming tasks used to evaluate code synthesis models in AI-supported coding tools. It contains a large number of problems in which functions are to be implemented based on a description and a function signature. The authors observed that GitHub Copilot was able to generate the correct code for 46.3% of the problems, while it was unable to provide a correct solution for 30.5% of the problems. The solutions generated for the remaining 23.2% of the problems were partially correct. [4]

Currently, the number of studies on AI programming assistance is still very limited. But the initial results for GitHub Copilot clearly show that AI tools in development lead to massive productivity gains. However, the interaction with the developer is crucial, as completely correct code is only generated in just under half of the cases. This means that in most cases further user input is required to generate functional code.

Personal experience

From my personal experience, using GitHub Copilot has changed my approach to programming projects. I use GitHub Copilot at all stages of my work. First of all, I get support in creating the general program structure by describing the use case. Then I have individual functions generated. If errors occur, I copy the error messages to GitHub Copilot and get help with debugging. Using GitHub Copilot feels like having an experienced developer sitting next to you, offering instant advice and examples, and has led to significantly increased productivity for me.

Introduction to the tutorial

Ff you are interested in integrating GitHub Copilot into your development workflow, I have created a 60-minute tutorial. It uses a simple example to show how to use GitHub Copilot efficiently. Whether you are a beginner who wants to understand AI-assisted programming or an experienced developer who wants to increase productivity, this tutorial will provide you with hands-on experience that you can then apply to your own projects. A prerequisite for the tutorial is that you understand the basic concepts of programming (e.g. what is a function, how do I combine different functions, what is Python?). I encourage you to this tutorial to get to know the possibilities of GitHub Copilot.

Summary and outlook

GitHub Copilot has established itself as a valuable tool for developers that not only speeds up programming, but also makes it easier to get started with new projects. The AI-driven platform supports both beginners and experienced developers in code creation and debugging by making intelligent suggestions and learning from millions of lines of code. Despite some challenges, such as the dependence on an Internet connection and data protection concerns, the response from the developer community shows that the advantages outweigh the disadvantages.

GitHub Copilot is just the tip of the iceberg when it comes to AI-supported programming. With ongoing developments, we can expect even more sophisticated tools that will further simplify the development process. At the same time, however, there still seems to be a long way to go before programming can take place completely autonomously; human input to control the process is still essential.

Sources:

[1] https://github.com/features/copilot
[2] https://www.dev-insider.de/wie-funktioniert-github-copilot-a-10941ef3768ed24068b36909fd430a1e/
[3] Bayer, F., Gonçalves, J., & Yang, J. J. (2023). The Impact of AI on Developer Productivity: Evidence from GitHub Copilot. arXiv preprint arXiv:2302.06590. Available at: https://arxiv.org/pdf/2302.06590
[4] Jiang, S., Zhang, R., & Williams, A. (2023). Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT. arXiv preprint arXiv:2304.10778. Available at: https://arxiv.org/pdf/2304.10778


talk to



Tim Küllmey
!

Julian Philippi