AI HUMANS Can AI write as well as humans 2021
Software Development

The pros and cons of using GitHub Copilot for software development [survey results]

date: 27 July 2023
reading time: 8 min

GitHub Copilot has sparked both excitement and debate within the tech community. During Future Processing's pilot program, 12 in-house developers tested GitHub Copilot and gave us feedback on the tool.

In the ever-evolving landscape of software development, emerging technologies continue to reshape the way we write code. One such innovation that has garnered significant attention is GitHub Copilot, a powerful AI coding support tool. With its promise of assisting developers in writing code more efficiently, GitHub Copilot has sparked both excitement and debate within the tech community.

During Future Processing’s pilot program, 12 in-house developers tested GitHub Copilot and gave us feedback on the tool. With this data at hand, we dive into the world of GitHub Copilot to explore its pros and cons, shedding light on its impact on software development, examining real-life examples, and helping you determine whether it’s the right fit for your development process.

What is GitHub Copilot?

GitHub Copilot is an AI pair programmer that offers autocomplete-style suggestions as you code. You can receive suggestions and code snippets from GitHub Copilot either by starting to write the code you want to use, or by writing a natural language comment describing what you want the code to do.

The tool’s main job is to understand users’ intent when they write something in natural language and then translate that into code, the language that computers understand. It is also able to translate code between programming languages.

Copilot is a paid software service that was created in collaboration between GitHub and OpenAI.

GitHub Copilot: its impact on software development and programming

GitHub Copilot has already made a significant impact on the landscape of software development and programming. By using code completion tools, Copilot offers developers a helping hand throughout the coding process. This newfound efficiency translates into increased productivity, allowing developers to focus more on higher-level architectures rather than getting caught up in the intricacies of syntax.

Moreover, GitHub Copilot serves as a valuable learning resource for non-developers as well. By observing and analysing the text being written and generating code snippets, Copilot provides real-time insights and suggestions to turn non expert language into code. This can potentially lower the bar for non-developers to start writing code in the future.

Is GitHub Copilot a threat to developers?

While the tool’s advanced AI capabilities can generate code suggestions, it’s important to view Copilot as a complement to developers rather than a replacement. Developers’ expertise, design mindset, and creativity remain essential in ensuring code quality.

Our test has shown that Copilot’s suggestions should be checked by an experienced developer to ensure the suggestions are not only correct, but also optimal and taking into account the bigger picture of the solution.

One developer commented:

‘Sometimes, if it didn’t know the context, Copilot would suggest utter nonsense because it had seen something similar once or use non-existent method names. But it was not something that was annoying or disturbing. I think everyone can benefit from it to some extent to make their work easier.’

Another one commented on its ability to learn:

‘In the initial phase of writing code it sometimes prompts nonsense, but with later it sort of learns and starts to be more accurate. In addition, the whole syntax is often prompted, so you don’t have to remember brackets and other characters.’

Rather than replacing developers, Copilot should be used with a developer’s oversight to speed up the creation of tedious, repetitive code.

Advantages of GitHub Copilot

Out of 12 developers who tested Copilot, 11 found the tool useful with 3 rating it as ‘very helpful’ and 8 ‘somewhat helpful’. One developer said it wasn’t helpful at all.

Most people agreed it was great with boilerplate/ repeatable fragments of code and autocompletion. It was also reported that the problem of remembering complex syntax was eliminated. Instead of struggling to recall every detail, Copilot offers a range of available options, allowing developers to choose the most suitable syntax.

Additionally, our developers found that Copilot’s ability to provide hints, guesses, and suggested syntax for the next code snippet accelerated their coding process. One developer remarked:

‘Suggestions were ‘smarter’ than I expected, saving me valuable time in searching for the right syntax.’

Furthermore, GitHub Copilot proved to be a valuable resource for our developers in terms of providing boilerplate code templates and predefined document structures. It also facilitated seamless transitions between coding languages.

Disadvantages of GitHub Copilot

While GitHub Copilot offers numerous advantages, it’s essential to consider potential drawbacks as well. One notable concern raised by developers and the wider tech community is the notion of “fair use” in the context of mining code protected by intellectual property for training AI systems. Questions arise regarding whether the developers whose code is used to train Copilot provided their consent for their work to be utilised in this manner. The ethical implications of using copyrighted code without explicit permission have sparked discussions around intellectual property rights and the need for transparency and consent in AI training datasets.

Also on the negative side, some of our developers have found certain aspects of GitHub Copilot to be less satisfactory. A feedback from one of the users is the absence of an integrated chat feature, which limits the ability to interact with Copilot directly. Additionally, some developers reported instances where Copilot suggests an excessive amount of code or code that is not relevant to the specific context.

Another person wasn’t happy with its performance with more complex functions, saying:

‘Copilot doesn’t understand if I want to create some advanced or complicated tasks, or I don’t know how to instruct it!’

GitHub Copilot: examples and real cases

GitHub Copilot has been put to the test in various scenarios with Future Processing developers providing valuable feedback based on their experiences. One developer shared their experience while using Copilot in Visual Studio Code (VSC) while working with React. They mentioned that while Copilot proved helpful overall, there were instances where it could be somewhat annoying.

Additionally, another developer noted that Copilot seemed to perform slightly better with C# compared to TypeScript.

In addition to assisting developers in professional projects, Copilot has also been embraced by developers seeking to improve their coding prowess through platforms like CodeWars, where users engage in coding challenges to hone their skills. One developer shared their experience of using Copilot to suggest code during CodeWars exercises, expressing amazement at how Copilot suggested a solution close to the correct one immediately after reading the function name.

These real-life examples highlight both the potential and the limitations of GitHub Copilot, emphasising the need for developers to exercise judgment and validation while leveraging its suggestions.

Does GitHub Copilot improve productivity?

Our developers estimated that by leveraging Copilot, they were able to save an average of 30 minutes of work per day per person, resulting in significant time savings across the team. This data underscores the tangible impact of Copilot on productivity and efficiency within the development process.

Weighing the pros and cons of GitHub Copilot for modern software development

When considering the pros and cons of GitHub Copilot, it becomes evident that this AI-powered coding tool has the potential to reshape the landscape of modern software development. With its ability to streamline the coding process, provide intelligent suggestions, and save developers valuable time, Copilot offers a glimpse into how AI will change the future of software development. However, it is essential to weigh these benefits against the potential drawbacks, such as the need for fine-tuning and addressing concerns related to fair use and copyright.

Is GitHub Copilot right for your development process?

When considering the integration of GitHub Copilot into your development process, it’s essential to maintain a clear, high-level view of the design challenges and constraints of your project. While Copilot can provide valuable code suggestions and streamline coding tasks, it cannot replace the need for a comprehensive understanding of your project’s architecture and design. Developers must still possess the expertise to make informed decisions and ensure that the code aligns with the project’s overall objectives.

In conclusion, GitHub Copilot presents a powerful AI tool that holds great potential for revolutionising the software development process. By leveraging its intelligent, time saving capabilities to generate code and suggestions, developers can enhance productivity and tackle complex coding challenges more efficiently. As Future Processing, with our 20 years of experience and ongoing experimentation in AI and ML services, we are at the forefront of embracing innovative technologies like Copilot.

If you are seeking to unlock the full potential of AI in your software development projects, we invite you to get in touch with us. Our team of experts is ready to guide you, leveraging our deep expertise and knowledge to provide tailored solutions that align with your unique business needs. Together, let’s embark on a journey, embracing the power of AI to transform the future of software development. Contact us today and witness the difference that Future Processing can make in your next development endeavour.

Read more on our blog

Discover similar posts


© Future Processing. All rights reserved.

Cookie settings