AI Helpers for Developers with Juan Pablo from CodeGPT |🎙️#52
In which spheres can LLMs already be really useful? What can you do to combat famous AI hallucinations? How to secure your information when working with AI tools? For episode 52 of DevOps Accents Leo and Pablo invited Juan Pablo from CodeGPT to talk about their project and the whole AI space in 2024. In this episode:
- The story of CodeGPT;
- The difference in AI tools usage in your company;
- The growth of efficiency for individual worker;
- The fears of AI taking peoples’ jobs;
- The progress of AI in 2024;
- How the industry reacts to all these changes.
You can listen to episode 52 of DevOps Accents on Spotify, or right now:
As artificial intelligence (AI) continues to expand its capabilities, 2024 has been a landmark year for its development and adoption. Discussions around combating AI hallucinations, workforce efficiency, fears of job displacement, and how industries react to these rapid advancements have dominated the tech landscape. Here's an overview of these key topics and their implications for the future.
Combating AI Hallucinations: A Critical Focus
AI hallucinations—instances where models provide false or nonsensical outputs—remain a significant challenge. Juan Pablo from CodeGPT explained how their team tackled this issue by integrating AI agents with specific contexts, such as repositories and APIs, to minimize inaccuracies. By using graph technology, they improved the connections between repository components, achieving nearly 99% accuracy in reducing hallucinations. This breakthrough emphasizes the importance of grounding AI tools in precise, contextual knowledge, making them more reliable and tailored for developers.
Something that we can learn from this year is that LLMs have a lot of hallucinations, especially when you're using them with your own code or information. This happens because the LLMs don't know your code, repositories, or information. That's where CodeGPT starts to have a huge impact on dev teams. Now, we boost the productivity of dev teams by allowing them to create AI agents with, for example, an API, their own repository, a framework, documentation, or anything they want.
These agents can be used for pair programming, becoming experts in specific APIs. This is incredibly powerful because when you ask an LLM directly—any LLM, as we are agnostic to the LLM—you can use Anthropic, Cohere, Gemini, OpenAI, or any other LLM in CodeGPT. If you ask the LLM directly, you're likely to encounter hallucinations. The problem is that when you're coding, these hallucinations could lead to mistakes—potentially significant ones—especially when interacting with APIs. For example, when you need to integrate Stripe into your webpage, you can now achieve more accurate results. The answers are more precise because the agent has the full context, exporting the API of Stripe and being used for pair programming. — Juan Pablo
AI Adoption Varies by Company and Function
Despite AI’s potential, adoption varies significantly across companies and departments. According to Juan Pablo, many organizations are hesitant to integrate AI, citing concerns about security, readiness, or skepticism about its immediate benefits. Pablo, the co-host, observed that tools like CodeGPT demonstrate high potential in streamlining development processes, yet the adoption often hinges on top-down decisions from company leadership. Interestingly, while developers may hesitate due to fear of redundancy, executives prioritize tools that boost productivity and reduce time to market.
This discrepancy underscores a broader reality: the success of AI integration depends on tailored solutions that fit specific business needs rather than a one-size-fits-all approach. Companies are excited but often require guidance to implement AI effectively in their unique environments.
It depends on what kind of company it is. What you're saying is, 'I am not going to introduce a new feature in my product with AI; I’m only going to use AI to code.' This is something different. For example, you’re going to reduce 30% or 40% of the time developers need to create something.
It's the same as when you have a ticketing tool. You use a ticketing tool because you need to create tickets for accessibility, and later on, it reduces incidents by 50%. When there's an issue, it helps fix the problem because you can track all incidents and discover, for example, that last time this happened, we did this, that, and the other thing.
In the end, these kinds of tools in companies are different. They're simple to sell to the C-level team by saying, 'Okay, we need to pay for this because...' This is different. For me, I don't see this as a SaaS in the sense of being a disruption to what the company is doing. In the end, it’s more like a helper. You have a company doing code, and now you have a helper. — Pablo Inigo Sanchez
Boosting Individual Efficiency Without Replacing Skills
One of the most significant benefits of AI in 2024 has been its role in increasing individual efficiency. Pablo highlighted that even non-developers can achieve remarkable results using tools like CodeGPT, transforming what might have been complex coding tasks into manageable projects. Juan Pablo likened this shift to the automation seen in warehouses, where humans now supervise advanced robots rather than perform repetitive tasks themselves.
However, both speakers stressed that human expertise remains essential. AI tools work best as assistants, enhancing productivity but not replacing the need for creative problem-solving or in-depth domain knowledge. This collaborative dynamic could redefine what productivity means in industries worldwide.
You can do more with the same developers. Why? Because we believe this is similar to the Amazon warehouse 10, 15, or 20 years ago. If you took a picture of the inside of an Amazon warehouse 20 years ago, you'd see some machines, some computers, and a lot of people. Now, when you look inside, you see experts operating a few machines, but also a lot of robots and automation everywhere in the warehouse.
This means you can have a bigger warehouse with the same number of people, but they’re doing more tasks and accomplishing more. And yes, that’s true. However, at this moment—and likely for the next five or more years—we still need people to make sure this entire process works. People are essential for listening, understanding, and managing what is happening with these agents and bots throughout the process.
That’s probably where the next big skill comes in: learning how to build these agents. Enter the role of the prompt engineer. This is something we’ve heard a lot about this year, with companies hiring prompt engineers to build AI agents. But beyond that, the real expertise lies in knowing how to control and manage all these machines effectively. — Juan Pablo
The Fear of Job Displacement: Myth or Reality?
The fear of AI replacing jobs is pervasive, and it's not entirely unfounded. Pablo shared anecdotes about how automation in industries, like agriculture and manufacturing, has displaced workers historically. Similarly, AI’s increasing capabilities have sparked concerns among developers and other professionals. For example, some developers resist AI tools because they feel it diminishes their unique contributions to their projects.
Nevertheless, both Juan Pablo and Pablo emphasized that these tools are not about replacement but augmentation. AI enables workers to focus on higher-value tasks, leaving repetitive and mundane activities to the machines. The adoption of AI requires a mindset shift, where workers see these tools as partners rather than threats.
Is it that fear, perhaps buried in the subconscious, is something I’m not consciously thinking about right now, but something happened inside me and I don’t know what it is? And the other thing, I think—this is something I heard from a CTO—he told us that people are going to think we’ll keep asking for more and more because we now have tools helping us.
At the beginning, we just wanted to be here and now. Then we wanted to be there, and next, we wanted to reach the next level. So it’s something that could affect people in the same way—they might fear what’s happening with their position, and they might fear what’s happening with what we’re being asked to do because the demands will keep increasing.
But for everyone listening right now, what can we do about this? I think productivity is something that is always sought after. From the C-level perspective, they always want more productivity. So for us, this can be a very good thing. If you have the opportunity to use a new tool to boost your productivity, use it. — Juan Pablo
The State of AI in 2024: Progress and Potential
While AI has come a long way, both speakers noted that we are still far from full autonomy. Current AI tools can efficiently enhance existing processes but struggle to innovate or create from scratch without human guidance. Juan Pablo highlighted that tools like CodeGPT excel in providing context-specific insights but rely on existing knowledge rather than generating entirely new concepts.
Pablo echoed this sentiment, observing that AI tools often lack the ability to execute complex instructions seamlessly. The gap between user expectations and real-world performance means there is still room for significant improvement.
I think we are getting close to the point where LLMs—or not just LLMs, but certain software—can create different things based on what you've done before. They base their outputs on what you do in your daily tasks, in your repositories, and that’s where I start thinking about our graph tool.
When you’re programming with an expert that understands your repositories, you can ask something, and it’s going to base its response on what you’ve done before. It will likely have the same style, probably the same way you write, and build upon that. So they are leveraging what you’ve already done.
When it comes to creating something entirely from scratch, I think we’re not quite there yet—but maybe we will be in a few years. — Juan Pablo
How the Industry Reacts to AI’s Growth
Excitement about AI’s potential is palpable across industries, but implementation challenges remain. Juan Pablo shared how many companies, especially those with specialized needs, seek support to integrate AI into their operations effectively. Collaborations with firms like Accenture are becoming crucial in bridging the gap between AI capabilities and practical application.
Despite these challenges, the industry is optimistic. As companies develop tools to connect AI with specific workflows, the adoption curve is expected to accelerate. The focus is shifting from theoretical capabilities to practical applications that drive measurable value.
I think they are excited, but they need help. Something that has been happening to us over the last two or three months is that companies look at our product and say, 'Wow, this is great, I like it very much, but I need help implementing it within my company.’
They often have super specific types of technology that only they use, or some other software that is highly specialized for their industry. So they need guidance on how to use LLMs and artificial intelligence effectively. That’s something we are seeing a lot of right now. — Juan Pablo
What’s Next for AI?
As we move into 2025, the industry must address lingering concerns around data security, tool adoption, and workforce adaptation. AI has proven its ability to enhance productivity and efficiency, but its potential will only be fully realized when organizations embrace it as a tool for collaboration rather than competition.
For businesses and professionals, staying informed and adaptable will be key to navigating this evolving landscape. While AI continues to reshape industries, its ultimate impact will depend on how we choose to leverage its transformative power.
- Our guest, Juan Pablo, on LinkedIn.
Podcast editing: Mila Jones, milajonesproduction@gmail.com