Scaling AI Without Losing Control
In a Nutshell
AI agents boost productivity but require strict human oversight—especially in high-trust domains like payments—because hallucinations and unchecked code volume quickly overwhelm senior engineers. Success hinges on bounded context, layered agent architectures, and shifting focus from “what agents create” to “what humans verify.” Organizations that master the 80–95 % accuracy gap and enforce observability, governance, and outcome-based metrics will gain the decisive edge.
These notes were generated by AI and may contain inaccuracies.
Misha Blinkin, VP of business development at JetBrains, hosts a pre-recorded webinar series featuring technology and engineering leaders discussing AI software development trends. The webinar includes live Q&A during broadcast.
Sendhil from Uber joins as an engineering leader and author of the book "Practical WebAssembly." He works on payments infrastructure, building the rails that handle payments to and from Uber globally.
Sendhil transitioned from full stack engineer to engineering manager after experiencing the satisfaction of helping an engineer on his team achieve a senior promotion. This led him to recognize that management was the right career path for him.
According to JetBrains' developer ecosystem survey of tens of thousands of engineering leaders and developers globally, while many developers are adopting AI, only 14% of AI adoption leads to excessive adoption in enterprise settings where measurable value is realized.
Sendhil views AI agents as junior engineers joining the team, enhancing productivity and agility. Current use cases include writing code, writing tests, authoring code reviews, and writing documentation. Documentation quality has improved significantly through AI adoption.
Initial AI adoption involves significant time investment to understand capabilities, educate the system, and provide context. During this period, organizations experience high rates of hallucinations and slop generation, increasing frustration for both engineers and leaders. Once through this phase, AI adoption improves company and team agility.
- 2024: AI assistants integrated into editor UIs
- 2025: Year of AI agents
- 2026: Adoption of remote cloud-run agents, moving from local to remote systems, enabling autonomous or semi-autonomous agents and agent swarms
Organization-wide push toward agent usage exists. Once developers become comfortable with agents, migration toward more autonomous systems becomes feasible. For personal projects, Sendhil selectively uses custom-trained agents built with nano chat for specific activities, while accepting cloud agents for generic tasks where data sharing is acceptable.
Sendhil equates payments with trust - users trust the source when providing credit card information. In payments and enterprise scenarios, AI agents can assist with problem analysis and design, but should not have autonomous power to deploy to production. Human-in-the-loop approval is required before any AI-generated changes reach servers.
A growing trend across organizations involves agent e-commerce, where agents book items and handle heavy search processes. The mandate and intent of the agent becomes critical. Explicit approvals and sign-offs deliver better non-hallucinating outputs for payments scenarios.
Human oversight serves as the essential guardrail for AI systems. This includes code reviews, verifications, and ensuring someone remains responsible and accountable for actions. All AI-generated code undergoes human review before production deployment.
Agents consistently suggest adding code but never recommend removing or refactoring code. This creates a significant issue as senior developers face overwhelming cognitive load from both junior developer code reviews and agent-generated code reviews. Senior developer capacity is limited while the volume of generated code continues increasing.
To manage increasing generated code volume, engineers must be specific about what they ask agents to do, providing bounded context rather than requesting agents to "create an application like X." This reduces slop generation significantly. The approach involves solving particular problems with AI agents rather than attempting to solve entire systems.
Beyond human-in-the-loop reviews, additional verification methods are needed to ensure architectural correctness, code quality, and absence of security vulnerabilities. AI agents can complement engineers by checking areas where they lack full context, such as iframes and content security policies.
The focus shifts from number of words typed to number of words reviewed. AI agents make it easier to understand code base nuances that traditionally required tribal knowledge. Agents accelerate onboarding by helping engineers understand team contexts and dependencies.
Engineers must maintain ownership and stamp approval on anything going to production. They remain responsible for the final output regardless of AI assistance.
Sendhil has experience with multiple migrations including Java to Kotlin, React to Angular, and Angular to React. Migrations present stamina problems where initial enthusiasm diminishes due to mundane grunt work. At scale involving millions of lines, testing coverage varies significantly across teams - compliance teams have extensive testing while growth teams prioritize speed over coverage.
AI agents address migration challenges by handling grunt work, understanding dependencies between systems, articulating relationships, and writing tests in both old and new libraries. Framework migrations become tooling problems rather than people problems.
When one agent migrates code and another writes tests, hallucination risks compound if the same LLM hallucinates in aligned ways. The solution involves fragmenting problems into smaller chunks, planning with agents, and having humans optimize and validate outputs.
Engineers must understand when hallucinations occur and remove them through better prompts, session restarts, or other optimization techniques. Human involvement remains essential for validating agent outputs.
Research and personal experience indicate that when agents achieve 80% accuracy, humans psychologically reduce verification efforts and begin trusting outputs more. This creates vulnerability when hallucinations occur at critical moments.
Anthropic has developed tooling for understanding agent internals. More observability is needed around what enters agents, what exits, and how transformations occur between inputs and outputs.
Startups easily achieve zero-to-80% progress with AI agents. Future success definition shifts to 80-100% completion. Organizations capable of tuning agents and converting 80% to 90-95% outputs will achieve market advantage.
Even enterprise organizations need zero-to-80 shifts despite complex codebases and monorepos. Organizations must evolve to use the right tool for each job. The 90-95% finishing capability will elevate market position.
Engineers will be evaluated on prompting quality, understanding of prompting mechanics, comprehension of how code gets generated, and validation approaches. These become the core competencies.
Sendhil supports using different LLMs or agents for different task components with handoffs between them. AI agents function like team members - each has strengths that complement others. The goal is creating a connected brain that delivers better outputs.
For experiments and niche problems, Sendhil proposes a three-agent architecture:
- Agent 1: Collects and structures relevant data
- Agent 2: Analyzes anomalies and builds probabilistic models
- Agent 3: Applies business context and team knowledge to combine outputs
This layered approach controls hallucination at each stage, reducing garbage data and improving final output quality.
Smaller language models (SLMs) generate better token output compared to larger language models (LLMs) because reduced context decreases hallucination probability.
Unlike IDEs with high switching costs, developers experiment with multiple models and agents simultaneously. JetBrains supports this ecosystem approach by integrating various AI solutions into their IDEs rather than pushing proprietary solutions exclusively.
Organizations need bounded flexibility - allowing innovation through multiple tool experimentation while maintaining standardized processes for sensitive data. Critical information should not be shared with external agents.
The transcript ends mid-sentence at: "you are allowed to pick any tool inside the team and inside the organization you can try"
Early-stage startups should experiment with multiple tools while maintaining focus on delivering specific outcomes. When a tool fails to meet requirements, teams should avoid retrofitting their workflow to accommodate it. Similarly, organizations should avoid adopting multiple tools that perform identical functions. Not all tools will deliver 100% effectiveness—some provide 80% or 80.1% of needed functionality. The critical decision is whether to restructure entire workflows for marginal improvements. Early-stage founders often pick a tool and add more as they encounter roadblocks. Instead, teams should periodically evaluate whether chosen tools are correct and cost-effective, considering both the financial cost and the prompting effort required. Once a tool demonstrates good progress, organizations should standardize on it before moving to automate or AI-enable other parts of their workflow.
Companies and developers must assess their security posture when working with autonomous agents. Running agents with root access on devices containing corporate data represents a significant risk. Despite security training provided in enterprises, vulnerabilities persist. As teams grant more autonomous power to agents, engineers need to develop skills for identifying security implications and ensuring only appropriate, non-proprietary information is shared. Closed models and small language models (SLMs) with narrow use cases provide advantages for controlled experimentation.
Before implementing governance, organizations need observability—clear visibility into all tooling, models, agents, data transfers, and affected projects. Current LLM and SLM tools provide prompt analytics and observability features that help track prompt patterns and information flow. At the prompt injection level, future filtering capabilities may help control data handling. Every prompt carries associated costs, requiring teams to prioritize and evaluate both inputs and outputs when working across multiple tools and systems.
Tool sprawl creates management difficulties, particularly in large organizations with tens of thousands of engineers who prefer standardization. Shadow AI—where employees use unapproved tools like ChatGPT or Codex through private accounts—presents significant challenges, especially in early-stage startups lacking compliance guardrails. The solution involves implementing enterprise accounts and applying zero-trust policies: limiting data access to only what is necessary and granting only required permissions. Leaders must identify which tools deliver results for their teams and guide adoption accordingly rather than allowing unrestricted tool usage.
Traditional metrics like counting diffs are inadequate for measuring team productivity in AI-augmented environments. AI agents can generate more diffs and accelerate development, making this metric obsolete. Instead, productivity should be tied to KPIs, delivery outcomes, incident reduction, decreased meeting time, and reduced support burden. The focus shifts from quantity to code quality. AI agents currently produce better code than most humans based on their training data, allowing teams to concentrate on finishing details, ensuring top-notch quality, and solving organizational problems.
Developer experience encompasses humanistic and human-oriented practices—approximately 5% tooling and 10% technology, with the remainder being HR-related factors concerning how engineers feel as humans. AI transformation agents have not yet reached this level of sophistication. In the evolving landscape, generalist engineers who understand system-wide paths will have advantages over depth-first specialists. The ability to orchestrate multiple agents, chain them effectively, and generate better team output becomes increasingly valuable for organizational growth.
Despite predictions that developers would disappear by 2026, demand for developers—particularly senior developers—continues to grow. The role is evolving rather than disappearing. Senior developers remain essential for developing AI systems used by citizen developers and for completing the remaining 20% of work not yet accessible to agents. While orchestration and agent guidance represent new aspects of the role, coding remains fundamental. Senior engineers' responsibilities include training AI agents while also transferring tribal knowledge and understanding what distinguishes better from awesome solutions. As systems grow more complex and companies pursue more initiatives with increased resources, engineers must grasp these nuances to drive organizational progress.
Rust's memory safety and performance characteristics make it valuable, with LLMs trained on Rust producing more secure code due to compiler constraints. However, Rust adoption faces challenges as teams encounter compiler boundaries and increased verbosity, requiring specialists. WebAssembly, initially viewed through a backend lens for bringing power to web applications, now shows significant impact in container and cloud-native environments. Startups are leveraging its small portable runtime to run any language. The vision includes moving server-side computation loads to client-side execution in secure, portable environments, reducing server costs and distributing computational load across regions.
As ground conditions change rapidly, engineering leaders' roles shift toward providing clarity and stability during transition. Leaders must communicate what metrics matter—focusing on delivery and outcomes rather than diffs—and assure teams that AI-validated work counts under their responsibility. Focus becomes critical amid tool sprawl and hallucinations; leaders must prevent cognitive overload by helping teams avoid solving identical problems through multiple directions. Psychological safety emerges as paramount as teams navigate AI adoption. Engineers experience fears about job security and role changes. Leaders must acknowledge uncertainty while providing assurances about known factors, maintaining open dialogue, and positioning themselves as sources of clarity and stability.
Payment solutions for leading IT companies require balancing security, compliance, and team productivity. JetBrains IDEs and AI solutions offer SOC 2 compliance and on-premises security options for enterprise AI adoption.
Keep JetBrains in your library
Save the videos and channels worth coming back to, and find them again in one place.
