JetBrains GameDev Day 2026
In a Nutshell
Game development requires treating community feedback and analytics as co-development partners rather than noise, while distinguishing between player-stated preferences and actual behavior. Static analysis combined with AI code review catches 33% more vulnerabilities than either alone, and fragment-based version control (Lore) reduces binary file transfer costs by 99% compared to Git LFS. Multi-platform debugging with correlation IDs and protocol-first verification catches cross-runtime bugs that single-side debugging misses, while ECS can be gradually integrated into existing engines without full rewrites.
These notes were generated by AI and may contain inaccuracies.
Creating your own community on platforms like Discord provides developers with a direct line of communication to discuss upcoming features and build audience retention for future projects. Unlike communities existing on Reddit or external platforms, having your own community allows you to pull players in and create bi-directional engagement. Developers have implemented ARGs and game extensions into their Discord servers, and themed their servers after their games to enhance the gaming experience.
Anton shares experience from working at King and VUGA in data-driven environments. The key approach is creating a mix of quantitative and qualitative data. Quantitative data involves tracking in-game events to understand what's happening in the game. This is blended with qualitative outreach by reaching out to players directly, including writing in Facebook communities to ask about their gameplay experience.
The goal is to understand the bigger ecosystem around the player being designed for. This involves watching notifications and understanding what players are experiencing. Even in indie development working on 3D action adventure games, analytics remain valuable for understanding how players progress through the game, what actions occur frequently, and to back up observations from community communication with data points.
Every Discord server Anton joined had channels for sharing pets or companions, with community members sharing dogs and cats.
Jason emphasizes that feedback from players is extremely useful based on how engaged developers are with the game. When developers are deeply passionate about their project and know exactly what they want, external feedback often becomes noise that's difficult to filter. Data points become essential for distinguishing valuable feedback from the loudest, most vocal players on Discord or forums who may push for bad ideas.
There are scenarios where developers don't play their games extensively, making user feedback and one-on-one interaction critical. Jason shares an example from working on a children's game where he interacted with elementary school kids playing the game daily to understand what they liked and found interesting, then led development toward those preferences. This approach worked very well, but caution is needed when listening to random forum or Discord feedback.
Direct conversation with the audience is very important compared to general community feedback from forums. Other indie developers learned these lessons through making mistakes. Having access to a built-in test group like family members provides exposure to the right age group and interest group for feedback.
Gennady discusses how community feedback affects work prioritization as a co-development partner. Community feedback can change the scope of work or prioritize certain items after discussion with the client. One example involved technical support on a project where Gennady discovered through Discord and forums that a player had created their own map because they had played the game so much they lost the excitement of undiscovered places. This led to a decision to expand the map and create more areas rather than adding more activities to the current map.
A key process is distinguishing between feedback that identifies an issue versus feedback that suggests specific implementations. Players might say a boss is too hard and suggest removing it, but the actual issue could be poor controls, insufficient checkpoints, or lack of feedback when the boss hits. The focus should be on explaining the purpose of specific game elements rather than the implementation.
When enthusiastic players become moderators, bug reporters, or community champions, studios often feel they owe something in return. The approach is to give rewards with no monetary value to avoid legal issues, such as special badges on forums, early access to content, special items, or free access to game content. While making special items for super helpful players occasionally caused drama, the consensus is to reward helpful community members.
On the Vanguard MMO project, the team had freedom to create special items for players, including unique items with player names on them. On June's Journey, VIP player clubs rewarded and highlighted special contributors. On a game creation platform, power players who gave extensive feedback saw their suggested features implemented into the product. Players appreciate both vanity rewards and recognition through shoutouts in changelogs, special roles for consistent contributors, or permanent recognition for long-term bug testers.
Some games implemented edit achievements with player nicknames and avatar systems as a way to recognize community contributions.
A significant challenge occurs when player statements contradict their actual in-game behavior. When tracking in-game events reveals different patterns than what players report, the approach involves following gut feelings first, then trying to understand the underlying problem rather than accepting suggested solutions at face value. Sometimes players use different terminology than developers, meaning something completely different than what they say.
Data should not be trusted blindly because tracking events may be implemented differently than expected, and charts may be filtered or set up in ways that don't reflect the actual situation. The advice is to trust the process and dig into the code when data seems contradictory.
On an MMORPG project, tracking showed high completion rates for a new event activity, but surveys indicated players found it boring. Through direct conversation, it was discovered players completed the activity only because it was the fastest way to progress, highlighting the need to communicate more and compare different information sources.
Jason notes that people often say they like things they don't actually engage with, making analytics essential. In one MMO, players claimed to like all three core progression systems, but data showed they only played the adventuring system while another system had almost zero players. This led to stopping work on the unpopular system entirely. Similarly, in a children's game, kids said they liked mini-games but data showed they primarily played pet battle content.
Issues affecting more than 1% of players become high priority depending on criticality. Save file issues require attention even if affecting only one person. Trophies and achievements should work as expected since players will be disappointed if they unlock everything except one bugged achievement. Graphic issues reported by single players may be configuration or software-related and can be addressed later.
When issues arise, the goal is determining if they're isolated or widespread by collecting user computer specifications and reproducing problems. Reaching out to affected players who consented to contact provides direct descriptions of issues. The more widespread an issue, the faster it needs to be addressed.
Community feedback and data analysis should work hand in hand. Understanding the size of affected player subsets helps determine whether feedback represents isolated incidents or widespread issues. Building data pipelines inside games helps distinguish between one-off feedback and broader player impact.
AI and LLM models can process community feedback and close feedback loops. One developer has an AI hooked up to Discord that takes user feedback and automatically implements bug fixes. The system attempts to fix bugs at submission time, reproduce issues, and provide users with test builds. While some clients don't allow AI usage, others permit it for analyzing code issues and suggesting fixes, with human programmers making final decisions on implementation approaches.
AI is actively used for gathering feedback, particularly after a port release. Within a day, ChatGPT is queried for feedback specifically about performance on PlayStation or Switch to gather what people are saying and lay groundwork for next updates and fixes. AI is used effectively for moderation both in communities and in games. GGWP is mentioned as a company that does this, and it's useful for handling large amounts of community feedback. However, AI is just a piece of the picture, and it's important to have humans looking at it and being in the community to get a broader picture of the whole puzzle.
The team is not yet at the point where they would use AI to interact with the community. They handle this themselves. Two concerns are raised: whether AI can pick up on game-specific lingo and understand how game features work together, which can be very specific to what is being built, and whether an automated summary could miss important blind spots. The second concern is that the gaming community is very sensitive to AI use, and this sensitivity is even stronger in games than in customer service contexts. The speaker feels not valued when receiving responses from a chatbot.
When interacting with a community, the goal is to sound human so people feel it's not just a chatbot replying and automating everything, which makes their feedback feel valuable. One panelist has been enjoying AI chatbots for support in the last three months because they resolve issues and answer questions immediately versus having to authenticate with a person who may get confused. However, games have many players who are anti-AI, and the question is whether the benefit of immediate responses will outweigh this concern.
When asked which decision they would never hand over to community vote, dashboard, or AI, responses included: the decision that the product is done, because the speaker wants the responsibility to check everything and see that everything is fine before release; creating the game vision and on the technical side creating the game architecture; game feel, which is a deeply human thing where personal style comes through in how a game looks and feels, and AI cannot be trusted to replicate that, though experimenting with community guiding every decision could be a fun project.
Advice for building a community includes starting by posting online, making videos about development, and sharing the process. Initially, a lot of things being done will be terrible and will need adaptation based on feedback from the few people listening, and the fan base will grow as the game improves and responses are made to that feedback. It's important to figure out who the game is being made for, where those communities are, and spend time in those spaces. Start early and start often, having the community there as the prototype is being built, because the second something is shown off, it could go viral. Make sure the community is posted on store pages, wish lists, and articles to capture the audience.
Beyond Discord, Reddit and YouTube are recommended places to build community. On Reddit, a big chunk of the audience was found, and YouTube also has a lot of activity. Another idea is to collaborate with other developers who have developed games in a similar style or genre and ask them to help promote the game.
Artem is a solution engineer at JetBrains for TeamCity, the CI/CD platform, and has been with JetBrains for about 12 years, most of that time as a release manager. Recently he joined the TeamCity team helping customers build delivery pipelines. The presentation covers a quick intro to TeamCity for game development projects, launching a build of a Unity project locally and on CI, investigating failures, and options to run investigation without leaving the Rider context.
A TeamCity cloud instance shows a pipeline with jobs connected to each other, ending with a deploy step to staging. Features for game developers include support for all agents including Unity and Unreal, and integrations with game tooling. For tests, there is support for unit tests with individual visibility, duration time, filtering by duration, ability to jump into exact test history showing all historical executions, statistics over time, and management flow to investigate tests with random failures or mute tests to ignore results while still executing them.
A Unity build step allows specifying necessary parameters, whether to run tests, selecting player modes, enabling or disabling graphics. Integration with Unity license is available. TeamCity agents are build machines where code is executed. Unity license can be specified once on the TeamCity side and applied to each TeamCity agent so builds execute successfully.
Jobs connected to each other allow enabling build cache for time saving. Artifacts or build cache can be obtained at the beginning of the pipeline and reused in all next steps. Pipeline triggers include being enabled on every new change in the repo, with TeamCity watching for all commits and running builds. Schedule triggers are useful for heavy tasks like performance runs or heavy deployments to run during the night so results are visible by morning.
Adding a new job is done through the user interface by giving a name, adding steps like Unity tests, assigning to the pipeline, and defining logic such as launching in parallel with other checks. TeamCity suggests committing to the repo because TeamCity configuration is stored as code inside the repo side by side with the game code.
After making changes and committing to the repo, TeamCity detected changes and executed the pipeline, which appeared green. The deploy configuration showed a staging URL on CloudFront deploying to AWS. However, when playing the game on the Cloud URL, the background was not available while the vehicle could still be controlled, shot, and driven. This created a situation where something worked locally but not on remote staging.
To investigate without jumping between contexts, TeamCity CLI and TeamCity MCP were introduced. The CLI is a command line to interact with the TeamCity server. After logging in, recent runs in the pipeline can be checked. The project ID can be copied and used to list recent executions. The build ID can be copied to explore job results. The overview shows whether the job was successful, duration, agent used, and link to browser logs. Logs can be viewed directly in the terminal. Searching logs for specific parameters like flavor revealed a game build flavor parameter set to diagnostic, which was likely the reason the background was missing.
The parameter is defined in the TeamCity YAML file inside the repo. Changing the parameter from diagnostic to game, committing with message update flavor param, and pushing the change allows TeamCity to detect the change and run the pipeline automatically.
After triggering a build through the browser, the pipeline list shows the build in progress with commit update flavor parameter. Code review and code analysis have passed, and tests are now running. The build requires time to complete.
There are two approaches for interacting with TeamCity from Rider. The first approach involves direct pipeline management. The second approach uses MCP (Model Context Protocol), where TeamCity provides an MCP endpoint that can be configured in Rider settings.
In Rider, navigate to Tools → Assistant → MCP to add a TeamCity connection. The configuration requires the MCP endpoint and authorization token. When configuring the token, the scope can be selected - either global scope or project-defined scope. Under project scope, specific permissions can be defined such as read permissions for a particular project. This ensures the AI agent operates with limited guardrails and cannot harm the configuration.
In Rider, a chat interface allows interaction with an AI agent (Claude or other available agents). A sample prompt asks the agent to check what happens to the background on staging and why it works fine locally. The agent uses the MCP connection to detect runs and identifies that the bit flavor was set to diagnostic, which caused the misbehavior.
The AI agent provides detailed explanations and detects that the fix has already been committed. The user can ask the agent to watch the pipeline and provide a link once the build is deployed to the cloud environment. The agent monitors recent pipeline status and provides staging environment links.
The pipeline shows approximately 50% completion with the web gel in process. The next check occurs in 30 seconds. Once completed, the fix can be verified by copying the staging URL and testing in the browser.
After navigating to the staging environment, the moon is now available. Vehicle selection works correctly, and the behavior matches the local machine. The fix successfully resolved the issue, allowing vehicle control and map exploration.
Two methods exist for investigating issues directly from Rider without switching contexts. The cloud notifies when the build finishes successfully. This approach allows staying within a single interface to resolve issues related to continuous integration and infrastructure.
TeamCity Pipelines use a YAML-based approach similar to GitHub Actions for quick configuration with good visibility. However, TeamCity offers game development specific features including support for all game agents out of the box, build cache, Unity test support, ability to mute and investigate tests, and support for complex multi-platform deployment pipelines.
A poll during the session revealed CLI as the top method for interacting with CI systems. This makes CLI integration valuable for working with MCP and AI. TeamCity includes an assistant for configuration help regarding dependencies, settings, and build features. Additionally, an AI build analyzer examines build failures using context from build configuration and repository to provide explanations and recommendations.
Builds can be hosted within TeamCity or configured with external storage such as S3. Pricing depends on configuration, with a calculator available on the web page to specify artifact sizes and determine costs.
Three deployment modes exist: on-premises installation running on user infrastructure with full self-management; cloud offering with hosted agents including Windows, Linux, and Mac machines; and hybrid mode where TeamCity management runs in the cloud while agents execute within user infrastructure, keeping code on premises.
Kubernetes integration allows automatic scaling of build agents. When jobs require execution, agents spin up in Kubernetes pods. When no jobs are running, agents close automatically, eliminating charges for idle resources.
The new pipelines feature improves understanding and configuration of build processes. Contact information for further discussion is available at jetbrains.com/teamcity.
The presentation covers evolving a live game engine by introducing Entity Component System without requiring a full rewrite. The focus is on integrating ECS into an existing engine, identifying obstacles, and overcoming them properly.
Alex is a game developer specializing in game logic standardization and game engine development with ECS experience. He led a custom ECS project for World of Tanks at Wargaming, worked with Unreal Engine's Mass Entity ECS framework at Splash Damage, and currently works at Meta as a game developer.
ECS focuses on batched processing of composite things with three main concepts: Entity, Component, and System. An entity is a lightweight identifier available through API that holds no logic itself. Components serve as data holders without logic, functioning purely as composition elements. Systems integrate logic by performing batched operations over entities matching specific criteria based on component presence, absence, or optional presence.
A recommended mental model views systems as integration points for logic performing operations over specified components. API implementation varies by library - some require manual traversal while others provide more abstracted interfaces. Systems function similarly to shaders, performing operations on all objects satisfying defined criteria.
ECS provides logical scalability through composition over inheritance and enables extending logic by writing new systems without affecting existing entity structures. Performance benefits arise from cache-friendly memory organization through archetype models where components of the same type are stored in continuous arrays. Multiple systems can be reorganized in parallel graphs based on component access patterns. Individual systems can execute in parallel using parallel-for operations over entities.
For engines with long histories, ECS-based approaches may unify gameplay setup methods across different existing approaches in editors. This unification is not a direct consequence of ECS but may improve tools when implementing such changes.
The engine may already be in production with a live game as a service, or the game may still be in development without release. Both scenarios present different considerations for ECS integration.
When introducing ECS to an existing engine, several engine models may already be in place. The engine might have deep inheritance hierarchies of actors with logic, a recursive data structure similar to Godot, actors with logic plus components like Unreal Engine, or game objects plus components like Unity. Existing content, tools, and code base that manipulates existing types must be considered before any rewriting occurs.
The core of the engine is assumed to be written in C or C++, though most integration decisions are not directly affected by this choice.
Before beginning any rewriting, goals must be established. The first consideration is whether ECS will be used only as a performance booster, or whether it will serve as a unification and replacement for how game logic is written, shifting to a new paradigm entirely. A critical decision involves how artists will be affected—whether a completely new workflow will be introduced to them, or whether the changes will be hidden so they remain unaffected.
The scope of integration varies based on the goals defined. Several crucial milestones cover most integration work. The ECS core must be implemented, which may involve using an existing GitHub library or implementing it internally. The core requires an entity component system, a memory model representing components properly, and a wiring system that links components of all necessary entities and passes them into systems.
Integration with the existing world represents one of the biggest challenges. If artists need to communicate with ECS, editor and serialization scope becomes necessary. For multiplayer games, networking considerations arise regarding how the ECS scene will be replicated. If the engine has scripting, decisions must be made about how scripting will interact with ECS.
The first approach involves grabbing a library and placing it to the side, writing all new logic in ECS. However, the old world remains unchanged, creating significant problems. While wiring the library to the existing ticking graph is straightforward, the lack of integration with existing code makes it virtually impossible to write any ECS-based feature that needs to interact with existing logic. In the worst case, ECS cannot be used until everything is rewritten from scratch.
Instead of isolated integration, gradual integration with existing code is the recommended path. Existing options will be exposed to ECS, new ECS components and systems will be added to interact with existing code, and responsibilities may be partially moved from the old world to the new world when necessary. Each integration step must support useful gameplay, avoiding postponement of the moment when ECS becomes functional within the engine.
The first gradual approach involves porting isolated islands of logic. Engines with age may have several component models already implemented. Some component models may be deeply entrenched in engine subsystems, making porting impossible, while others may be more localized and represent gameplay logic suitable for porting to ECS. Isolated pieces of actors that do not interact heavily with other subsystems can also be identified and ported.
During porting, components are added to entities. Existing component code bases are reused by declaring these components as ECS components added to entities. When scene representation includes actors with logic, actors themselves may be treated as special components. Additional ad hoc systems must be written to access methods within these components for operations like triggering.
Porting works if components are easy to move to ECS, particularly if the ECS library allows arbitrary class types as components. Ownership changes from actor to entity with minimal code rewriting. However, if actor or component lifecycles are deeply entrenched in the engine, especially involving custom memory allocations or allocators more complex than standard new operations, extraction becomes difficult.
Object-oriented component models often store pointers to other components, including raw pointers. Performance-oriented ECS implementations are unsuitable for storing raw pointers because memory is organized as packed areas. When holes appear in packed areas, swap remove operations move the last component into the hole, leaving pointers pointing to invalid memory locations.
Solutions include rewriting all pointer access locations, replacing raw pointers with shared pointer data structures that hide raw pointer access by storing entity references and finding appropriate component types, or adding support for multiple memory models within the entity component system.
API design for systems can avoid exposing memory model details. Systems simply request access to component types A, B, and C without memory model information. Buffers storing components may differ—new ECS components stored tightly packed while old components remain scattered throughout memory. This approach may reduce performance but enables gradual migration.
The bridge access approach avoids directly porting components. Instead, a mapping from actor to entity is created, where each actor in the scene has a corresponding entity association, though some actors may remain unassociated depending on requirements. The system API can be extended to allow both ECS components and actors or actor components to be mentioned together.
Bridge access uses component holder components as proxies. Special holders store pointers to existing components A, B, and C, while a special actor component holder stores the pointer to the actor. This approach avoids touching existing actor implementations entirely, which proves useful when actors are deeply entrenched or when engine modification access is limited.
Unreal Engine's Mass Entity ECS demonstrates this requirement when accessing actors, as porting all actors to ECS is impractical due to their deep entrenchment in the codebase.
Both approaches enable connecting ECS and non-ECS worlds. New ECS logic can interact with non-ECS code through mixed systems accessing ECS components, actors, and actor components simultaneously. From the developer's perspective writing final system code, there is no difference between these access patterns.
New ECS logic can be added to existing worlds. Performance optimizations can rewrite actor components to ECS without changing the calling code.
Writing new systems and components in ECS achieves the same performance as standard ECS since they operate in their own ECS world without interacting with existing code. Access to existing actors or actor components typically results in cache misses since memory arrangement is not guaranteed to be cache-friendly, potentially causing scattered heap storage and performance degradation compared to clean ECS.
Schedulers can build complex dependency graphs within ECS systems, making it straightforward to write systems working with both ECS and non-ECS worlds. However, existing actor codebases may not support multi-threaded access well. In Unreal Engine, actor components often cannot be accessed from non-game threads in many situations, forcing mixed systems onto the game thread and limiting ECS performance benefits.
Component data can be copied in special synchronization systems. This requires determining the source of game data—if data is copied from actors to ECS and then modified on the ECS side, synchronization back to actors may be necessary. Ownership responsibility must be established when data differs between representations.
This approach becomes necessary when non-ECS worlds need to access ECS data for backward compatibility reasons. The strategy involves actors with components ABC and entities with different components AB, connected through synchronization systems performing data copying.
When ECS serves as the source of truth, actor mirroring components can store entity handles and find ECS components under the hood during data access. This eliminates the need for additional synchronization systems but requires careful handling when actor mirroring components are used on non-game threads, since ECS memory can be affected by multiple game threads simultaneously.
Networking presents two main options. Existing network replication can be reused by employing the mirroring pattern to use actors as bridges for network replication. This approach is easier but adds data copying layers and increases factor count when replicating many entities.
Alternatively, a fully isolated ECS replication layer provides greater benefits and control, enabling batching of data transmission from ECS components. However, this resembles developing new engine network replication from scratch, creating resource challenges and requiring two competing network replication models.
If existing network replication is abstracted from actors, ECS components may attach to the abstracted network system. Deep entrenchment of network replication with actors increases difficulty.
Scripting support may be omitted when ECS serves primarily as a performance boost. However, scripting becomes essential when ECS replaces the fundamental logic writing approach, especially when scripting is already heavily used and ECS access is unavailable.
Three considerations guide scripting decisions: target audience qualification, whether artists and game designers with less technical knowledge will use scripting, or whether core logic developers will use it for rapid prototyping.
The first approach hides all ECS patterns, presenting an object-oriented model on the scripting side with direct one-to-one mapping of C++ ECS components to scripting components, including methods for operations and logic. This avoids exposing new paradigms to existing scripting users but sacrifices ECS flexibility.
The second approach exposes all ECS patterns, including components, systems, and queries, enabling writing of systems that find appropriate components and entities. While performance benefits like cache friendliness and multithreading cannot be achieved in scripting, logic writing flexibility through conditional systems becomes available.
The third approach combines ECS pattern exposure for system writing with allowance for conditional logic within components. This mixed model is less verbose since scripts often handle additional setup for concrete game instances, where writing systems for each corner case would be impractical. Callback methods like onEnterRole, onLeaveRole, and onTick become possible despite deviating from pure ECS principles.
Authoring tools offer two approaches: integration into existing component editors or introduction of completely new editor flows. From an artist's perspective, no substantial difference exists between ECS and non-ECS component models—component addition appears identical regardless of underlying implementation.
Integration can dispatch components appropriately: old actor components added to actors, new ECS components added to ECS counterparts. This approach fails when editors lack component model support, such as actor-based models without components, or when ECS entities exist without actor counterparts.
You may modify your existing editor, but it may be not so easy. In this case, we will introduce a new authoring tool, which will, of course, because it's a new tool, it will give you grants more freedom of implementation in your editor. And you can make it fancier, more beautiful, more modern, et cetera, et cetera. And this may be the only solution if you don't have component model within your engine at all. Obviously. But you need also to understand that this introduced different features. if you want to have a different authoring flow now, and now you have to teach your artists about the new flow and describe them when they need to follow the new flow and when they need to follow the old flow. So this is a consequence.
Rollout is an exceptionally important part of all of this, because it determines whether all our endeavor, whether it's successful or not. And each stage needs a usable result and a way of verifying this result. And without proper rollout, we just won't have any kind of success.
In the first stage of rollout in our ECS, in our new engine, maybe suddenly not rolling out of the ECS itself, but we may, for example, roll out initially only the authoring tools like editor modifications, without, without backend. What can be the reason of this? We can test, especially if the flow dramatically changes from the existing flow of manipulation of options. We can test whether artists understand this properly first. And second, it allows us to, for example, have some intermediate layers, which will save the data from new authoring tools into old formats, which allow you, for example, to avoid modification of the, of the binary, which is distributing, distributed to clients, which itself, like, makes, makes it a lot less dangerous, because otherwise we're talking also about modification of the ECS itself itself. Oh, sorry, the modification of the binary, which, of the game itself, which is being distributed and so on. So this, here we are testing the, of the tools, first and foremost.
Then we are talking about rolling out of ECS core finally. Yeah. Here we will roll out the integration with the world either. So we can test that existing logic, which we have ported to ECS. It is not broken. We do not write new logic here. We just test that a presence of ECS core doesn't contradict the existing world, doesn't contradict the existing engine. And we do testing for regressions here. First and foremost.
Then we can do eye candy features testing. So we can do features which enhance appealing of the game. So at the same time, because it's about eye candy, they won't break any kind of complex gameplay related subsystems. So in worst case scenario, we can just turn it off. And the main purpose of this stage is to test new authoring tool workflow combined with the ECS core. We test both whether our artists are capable of producing content, and whether this content has been produced properly, and whether our engineers implement new features with ECS properly as well, and whether all of this combines together. So this is real testing ground of the new flow.
And the last part. Ta-da! Gameplay features. If our game is a network game, we also are talking about network integration rolling out. And here we write complex gameplay features with complex long-term consequences. And here we do full stress test of everything we have implemented before. And we're happy when everything is okay. I hope everything is okay in this stage. Because otherwise it's not very good. And here, essentially, at this stage, we may say that... Yes, and here we understand that, well, we have achieved our goal.
ECS can be integrated into existing engine. It will require more efforts from you than building it from existing engine, potentially. At the same time, integration work will be noticeable. So it's not for free, but it's possible. You must avoid every synchronizing approach. So if you sit and wait till you rewrite all your engine to the new ECS, and before that you won't release it, you just will do it never. And you should understand this. It is unlikely that you will get rid of old approach to 100%. This may be unfortunate, but you should just understand and agree with yourself that you will never reach such kind of state. But to be honest, it's not necessary. Because all of this is a gradual evolution. You gradually evolve your engine. You gradually evolve it to a new paradigm. And at each point, because of this, at each point, you can, for example, stop and say, okay, we don't proceed further. Or you can say, okay, we invest more resources. When we make our life even better, more performant, more scalable, et cetera, et cetera. So this is evolution.
We've had a number of questions in the chat. And I think one of these, the big one which kind of aligned with something I wanted to ask really is, the talk's quite agnostic. So the question was, is ECS supported natively in the main engines? It says from the talk, it sounds like there is a non-trivial effort to maintain and respect the boundaries of a hybrid actor or ECS, especially with many engineers.
We have here a situation that first, it's three directions. We can look at existing third-party engines like Unreal, Unity, for example, I will focus on Unreal and Unity here first. Because they, both of them, have their own ECS integrations as well. Unity have their DOTS stack, and for example, Unreal has their mass entity stack, yeah? At the same time, they have this, let's say, they also face the same problems I'm describing here in the talk. For example, if you try to write something with mass entity in Unreal, you will also have this situation that you need some kind of features from existing actor world and you don't have them in mass entity, for example. So yes, this is in third-party engines. First, they have libraries which allow you to write ECS, but yes, you have these troubles when you need to think about it. Also, when it comes about all proprietary engines, you also will need to do such kind of work. But also one other thing, I have spoken like, I periodically speak with other people within the industry, and what I noticed that a lot of new engines, for example, which have been developed from scratch or they all even not developed from scratch, but they gradually have integrations of ECS. So when it comes about ECS itself, it's gradually being introduced into existing engines over the places and people have it. But yes, obviously, you need to spend additional efforts on maintaining several work representations in this regard. So it's not for free. It's not for free integrating into existing engine. You definitely should understand what's going on. But yes, it's doable.
We had a poll and most people are using OOP, some using ECS, but quite a few doing hybrid approach. So would you recommend the hybrid approach or would you prefer pure ECS? Wonderful, wonderful question. So if you are starting building a new engine from scratch, and again, I would highly advise to stick to ECS approach at least in its core. Again, you can, and again, this is something I'm observing like while speaking to these people with industry as well. So in this case, I would advise build the ECS core. You always can, for whatever reasons, wrap it into more object-oriented approach for whatever reasons you face in the future, because sometimes it may be more useful. You can always wrap it up. But under the hood, stick to ECS, because again, ECS itself, what's very funny, what I mentioned, it's about budget processing of objects, of things. And this is, if you just look from high perspective for the code base of a game development code base, this is the way how we were trying to write a game engine code base for quite a lot of time, even before ECS. ECS standardizes this way of writing. And again, it is more useful for modern architectures. So I would advise stick to this. Again, if we're talking about engine, which are already present, you just cannot rewrite it fully. In this case, you will stick to some hybrid approach similar to what I have described.
Do you think the best reason for ECS is the performance reason? That's the one which kind of springs out immediately to me. But if you want to, if you're suggesting you should perhaps even have a core of your engine as ECS, is that for performance reasons? Or do you have other main reasons in there as well? Yes, that's a wonderful question. So see, first of the reasons why people mention ECS, of course, it's performance because it allows you to organize the code base in a more friendly way and a more suitable for multi-threading play as well. But at the same time, it gives you opportunity of writing more flexible game architecture because of this budget, because of this because you, again, your way of thinking changes a little bit. You say, okay, my scene consists of entities and components and I'm inquiring in this piece of logic, I'm inquiring entities satisfying this criteria. Here, I'm inquiring objects satisfying this criteria. And it gives you additional layer of extensibility of the way how you write this kind of logic. So I would say architectural benefit here is also being achieved as well. And in mixed hybrid approach, this essentially will be the first benefit here because performance will be, you know, in a hybrid approach, performance benefit will be a little bit harder to achieve for everything. Like for subsystems really in ECS, of course, you will have it, but for intermediate situations will be different. And again, as I mentioned, it's possible to write a script on scripting site in ECS paradigm as well. And of course, you won't have performance benefits here, but you will have flexibility. And you just have to teach everybody outside of the dev team how to use ECS, which is awesome. Yes, but this is, this is literally true. But again, the funny thing is if you ask yourself, like, how guys you were writing, how folks you were writing before, suddenly you may discover that people tend to write some kind of managers which grab objects with their bespoke ways and do some kind of operations. So suddenly they were trying to write something ECS like but without ECS. So and and suddenly and then click and then they understand that it's actually the same factors. Exactly. very familiar with adding a component. So it does make sense.
Unfortunately, we're going to have to wrap up there and move on. There's still a couple of questions left in the chat. I wonder if you could pop into the YouTube chat there and see if you can answer them there. There's a another good one, especially about the convincing executives to move on to an ECS system. Exactly. It's a whole other hour. That one is. This is good. Alex, thank you very much. Thank you for we can move on now. Yeah. Let's bring on Kai, a member of JetBrains Cadent team with the talk called Code Quality Never Changes How Static Analysis Can Help in the Age of AI. Hi, Kai. Hey. Hello. Good to see you both. Nice to have you here. Always good never changes never uh it's always relevant yeah it's probably not as sexy as ai and all that um but it's still very very relevant and i will explain why in my presentation we'll be like quality guy we can see us see your slides um please take it away right thank you cool yeah thanks everyone for listening um to me talking about code quality um just full disclosure up front i'm not an engineer i'm from the business side of things but i promise i'm not going to show a single marketing slide and i'm around in this area for almost 20 years so even i can't have to pick something up here or there cool but going on code quality um what am i going to talk about today first we are going to define what we actually mean with code quantity then we have a quick look about at how it impact business outcomes where ai comes in then ai's potential impact on quality and how to safeguard said quality as well this talk is fairly generic so means this is not geared to one one particular solution in this area so you can take uh the methodology is described in here and use it for all sorts of different solutions however at the very end if you have time i will also show you an example how that looks like with katana code quality um again is a topic that tends to get overlooked at times especially in in game development where you might have other issues on your plate however speaking with a lot of you at for example at a game the weather was conference this and last year there is an awareness that this is an important topic however i also found that there doesn't seem to be one particular set of industry best practices around this area in specifically in the gaming industry and maybe their gaming can learn a little bit from how it works in other industries and i will share some of these best practices here in my presentation code quality so what do we mean actually with that um it's interesting if you find these things interesting but for such an important topic it is actually very fuzzily defined so if you speak with 10 of your peers what do you mean by code quality it could very well be that you get a lot of um different answers however for the sake of what we are talking about today i want to want you to focus on a couple of different points the first one would be correctness fairly straightforward so does my code does what what i designed it to do then performance super important obviously for for gaming um is it fast enough stability um is it fast enough stability um uh always a big topic doesn't have to be an ici or there is no c or there isn't a to go or there isn't a to go or there isn't a to go answer it's but that's really important because it's a critical period of time for focus on what we're talking about today i want to share a few different questions that. does it crash does not crash and how stable it is for my players security obviously always very important as well so you don't want to show up at in in an industry newspaper because you just got hacked and maintainability so just does my code work today or is it going to work in a couple of years as well especially with some games no going on for 10 12 years this is something that definitely should be looked at and last but not least reusability and this is a topic where we can that we that i now see more and more when i speak especially with larger studios that can i reuse the code that are already developed for other projects as well and all these kind of give you an answer an overview about what when i talk about code quality and what a lot of my peers speak about code quality which points we actually want to cover with this now if you go in onto the wide west that is um x or linkedin for example you can see that if code quality is important is kind of a hotly debated topic so from one person's saying i don't care how my code looks like it's just as long as it works to the other extreme where someone says no my code needs to be beautiful and spotless before i release it and then anything in between there are loads of different opinions out there obviously we are biased but we think in the kadana team that code quality can have a real impact on your business and on how your game is received To give you a few figures here, and I'm sure you know all of them already, but there was a AAA studio that lost 33% of its value a week after a buggy launch. I'm pretty sure we all know which one that was. Then 70% was cut from GTA online load times by just fixing two small code flaws. So the time to load was cut from six minutes to two minutes. And by the way, those code flaws were actually fixed, not by the studio themselves, but it was fixed by a player. And then Dark Souls was offline for two months because there was a remote code execution flaw, so a security issue. And you can imagine, first, if you lose players over these two months, most likely they're not coming back and also real life impact on revenue. So that all said is, then in games, bad code does not only stay on your side, so it's not something that is there, it's annoying, but it doesn't really hurt the business, but it lands with your players, meaning that can cause refunds, bad reviews. Also always a topic when I look at something like IGN or PC Gamer or any of those that are out there, quality, bugs, how stable is it? Well, I think it's always influencing the score as well, and the score as well will also influence that, you know that better than I do, how many players you will attract to play your game. That means buggy games will also, last but not least, lead to disgruntled players. I mean, there are some examples of big studios who seem to be able to get away with it. If we have a look at something like Bethesda. But this is something you really need to look at. You really shouldn't risk. Then moving on, why is that, that sometimes bad code is getting released? Again, in a lot of conversations, I find that just the pressure of getting your game to market and getting updates to market is immense. So scheduling is a real issue for a lot of studios. And that, for example, is reflected in the number of on the percentage of game developers who report that they had to do crunch periods or extended hours, which is over half of you guys. That means and that entices a lot of studios to figure out how they can use AI to help with this topic. And again, as I said, AI, especially in game development, is a very important part of the game development process. It's a very hotly discussed topic. However, 95 percent, for example, of developers already use AI at work, and most of developers are using AI or assisting for assistance in the coding process. So this shows that AI, if it is used correctly, can be a helpful tool because what you can do with it, for example, is you can. You can. You can prototype much faster, so try out a new game mechanic, whereas in the past you might have had to spend a couple of months. You can you can now try in a couple of days, and if it doesn't work, you just throw it away and try something else. You don't have to spend so much time on border plate issues, so you can leave that to the AI to figure that and sort it out for you. And this resides in an increased efficiency. So here. If you use correctly, from my point of view, the goal is not necessarily to cut to cut developers from the teams. This is something you shouldn't do, obviously, because you should just enable developers or enable your team to use AI to be able to do something much more interesting than doing boilerplate stuff. However, this speed and. And the usage of AI comes with a potential downside when it comes to code quality. So AI can generate a lot of code real fast. However, AI code is not necessarily better or worse than human generated code, but it would just exasperate the amount of issues that would go into your code. So you can imagine where in the past you may have written a couple of hundred lines now in the same time. AI can write 10,000 lines. And if you have the same average amount of issues in your code, it's kind of self-explanatory where that issue is coming from. And that leads again to real life issues in terms of, for example, slow code. So is my game up to scratch when it comes to performance? Does it crash or is it stable? And is it exploitable or does it has potential security issues. The good news and the bad news at the same time is that it said AI code is not necessarily worse than human generated code, it is if you have a look at studies, it is kind of on par, more or less when it comes to the overall amount of issues that it creates. But it fails. In different ways compared to human generated code. So, for example, when it comes to logic and correctness errors, it does 1.7 times more, it creates 1.7 times more issues than human generated code when it comes to general code quality and maintainability issues, coming back to the issues that we are to the focus areas that we spoke about, it is worse and most concerning. It will also. Eventually create more security issues than a human developer would do. This is coming from a study by CodeRabbit, so CodeRabbit, I'm sure many of you have already heard about them, they are an AI powered code review tool, so meaning this is not coming from us trying to make AI look bad, but this is actually coming from an AI vendor themselves. The overall question. It's not necessarily what kind of issues are getting created by whom. So created by humans or created by AI. But the only question that we pose is, is my code overall up to scratch and does it meet my quality standards or not? That is exactly where static analysis can come in to help. So if you just take the points and regarding code quality that I presented at the very beginning. The good news is again that you can use static analysis to basically cover all of these six areas. So when it comes to correctness, then you have inspectors just standard inspections. You can also check for code coverage thresholds. So code coverage again becoming more and more important. In not only covering, but in how to use them as well. Then performance is an area that static analysis can cover. Stability, obviously a couple of examples would be null safety, resource leak, exception inspections. Security, meaning security inspections and SCA, so software component analysis. Meaning checking for outdated dependencies. Plus data. Doing taint analysis as part of your static analysis to find more complex security issues like cross-site scripting, SQL injections and so on. Maintainability. So here a very important topic is complexity and code smells and complexity. And I would also add code duplication on top of it as well. So this is something that AI is very well known for. That it really likes to do code duplication. And what it really doesn't like to do, if you do not specifically tell it to do, is refactoring. And if you do not have a way to keep that in check, this will really hurt your maintainability going forward. Because nobody will be able to understand your code anymore at some stage. And the same with reusability. So do I have duplicate code, which makes it more complicated. And also static analysis can check if your code adheres to your own internal coding standards as well. Coding standards in order to have the same approaches regardless of the project that you're working on. Resulting in being able to share your code much easier. On one side note, just to quickly add why. AI creates this kind of code and these kind of issues. And the most common reason is simply because it got trained on outdated sources. So as we all know, basically the sources are all sucked dry. So there isn't really a lot of new stuff coming up. And this is one reason why, for example, AI really likes to put in outdated dependencies and similar things. Moving on. Static analysis. I want, before I go to why you should use static analysis in conjunction with AI. Quickly explain the advantages of using it. So again, the downside is not really sexy. Because this kind of analysis is around for ages. But it is around for ages. For very good reasons. So one is the results are deterministic. So if you give static analysis to the same piece of code 10 times, it will give you the same set of results 10 times. And also, by the way, if you run it 10 times, it doesn't cost you any more or less because static analysis doesn't consume any tokens. Whereas if you, and I'm sure at least I made the experience, if you give AI the same piece of code 10 times, you will get 20 different answers out of it at the other end. It is fast and cheap. Again, doesn't consume any tokens. And it's usually also very fast. So meaning if you compare an AI review tool or an AI scanning tool with static analysis, usually the static analysis is much faster. Last but not least, it is auditory. So coming back to being deterministic. You can tie every result that static analysis delivers to you. You can tie that to a specific inspection. You can see why that result was delivered to you and why that is an issue. Also on a side note, this will become more and more important with more and more compliance topics coming our way. One big one, especially for European-based vendors or not even for global vendors who make their code and their games available in the EU.
The new EU cyber resilience act is already in effect for some areas and will come into full effect in November. Part of this act requires creating an S-form or software build of materials. Using AI makes this difficult because AI-generated work is not auditable. Static analysis provides a one-to-one relationship between the result and the source, making it suitable for compliance requirements.
Static analysis excels at catching run-of-the-mill issues by checking patterns like whether A plus B equals C. It is fast, auditable, and serves as an effective sanity check early in the development cycle. However, static analysis cannot understand the intent of code because it only looks for patterns.
AI analysis complements static analysis by reasoning across files and across projects, checking for logic-related issues that static analysis cannot detect. The recommended approach combines both tools: static analysis to cover the base cases, then AI layered on top for deeper analysis.
The workflow follows a specific loop. First, the human writes code or uses AI to assist with writing code. Second, static analysis checks for issues and can create quick fixes automatically. These quick fixes are not generated by AI, making them fast, cost-free, and reliable. Third, AI reviews remaining complex issues like logic problems or refactoring needs. Once AI completes its work, the code returns to static analysis to reconfirm it passes security and code quality standards before merging into the main branch.
Studies demonstrate advantages of combining static analysis with LLM calls. One study showed token usage or LLM calls decreased by 72 to 97% while actually increasing code quality. A second study found this combination reduces vulnerabilities in LLM-written code going into the master branch by up to 33%.
Kodana supports the most common gaming engines out of the box. In the demonstration, a banking frontend written in Java was wipe-coded in minutes. The first Kodana scan found 126 issues. Quick fixes were applied through GitHub Actions pipeline, with remaining problems pushed to baseline so they do not slow future development. Subsequent scans showed five new issues, with three fixed automatically, leaving only two visible problems.
Kodana integrates with IntelliJ, Rider, PyCharm, Visual Studio, Visual Studio Code, and Cursor. For IDEs without ready-made plugins, the CLI tool remains available. Issues like duplicated code can be reviewed in the IDE, with AI actions available through More Actions menu to fix problems either manually via chat or agent, or semi-automated in the background.
Large companies using fully agentic cloud platforms to write code still incorporate static analysis at the end to verify code quality. The combination of deterministic static analysis as baseline with AI analysis for deeper review is considered effective.
When organizations add multiple tools to their pipeline without review, developers may need to jump through ten different hoops before releasing code. Static analysis should not slow development. Kodana's pull request mode checks only changed files and takes one to two minutes. There is also a distinction between running linting in continuous integration versus running it as you type in the editor, where only the current file gets linted.
Rider has a feature called hooks where every time an agent generates code, it calls hooks in Rider. One hook reformats code while another checks for current problems, performs linting, and reports back to the agent. The agent can then regenerate code to meet coding standards or fix inspections. These hooks are deterministic—the agent is forced to use them rather than merely advised to do so, ensuring execution every time the agent writes code.
There was a case where an agent asked to remove inspection results instead disabled the inspections rather than fixing the underlying issues.
Six-month Ultimate licenses were raffled, including Rider and all profilers (memory profiling, performance profiling). Two questions were posed: which sport combines with Tetra to form Tetris (answer: tennis), and which game engine was used to build Disco Elysium (answer: Unity). Winners were instructed to email [email protected] to claim licenses.
Tom Lowman creates Unreal Engine tutorials and courses focused on C++ and game optimization with 15+ years of Unreal Engine C++ experience. He teaches indies and AAA studios proper C++ usage within Unreal Engine, including Unrealisms specific to the engine.
A setup guide covers configuring JetBrains Rider with C++ and Unreal Engine, including required build tools and specific Unreal components. An errors and troubleshooting section addresses common issues encountered during setup.
Project Orion is an open source C++ co-op action roguelike sample game for Unreal Engine 5. It serves as the reference project for the Unreal Engine C++ course. Features include an action system, abilities, enemy monsters with behaviors and ranged attacks, pickups, power-ups, a save game system, and data-oriented design for spawning thousands of coins and projectiles efficiently. Documentation explains these features, and source code is available on GitHub.
Rider integration with Unreal Engine proves valuable for teaching. When binding to a delegate trying to connect to component begin overlap, Rider warns when the function lacks the UFUNCTION macro. Adding the macro resolves the warning and exposes the function to Unreal's property system.
The course builds Project Orion while explaining every code decision and the reasoning behind specific implementations. It covers traps and tricks, debugging game code, and handling compile errors in realistic game development scenarios. The goal is developing independence so students escape tutorial hell and feel confident implementing features without uncertainty about best practices.
Assignments challenge students to explore engine source code with hints while building gameplay features. The course was completely rebuilt and relaunched in Unreal Engine 5 with all versions updated to include latest Unreal Engine systems introduced over recent years. Existing students receive this upgrade for free, reflecting a long-term commitment to the course. Over 50 professional game studios use the course for training and onboarding.
Sami introduces the session focused on connecting open source tooling, modern IDE workflows, and local agents with cloud language models directly into the game engine to create an AI editor-time co-builder. The presentation addresses real production engineering challenges rather than hype or gimmicks for indie developers, students, technical artists, and world builders.
Sami (Sharmishta) is a game designer, developer, and QA professional with experience across console, mobile, and indie development. She completed her master's degree at RMIT University in Melbourne, where her research focused on virtual reality and gesture-based interaction design. Her background in QA and system design drives two core priorities: developer ergonomics and rock-solid performance. She emphasizes that technology must function reliably in production builds rather than just appearing impressive in short demonstrations.
Building games in small teams or solo development presents four major constraints. First, handcraft burnout occurs when placing assets into 3D worlds individually, draining creative bandwidth from mechanics, systems, and gameplay loops. Second, SaaS and budget barriers lock developers into expensive monthly enterprise subscriptions or heavy server clusters, whereas indie pipelines must run on consumer-grade or mid-tier hardware with zero subscription fees. Third, traditional procedural generation using simple Perlin noise creates the "soulless void" - repetitive worlds lacking landmarks, natural choke points, and paced gameplay, while unconstrained noise generates chaotic, unplayable geometry. Fourth, environmental QA bottlenecks require days manually inspecting chunks to locate floating trees, broken collision, or impossible geometry.
Godot 4's dynamic scripting struggles with iterating through hundreds and thousands of vertices and coordinates in real time, making C-Sharp using the .NET framework essential for PCG core development. High-performance geometry threading generates terrain meshes using Godot's low-level ArrayMesh and FastNoiseLite, with heavy noise sampling, analytical normal evaluation, and vertex coordinate mapping offset to asynchronous background tasks to keep the main thread responsive without editor freezing or UI lockups.
GPU-accelerated batch scattering addresses CPU scene tree overhead by calculating candidate positions, checking slope limits, and serializing transform matrices directly into a packed Float32 array buffer fed into a MultiMeshInstance3D, batching over 30,000 environmental instances in a single GPU draw call. Managed proceduralism via authored anchors prevents the soulless void by defining authored Marker3D nodes in scenes, using C-Sharp math to calculate vertex distances to these anchors and applying linear interpolation to flatten dynamic noise into smooth foundations for roads, dungeons, and tutorial hubs while automatically carving vegetation and clear zones.
Web-based LLM models like ChatGPT, Claude, or Gemini exhibit "AI runtime blindness" because browser models lack access to active scene node hierarchies, script dependencies, and node names, leading to immediate null reference exceptions. These models are trained on older Godot versions, producing Godot 3 syntax like yield, KinematicBody, or old connect signal signatures that fail in Godot 4 C-Sharp code. Silent runtime failures occur because web chatbots cannot read engine console or runtime debugger output, creating mismatches between collision masks, array bounds, and actual editor states.
The pipeline implements a dual agent architecture using Rider IDE with dedicated editor bindings via C-Sharp syncing. The local AI runs Ollama on localhost:11434, while cloud Gemini 1.5 connects via the Rider plugin. The Godot MCP plugin serves as the engine context bridge, running on WebSocket server port 6505 to expose live scene hierarchies to both AI instances. The custom C-Sharp PCG core handles ArrayMesh construction, vector math, normal calculations, and Marker3D height blending, while the MultiMesh3D pipeline batches foliage with custom Godot shader code for real-time procedural height, slope, and biome texture blending.
Rider was selected for its lightweight, fast processing and heavy procedural generation requirements. C-Sharp on modern .NET executes mathematical operations significantly faster than dynamic scripting. Rider's dedicated Godot plugin (Continue plugin) provides deep engine integration for instant error checking and solutions without reloading. Advanced IDE profiling includes performance profilers and memory allocation trackers essential for managing memory spikes when scattering 30,000+ trees or 50,000 elements in single GPU draw calls.
Cloud AI (Gemini 1.5) handles refactoring, special shaders, algorithms, and optimization, providing maximum reasoning power and deep context without consuming excessive local resources. Local AI (Ollama with Gemma 3 or Qwen 2.5 Coder) serves as inline copilot for auto-completion, helper functions, and PCG parameter tweaking, running 100% offline for complete privacy at zero cost and API token usage. Both models access the Godot MCP server via WebSocket port 6505, ensuring consistent scene hierarchy visibility regardless of which model handles specific tasks.
The setup accommodates limited hardware: Intel Core i7 CPU with NVIDIA 1060 6GB GPU requires only 4-8GB VRAM via Ollama, enabling smooth pipeline operation on standardized gaming laptops without high-end PC investments or API rate limits.
An example prompt to the Continue plugin requested updating min and max scale properties to use export property ranges from 0.1 to 5.0 with 0.05 step increments, keeping defaults at 0.2 and 0.5 without modifying other properties. The agent generated precise Godot 4 C-Sharp attribute syntax that avoided rewriting entire files or introducing deprecated Godot 3 syntax, with changes instantly saved to C-Sharp solution and ready for Godot inspector.
MCP installation via NPX (rate-coding-solo/godot-mcp) establishes bidirectional communication by pointing directly to the Godot executable without major configuration hassle. Live scene awareness allows AI to query the server for active scene tree node hierarchies, static content parents, mesh nodes, and verified node paths. Automated debugging loops enable MCP servers to launch Godot editor, run projects in debug mode, capture live console output and runtime stack traces, and self-correct C-Sharp code when engine exceptions occur.
MCP provides six specialized tools including structure analysis, scene exporting, mesh libraries for GridMap workflows, and managing UID references for Godot 4.4+ versions.
When procedural foliage clumped too closely, a prompt instructed the Continue plugin to update the spawn foliage layer method to prevent clumping by maintaining spawn position lists, checking distances between current vertices and previously spawned positions against minimum radius thresholds. The AI implemented distance squared checks instead of standard Euclidean distance, eliminating expensive square root calculations across thousands of candidate positions per loop iteration, preserving frame rates during foliage scattering.
A strict four-stage data cycle prevents garbage data from burdening CPU resources. Rule-based engine access defines explicit .continue rules instructing AI to query Godot MCP on port 6505 before generating any code. Context-aware fetching requires agents to inspect scene trees and project dependencies before grounding script modifications in live state. Zero hallucination execution combines strict system prompts with verified MCP data to eliminate version confusion, barring AI from using GDScript or Godot 3 syntax. Closed-loop verification feeds console logs back into AI context for immediate self-correction when compilation or runtime errors occur.
Strict system rules include: inspector parameters export with noise frequencies at 0.015f, maximum terrain height at 8.0f, random seeds, and Marker3D blend values; engine scene tree and node paths with live verified string paths to static content parents and tutorial markers ensuring zero invalid code path errors; procedural geometry buffers containing ArrayMesh vertex coordinates, counter-clockwise quad indexed arrays, and analytical normal vectors; GPU foliage logic packing Float32 arrays containing 3D transformation matrices with position, rotation, base, and scale data fed directly into MultiMesh3D.
Rules mandate exclusive use of C-Sharp code, Godot 4 Node3D and CharacterBody3D, export-tunable properties via Export attributes, CCW winding for mesh construction, and MCP verification of scene paths before generating references.
Continue plugin workspace rules named "Godot 4 C-Sharp World Generation" automatically inherit across every prompt once saved, eliminating repetitive instruction about Godot 4 usage versus GDScript. This produces consistently clean code for real-time terrain generation, procedural generation, and foliage mechanics.
Dynamic chunk loading memory divides worlds into infinite grids where chunk managers monitor player coordinates, dynamically instantiate chunks within target areas, cull and free out-of-bounds chunks, maintaining steady 60+ FPS. Off-thread mesh generation runs FastNoiseLite calculations on background threads using Task.Run, with completed buffers passed to main thread via CallDeferred to prevent engine crashes from background thread node modifications.
Procedural blending continuously interpolates dynamic height maps around Marker3D anchors, ensuring critical gameplay foundations remain stable regardless of procedural seed changes.
Dual vector math implements geometric cross products for each triangle using edge vectors to calculate uniform normals, creating clean low-poly aesthetics in single-color terrain. For sandy or dirt terrain with desert dune rolling, cross-product edge generation resolves chunk border lighting seams and visible lighting splits.
Slope awareness GPU foliage samples y-components of surface normals, checking if normalized vector y equals 1.0 and normal dot product falls below max slope thresholds to prevent tree and rock spawning on vertical surfaces. Real-time physics extracts collision shapes from generated ArrayMesh, creating shapes attached to CollisionShape3D with explicit concave backface collision enabled to prevent player character clipping.
A single line of code ensures the player character will never clip or fall into procedural terrain. When rendered into a Godot viewport, the foliage distribution appears clean with tree clusters naturally positioned in valleys and flat areas. As terrain rises into steeper ridges, the system automatically keeps slopes clear while maintaining consistent lighting throughout.
The entire chunk system runs on a locked frame rate of 60 plus FPS. In Godot, complex manual wrapping is not required for procedural worlds. A custom shader can be used, and instead of hot reloading, the Godot code can be edited directly in Rider. Pressing Ctrl+S instantly reloads the shader material inside the engine.
When generating procedural meshes, visual bugs can occur with invisible or curl triangles. If an array is indexed or locked into clockwise instead of counter-clockwise, Godot will render it incorrectly with transparent holes. Adding a render curl mode disabled guarantees both sides are drawn, eliminating transparent holes in terrain or glare when the sun hits the terrain.
Inspector-driven controls use AI to structure shader parameters with group uniforms and hint ranges, automatically building clean sliders. Godot allows the inspector to adjust sand, grass, building, noise patch scales, and roughness visually without touching code. The shader code implementation includes Gemini 3.6 with logic to generate organic orange dirt patches, render mode curl disabled, uniform vector 4 for sand grass, patched albedo, and a 3D value noise function based on world position to mask patches primarily over grass.
Noise calculation and vertices array generation occur without thread safety concerns. Tasks run safely returning to the main thread where call.deferred comes to GPU draw call batching. Through multi-mesh instances, 20,000 to 30,000 trees, rocks, and grass tops are rendered in a single draw call by serializing transform matrices into a flat memory buffer without allocating individual tree nodes.
Chunks are streamed based on player proximity and immediately freed when out of bounds to prevent RAM bloating and memory leaks. The final outcome shows smooth terrain with custom shaders running smoothly from warm sand areas through autumn graphs with organic dirt patches. Thousands of stylish bridge trees and boulders populate the landscape.
The entire world was generated and texture populated using editor time AI core builder inside Rider and Godot, built on open source tools running locally with zero manual asset pricing or fatigue. This achieves both scalability of procedural generation and deliberate structure of handcrafted level design.
The first takeaway is the editor time co-builder approach. Stop treating AI as a runtime player gimmick and use an MCP bridge to connect your IDE with Godot. Manage proceduralism and performance, and don't let AI go wild without using rules, marker 2D, and math logic. In the future, more foliage buffers can handle data more efficiently. Currently this is a prototype, but more loops and runtime debuggers are planned.
Performance problems rarely come from one obviously expensive piece of code. Gameplay code often contains many chunks that look perfectly reasonable and may be super cheap individually, but when combined can create situations where expensive features cannot be worked on effectively.
Games don't have abstract performance budgets. Everything must complete within a frame. A frame is a snapshot of the game at a particular moment showing object positions, animations, gameplay state, UI, camera position, and effects. Games produce these snapshots repeatedly to create continuous motion illusion.
Frame rate (FPS) tells how often frames are produced, while frame time tells how long frame production takes. At 30 FPS, roughly 16 milliseconds are available per frame. At 60 FPS, only 8 milliseconds are available. For 60 FPS targeting, approximately 16 milliseconds are needed to produce each frame.
Not all 16 milliseconds are available for gameplay code. Frame production involves CPU work, GPU work, physics, animation, engine systems, audio, networking, and other resources competing for time. Gameplay is only one contributor to frame time.
Performance problems typically manifest as many individual reasonable pieces rather than single expensive functions. Three optimization levers exist: cost, frequency, and scale. Cost involves making operations cheaper. Frequency involves executing work less often. Scale involves reducing how many objects perform the work. These levers are not mutually exclusive and must be considered together.
Rather than micro-optimizing at function level, the approach progressively changes how gameplay systems schedule work. For Unreal Engine, the obvious frequency control point is the tick function. For Unity, this would be updates. Tick itself isn't problematic, but putting code in tick implicitly answers the frequency question as every frame.
A shooter example places the player on a line that can move left to right while shooting enemies. Enemies move in lines toward the player in a queue system where each enemy stays on its assigned line. Enemies spawn at line beginnings and move toward the player. If an enemy reaches the end, the game is over. Enemies behind the line shoot at the player but require targeting and line of sight checks within a cone.
Each enemy needs movement toward the player, distance calculation to the player, gameplay state updates including alive status, and lane progress tracking to determine when the enemy reaches the end.
Each enemy is an actor with its own tick. Every tick performs movement forward, distance calculation to player, attack range and line of sight checks, state updates including health and UI updates, and lane progress tracking. None of these operations appear expensive individually.
Early profiling shows approximately 84 microseconds per tick, which wouldn't immediately concern developers. However, with 50 enemies executing the same cheap tick, the total reaches around 4.2 milliseconds. This becomes significant since gameplay receives only a fraction of the 16-millisecond frame budget.
Four pieces of gameplay logic exist in the tick: position updates, distance to player calculations, state updates, and lane progress tracking. Position updates must remain every frame for continuous movement. Distance to player calculations don't require frame-perfect precision since attack timing includes cooldowns and delays that mask frequency changes from player perception.
Instead of executing distance checks every frame, the operation can be performed periodically. Four frames of illustration show movement continuing every frame while distance checks occur only on specific frames. This reduces frequency without changing individual operation cost.
When all 50 enemies skip distance calculations for three frames and execute together on the fourth frame, work becomes concentrated into single frames. This creates frame pacing issues where average CPU usage may appear acceptable, but individual frames exceed budget causing visible hitches.
Players experience sequences of individual frames rather than averages. Even if average frame time appears healthy, synchronized work onto the same frame can exceed budget and produce visible hitches. Three performance dimensions must be considered: frame time, frame rate, and frame pacing.
Instead of every enemy waking on the same fourth frame, the population is distributed across frames. With 100 enemies divided into four groups, frame one handles enemies 1-25, frame two handles 26-50, and so on. Each individual enemy still performs distance checks every fourth frame, but from the frame perspective, only a quarter of the population executes the check each frame.
After distribution changes, profiling shows improvement from 4.2 milliseconds to 3.8 milliseconds. The optimization didn't rewrite algorithms, introduce data structures, or make individual calculations faster. Work was simply stopped when unnecessary, performed as often as necessary rather than as often as possible, with measurable improvement confirmed through profiling.
The speaker emphasizes that optimization requires profiling before and after changes rather than working blindly. The process involves: profile once, make a change, profile again, and only continue if necessary. This systematic approach ensures actual performance improvements.
The first optimization cut addressed frequency by examining work happening every frame and questioning whether it actually needed to occur that often. This approach reduced execution frequency but still involved asking the same question repeatedly at intervals, leading to the next optimization question: whether to stop asking altogether.
After addressing update distance to player, the speaker examines the update state function, which checks enemy health, updates health bars, and destroys enemies when health reaches zero. The current implementation calls this function every frame, repeatedly checking if enemies are alive.
The speaker presents a table showing that out of five calls to update state, only one frame contained an actual health change (from 100 to 75), while the other four calls occurred when health remained unchanged. This revealed that most calls were unnecessary since they only checked for changes that hadn't occurred.
Instead of continuously asking whether health changed every frame, the solution involves doing nothing until health actually changes, then calling update state only when the change occurs. This transforms the system from polling-based to event-driven, where the health change itself triggers the necessary work.
The speaker warns that events aren't free, mentioning delegate invocation costs, subscription lifetime concerns, and potential difficulties with debugging and ordering when many events fire simultaneously. This approach shouldn't replace everything with delegates, as the number of operations will decrease but architectural complexity increases.
The lesson states: if work only exists because something changed, consider executing it when the change happens rather than checking periodically. This reduced processing time from 4.2 milliseconds to 3.2 milliseconds for 50 enemies, saving another 0.4 milliseconds.
The speaker introduces ownership as the third optimization cut, referring not to C++ memory ownership but to which part of the gameplay architecture is responsible for knowing about something and deciding when it should be processed. This addresses the question of why every enemy needs to own certain work.
The update progress function tracks how far enemies have progressed through their lane and displays this information on UI. Currently placed inside enemy tick functions, this seems intuitive since enemies move every frame, but the speaker questions whether enemies should calculate their own progress.
Enemies belong to lanes rather than existing in isolation. Instead of each enemy independently asking about its position on its lane, a lane system can manage the whole operation. This provides control over the population, allowing decisions that individual enemies couldn't make about which enemies actually matter for specific operations.
The lane system understands how many enemies exist, where they belong, and which matter at any moment. This enables distributing work across the whole population rather than having every enemy check its own progress independently. For determining whether an enemy approaches the end of a lane, only the leading enemies need checking rather than all 100 enemies.
Moving responsibility to the lane system reduced processing time from 3.2 to 2.9 milliseconds, saving another 0.3 milliseconds. The total optimization achieved a reduction from 4.2 milliseconds to 2.9 milliseconds without optimizing math, replacing algorithms, or making individual functions faster.
The optimization approach asks three questions: Does the work need to run? How often? Who should own the work? The speaker started with 4.2 milliseconds and reached 2.9 milliseconds through different decisions about when and where work happens.
The first cut stopped assuming every update needs to happen every frame. The principle is to do work as often as necessary, not as often as possible. Average frame time isn't everything - moving work onto the same frame can create spikes, so scheduling requires thinking about distribution.
The second cut revealed that update state was asking the same question repeatedly when the answer was usually that nothing should be done. Since the exact moment of health changes was already known, the system could react to changes rather than constantly asking for current state. The principle is: don't act when you can react.
Ownership changes who decides when operations should happen. Individual enemies have limited world views, but moving responsibility to the lane system provided knowledge about the whole population. This architectural change enabled more precise decisions through better information spread.
The speaker notes that everything discussed assumed individual operations were reasonably cheap. When work itself is expensive - due to non-scaling algorithms, memory jumping, or large batchable workloads - different approaches become necessary, including data-oriented design, ECS, or MASS in Unreal.
When facing performance problems, return to three questions: How expensive is the work? How often are we doing it? How many things should do that work? Cost isn't always the dimension to optimize - sometimes reducing frequency or ownership scope provides bigger wins.
The cheapest work is the work you don't have to do. Making something twice as fast is good, but avoiding 90% of it is a much bigger win. The best optimization comes before the function even starts executing.
When asked about ideal distribution of work per frame (50% gameplay, 15% engine), the speaker responds that no magic rule exists. The distribution depends on the specific situation and current bottlenecks. Profilers should guide decisions about where optimization efforts should focus.
Regarding batching and whether it affects mechanics or frame rate tying, the speaker notes that optimization topics vary by engine and situation. Batching decisions relate to what players see, and sometimes work can be delayed by one or two frames without player noticing, effectively reducing frame time.
No single gold solution exists for satisfying both low and high-end PCs. The approach involves checking performance on different machines and adjusting schedules based on actual computer data. Sometimes conditions can determine different scheduling based on average frame times measured from the specific hardware.
Converting Figma designs to Unity UI presents significant challenges. Converting the whole screen at once produces interesting results but creates problems. The Figma JSON is very heavy and the prefab itself is stored as a large YAML file. The agent spends considerable time figuring out the structure and often writes scripts for itself. This approach takes a lot of time, uses many tokens, and produces slightly different results each time. The process is slow, expensive, and unpredictable.
Multiple companies have attempted to connect Figma to Unity to reduce UI building costs. Figma has its own official MCP that gives the agent the context of the mockup so the agent can create prefabs by itself, but it has no integration with Unity. Unity AI Assistant builds screens directly in the editor from Figma links and works with both Unity AI and UIToolkit, but as an out-of-box tool, it does not always work as expected on large major projects, and this level of control was insufficient.
Open-source tools like Unity Figma Bridge and paid converters exist, but they are similar to existing converters and the team wanted more than simple conversion functionality.
The team decided to focus on three specific requirements. First, UIKitSync ensures the kit in Figma and the kit in Unity work as one system, with all screens built from that kit. Second, pure manual changes means no fixes are required afterwards. Third, rebuilding prefabs with the agent instead of by hand allows designers to change mockups and have the agent apply changes while keeping company references and existing fixes in place.
The foundation of this pipeline is UIKit. For automation to work well, the kit needs to be structured and predictable. Components need clear names, states should be organized consistently, and repeated elements should be reused instead of copied. To close the FigmaKit to the UnityKit, the agent has to guess. Even without AI, a well-structured and synchronized UIKit already makes UI implementation much faster.
For Figma and Unity to understand each other, they need a shared dictionary. One decision in Figma should mean one specific scene in Unity. Components become prefabs, variants become prefab variants, auto layouts become layout groups, and constraints become anchors. When this mapping is clear, the agent no longer has to guess what the designer meant.
Three simple rules improve results. First is clear and consistent naming where element names should explain what they are. Second is correct use of component and component variants where repeated UI elements should be reusable components, not copies. Third is correct adaptive layout using constraints and auto layouts so the mockup already shows how elements should behave on different screen sizes. These rules help the agent understand the designer's intention, use fewer tokens, and make fewer decisions on its own.
Figma agents or plugins can help with naming, adaptive layout, and finding touch components. A good mockup works like a good prompt where clear input gives more predictable results.
When these rules are ignored, the agent improvises instead of matching the designer's intention as closely as possible. The designer defines the solution in Figma and the agent handles repetitive work, making the mockup the specification. If the structure is not clear, the agent can choose the wrong icon, match wrong prefabs, or rebuild something because the original component was detached. This is not only about AI. A clear and predictable structure also makes the project easier to understand and maintain for the whole team. A frame called frame333222 will not hurt product metrics today, but many small inconsistencies make the system harder to support over time.
Before giving the task to the agent, there is one final check. The team checks naming and hierarchy, removes unnecessary elements, and ensures masks and outer layouts are used correctly. They also check optimization components and component variants, assets, and adaptive behavior on different screen sizes and content. This can be done manually or with the Figma agent, but the main goal is to give the agent a clear and reliable specification before it starts work.
Figma mockups are always the source of truth and Unity prefabs just reproduce them. Unity also keeps the mapping between components and prefabs and between images and sprites. Any change in Figma must be reflected in Unity. For example, updating a widget component in Figma requires updating the connected prefab in Unity while keeping all logic and existing references untouched.
A Figma component finds its prefab in Unity through clear names and stable component IDs. The mapping connects component IDs to specific prefabs. For example, button primary in Figma matches primary button in Unity. If there is no match yet, a skill runs to look for the right prefab in the project. If nothing is found, the prefab should be built from scratch. Users can set up this pair by hand or let the same skill do that.
The base is an existing tool that works with Unity UI and is split into two layers. The low-level layer is the core, which is a simple Unity package that reads a Figma frame into a single tree model and runs it through a sequence of steps called the pipeline. The steps can be extended and customized for specific projects. The high-level layer is the interface, which uses MCP instead of an editor window so the agent can control the core from chat. A normal editor window can be added if necessary without changing the core.
The MCP server has 13 tools split into five groups. The first group handles reading the mockup by loading nodes hierarchy from Figma, getting information about specific nodes, and exporting images. The second group handles building the prefab by getting the list of pipelines and building prefabs from node hierarchies through specific pipelines. The third group handles editing by changing the prefab and getting the prefab as compressed JSON. The fourth group handles assets and mapping by saving sprites into Unity, linking assets to Figma component IDs, and seeing what is already in the project. The fifth group checks connection status.
Six skills operate on top of the tools. Setup project checks that everything is connected including the token, editor, and pipelines. BinFigma Unity looks for prefabs and sprites that already exist in the project and links them to components in Figma. Build prefab builds widgets and screens from scratch. Update prefab applies all changes from the mockup to an existing prefab. Inspect shows what is different between the mockup and the prefab without changing anything. Download sprites pulls images from Figma into the project.
The build prefab skill shows a preview and the hierarchy of the feature screen. Users pick the right pipeline and hand the job over to MCP to build. When the build is done, the agent gets the prefab as compact JSON with only what it needs. After that, it looks at the resulting prefab, finds rough spots, and fixes only those spots just like manual fixes would be done.
Prefabs are stored in YAML, which is hard to read for both agents and humans. The team converts YAML into a format that is easy for agents to work with. Taking the raw prefab file as baseline at 100%, converting to JSON made it bigger at about 135%. Compression steps include minifying, shortening keys while keeping them clear for the agent, storing objects as plain arrays, and removing defaults and null values. This brought the file down to about 8% of original size.
Testing 7 formats including plain JSON, Toon, and custom formats showed the custom format was smallest but only by a few percent compared to optimized JSON. The custom format needed its own parser and the agent worked with it worse than with JSON. The team kept JSON and optimized it through minification, short keys, flat arrays, and removal of defaults and null values, resulting in 92% smaller than the original prefab. Any standard parser reads it and agents have no trouble with it.
Product changes always lead to UI changes. Before, even small layout updates meant manually comparing mockups with existing screens in Unity and transferring every change individually. As products grow, these become ongoing maintenance costs.
To rebuild screens with minimal side effects, the update prefab skill works differently than rebuilding. It compares the current prefab with the mockup and applies only the differences in structure and properties. When designers move elements in the mockup and ask the agent to rebuild the widget, the update prefab skill finds what changed and applies it. The widget updates while scripts and references remain connected and all manual fixes survive.
Real production screens were used to check visual quality of pop-ups, how close results were to mockups, structure and component reuse. Alina measured actual implementation cost and time. Four approaches were compared for building the same screen: by hand, with converter, with plain agent, and with the full setup where converter, agent, MCP, and skills work together. Metrics included human time, agent time, total building time, and agent cost.
Manual implementation carries risk of human error depending on how careful the person is. Converters can struggle to find assets or prefabs when links are missing and sometimes rebuild components that already exist. Plain agents can give slightly different results from run to run, struggle with large screens, suggest manual fixes, and break script references. The tool was designed to solve these problems but is not perfect like any agent-based system, has limits, and some cases still need human intervention.
The tool makes UI implementation faster and removes repetitive work from designers and developers. It works best when projects have at least a basic design system with reusable components, clear naming, and predictable layout. UI implementation became about four times faster on average across measured screens. The repository is open source and free to use with a QR code available for access.
Dev mode on Figma is required to set up this automation because Figma has IP requests in free mode. The mapping between Figma concepts and Unity concepts uses assets with connections to Figma by component ID. All elements from the UI kit in Figma should be presented in this mapping asset. A skill handles this mapping because mapping all elements is a difficult task.
The designer checklist can use skills to fix mockups for automated implementation, find inconsistencies, and remove problems. The tool was built specifically for legacy projects. New projects can use UI Toolkit where agents can create UI without MCP. Legacy projects work well with this tool.
The game project shown is the pet project "Catch the Light," a game for people who lost part of their visual field. The tool helps with visual field loss by training reading and finding things in the world, helping users see people and cars in blind spots when walking across streets.
Figma is the source of truth. Changes should be made in Figma and agents should implement changes in Unity. Changes made in Unity should not affect Figma.
Rider aims to be the ultimate development environment for game developers. It supports Unreal, Unity, Godot, or custom C++ engines. Users can write C++, C-sharp, GD script, shaders, build scripts, and ship to desktop, mobile, consoles, or XR devices. Rider understands the environment around code rather than treating game development as generic software development.
Rider supports the whole game dev loop including writing code, debugging, profiling, running tests, and deploying games as one connected workflow. AI is becoming another layer across this workflow rather than a separate tool.
Three key benefits are understanding the whole project including source files, assets, engine concepts, project structure, and runtime behavior; connecting the development workflow so context carries across code, engine, debugging, profiling, build systems, and target platforms; and iterating faster by finding problems earlier, understanding root causes faster, making changes, and validating with less friction.
JetBrains takes an open approach to AI where developers should not have to choose between trusted development environments and preferred AI agents. The idea is using agents with Rider intelligence through three pieces: the agent itself through ACP (Rider Agent Client Protocol) allowing work with different agents; Rider tools through MCP giving agents access to code intelligence, navigation, inspections, debugging, and project understanding; and for game development, engine-specific knowledge and tools including recently released Rider skills for code refactoring, debugging, agent hooks for code quality checks.
Instead of giving agents just folders full of files, they get access to the actual development environment. Unreal projects are not just C++ repositories. Game behavior spreads across C++, blueprints, assets, various configs, actual Unreal editor state, build targets, logs, and runtime behavior. Generic coding agents can be good at reading and generating C++ but may miss the broader picture if they only see repository files.
RiderKit is an AI chat directly inside Unreal Editor. Rider installs a plugin into Unreal Editor that connects the editor back to Rider. The goal is not to build another AI agent or reinvent models. Users connect their preferred agent via ACP, such as Cloud Code, and RiderKit enriches that agent with the development environment around it.
Rider provides understanding of code base, symbols, usages, inspections, navigation, debugging, and project model. Unreal Editor provides viewport, blueprints, assets, logs, and current editor state. RiderKit connects these two environments and gives agents access to both sides of the context. The interaction happens right in Unreal Editor where many tasks and problems actually start.
Workflows are grouped into three categories: understanding, implementation, and investigation. For understanding, agents can answer questions about real projects and trace gameplay logic across C++, blueprints, assets, and project structure. For implementation, agents can
For implementation, agents can make changes using both project knowledge and Unreal Editor context rather than treating every task as isolated code generation. When it comes to investigation, the agent can reproduce the issue itself, inspect around time, state, and logs, use the debugger from writer, find the cause, make a fix, and then validate it even. Underneath all three is the same advantage: more context, deeper project understanding, and the ability to act across writer and Unreal Editor. This is the writer kit advantage being explored - actions across writer and Unreal Editor, not just answers in a chat window.
WriterKit is almost ready product that is going to ship as a private preview. For the first demo, three small examples show how the agent can reason about an Unreal project beyond just reading C++ files. Most work here is around blueprints, but it also crosses into C++ when relevant. The agent is asked to find all blueprints derived from two Wira types. The agent uses writer's project knowledge and gives the actual blueprint hierarchy and actual asset locations.
In the second prompt, the agent is asked to create a new blueprint and then asked a much broader question: find what references the existing base blueprint has, understand how those assets use it, and tell what it could break if changed. The agent calls MCP tools from writer and does the job. It doesn't stop by simple reference search, so relationships are obtained that show the potential impact of the change across all blueprints.
The third task asks how a pistol asset is connected to its weapon instance. The answer crosses the blueprint boundary. The agent follows the asset relationship and then moves into the C++ code that actually consumes them at runtime. The agent chose the right answer and showed a comprehensive answer of how it's wired. The agent was using Writer MCP under the hood, Writer project knowledge and Writer search methods without doing full text search always. Unreal relationships often span assets, blueprints, and C++ code. Writer can use Writer's structured project understanding and project index to follow those connections instead of treating the project as a collection of unrelated files.
The agent is asked to create a new dash gameplay ability in C++ using Unreal's gameplay ability system, expose a force property of that ability, create a blueprint based on it, and add this ability to the player. The agent first uses Writer to understand the existing project understanding, then finds the right classes and files where to put this C++. It creates the C++ implementation. The agent starts the Unreal build through Writer. Writer understands live coding and makes the build through UBT. Unreal live coding applies the change without restarting the editor. The agent creates the blueprint, configures the property and wires the ability into the player's default abilities. At the end, the agent notices that nothing actually triggers the dash yet and explicitly tells that the next step would be to bind this to an input action. This is a good example of one connected workflow: understand the existing architecture, change C++, put the files in the proper place, build it correctly through Unreal tooling, create the blueprint side, configure the player, and understand what is still missing.
The agent is given a human-level instruction: set a breakpoint on the weapon pickup attached to the game, simulate the player moving forward, and see what happens. The agent uses writer search and project understanding to figure out what weapon pickup actually means in this project. It finds the relevant code in shooter.cpp and identifies the actual pickup path. Then it sets the breakpoint and attaches writer's native debugger to the running Unreal editor process. WriterKit uses Unreal side input simulation to reproduce the scenario itself. The agent starts playing editor, moves the player forward, and lets the character reach the pickup. The breakpoint hits exactly on the weapon pickup. The agent reads the debugger state and reports the relevant runtime variables back in the chat. This started with a very high level description of the behavior to inspect, then the agent translated that into the set of actions like write code location. It used writer's debugger, reproduced the behavior inside the Unreal editor, and brought the runtime state back in the conversation.
A real game issue is observed where a weapon is picked, another weapon is picked, switch weapon is clicked, debug log is printed but nothing happens. The agent is given the instruction with the problem in plain language and asked to run a standalone game, attach writer's debugger and figure out what's going on. The agent uses writer to find the code responsible for weapon switching without being given any files. It finds the relevant implementation from the behavior described, sets the breakpoint, and launches the game in standalone game mode. The agent attaches writer's debugger to the new game process. The breakpoint is hit, then the agent inspects the actual runtime state and continues the investigation through the debugger, stepping through the stack. The root cause is found: there is an inverted condition in the weapon selection loop. The agent tells that it proved this with the live debugger. It proposes the fix, explains the issue, and proposes the actual patch directly in the chat and asks whether to fix. After applying the fix, the agent builds the update through writer using the writer build command and real-life coding via UBT. This brings back to play in editor to validate the result. The agent found the relevant code, launched the runtime, attached writer's debugger, observed real behavior, found the root cause, proposed the fix, applied it, rebuilt the project, and helped validate inside Unreal editor.
WriterKit is still early and not publicly available yet, but a private preview is opening shortly. QR code is available to sign up for the private preview. WriterKit is maintained by JetBrains. The agent uses the real project model, the real assets, the real index from the project at any given moment of time. Even if something is changed and then the agent is asked to check this or that, it would use the actual state. Dynamic links to the actual blueprints or code can be included so that jumping to and from between the chat and the actual code or blueprint node is possible.
Having an AI assistant right at fingertips in chat has changed the relationship with the editor. It shortens the development loop. If something is validated in Unreal Editor and an agent is needed to help with debugging or with that asset, a task can be spawned for an agent right there in Unreal Editor without leaving it. Either continue in Ryder or stay in Editor, free to choose where to go.
The agent handles organization and moving assets around pretty well, as well as Ryder, because agent uses Ryder tooling under the hood via MCP. As if refactoring something or reorganizing something in the IDE would result in the updated state, then working with this updated state, the same would go for an agent. The agent would know about any reorganization of the files done at any given moment of time.
A new version control system doesn't come every day. This talks about files that most source controls really never built for: textures, meshes, audio levels, all the big binary stuff that games are made of. An artist opens a two gig source atlas, one huge image, uncompressed, stored all the tiles. They repaint one small square of it, maybe 1% of the image, save it and commit it. Send that same edit through three kinds of version control: Git with or without LFS, centralized server, and lore. Same file, same edit, same network, and the three other sites that need it.
Rin Slack is a game solver, senior game solutions architect with AWS out of Arlington, Virginia. Ben Cook is a senior game solutions architect from Austin, Texas. Ben will cover how it works, as well as what was measured. Rin will discuss how to run it, as well as being the voice of the studio and whoever happens to get paged at 3 in the morning. Three things to leave with: mental model of fragment-based storage, why the architecture looks the way it does, and a starting point to run it and what it costs. Whenever a number is shown, where it came from will also be told.
Every studio already has version control, but a game repo is not a source repo. It's gigabytes of bandwidth, binaries that change every day, and tools that are designed around source code. Git was never built for that. In an Unreal-style project, the biggest tile is textures, the images painted on every surface. Next is source art, layered image files and sculpts that the artist made those textures and models from. Then meshes, 3D models, engine binaries, and audio. The thin orange sliver on the right is the source code, about 2%. Nearly all of it's binary. Some single files run to gigabytes. Artists re-save them all day. Line diffs and merges are great for that 2%, but the other 98% are opaque bytes. You can't line diff a texture and you can't merge one.
When naming Git or Git LFS, default settings are meant. With LFS, big files move out of the history and leave small pointers behind. But LFS can't send part of a file. A changed byte means a new full object. Uploaded whole, downloaded whole, by every checkout that's needed. Plain Git can send deltas, but past 512 megs, it stops trying. A default clone carries all the history. The second approach is a centralized model where one authoritative server exists, and to scale it out, edge servers are added near the team. Each site runs a full replica. With the default binary file type, every change is a whole new revision. Each replica pulls the whole thing. Some centralized servers can now send only the changed parts of uncompressed binaries, but they store every version whole. Replicas do scale reads, but three standing bills exist: capacity at every site planned in advance because each replica holds the whole archive, seeding a new full edge that needs its metadata restored from a checkpoint, and the morning rush when the whole studio syncs at once.
Lore is centralized as well, but what changes is the unit. Lore doesn't store whole files. It stores pieces of files called fragments. When an artist commits, lore works out which pieces changed. Only those pieces move. Copies that held the old version already hold that file's unchanged fragments. Nothing else needs to move there. Two of them version whole files. One versions only what changed. With the atlas example and real numbers, one small square repainted is about 2% of maybe a 2 gigabyte atlas, uncompressed and tiled. Each bar is what one site moves: the upload and any site that pulls that change. Git LFS moves 2 gigabytes - the whole file goes up, every site that needs it pulls the whole 2 gigabytes down again. Neither sends the part that changed. In the centralized model with default binary type, it's the same 2 gigabyte up to the server and then 2 gigabytes out to each replica at the edge. The newer change parts option would shrink the bar, but each version is still stored whole. With lore, only about 20 megabytes move, just a hair over 1%. The fragments at each end have to move whole, so partial fragments exist in the change. The gap stays about 100 to 1. These are uncompressed sizes. Every system compresses something. Already compressed outlets gain only a little. This is just illustrative at this point, not a benchmark. Multiply this by every site, every artist, every commit, every day. With the defaults most run, one change byte means a whole new version, stored whole, sent whole to every site that needs it. Studios pay for this in three ways: bandwidth on every commit, storage of whatever version of a big binary kept whole, and waiting at every site. Every time someone syncs, the bigger the files get, the bigger all three costs get.
Lore's answer is to stop versioning files. What does it version instead? It versions fragments. Lore doesn't version these files, just versions the pieces of the files. Only the pieces that change ever move. From a developer point of view, it has the same action as a file. It doesn't change any other version control system. The same kind of verbs are used: pull, push, commit, sync. But under the hood, it operates very differently. When committing, the client on the workstation cuts up the file into fragments using content-defined chunking. Instead of cutting every so many bytes, the content itself decides where the cuts go. With fixed-size cuts, one inserted byte would push every later cut along, and every fragment would then look new. The cuts move with the content. After every insert, most fragments come out exactly the same as before. Each fragment is named by a Blake 3 hash of its bytes, a fingerprint. Same bytes, same name. A fragment with a given name is stored exactly once in the system, across files, branches, history. Deduplication falls out of this naming. When that artist repaints one region, the two fragments that get new names, the other 12 keep theirs because their bytes didn't change. Only fragments with the new names upload. The 12 gray ones are already in the store from the first version. Just two of the 14 move here. A version is really just a list of names. Version 2 is version 1's list with two entries swapped out. Only the two new fragments and their new list were written. Versions are a ton. The expected size of a fragment is about 64 kilobytes. The fragments are never more than 256 kilobytes. That 2 gigabyte texture atlas is tens of thousands of fragments, not 14. An edit to one patch of that atlas touches only the fragments that it covers. That's why roughly 20 megabytes from a few slides ago comes from. Every fragment has its own request. Thousands of small requests have to keep moving.
Lore uses the Quick protocol, a modern Web 3 transport that runs over UDP. One connection carries eight streams, with two kept clear of bulk data for small tree reads and sync needs. The client can keep up to 10,000 requests in flight, with replies coming back in any order matched by ID. When a packet is lost, only its own stream waits for the resend while the other seven streams continue moving. Over TCP, one lost packet stalls everything on the connection.
Quick runs in the application rather than the OS, allowing Lore to tune itself. BBR congestion control uploads one stream at a time so whole fragments land sooner. Each window is bandwidth times round trip. For clients, this means a gigabit per second over 100 milliseconds, or about 13 milliseconds, which is approximately 13.4 megabytes. When a client opens one connection by default, it tops out near a window divided by the round trip. At 100 milliseconds, this is about 134 megabytes per second, and about 200 at 67 milliseconds. These are computed figures, not measured.
All testing was conducted in VPC using EC2 instances that cap out at about 5 gigabits per second, or about 625 megabytes per second. Every client number in later sections reflects just one connection. If a distant client's link is faster than this ceiling, the maximum connections can be raised up to 10. Chunking, hashing, compression, and Quick's encryption all run on the client's CPUs.
A client gets its file by talking only to a lower server, an edge pod, or the right tier directly. There are no AWS credentials or AWS SDK involved. Fragments travel over the Quick connection. Branch pointers and locks go over gRPC on TCP. The authoritative tier uses S3 as an object store with one object per fragment named by its hash. DynamoDB serves as a managed key-value NoSQL database holding the fragment index and metadata. In lower .8's layout, only the right tier reaches the branch pointers and locks, with only lower servers having access to that data.
When a client finds a revision, it asks for a branch pointer and the server reads it from DynamoDB. The client then walks the tree, which consists of fragments, following only the paths in its view. The fetch operation uses one get per fragment, 256 kilobytes at most, with many in flight simultaneously. When a pod cache misses, it asks the right tier, which checks its own cache before pulling from S3, creating two layers of cache. During rebuild, the client checks each fragment's hash, assembles the file, and writes it to the workspace so rereading does not go back to the server.
Every byte comes through a lower server. For big sinks and imports, server throughput is the ceiling measured. Every miss reaching S3 becomes one S3 get call per fragment, plus a DynamoDB lookup. A client needs a Lore token for sign-in, and AWS permissions are invisible to the system since everything is managed within Lore.
The durable tier consists of Amazon S3 and Amazon DynamoDB. S3 provides 11 nines of durability and multi-layered deletion control mechanisms. DynamoDB is a serverless NoSQL database delivering single-digit millisecond performance at any scale. Together they form the authoritative copy of the repository. The right tier takes every commit and serves reads, running as a container on EC2 with a local NVMe device as the first layer of caching. Edge pods are plain EC2 instances with NVMe cache, placed where reads occur such as next to CI fleets or teams regionally, serving as the second layer of cache.
If asked what could be deleted without losing committed data, the answer is everything to the left of the durable tier. Losing caches costs time but not data. The right tier is replaceable as it essentially functions as a cache, though it serves as the single door for commits. In the example, it runs in an auto scaling group that automatically replaces failed instances, with commits waiting until the replacement is back. S3 and DynamoDB have multiple layers of protection from deletion, while everything else to the left can be lost without impacting the data.
Lore uses the same verbs and actions as other version control systems. For commit, the heavy lifting happens on the artist workstation. Artist A chunks files, hashes and compresses fragments, then uploads ones not already present. The server stores each fragment only once. Push is very small because the commit has already uploaded the fragments. Push checks they are present and moves the branch pointer forward in DynamoDB from one version to another. In testing, this is usually under a half second.
For sync, Artist B pulls only the fragments that are missing through an edge pod. The pod caches keep copies for the next person. The client fetches every file in its view and nothing outside it. Moving fragments is heavy and grows with the size of the change, while moving a pointer on the push side is tiny.
Four principles describe the model. Content is chunked into small fragments and deduplicated, so edits to huge files only touch changed fragments and identical fragments across versions are stored once. Commits upload the fragments while push checks they are present and moves the branch pointer. One operation is heavy while the other is tiny. Sync fetches only the client's view. Pods never hold the only copy of committed data, so a pod can die and a new one refills through the right tier as fragments are requested. The store is append-only, caches evict, but nothing sweeps S3 storage. Every fragment in a commit stays unless deliberately pruned. Lore can prune history but does not do so by default, so storage keeps growing.
A cache that can be thrown away is one that never needs seeding or backing up. A new location syncs through an edge pod instead of waiting for a full archive edge. CI agents can also sync through this setup. For a 500 gigabyte repo over a 1 gigabyte per second link, provisioning and restoring a checkpoint takes about 20 minutes, copying the archive over the site link takes around 67 minutes at line rate, and verification adds another 10 minutes, putting the first useful commit at the 97-minute mark.
A caching proxy or cache mode edge shortens this significantly. A pod drops the checkpoint and edge database. Launching a pod takes five minutes, the first sync pulls only the slice the site works on, adding a couple more minutes, with the first useful commit coming at about eight minutes total. The 89-minute weight of a full archive edge is eliminated. The pod's cache fills as people work, with nobody waiting for completion. The first sync of a file fetches it from the right tier, and everyone after gets it from the pod.
Migration A brought in a 105-gigabyte media snapshot of Netflix's Meridian, a Creative Commons film with uncompressed UHD video frames and Dolby Atmos masters. Sent directly to the right tier, the first attempt did not complete. S3 throttling on a fresh account and fresh bucket throttled the rate about ninefold and never recovered. After a little more than an hour, the attempt was stopped. Through an edge pod, the same import finished in about 8 minutes and 45 seconds.
The pod does not buffer and confirms each fragment only once. The right tier has S3 access and never holds the only copy. This was tested with Lore 0.8.6. One of the guardrails routes first large imports above 50 gigabytes through a pod because the back pressure and control is much better. Lore is currently pre-1.0, working toward a 1.0 release. Ten-gigabyte repos went direct without problems, but the 105-gigabyte direct import on stock settings with a fresh account did not complete.
Once the server image is in the registry and there is a network path into AWS, deployment takes about an afternoon. Only one Terraform module is needed to stand up all three tiers, coming from the Cloud Game Development Toolkit, an open source AWS project in three parts. Assets are reusable pieces including Packer templates, pipeline definitions, and post-deploy playbooks. Modules are the Terraform itself called from user code. Samples are complete configurations showing modules working together. The modules cover version control, Unreal Horde, Unreal Cloud DDC for catching derived data like compiled shaders, build servers, virtual workstations, and Unity's tools. Lore is the newest addition.
The Terraform applied builds clients connecting over networking services as a private line into AWS. Pods and right tiers sit in private subnets, not publicly accessible to the internet, admitting only allowed office or VPN ranges. Clients sync from edge pods, with two pods in separate locations within the region sized for about 20 to 40 people. The right tier takes every commit and auto scaling groups replace it if the machine fails. The durable tier includes S3 holding fragments and DynamoDB holding the index and branch pointers.
A handful of people can skip pods when building out. The durable tier is the same in both shapes, so adding pods later does not move any data. Adding two pod blocks, running apply, and pointing clients at the pods accomplishes this. If the right tier grows, that can be accounted for with a simple variable change.
Three decisions need consideration on day one. The first covers environment, container image, and allowed ingress CIDRs. Instance types for the right tier and each pod, pod count, and region are the key choices. Region lives on the provider, not in the module. Each pod defaults to an instance with about 32 vCPUs running around $1,000 a month depending on region. The right tier defaults to an instance around $250. Smaller boxes work as well. The default and minimal examples ship as dev setups with no sign-in and no deletion protection. For production, set auth mode to Cognito or an external identity provider, turn on deletion protection, and turn off force destroy. The bucket ships with versioning off, so backup planning is required. With pods, the stock pod block has no sign-in inputs, so a patched pod block is needed.
LoreBench was built to measure performance by driving real Lore clients and real deployments. It packages everything from the architecture diagram along with test clients to iterate through instance sizes and available cache levels. Testing covered 21 different server configurations with one to 29 clients across two corpora. Stacklebot, Epic's sample project for Unreal Engine, contains about 2,000 files and about a gigabyte. The UHD media corpus has multi-gigabyte files of about 10 gigabytes each, totaling about 105 gigabytes for import or migration testing.
All latency testing was in VPC in the cloud. Every file cloned and synced was verified outside of Lore, not just trusted by an exit zero from the client, with every file fully independently checked for data integrity.
Testing the right tier across six shapes using C8GD and C9GD compute optimized instances from one vCPU up to eight vCPUs showed that import speed depends on available vCPUs. However, P50 latency for small edits hardly changes regardless of server size. Server size does not drive faster commits but makes a difference for large bulk imports. The bottom line for everyday work remains incredibly fast regardless of server size. This is because small commits are mostly metadata round trips to DynamoDB. A single box without DynamoDB takes maybe four tenths of a second in VPC for the same commit. Big asset commits differ because the workstation must chunk, hash, and compress them, with client CPU setting the pace at roughly 170 to 290 megabytes per second from two to eight client vCPUs.
Upsizing the right tier provides more benefit for first imports and cold hydration but does not deliver faster everyday commits.
Edge pods are a major feature of Lore for distributing work and bringing data closer to users. Outside of a couple moments, pod size is effectively invisible. With steady state small drips, the slowest 1% of all calls is about 100 to 170 milliseconds on every pod with two or more vCPUs. The one vCPU case is disqualified. For either right tier or pods, at least two vCPUs or more is recommended. Single vCPU instances caused issues.
During sync storms and big imports, pod egress going flat indicates the data out upstream is flat and CPU is effectively pinned. This is the point where scaling up makes sense for more throughput. CloudWatch logs with a CloudWatch agent on pods or right tier collects CPU and network data every 10 seconds. Alarming on the ENA counter for bandwidth out of allowance exceeded indicates when moving to a larger pod size makes sense.
Edge pods serve two primary purposes beyond the right tier baseline. The first is locality, where pods are deployed globally to reduce latency for users far from the US East 2 region where the main infrastructure resides. The second is handling big sinks or large data imports that benefit from proximity caching.
All lower-tier servers require NVMe instance store drives, which provide the fast storage essential for the version control system's performance. Direct routing to the right tier fails with large imports, while edge pods become valuable once client count reaches approximately 15 heavy users per location.
For 1-5 users, edge pods are optional and can be scaled up or down as needed. They provide buffering for big imports and cache placement near reader fleets like CI systems. Multiple pods can fan out to approximately 15 clients per warm pod, with testing conducted up to 25-30 clients.
Monthly costs using US East 2 on-demand pricing show compute as the dominant expense, with S3 and DynamoDB storage representing only a fraction for a 20-gigabyte repository.
- Small team (1-5 people, 20GB): C8GD.large right tier costs approximately $77/month
- Medium setup (2 edge pods, C8GD.XL right tier): approximately $312/month
- Large team (storm-heavy, distributed pods): under $500/month
Additional costs of $40-50/month cover VPC plumbing, NAT gateway, and CloudWatch logging, with site-to-site VPN and other services potentially adding more.
Four critical guardrails emerged from Lore Bench testing:
- Never use single vCPU instance shapes for right tier or pods due to performance and reliability issues
- Cold pods can only onboard 5-6 clients at a time due to empty cache state
- Verify after every clone and sync operation, checking expected file counts and ensuring no temporary files remain
- Route imports over 50GB through pods rather than direct connections
The 105GB import failure resulted from S3 throttling at 3,500 writes per second per partition on fresh buckets. Partitioning the bucket by the first character of file names (hexadecimal hashes) increases capacity to 56,000 puts per second.
DynamoDB on-demand tables throttle at 4,000 writes per second before scaling. For large imports, pre-warm tables using command line tools to maintain per-second rate limits. This issue primarily affects imports over 50GB; smaller operations with Stack-a-Bot don't require pre-warming.
The Qwik protocol encounters DNS resolver throttling at 1,024 packets per second per network interface. Linux and Mac have DNS caching enabled by default, but Windows clients require explicit DNS caching activation to prevent throttling during fragment requests.
Three key variables matter for deployment:
- Right tier variable: Start with C8GD.large instance type for baseline performance at approximately $77/month for small groups
- Edge pods: Define 2 C8GD.large pods plus C8GD.XL right tier for 20-40 users with 50-200GB heads, costing approximately $311/month
- Scaling signals: Monitor 10-second pod CPU metrics and egress flatlining during morning syncs
Replace pods with .XL instances one at a time when CPU pins at 100%. Both configurations use Graviton ARM64 sizes, requiring ARM64 server images.
For 100-person studios with up to 1TB heads, projections include 7 XL compute-optimized pods plus at least 2XL compute-optimized right tier for imports and CI, totaling approximately $1,500/month. Each additional 100 concurrent syncing clients requires roughly 7 more pods, adding approximately $1,000.
Open questions remain for multi-terabyte repositories exceeding right tier local disk capacity and 50+ CI agents creating secondary morning rush patterns. Multi-site deployments are planned for future work.
Each region requires one module call creating separate repositories. NAT Gateway, disks, and logs require DNS endpoints adding approximately $200/month. The Cloud Game Development Toolkit contains these modules for deployment, with issues filed and pull requests accepted for improvements.
Lore Bench provides stability verification and helps answer questions about sync storms, onboarding waves, and big imports. Release will be announced through blog posts.
Additional data on daily repository change rates, user roles, and churn patterns would improve sizing models. Current models lack grounding in these metrics despite working with multiple game studios.
Three major implications stem from versioning fragments rather than files:
- Every cache holds pieces, not versions - warm pods contain most pieces of current files while dead pods refill without losing committed data
- New sites or CI agents pull only missing pieces from their view, making them useful after first sync
- Storage grows only with changes since each piece stores once, though manual pruning is required to prevent perpetual growth
Storage costs remain a fraction of overall expenses, with compute driving the majority of costs.
A client-side inventory bug initially appeared to show two items collected instead of one when opening a chest. The visual glitch suggested client rendering issues, but investigation revealed the problem originated on the server side.
The inventory code on the client simply displays totals from entries provided to it. Since the client received two items, the issue traced back to server logic sending incorrect data.
Server inventory logic sends addition packets with ID, item, and count parameters. The send function correctly transmits count as one, but the client receives two items due to resend loop behavior.
The resend logic at lines 41-44 checks acknowledgment after sending, creating duplicate sends when the first packet isn't acknowledged before the resend check occurs. The acknowledgment gets read and checked after the addition is sent, causing the duplication.
When debugging across client and server, assume both sides are lying about their state. The wire protocol represents truth - if the client shows two items and the server reports one, the client is accurately reporting what it received, indicating the server sent duplicate data.
Breakpoints on both sides cause heartbeat timeouts and state loss since stepping through debuggers prevents proper communication. The stutter glitch example showed backstepping every tick, requiring correlation IDs to track packets across the wire.
Packet headers include game tick and sequence values serving as correlation IDs. These track packets on both send and receive sides, ensuring correct packet identification even when packets arrive one tick late.
Client-side header reading displays tick and sequence values by default, enabling quick identification of packet timing issues during reconciliation loops in server-authoritative systems.
Being able to read your current memory and your iteration and all of the information that your client or your server are actively reading at that point. The fun part of this is now that we have a correlation ID, we have a really good idea of which packet the problem is coming across. In this very simple demo, you're seeing stuttering which shows that something is causing the client to jump back and then forward again.
The easy way would be to go and blame the client. In this situation, you'd actually be right, but you're going to be pointing at the wrong problem. You're going to go look at the movement code and ask why is movement jittering. When the movement is returning the correct value every time, then you go look at the renderer and go why are you updating slow. That is the difficulty of this bug itself. Tick skew is the issue when the response from the server is updated arguably late, but technically as soon as it's supposed to arrive.
You can set a trap to prove this, but there's actually an even easier way to look at this. If you set a trap by putting conditional breakpoints on both sides, which being able to set a conditional breakpoint is helpful. Conditional breakpoints in Rider specifically are even better because we can set a condition where snapshot.tick is equal to 400. This gives us one explicit tick that we're going to look at. We're going to look at tick, and then we're going to do the same on the server side. This is where Riders and CLion's ability to have multi-platform debugging becomes so powerful.
By setting a snapshot on both sides of the problem, you're able to quickly and easily look at where the problem comes from. We have a simulation running on our server side. That is very helpful for us because we can go point at the exact point in the simulation where the reconciliation runs, which is going to be inside of our run tick method. Looking through all of our code here, we're going to be placing a breakpoint at line 135. We're going to be setting a condition where tick is equal to 400. That's where the power of multi-platform debugging comes in from CLion and Rider. If you run this as a debug, you get the state at both ticks equaling 400.
The problem is that heartbeat that we talked about before. The solution for most people is in a development environment, don't require a heartbeat. You don't need it. So at a client running at 20 hertz, which is 20 ticks per second, at 60 seconds or three seconds, it times out because it didn't receive its heartbeat from the server. Now we have to figure out how do we design around this problem.
The easy way to design around this problem is another trick that the debuggers give us, which is to control exactly how your debugger breaks. The easy way to do this is to just set your suspend execution to not. That way your server keeps running over and over and then you still get the breakpoint information, but you don't get the heartbeat crash that prevents your client from ever receiving the information that it needs to continue.
The other really awesome thing about debugging in this platform specifically is that if you're having issues with a specific debugger, then over on the Rider side, you're able to even tell it to dump you a stack trace when that happens so that you get all of the information about where that breakpoint was hit, and you can store that later. That also applies on the CLion side, but in native debugging, sometimes that's a little bit buggy. So I suggest you just use this suspend execution.
Another really smart way to build around this is with threads. If you're building with threads, then you know immediately that you're going to be suspending a thread. I actually built for this by design. In the simulation, we're running the entire simulation off of a single thread. The simulation lives in its own thread separate from the main and the network thread that would be suspended, meaning the heartbeat that runs in a loop on the network would never stop being sent because you can suspend a specific thread in either CLion or Rider, which are the only two I'm showing today. However, you can do this in other platforms. The terminology may just differ a little bit.
One of the really awesome things about being able to control individual pieces of your debugger is being able to also control the conditions as you go on. If you notice a condition in a specific tick, let's say tick 193 is landing on the client, but the server's only sent tick 191, then you know that there's an issue on your server side sending things in the wrong order. And then you can chase that error just a little bit differently at that point.
The bug is easy to find in this issue. Because we know that there's a reconcile loop, the server is going to be sending its information one tick late. And with your debugger in debugging each thread, you're able to quickly and easily land on the problem being the tick arriving late.
We've talked a lot about how you land on a problem and how you use a debugger to find a problem. But how do you find the problem when it's actually the wire itself. In our bugs so far, we've talked about how you can chase down any individual problem using a debugger. But how do you chase down wolves that are behaving like chests and a chest that's behaving like a wolf.
This looks like a data problem. In this situation, you're quick to blame the client. Because the client is obviously showing that a chest is a wolf and two wolves are chests. Being able to identify that problem quickly points you at the client. We go and we pull up the client. We take a look at our information. We go over to our wire code. We know that our wire is reading in our information correctly. So we go look at our protocol. Our protocol defines our entity type in a number format. We have our player, our crate, our wolf, our chest, and our torch. These all look right.
Now that we're getting the right information, now it sounds like the server is sending the wrong information because the entity types are correct. So why is the server sending the wrong entity type. If we hop back on over to our server here, now comes the question of where's the problem here. In our setup loop, we are setting up our information. Then we hand it over to the server. We hand it over to our simulation. Problem must be in our simulation. Our simulation here sets up a crate, a wolf, a chest, and a wolf. So our two wolves are set up correctly and so is our crate. But they're being shown backwards. The server is sending the right information.
In this situation, now that we know that the server and the client are both sending information that they believe to be the correct information, we can now ask the question of where is the wire wrong. To do that, it's first easiest to look at the protocol itself. Our protocol is being generated in a very weird way, where it has to be in two languages. We have to have one side in C sharp and we have to have one side in C++. Obviously, you're generating that information when you need it. While you don't need it, you don't need the C++ on the C sharp side, and you don't need the C sharp on the C++ side.
If we look at the protocol here, it player, crate, wolf, chest, torch, right. On the other side, player, crate, wolf, chest, torch. It looks exactly correct. The problem is that the generated files that we're building out of this on the C++ side have a bug. Our C++ side is sending this information just slightly backwards. Our C++ protocol shows crate and then wolf, right. But on our C sharp side, which I believe it's currently being overridden. It's being shown as wolf and then chest. Obviously, I had to create this in two different ways. There's a stale version because we didn't regenerate the client code. You'll notice that it goes wolf and then chest here, but it goes crate and then wolf here because we added the crate type to the server and then never regenerated it. We just generated the client to adapt for that. That's an issue with protocol generation.
There's a lot of solutions to this specific problem. One of the best is to generate both sides of your wire using a single centralized protocol type and then just generating for whatever language you need. There are tons of languages for this. Things like protobuf. I know you can do it in C sharp. You can write your protocol in a language and then generate binding types for it as you need to. You can do that in C++. You just decide one side needs to be the answer. The other side needs to listen to that answer. If you're rushing that a little bit every time, then you run into the point of how do I then solve the problem when I need it in two languages. That's when you write bindings and that's when you write new pieces. That's when you pick a side to be true. This exposes that problem firsthand because your client is currently in the process of believing that your wolves are chests. That's because the server believes that it's sending the right enum value, but two is a chest in our client and wolf is one. Our chest gets turned into a torch. This gets turned into the number four because it doesn't even support it. That enum doesn't exist.
That's another really quick way to look at it is to see something that just doesn't look right. That tells you the problem pretty quickly. I talked through this entire problem in five minutes. Instead of spending my time going server's sending it right, client's sending it right, I need to look in my wire, there's the problem. That's the really quick way to do this is to again, go back to that bisection point. Look at what it believes about the entity. Both sides believe it has the right information and both sides are getting the right enum value. Both are getting the number two or the number one. It just believes they're different. That is when your protocol changed. When your protocol isn't right, then you're able to go look at it and find the problem. Notice how the last thing I do was to go look at the protocol and look at the problem in the protocol. That goes back to the original method I talked about. If you chase the protocol first, then you end up wasting a lot of time. If you chase one side first, then you waste a lot of time. The method is to look at what both sides believe before chasing one side or the other.
There's a second way that this happens. This one is really hard to actually identify. When you have an issue that shows like this, where everything's just stuck up here on the top, and nothing's happening, you glitch around for a second, and you're back in the top. This is a really weird issue because again, now it looks like the protocol's lying. It's hard to identify this problem explicitly because the server's obviously sending information because things appeared on screen. The client's obviously rendering things, but it's not coming across the way the server was told to send it. In that situation, you're forced to look at the protocol. This is the really big bug that I want to cover here is when not to look at the protocol. Entities are genuinely in impossible places. That's hard to show without some sort of debug on the actual client itself. But everything's just a little messed up. Everything's just a little off.
How do we fix this. The easiest way to fix this is to first go and look at what each side is sending. We're going to go over first to our simulation. Our simulation is responsible for all of the information that's being shared across. When you look at the information that you're sending across the protocol, obviously, your protocol is a source of truth. It has what things need sent, what types they need to be sent as, and how in the protocol they need to be done. It also contains a list of messages that you're sending. In here, I'm not seeing any issues. Everything's being sent across correctly, and there's no problems. In our simulation, everything's being sent. We're writing our header. We're sending our header and the correct information. We're sending our entity with an ID, our type, our X and our Y, health. We're just sending our X and our Y values. The wire entity is being set correct. This is when you now want to go look at the client, because while our X and Y is being set correctly, where does the problem actually live.
If we go over to the client, and we go to our program. This will be in our netcode. In our receive, we're going to be pulling information in and reading it, so that we have the information about what's going on. The easiest way to look at that is our applied entity. When we apply an entity, we take in the X and the Y values and look at them simply. To do that, let's run a console dot write. Let's do write lines so it doesn't look terrible. Let's print all of the values that we get from the server. If we run the client, you'll notice these numbers are ridiculous. We have 3.6 times 10 to the negative 41st, negative 13 billion, like, these numbers are absurd. When numbers are this wrong, there's a problem, and there's a problem somewhere. We've now identified the issue. The server is sending its X and Y values, and the client is reading them, and the client is reading absurd values. Let's go log what the server side says. When you log these, the server's going to show that it's actually sending the correct X and Y. It's going to show a proper X and Y value, because to it, the X and Y is correct.
Where's the problem. The problem is on this side of our write loop, we're actually writing a padding value to make sure that the information that we send is padded. The padding value is actually what's causing the problem. The packet is supposed to be 106 bytes. It's supposed to send your type, your ID, your X, your Y, and your health. Then it writes the entity to the wire. It's supposed to be just that simple written on the header. But it's actually being padded. That's what's happening over here. When you're reading this, your X and Y values are actually being read incorrectly. Because we can debug the client, the easy way to do this would be to run the debugger here. If we debug, obviously, we're going to hit our write line here and break. We'll find that these values are absurd because we're just printing them. We're showing that they're just ridiculous. The issue is that the padding is writing over top. The client is reading over top of that value because it's being padded in such a way that you run into that problem.
This bug loves to hide places where this isn't something written on your protocol. Your wire entity is being defined in a type as an entity record here where you have an ID, a type, two floats, your health, and then the raw data. On the server side for your wire, your entity is ID type padding XY and health. You can kind of see where the issue is. There's padding on the server side and not on the client side. Now you found your problem. This one's a lot harder to catch, grab, and do something with. Because it's hard to prove. It's hard to look at and find the problem. This is one that if you run into, it's hard to point at the problem. Because it's not the protocol. It's not the server. It's not the client. They're both reading and understanding exactly what they're supposed to. The actual issue is in something you didn't define in the protocol. It's something you defined on the wire. Defining more on the protocol and not on the wire fixes this problem. As long as you're focusing primarily on writing everything on the protocol, then this becomes a protocol issue. Then your double-sided debugging actually works. The method works. The problem is when you're writing things that aren't on the protocol and sending them across.
The last thing we're going to talk very briefly about is about failure patterns per language. Obviously, each language has a set of failures that at least look very similar in a pattern. If you look at a symptom, you can find which runtime is guilty easily. I'm only going to talk about C-sharp and C++. But as you do more and more languages, you're able to cover them easier yourself.
In C-sharp. Tasks are really good at swallowing exceptions. If you never await a task, the exception just disappears. It's easy down it can take time. When that happens, you run into a point where it looks like your network is lagging behind, like your server sending packets slow. In just situations like that. You can run into continuation, resuming on the wrong thread, and then you have a context issue. That one's really hard identify. But because the Rider debug now allows you in your threads and variables, you can look very quickly at each individual thread that's running, you can help identify if something resumes on the wrong thread by looking through its state and finding at which point the problem occurred. This is one of the big benefits of using the Rider client and the .NET tools that JetBrains has, is you're able to really easily look through each of these individual problems and find if your state diverges.
If we go back and talk about C++, things are a little bit harder. C++ in native code is really hard to debug. You run into padding issues across the wire, if your buffer is being reused, then you can write garbled information across. Finally, use after free. If you free an object in a loop and then you have another thread that's using it, it looks like data corruption and it's really hard to identify. So I really suggest taking the time to trace your usage of malloc and free so that you don't run into these problems.
That's about it. That's the entire method. You place the bug before you read any code. You go in and read your padded information or your wire information when the two debuggers disagree with each other, and you check the boundary last, not first. Checking your protocol, checking your wire, checking your boundary first leads you to debug more time on the boundary of the protocol than you would spend just debugging one side or the other. Take your boundary as the source of truth. Take it as the answer to the problem, and then go from there on each side based on which side is telling the truth. If you read each side to figure out where the truth is when you place your bug, then when you get down to the boundary, it should be the last thing you've looked at, because you placed both sides and they both have the answers you need. This method has worked for me all the time, and I really hope it works for you all. Making sure you place your code, correlating which packet corresponds to which, reading the bytes when the debuggers are disagreeing with each other, putting all of this together is a method that really helps me out a lot.
For the protocol generation side of things. Do you tend share between those two languages to create that common abstraction. Both sides look really good. If I put it on the client, my client will never be wrong. If I put it on the server, when I'm writing features on the server, it's easy to write part of the protocol there. A shared library means if I write something on the client, I have to look into the shared library. You get the tradeoffs on both ends. Generally, I find that a shared library is normally my least favorite way to approach the problem. The problem itself is normally that the server has to conform to this protocol. I like that a little bit better. Obviously, your client is doing things like rendering and reconciliation. If you choose to make the server crash instead of your client, you can quickly expose it with better logging on a server side. I would stray away from a shared library because that's just more overhead.
What do you think about generated protocol like protobuf and those types of solutions which do some of that heavy lifting for you. I've touched on it slightly because I know that that's used way more than writing it on one side versus the other. I've seen it a lot more. It's a really good solution. I personally prefer to write my protocol once in a language I'm using. If I'm going to write my protocol for the client, in this instance, I would write it in C sharp. It's specifically done on the side I want in the language I want so that it's more native and a little closer. You only have that translation in one point. Yes, it has bindings for every language under the sun. I'm not a huge fan of protobuf because it's one more language you have to deal with. It's one more runtime. It's one more runtime you have to debug. If you're writing your protocol in C sharp, when you're debugging in C sharp, you're debugging the protocol in C sharp. When you're running the protocol in C plus plus, you're debugging the protocol in C plus plus. If both of them lie, then you might have to deal with clients and servers in slightly different versions or have to update them over some period of time versus something where you're in an environment where you can force people to update or control both sides of the equation. Protobuf kind of fails down a little bit because you can version a protocol on the client well and have a server running with that version in mind. Within reasonable timeframes, of course.
Daniel confirmed there was nothing additional to add to the session. He thanked the host for their hosting duties and expressed that it was great to be present at the event. Daniel mentioned genuinely enjoying all of the talks presented throughout the conference.
The host responded by thanking everyone in attendance and expressing appreciation for their participation. They stated they would see attendees at the next event and wished everyone to take care.
The session concluded with a simple farewell as participants signed off.
Keep JetBrains in your library
Save the videos and channels worth coming back to, and find them again in one place.
