rust-analyzer vs. RustRover: Clash of the Titans
In a Nutshell
rust-analyzer and RustRover both re-implement the Rust compiler's front-end for IDE features, but differ fundamentally in architecture: RustRover uses map-reduce indexing with PSI stubs and eager processing, while rust-analyzer employs a Salsa query system with fine-grained dependency tracking and lazy computation. The core technical challenge is Rust's cyclic module graph, macro expansion complexity, and name resolution requiring fixed-point iteration, forcing both tools to balance eager upfront work with lazy per-request performance. Both systems isolate procedural macros in separate processes for safety, but differ in memory management—RustRover leverages Java heap LRU caching while rust-analyzer uses mark-and-sweep collection for type interning and LRU memoization for query results.
These notes were generated by AI and may contain inaccuracies.
Vitaly Bragilevsky hosts a RustRover livestream discussing Rust analyzer and RustRover internals. The panel includes Lukas Wirth, Rust engineer at Zed and team lead for rust-analyzer, and Vlad Bezborodov, engineer at JetBrains working on RustRover. The livestream serves as an extended version of a talk delivered at Rust Week in Utrecht one week prior, focusing on implementation details rather than direct IDE comparisons.
Lukas Wirth began coding around age 12 in high school, writing Java mods for Minecraft. He started writing Rust in 2016 upon entering university and has accumulated roughly 10 years of Rust experience. Vlad Bezborodov discovered Rust in 2014 during high school but began actively writing it around 2017 when he joined JetBrains to work on the predecessor of RustRover, the IntelliJ Rust plugin.
An IDE integrates multiple development tools including version control, debuggers, and file explorers into a single product. The most notable component is the smart text editor providing features like completion and go-to declaration. In modern Rust development, rust-analyzer and RustRover serve as the two production-ready backends for these smart features.
Rust-analyzer operates across various editors through the LSP protocol, while RustRover maintains its own code analysis engine. Both systems must re-implement approximately half the compiler, specifically the front-end components from parsing through MIR generation.
The fundamental difference between compilers and IDEs lies in optimization focus. Compilers prioritize throughput, measuring how many millions of lines of code can be transformed into machine code per second. IDEs prioritize latency, focusing on immediate response times such as displaying completion variants when a dot is typed.
When requesting IDE features like completion, users care only about the specific context and want instant results, without concern for other function bodies, files, or errors elsewhere in the project. This different optimization focus influences algorithm selection and overall architecture, with IDEs preferring lazy algorithms while compilers tend toward eager processing.
Compilers operate as one-shot processes, performing parsing, semantic analysis, code generation, and linking before exiting. IDEs maintain long-running processes that continuously re-analyze codebases with each edit, creating fundamentally different lifecycle requirements.
This lifecycle difference affects data structure choices. Compilers use distinct phase approaches that enable simple memory allocation patterns through arenas that can be discarded after phase transitions. IDEs lack this phase distinction, requiring different approaches to memory usage and garbage collection strategies.
Rust compilers use abstract syntax trees that exclude semantically irrelevant tokens such as whitespace and comments. Rust-analyzer and RustRover preserve these tokens to enable source code surgery without losing significant information like comments.
Macro expansion handling differs significantly between approaches. The Rust compiler flattens parse trees during macro expansion, creating one final fully expanded tree per pass. Rust-analyzer and RustRover maintain separate trees for macro expansions, creating a tree-of-trees structure.
The timeline of Rust IDE development begins with IntelliJ Rust as the first complete engine. RLS (Rust Language Server) followed as an attempt to build a server on top of the Rust compiler itself using save analysis, which compiled everything and dumped large JSON files containing name resolution and type inference results.
RLS suffered from high latencies and proved nearly impossible for implementing completion since rustc works poorly with incomplete code. It relied on Racer, a heuristic-based tool providing poor completion quality. RLS was deprecated in 2022 after being superseded by rust-analyzer.
IntelliJ Rust began in 2015 when Alex Kladov, working at JetBrains, and Alex Kudinkin started the project as a classic plugin architecture written in Kotlin. Vlad Bezborodov joined in 2017, initially working on type inference and macro expansion, with macro expansion implementation completed in 2018. RustRover was announced as a separate product in 2023 and released in 2024.
Rust-analyzer was started by Alex Kladov shortly after IntelliJ Rust, beginning as a syntax or parser library before evolving into the full rust-analyzer. It was written entirely in Rust, receiving type inference and macro expansion capabilities a year after its initial creation. Four years after its start, it became part of the official Rust project, replacing the Rust Language Server.
Rust's module graph within a crate is cyclic, requiring complete module resolution before any useful name resolution can occur. This creates significant upfront work that contradicts typical IDE preferences for lazy processing.
Name resolution in Rust operates through a fixed point iteration algorithm. The process collects all items including modules and their children, collects all imports, attempts to resolve imports with current knowledge, resolves macros using the same mechanism, expands macros, collects newly generated items, and repeats until reaching a fixed point where no further progress occurs.
Several language features break the assumption that all relevant files exist within the crate root directory. The path attribute allows module declarations to point to arbitrary filesystem locations. The include! macro similarly accepts arbitrary file paths. Procedural macros can access any filesystem location, creating additional complications for determining which source files belong to which crates.
A particularly unusual pattern involves module declarations inside local scopes, which compiles on stable Rust but requires extensive upfront work that rust-analyzer and RustRover choose not to support, accepting coverage limitations rather than performance penalties.
Macro_rules items behave differently from normal items due to textual scoping, meaning they only define themselves after their declaration point, unlike other items that exist even before their declaration. This textual scoping enables macro shadowing scenarios.
When a macro creates another macro with the same name, and the created macro is invoked again, Rust produces an ambiguity error. The compiler specifically reports conflicts between macro-expanded names and less macro-expanded names from outer scopes during import or macro resolution. This restriction prevents fixed point resolution from leaking between different macro expansion depths.
Procedural macros present significant challenges as they function as dynamically linked libraries capable of arbitrary host system operations. They can produce side effects through reading files, writing files, and reading environment variables, theoretically preventing compiler caching since caching would eliminate these side effects.
IDEs assume good will and treat procedural macros as pure functions for performance reasons, enabling caching despite theoretical side effect concerns. However, procedural macros can crash or abort processes, potentially bringing down entire IDEs. Both rust-analyzer and RustRover address this by spawning separate proc macro servers as distinct processes, communicating through custom JSON-RPC protocols. Process crashes affect only the proc macro server, which can be restarted without terminating the IDE session.
Rust's sophisticated type system includes local type inference within function bodies, enabling lazy type inference without examining other function bodies, only requiring signature information. This locality supports the IDE preference for lazy processing.
However, Rust's type inference contains thousands of arbitrary rules that must be meticulously replicated. One example demonstrates that method receiver types must be known during method call resolution, with rustc processing statements sequentially. Another example shows that function invocation argument processing doesn't follow written code order, instead using a loop that processes non-lambda arguments first, then lambda declarations.
Method resolution can depend on const arguments, requiring constant expression evaluation at compile time. The compiler performs this constant evaluation using MIR. Implementing MIR within an IDE represents the most feasible approach for supporting such cases, though this particular example currently works in neither rustc nor rust-analyzer, with ongoing development aimed at future support.
RustRover architecture begins with project model definition through Cargo metadata invocation, which generates extensive JSON describing project structure including packages and dependencies. Cargo build execution follows, compiling for smart binaries and executing build scripts necessary for macro expansion and generated Rust code.
Multiple Cargo invocations obtain versions, nested dependency roots, CG options, and other configuration data. The primary output is a crate graph listing crates and their dependencies, with each crate associated with source folders for indexing purposes.
RustRover uses PSI (Program Structure Interface) as an abstraction over syntax trees, implemented as a set of Kotlin
PSI is a set of Kotlin interfaces with one interface per each kind of syntax node. An actual PSI tree for a Rust function shows keywords, whitespace, and comments directly mapped to text. This direct mapping enables immediate text updates in the editor when PSI is manipulated. All IDE features that change code—including refactoring and completion—manipulate PSI directly rather than text.
PSI is backed by two different data structures. The concrete syntax tree consumes large amounts of memory, making it suitable only for files currently open in the editor. The alternative data structure is called PSI stub, which resembles a compiler's AST but is more compact since it excludes whitespace, comments, and even function bodies. Stubs contain only declarations and signatures needed for cross-file analysis.
PSI supports lazy switching from stub to syntax tree on demand. When requesting PSI that doesn't exist in the stub, such as a function body, it is lazily parsed and obtained.
When opening IntelliJ IDEA, an indexing progress appears, planned to be renamed to "analyzing project." The index functions as a map-reduce system where each file from the project model is parsed in parallel and mapped to sets of key-value pairs, which are then joined into a key-value database.
Key index features include parallel file processing for speed, persistence with disk storage to avoid recomputation on server restart, and fine-grained incrementality. When changing a file, only the index portion computed from that file gets updated.
Multiple indexes share the same architecture and indexing phase but differ in their mapping functions. The three-gram index optimizes text search and find usages. The symbol index enables quick navigation to functions, types, and search everywhere. The stub index is most important for RustRover.
Stubs are constructed from syntax trees during indexing and stored on disk in the stub index. Accessing PSI loads from the stub index rather than parsing files again. During indexing, function bodies are not parsed—only lexing occurs with bodies skipped by counting braces, speeding up parsing three times or more.
The dotted line in the architecture diagram represents the critical boundary between eager and lazy processing. While IDEs should ideally operate completely lazily for low latency, delivering decent performance in languages simpler than Rust requires some eager processing. The art of IDE design lies in positioning this boundary correctly.
The eager phase is inevitable but must be incremental in a highly fine-grained manner. Map-reduce indexes fit this requirement naturally since files don't depend on each other, so changes invalidate only one file's index portion.
For many programming languages, map-reduce indexes suffice. Java demonstrates this through package declarations that enable indexing key construction by concatenating package name and item name. Name resolution in Java reduces to hash map lookups.
Rust cannot use the Java approach for name resolution. Rust lacks package declarations, making it impossible to determine where files are included in the module tree or construct indexing keys for declarations. Even with package declarations, re-exports would prevent determining the correct index key to look up.
Rust requires an additional subsystem beyond the name resolution index. This subsystem must maintain incrementality and fine-grained properties to avoid 10-second completion delays after code changes. Name resolution in Rust fits these requirements due to crate-level granularity—changes require recomputation only for the affected crate and dependent crates. Function bodies do not invalidate crate name resolution. Fixed-point iteration performs well with cached macro expansions.
Each diagram component uses different caching, invalidation, and garbage collection strategies. The index uses typical map-reduce incremental updates: file deletion removes that index portion, file changes update that portion. Index data stores on disk.
Macro expansions also store on disk without explicit invalidation. Instead, cache lookup computes hashes of macro declaration inputs and macro invocation bodies to determine cache hits.
Name resolution data (crate map) stores in memory with change tracking via input hashes. The fixed-point iteration is treated as a pure function with hash computation for all inputs, invalidating cache when inputs change.
All other computations use PSI modification tracking with an integer counter incremented on each PSI modification. This tracker attaches to scopes, typically project-global scope and function scope. Typing inside a function invalidates caches for all type inference and other function-related computations. Global scope changes invalidate all lazy computations everywhere. These values store in memory with LRU eviction order.
The Java heap serves as the LRU cache, defaulting to 4 GB for most systems.
Smart IDE features like inspections can access PSI, indexes, and lazily computed types. All lazy computed values are managed and validated through PSI modification tracking, with memory usage managed by the Java heap.
Rust-analyzer uses a query-based frontend similar to rustc rather than map-reduce algorithms. All semantically interesting functions are behind queries that memoize results keyed by inputs while tracking dependencies. This enables fine-grained invalidation tracking—when a dependency query changes and a parent query is invoked, it invalidates and recomputes if the dependency result changed since the last edit.
The definition map query (module graph resolution) invokes item tree queries, which invoke macro expansion queries, parse queries, and SID queries. Dependency management requires care: if an item tree query result remains unchanged after recomputation, the definition map can reuse its previous result. However, depending on parse queries would cause recomputation on every keystroke since parse results always change on edits.
Queries appear as functions annotated with attributes like #[salsa::invoke]. Salsa handles memoization, key setup, and dependency plumbing. Any function call within a query body implicitly sets up dependencies, requiring careful attention to avoid accidental dependency edges.
Rust-analyzer implements three garbage collection approaches. Salsa provides LRU-based garbage collection for memoized query results, limiting stored results per query. For example, at most 128 parse trees remain in memory simultaneously due to their size.
Symbol interning uses a global Arc-based scheme where symbols are shared Arcs via a global hash map for deduplication. Additional unsafe pointer tagging enables thin pointers for cheaper movement.
The most complex garbage collection is a mark-and-sweep tracing collector used exclusively for type IR interning. Types require copy semantics backed by IDs for cheap copyability. During revision bumps (edits), the collector traces from all roots and collects unreachable objects.
The project model implementation is nearly identical to RustRover, reflecting their shared origins. The IDE layer above query structures consumes semantic information by querying the frontend for expression types, symbol definitions under references, and similar operations.
Symbol indices and import maps are maintained differently than RustRover's map-reduce indexes. Rust-analyzer uses large symbol lists per crate or module with state machines for browsing name trees to locate symbols.
The LSP server layer operates as a giant select loop responding to language server protocol messages, task completions, and virtual file system access. This layer handles cache priming, which corresponds to indexing in other systems.
Cache priming is not strictly necessary since there are no indexes to populate. It maximizes system resource concurrency to pre-populate query results by walking the crate graph in topological order and filling early name resolution queries, module graph queries, and symbol indices.
RustRover's debugger integrates a fork of LLDB with two parts: the frontend as part of RustRover's Kotlin codebase and an external LLDB process with custom adjustments.
The evaluate expressions feature allows typing any Rust expression in the evaluate field for interpretation within the running process scope. Implementation involves building MIR for the typed expression on the frontend, including monomorphized MIR for the entire invocation graph. For virtual function calls, the complete call graph gets processed. This MIR is serialized to JSON and handed to LLDB, which contains a custom MIR interpreter.
Rust's complexity creates convenience for language users while increasing implementation complexity for tooling developers. Features that improve language usability often introduce significant implementation challenges. Some Rust features arrived too early without considering implementation consequences, such as procedural macros lacking sandboxing from the start.
Async task visualization support similar to Rider remains a desired feature but requires significant development effort. Debugging asynchronous code represents a major current challenge for Rust developers.
Keep JetBrains in your library
Save the videos and channels worth coming back to, and find them again in one place.
