This article introduces the notion of highlighting complexity and provides recipes for making your code highlighting-friendly, resulting in faster, more efficient highlighting.
Code style is not just for style – it impacts the physical world! The benefits of highlighting-friendly code include:
- Better responsiveness
- Optimized CPU usage
- Efficient memory usage
- Cooler system temperatures
- Quieter operation
- Longer battery life
While monads are burritos, you shouldn’t be frying eggs on your laptop!
Consider highlighting complexity
Imagine you’ve written this function to compute Fibonacci numbers using naive recursion:
def fib(n: Int): Int =
if (n <= 1) n
else fib(n - 1) + fib(n - 2)
It is predictably slow, but you wouldn’t blame Scala for that. The issue is more fundamental and not specific to the programming language. However, this doesn’t mean that the function cannot be made fast. There is a way to adjust the code so it outputs exactly the same sequence much more efficiently.
The same is true for highlighting code. If highlighting is slow, the IDE is not always to blame. Some code is inherently difficult to analyze. However, this doesn’t mean that highlighting cannot be fast. Minor code tweaks can make highlighting significantly more efficient, even if the code stays essentially the same.
So far, so good. However, while algorithmic complexity is “CS 101”, developers rarely think about highlighting complexity. (The two differ: Code might run slow but be easy to highlight, or run fast but be difficult to highlight.) Even if you study compiler construction, it’s primarily not about performance, and parts that are about performance refer to compilers rather than source code. Furthermore, batch-compiling code is not the same as editing code.
Following software engineering best practices may often speed up highlighting. It’s also useful to do in general: keeping your classes and methods small and focused, preferring clarity over cleverness, etc. However, these principles are mostly about cognitive complexity. In contrast to algorithmic complexity, cognitive complexity often correlates with highlighting complexity. Still, they are not the same and sometimes can differ significantly.
When writing code, you should also consider highlighting complexity. If you ignore algorithmic complexity, your code will perform poorly. If you ignore cognitive complexity, your code will be difficult to understand. If you ignore highlighting complexity, your code will take a long time to compile or highlight and will consume excessive resources in the process.
Good code should be good in all respects. Fortunately, the principles for making your code highlighting-friendly are simple and easy to apply in practice. (Most of the recipes are not Scala-specific and can be useful for other languages as well.)
Separate code into modules
Most Scala programmers divide code into packages, but fewer divide code into modules. There’s one and the same reason for both.
In contrast to a language like C, Scala supports packages, and most Scala projects naturally use them. Modules, however, are a concept of IDEs and build tools rather than the programming language, so they are used less often. Even the Java Platform Module System is mostly about compiled classes and JARs rather than source code.
Modules limit the scopes of bindings and introduce an explicit graph of dependencies – otherwise, any source file could, in principle, depend on any other source file. This limits the scope of incremental compilation and analysis, which makes compilation faster, reduces peak resource consumption, and allows modules to compile in parallel.
Likewise, modules improve the performance of highlighting – an IDE can search for entities and invalidate caches more efficiently. Moreover, this improves the UX by making autocomplete and auto-import more relevant, reducing clutter. Another benefit is that you can compile (or recompile) only part of a project when running an application or a unit test in one of the modules (even if other modules don’t compile cleanly).
Packages are often natural boundaries for modules. If there’s only a single module in your project, or if some modules are too large, consider extracting one or more packages into a separate module. Since the refactoring doesn’t affect packages as such, this should be backward-compatible. Furthermore, you can still package the classes into a single JAR – the refactoring is for the source code, but not necessarily the bytecode.
Note that you must use true modules – using multiple directories or multiple source roots is not the same thing. (See multi-project builds for sbt.)
Put classes in separate files
The Scala compiler doesn’t limit how many classes you can add to a source file (or how you name that file). This can be useful, but you shouldn’t overuse this capability.
If you modify only one class in a source file, the Scala compiler cannot compile that class separately – it has to compile the entire source file. The same is generally true for IDEs: You open a file rather than a class in an editor tab, which analyzes the entire file. (However, you can use incremental highlighting to overcome this limitation.)
Furthermore, when each class has a file with a dedicated name, it’s easier to find classes and navigate around the project, even without an IDE. You should put classes into corresponding files the same way you put packages into corresponding directories.
Another reason is import statements. While each class requires its own set of imports, defining multiple classes in a single file merges these imports and makes them common. This can slow down the resolution of references. (If there are many imports and imported entities that, in turn, depend on many imports, then there could be a combinatorial explosion.)
If you notice many relatively large classes in a single file, consider extracting classes into separate source files. It’s easy to do and doesn’t affect backward compatibility. (Obviously, companion classes and sealed class hierarchies should remain in the same file.)
Define classes in packages rather than objects
In Scala, packages and objects are similar, and there are even package objects! This makes it possible to put classes in objects rather than packages. However, there are good reasons to avoid that.
First, since each object is contained in a single source file, multiple classes in an object implies multiple classes in a file, which, as we’ve already seen, is not ideal.
Second, this also affects compiled code, not just source files. While every class is compiled to a separate JVM .class file, as if they were defined in a package, there’s only one outline for the object – pickles or TASTy. As a result, both the compiler and IDE have to process multiple classes even if they need to access only one.
Thus, you should normally define classes in packages rather than objects. Leave objects for methods, variables, and types. (And in Scala 3, even top-level definitions can reside in a package.)
Favor small classes and methods
Yes, yes, you already know this. But there’s a twist. When you normally think of “small”, you often think of “simple”. For example, if a class contains only a few methods with descriptive names, the class looks simple, and you don’t have to analyze the code of these methods to understand what they do.
This luxury, however, doesn’t apply to compilers or IDEs. If you open the file, the entire contents will be analyzed, and if the methods (and consequently the class) are large, the analysis will consume time and resources.
Consider splitting large classes and methods into smaller ones, even if they are simple. For highlighting, “lines of code” matter; even a single class or method can be too much if it’s very large.
This also applies to generated sources: If a source file is generated and other sources depend on it, you don’t need to look into that code, but IDEs and compilers still do. When generating code, divide the output into smaller parts – files, classes, and methods; don’t mix everything into one blob.
Depend on interfaces rather than classes
It’s good to “program to an interface” in general, and this can also help with highlighting.
Suppose there is a large class with a few methods that comprise its API. Even if you access only the API, reading the source file requires parsing the entire class, including all the implementation details. And even if you specify the types explicitly, resolving the corresponding references requires processing many imports.
Therefore, if a class is very large, consider extracting an interface instead of referencing the class directly.
Avoid wildcard imports
Using named imports rather than wildcard imports is a well-known best practice. It makes code more readable – you can clearly see where symbols come from. It also makes your code more robust. (Otherwise, code might stop compiling after a library adds a class that conflicts with another imported class.) And there’s less clutter – autocomplete will show only relevant symbols that are actually in use.
Furthermore, named imports can speed up code analysis. When resolving identifiers, each wildcard import has to be checked, and import expressions might, in turn, depend on wildcard imports above. There might be imports from objects, which themselves depend on imports elsewhere. All of that is not limited to the file being highlighted. Even if your code depends only on signatures in other files, because paths in the type annotations are not absolute, the analysis still has to process imports in those files.
Wildcard imports are especially problematic for implicits. Because implicits are, well, implicit, and might require other implicits, searching for them can be computationally intensive. And if implicits are imported using a wildcard, then both the usage and the import are implicit. This complicates the task even more – not only does the analysis need to find some vague entity, but it also has to look in a blurry scope.
Therefore, prefer specific imports to wildcard imports. Convert existing wildcards to named imports. In Scala 2, consider importing implicits by name. Although given imports in Scala 3 are an improvement, they are effectively wildcard imports and thus rely on good library design. To be on the safe side, prefer by-type imports to plain given imports. (And if you’re designing a library, define implicits in a separate package or object.)
Prefer imports to mixins
It’s possible to use inheritance instead of imports. We can see this even in Java: Every TestCase is also Assert, so you can access methods such as assertEquals without having to import them. This might seem convenient. However, this is effectively a forced wildcard import, with all the usual drawbacks. It’s better to import Assert.assertEquals selectively (or import Assert.*, as an option).
Furthermore, the approach with subclassing or mixing in traits is slower compared to regular wildcard imports. Analysis has to take inheritance and linearization, as well as overloading and overriding, into account. And if you modify the trait, classes that use it have to be recompiled.
If some definitions are effectively static, put them in an object rather than a trait, so that clients import rather than inherit them.
Declare classes and methods private
There are many good reasons to minimize the accessibility of classes and methods: to distinguish between API and implementation, to maintain source and binary compatibility, to prevent clutter in autocomplete, and to reduce cognitive load.
What’s less known is that declaring classes and methods private, whenever possible, improves the performance of compilation and highlighting. Incremental compilers don’t include private members when determining APIs and thus don’t need to store and compare them. In the process of resolving references, IDEs can skip inaccessible elements faster. When you write “Foo”, you already know which Foo is implied. However, you might be surprised by how much computation resolving a reference often involves. Declaring unsuitable Foos inaccessible helps make analysis faster.
The Scala plugin can help by automatically detecting declarations that can be private.
Specify types of public or complex definitions
Each non-local definition should either be private or have a type annotation. Definitions that are accessible to clients comprise an API. APIs are boundaries of abstraction and thus must be explicit; clients shouldn’t have to study the implementation – the right-hand side – to understand the signature – the left-hand side. In contrast to implementations, APIs must be stable and must not depend on the contents of the right-hand side. Type annotations make APIs both explicit and stable.
Type annotations greatly help incremental computations. When signatures are stable, fewer classes need to be recompiled after a code modification. Likewise, more caches can be reused when you edit code in an IDE, making highlighting faster and reducing resource consumption.
Thus, it’s best to always specify the types of non-private members explicitly. Note that you should specify the type even if there’s overriding because the inferred type might be more specific, at least in Scala 2. (For example, if a superclass method returns Seq[Int] and the subclass method is just = List(1), the type of the latter would be List[Int], which might affect clients that use the subclass directly.) You should also specify the types of protected members, not just public ones – subclasses are also clients. (As an exception, you may omit types when the right-hand side is both simple and stable, e.g., a literal. That said, having the type spelled out explicitly is often better, both for humans and compilers.)
Furthermore, explicit types can benefit even private and local definitions. While an incremental compiler recompiles the entire file, an IDE can invalidate caches more gradually and within a narrower scope. Thus, add type annotations to private members if they are complex – this can make editing code more efficient. Also, specify the types of complex local variables. (Sometimes you may first need to extract a method or introduce a variable to specify the type.)
Code Style | Type Annotation in the Scala plugin requires type annotations for public and protected members – they are automatically added by refactorings and code generation, and are checked by the corresponding inspection. However, there are exceptions for simple expressions, and they are not required for private or local definitions, regardless of complexity. You can make these settings stricter to be on the safe side.
Favor standard language features over macros
The concept behind macros might seem tempting – you do computations at compile time rather than at runtime. However, “compile time” is also “highlighting time”, which is true regardless of whether you use a compiler or an IDE when editing code… unless you always write everything in one go, without any assistance. So, macros might interfere with writing and editing code, making feedback slower and consuming more resources. Note that this applies not just to defining a macro, which requires a feature flag, but also to using macros, which doesn’t require a feature flag.
Macros are rarely actually needed. Take, for example, Lisp: The syntax is very limited, and the language is dynamic, so no static analysis is performed anyway. Scala, however, is a very expressive language as it is, and it’s statically typed. In Scala, the standard language features are sufficient for most tasks. In such a case, macros only make static analysis, as well as understanding code, more difficult. Thus, when writing code, reach for the standard language features first: type parameters, implicit parameters, etc. Macros are supposed to be the last resort, not a go-to solution.
This can be generalized: Don’t use complex language features just “because you can”, only when they are really needed; prefer the least powerful solution that solves the problem. For more details on this topic, see Lean Scala by Martin Odersky.
Apply these principles to AI-generated code
Even if you use AI to generate 100% of your code, you still read that code. (Right?) Therefore, producing highlighting-friendly code is as relevant as ever – the code is generated in a data center but is highlighted on your machine. This also improves incremental compilation, reducing system load when using agents. Moreover, it prevents context stuffing (when a model loads irrelevant information), which improves accuracy and reduces costs.
The first thing you can do is lead AI by example, because models tend to propagate existing conventions and coding styles. In a new project, you can explicitly add recommendations to AGENTS.md. Last but not least, you can always refactor your code, whether it’s written by a human or AI.
Summary
That said, the performance of your IDE is also important. We’re constantly working on improving the performance of both IntelliJ IDEA and the Scala plugin, and there are tips for improving performance that you can apply in practice. However, just as no amount of compiler optimizations can fix the example with naive recursion, highlighting may sometimes require assistance from your side.
As with everything, highlighting complexity is not the only factor; you need to balance different considerations. But often, there’s no contradiction: Clean code improves highlighting complexity, and improving highlighting complexity results in cleaner code. In any case, it’s useful to always consider highlighting complexity and having the recipes at hand.
For more details, see the corresponding ticket in YouTrack. It also lists features that can help you apply the refactorings more easily. If you find them useful, vote for the tickets so we know there is demand.
If you have any questions, feel free to ask us on Discord.
Happy developing!
The Scala team at JetBrains
Log in
