Stub Indexes
Stub Trees
A stub tree is a subset of the PSI tree for a file; it is stored in a compact serialized binary format. The PSI tree for a file can be backed either by the AST (built by parsing the file) or by the stub tree deserialized from disk. Switching between the two is transparent.
The stub tree contains only a subset of the nodes. Typically, it contains only the nodes needed to resolve the declarations contained in this file from external files. Trying to access any node that is not part of the stub tree or perform any operation that cannot be satisfied by the stub tree, e.g., accessing the text of a PSI element, causes file parsing to switch from the stub to AST backing.
Each stub in the stub tree is simply a bean class with no behavior. A stub stores a subset of the corresponding PSI element's state, like the element's name, modifier flags like public or final, etc. The stub also holds a pointer to its parent in the tree and a list of its children's stubs.
To support stubs for a custom language, first decide which of the PSI tree elements should be stored as stubs. Typically, stubs are needed for things like methods or fields visible from other files. Usually there is no need to have stubs for things like statements or local variables, which are not visible externally.
Implementation
Stubs Setup
The following steps need to be performed only once for each language that supports stubs:
Change the file element type for the language (the element type returned from
ParserDefinition.getFileNodeType()) to a class that extendsIStubFileElementTypeand override itsgetExternalId()method (see also following item).In the plugin.xml, define the
com.intellij.stubElementTypeHolderextension and specify the interface which contains theIElementTypeconstants used by the language's parser.Define the common
externalIdPrefixto be used for all stub element types (see Adding Stub Elements). SeeStubElementTypeHolderEPdocs for important requirements.
Examples:
JavaStubElementTypesregistered inintellij.java.psi.impl.xmlsee
Angular2MetadataElementTypesfor Kotlin sample
Adding Stub Elements
For each element type that needs to be stored in the stub tree, perform the following steps:
Define an interface for the stub, derived from the
StubElementinterface (example).Provide an implementation for the interface (
example).Make sure the interface for the PSI element extends
StubBasedPsiElementparameterized by the type of the stub interface (example).Make sure the implementation class for the PSI element extends
StubBasedPsiElementBaseparameterized by the type of the stub interface (example). Provide both a constructor that accepts anASTNodeand a constructor that accepts a stub.Create a class that implements
IStubElementTypeand is parameterized with the stub interface and the actual PSI element interface (example). Implement thecreatePsi()andcreateStub()methods for creating PSI from a stub and vice versa. Implement theserialize()anddeserialize()methods for storing the data in a binary stream.Override
getExternalId()according to common usedexternalIdPrefixfor the language (see Stubs Setup).For always-leaf stub nodes return
truefromisAlwaysLeaf()(2023.3). "Container" stubs that do not serialize any data of their own may implementEmptyStubSerializerto optimize storage (2023.3).Use the class implementing
IStubElementTypeas the element type constant when parsing (example).Make sure all methods in the PSI element interface access the stub data rather than the PSI tree when appropriate (example:
Property.getKey()implementation).
By default, if a PSI element extends StubBasedPsiElement, all elements of that type will be stored in the stub tree. To have more precise control over which elements are stored, override IStubElementType.shouldCreateStub() and return false for elements that should not be included in the stub tree. The exclusion is not recursive: if some elements of the element returning false are also stub-based PSI elements, they will be included in the stub tree.
Serializing Data
For serializing string data in stubs, e.g. element names, use StubOutputStream writeName() and readName(). These methods ensure that each unique identifier is stored only once in the data stream. This reduces the size of the serialized stub tree data. See also DataInputOutputUtil.
To change the stored binary format for the stubs (for example, to store some additional data or some new elements), make sure to advance the stub version returned from IStubFileElementType.getStubVersion() for the language. This will cause the stubs and Stub Indexes to be rebuilt, and will avoid mismatches between the stored data format and the code trying to load it.
It is critical to ensure that all information stored in the stub tree depends only on the contents of the file for which stubs are being built, and does not depend on any external files or any other data. Otherwise, the stub tree will not be rebuilt when external dependencies change, leading to stale and incorrect data in the stub tree.
Stub Indexes
When building the stub tree, the plugin can, at the same time, put some data about the stub elements into a number of indexes, which then can be used to find the PSI elements by the corresponding key. Unlike file-based indexes, stub indexes do not support storing custom data as values; the value is always a PSI element. Keys in stub indexes are typically strings (such as class names); other data types are also supported if desired.
A stub index is a class which extends AbstractStubIndex. In the most common case, when the key type is String, use a more specific base class, namely StringStubIndexExtension. Stub index implementation classes are registered in the com.intellij.stubIndex extension point.
To put data into an index, implement IStubElementType.indexStub() (example: JavaClassElementType.indexStub()). This method accepts an IndexSink as a parameter and puts in the index ID and the key for each index in which the element should be stored.
Accessing Stub Indexes
To access the data from an index, the following instance methods are used on the singleton instance managed by the implementation:
Keys
AbstractStubIndex.getAllKeys()/processAllKeys() returns the list of all keys (processes all keys) in the index for the specified project (for example, the list of all class names found in the project).
Elements
StubIndex.getElements() returns the collection of PSI elements corresponding to a certain key (for example, classes with the specified short name) in the specified scope.
Example: JavaAnnotationIndex