# Introduction to Integration Tests

> For the complete documentation index, see [llms.txt](https://plugins.jetbrains.com/docs/intellij/llms.txt).

> **Tip:**
> This page is part of the [Integration Tests](integration-tests.html) tutorial.

## Adding dependencies

Integration testing framework consists of two main components:

* Starter: Handles IDE configuration, test project setup, IDE startup, and output collection.

* Driver: Provides additional functionality like UI interaction and JMX calls of IDE's API.

The Starter framework exclusively supports JUnit 5, as it leverages JUnit 5's extensions and specialized listeners that aren't available in JUnit 4.

To create a new task - `integrationTest`, define new test source roots - `integrationTest`, and add required dependencies, update the `build.gradle.kts` file:

```KOTLIN
import org.jetbrains.intellij.platform.gradle.*

plugins {
  //...
  id("org.jetbrains.intellij.platform") version "2.19.0"
}

sourceSets {
  create("integrationTest") {
    compileClasspath += sourceSets.main.get().output
    runtimeClasspath += sourceSets.main.get().output
  }
}

repositories {
  //...
  intellijPlatform {
    defaultRepositories()
  }
}

val integrationTestImplementation by configurations.getting {
  extendsFrom(configurations.testImplementation.get())
}

dependencies {
  intellijPlatform {
    //...
    intellijIdea("2026.1")
    testFramework(TestFrameworkType.Starter, configurationName = "integrationTestImplementation")
  }

  integrationTestImplementation("org.junit.jupiter:junit-jupiter:5.7.1")
  integrationTestImplementation("org.kodein.di:kodein-di-jvm:7.20.2")
  integrationTestImplementation("org.jetbrains.kotlinx:kotlinx-coroutines-core-jvm:1.10.1")
}

val integrationTest by intellijPlatformTesting.testIdeUi.registering {
  task {
    val integrationTestSourceSet = sourceSets.getByName("integrationTest")
    testClassesDirs = integrationTestSourceSet.output.classesDirs
    classpath = integrationTestSourceSet.runtimeClasspath
    useJUnitPlatform()
  }
}
```

The following dependencies are required:

* `testFramework(TestFrameworkType.Starter)` will add all required dependencies for writing integration tests - Starter and Driver frameworks.

* `org.kodein.di:kodein-di-jvm` is a dependency injection framework used by Starter for configuration.

* `org.jetbrains.kotlinx:kotlinx-coroutines-core-jvm` is required for Starter framework which is implemented using Kotlin coroutines.

* `intellijIdeaUltimate(...)` the IDE version for which the plugin is compiled

This configuration does the following:

* Imports test implementation dependencies to `integrationTest` implementation dependencies

* Defines new test source roots in `src/integrationTest`

* Creates a new task `integrationTest`

* Makes the test task depend on `prepareSandbox`, ensuring the plugin is built before tests run.

* To test a plugin, the Starter framework needs to know where to find the plugin distribution for installation in the IDE. The `path.to.build.plugin` system property points to the plugin distribution file.

* Enables JUnit Platform for test execution.

For more details about configuring integration tests, please refer to [Gradle docs](https://docs.gradle.org/current/userguide/java_testing.html#sec:configuring_java_integration_tests).

> **Note:**
> Note: Driver and UI components are in the experimental state and subject to API changes.

## Creating the First Integration Test

Now that the configuration is complete, it's time to write the first integration test, which will:

* Start the IDE with the plugin installed.

* Wait for all background processes to complete.

* Perform a shutdown.

Create a new Kotlin file in `src/integrationTest/kotlin` with the following code:

```KOTLIN
import com.intellij.ide.starter.driver.engine.runIdeWithDriver
import com.intellij.ide.starter.ide.IdeProductProvider
import com.intellij.ide.starter.models.TestCase
import com.intellij.ide.starter.project.NoProject
import com.intellij.ide.starter.runner.Starter
import org.junit.jupiter.api.Test

class PluginTest {
  @Test
  fun simpleTestWithoutProject() {
    Starter.newContext(
      testName = "testExample",
      TestCase(IdeInfo.IdeaUltimate, projectInfo = NoProject)
    ).apply {
      val pathToPlugin = System.getProperty("path.to.build.plugin")
      PluginConfigurator(this).installPluginFromDir(Path.of(pathToPlugin))
    }.runIdeWithDriver().useDriverAndCloseIde {
    }
  }
}
```

Each part of the test is described below:

### 1. Context Creation

262+:

```KOTLIN
Starter.newContext(
  testName = "testExample",
  TestCase(IdeInfo.IdeaUltimate, projectInfo = NoProject))
```

Pre-262:

```KOTLIN
Starter.newContext(
  testName = "testExample",
  TestCase(IdeProductProvider.IU, projectInfo = NoProject))
```

> **Note:**
> Starting with version 262, the recommended way to indicate the IDE has changed from `IdeProductProvider.IU` to `IdeInfo.IdeaUltimate`.

The Context object stores IDE runtime configuration:

* IDE type (e.g., IntelliJ IDEA, PhpStorm, GoLand).

* Project configuration (using `NoProject` for this example).

* Custom VM options, paths, and SDK settings.

By default, the IDE version provided by the sandbox is used for tests. A different version can be specified with:

262+:

```KOTLIN
TestCase(IdeInfo.IdeaUltimate, projectInfo = NoProject).withVersion("2024.3")
```

Pre-262:

```KOTLIN
TestCase(IdeProductProvider.IU, projectInfo = NoProject).withVersion("2024.3")
```

The `testName` parameter defines the folder name for test artifacts, which is useful when running multiple IDE instances in a single test. The test case starts the IDE without any project, so the welcome screen will be shown.

### 2. Plugin Installation (Optional)

```KOTLIN
.apply {
  val pathToPlugin = System.getProperty("path.to.build.plugin")
  PluginConfigurator(this).installPluginFromFolder(File(pathToPlugin))
}
```

This step configures the installation of the plugin being developed, using the plugin distribution path defined in the Gradle configuration with the `path.to.build.plugin` system property.

> **Note:**
> [PluginConfigurator](https://github.com/JetBrains/intellij-community/tree/idea/262.10968.63/tools/intellij.tools.ide.starter/src/com/intellij/ide/starter/plugins/PluginConfigurator.kt) can install plugins from local paths or Marketplace.

### 3. IDE Life Cycle Management

```KOTLIN
.runIdeWithDriver().useDriverAndCloseIde {
}
```

These two methods:

* Start the IDE instance (`runIdeWithDriver`).

* Shut down the IDE (`useDriverAndCloseIde`).

The empty lambda is used for IDE interactions (`useDriverAndCloseIde`).

> **Note:**
> When the test is run for the first time, it may take longer than expected since it needs to download the IDE. Subsequent runs will be faster, using the cached IDE version.

## Understanding the Test Architecture

The test can now be run and it should pass.

Integration tests operate across two separate processes:

* Test process: * Executes test code, sending commands to the IDE. * Manages the IDE life cycle. * Controls test flow and assertions.

* IDE process: * Listens and executes commands from the test process.

This dual-process architecture explains several key aspects of integration testing:

* Why debugging requires special considerations.

* The need for a communication protocol between test and IDE processes.

* Why a built plugin distribution is required.

* The origin of certain test-specific exceptions.

## Debugging the Test

> **Tip:**
> To attach the debugger from the console to the test IDE instance automatically, enable the `debugger.auto.attach.from.console` registry key (open `Help | Find Action` and search for Registry).

Since the IDE runs as a separate process from the test, the test cannot be debugged directly. To debug a test, it is required to attach the debugger to the running IDE process.

General debugging workflow:

1. Run the test in debug mode.

2. In the Console tab of the Debug tool window, click the Attach debugger link that appears once the IDE starts listening for a debugger connection.

## Modifying VM Options

The VM options of the IDE under test (such as heap size or system properties) can be adjusted from the test at two levels:

* `IDETestContext.applyVMOptionsPatch()` applies them to the whole context, so they are reused across every run of that context.

* `IDERunContext.addVMOptionsPatch()` applies them to the current run only.

The patch block exposes a `VMOptions` receiver with helpers such as `withXmx()` and `addSystemProperty()`:

```KOTLIN
context.applyVMOptionsPatch {
  withXmx(2048)
  addSystemProperty("idea.trust.all.projects", true)
  addSystemProperty("ide.show.tips.on.startup.default.value", false)
}
```

* `idea.trust.all.projects` trusts opened projects automatically, so the trust confirmation dialog does not block the test.

* `ide.show.tips.on.startup.default.value` disables the Tips of the Day dialog on startup, so it does not interfere with UI interactions.

## Opening Projects in Tests

While starting the IDE with an empty project is useful, often it is required to use actual projects to verify real-world scenarios. The test can be modified to open a project.

The framework supports several ways to specify test projects:

* [GitHub projects](https://github.com/JetBrains/intellij-community/tree/idea/262.10968.63/tools/intellij.tools.ide.starter/src/com/intellij/ide/starter/project/GitHubProject.kt): ```KOTLIN GitHubProject.fromGithub( branchName = "master", "JetBrains/ij-perf-report-aggregator" ) ```

* [Remote archives](https://github.com/JetBrains/intellij-community/tree/idea/262.10968.63/tools/intellij.tools.ide.starter/src/com/intellij/ide/starter/project/RemoteArchiveProjectInfo.kt): ```KOTLIN RemoteArchiveProjectInfo( "https://github.com/example/repo/archive/master.zip" ) ```

* [Local projects](https://github.com/JetBrains/intellij-community/tree/idea/262.10968.63/tools/intellij.tools.ide.starter/src/com/intellij/ide/starter/project/LocalProjectInfo.kt): ```KOTLIN LocalProjectInfo( Path("src/test/resources/test-projects/simple-project") ) ```

The following example shows how to open a GitHub project:

```KOTLIN
@Test
fun simpleTest() {
  Starter.newContext(
    "testExample",
    TestCase(
      IdeInfo.IdeaUltimate,
      GitHubProject.fromGithub(
        branchName = "master",
        repoRelativeUrl = "JetBrains/ij-perf-report-aggregator"
      )
    )
  ).apply {
    val pathToPlugin = System.getProperty("path.to.build.plugin")
    PluginConfigurator(this).installPluginFromDir(Path.of(pathToPlugin))
  }.runIdeWithDriver().useDriverAndCloseIde {
    waitForIndicators(5.minutes)
  }
}
```

While simple, this test verifies a critical aspect: the plugin doesn't interfere with IDE startup.

> **Note:**
> `waitForIndicators` inside the lambda ensures that the test waits till all indicators are gone before exiting the IDE.

## Catching Exceptions from IDE

The test has one critical limitation: it won't detect exceptions or freezes occurring within the IDE process. The reason and the fix are explained below.

Due to the two-process architecture:

* Exceptions in the IDE process aren't automatically propagated to the test process.

* A bundled plugin collects exceptions from the IDE's [MessageBus](https://github.com/JetBrains/intellij-community/tree/idea/262.10968.63/platform/extensions/src/com/intellij/util/messages/MessageBus.kt).

* These exceptions are stored in the error folder within the logs.

* The Starter framework collects exceptions from the IDE and pass them to the method `reportTestFailure` in object registered as [CIServer](https://github.com/JetBrains/intellij-community/tree/idea/262.10968.63/tools/intellij.tools.ide.starter/src/com/intellij/ide/starter/ci/CIServer.kt) in DI.

TeamCity reporting is used by default, falling back to [NoCIServer](https://github.com/JetBrains/intellij-community/tree/idea/262.10968.63/tools/intellij.tools.ide.starter/src/com/intellij/ide/starter/ci/NoCIServer.kt) for other environments. However, this can be customized by using Kodein Dependency Injection.

The following example shows how to make tests fail when IDE exceptions occur:

```KOTLIN
init {
  di = DI {
    extend(di)
    bindSingleton<CIServer>(overrides = true) {
      object : CIServer by NoCIServer {
        override fun reportTestFailure(
          testName: String,
          message: String,
          details: String,
          linkToLogs: String?
        ) {
          fail { "$testName fails: $message. \n$details" }
        }
      }
    }
  }
}
```

The code above:

* Overrides a default `CIServer` object with a custom implementation.

* Provides `reportTestFailure` method which fails the test with detailed error information if any IDE exception is recorded during the test run.

This extensibility pattern can be applied to customize other aspects of the Starter framework as needed.

## Complete Example

Here's the complete test implementation that forms the foundation for future plugin testing:

```KOTLIN
import com.intellij.driver.sdk.waitForIndicators
import com.intellij.ide.starter.ci.CIServer
import com.intellij.ide.starter.ci.NoCIServer
import com.intellij.ide.starter.di.di
import com.intellij.ide.starter.driver.engine.runIdeWithDriver
import com.intellij.ide.starter.ide.IdeProductProvider
import com.intellij.ide.starter.models.TestCase
import com.intellij.ide.starter.project.GitHubProject
import com.intellij.ide.starter.project.NoProject
import com.intellij.ide.starter.runner.Starter
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.fail
import org.kodein.di.DI
import org.kodein.di.bindSingleton
import kotlin.time.Duration.Companion.minutes

class PluginTest {
  init {
    di = DI {
      extend(di)
      bindSingleton<CIServer>(overrides = true) {
        object : CIServer by NoCIServer {
          override fun reportTestFailure(
            testName: String,
            message: String,
            details: String,
            linkToLogs: String?
          ) {
            fail { "$testName fails: $message. \n$details" }
          }
        }
      }
    }
  }
  @Test
  fun simpleTest() {
    val result = Starter.newContext(
      "testExample",
      TestCase(
        IdeInfo.IdeaUltimate,
        GitHubProject.fromGithub(
          branchName = "master",
          repoRelativeUrl = "JetBrains/ij-perf-report-aggregator"
        )
      )
    ).runIdeWithDriver().useDriverAndCloseIde {
      waitForIndicators(5.minutes)
    }
  }
}
```

This test provides a robust foundation for more elaborate tests by:

* Downloading and opening a real project.

* Starting the IDE with a plugin installed.

* Waiting for all background processes to complete.

* Monitoring for exceptions and freezes.

* Performing a shutdown.

Example integration tests are available in the [ide-starter-examples](https://github.com/JetBrains/ide-starter-examples/tree/main) repository.