EDC 开源代码，connector main

# Integration testing ## Decision Extend the existing `EdcExtension` JUnit facility (that can currently run a single EDC runtime and supports stubbing and extension of runtime services) as follows: - Load more than one EDC runtime. - Load EDC runtimes in separate Class Loaders. - Run each EDC runtime Class Loader with its effective runtime class path, based on its module's Gradle configuration. ## Rationale The need to provide an integration test harness that supports multiple runtimes emerges from multiple needs: - Stabilizing [samples](/samples) that run multiple connectors, which have been breaking frequently. - Testing system behavior when multiple connectors interact, e.g. the contract negotiation process. - Testing system behavior upon component failure. - Providing a test facility for factoring out application components to separate runtimes (e.g. [DPF](https://github.com/eclipse-edc/Connector/issues/463)). Key drivers for the choice are: - Fast and efficient run in CI. - Fast "inner loop" (i.e. ability to quickly rerun tests after changing code) and debuggability for developers. - Use of existing frameworks, stability and portability. We have performed technical spikes testing multiple approaches (detailed further below), including various combinations of: - JUnit - [Docker compose](https://docs.docker.com/compose/) - Starting custom Class Loaders for separate threads for the Provider and Connector with separate class paths. Using distinct Class Loaders (and threads) for each service effectively provides full runtime isolation within a single JVM - [Testcontainers](https://www.testcontainers.org/) with custom containers was also evaluated, but support for bidirectional communication to host is complex, and we didn't manage to get it running. Spinning additional Class Loaders for runtimes with JUnit provides very fast inner loop. Using the Gradle Classpath is [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) and ensures the runtime under test exactly matches the standalone one. In contrast, approaches based on Docker have a slow inner loop and require rebuild between runs. The approach used is not limited to the Dataspace Connector, it can be used to run any Java module if required in the future. ## Spikes We have performed technical spikes on [sample 04.0-file-transfer](/samples/04.0-file-transfer/README.md), that runs two EDC connectors, a Consumer and a Provider. [Spike code is in a forked repository](https://github.com/agera-edc/DataSpaceConnector/pull/3/files). The test requires three components: - Consumer EDC connector - Provider EDC connector - HTTP client code to interact with the Consumer provider API We have written code for the following three options: - **Docker-compose** which works but provides an inconvenient inner loop. - **Class Loader with Gradle Classpath**, which uses Gradle to determine the effective class path for each runtime, and is very efficient. It is therefore the base for the recommendation above. - **Class Loader with Shadow JAR**, a variant of the above which uses the Shadow JAR, and provides a less efficient inner loop. ### Docker-compose In this setup, we run both connectors in Docker containers. Once they are up, we run HTTP client code in a JUnit test running on the host. ```yaml # docker-compose.yaml services: sample04-connector-provider: build: context: . target: sample04-connector-provider ports: - "8181:8181" volumes: - /tmp/provider:/tmp/provider - /tmp/consumer:/tmp/consumer sample04-connector-consumer: build: context: . target: sample04-connector-consumer ports: - "9191:9191" volumes: - /tmp/provider:/tmp/provider - /tmp/consumer:/tmp/consumer ``` ```shell $ (cd samples/04.0-file-transfer && docker-compose up -d) [...] Successfully built 26b639e8f852 Successfully tagged 040-file-transfer_sample04-connector-consumer:latest [+] Running 3/3 ⠿ Network 040-file-transfer_default Created 0.0s ⠿ Container sample04-connector-provider Started 0.7s ⠿ Container sample04-connector-consumer Started 0.6s ``` ```shell RUN_INTEGRATION_TEST=true EDC_PROVIDER_CONNECTOR_HOST=http://sample04-connector-provider:8181 time ./gradlew cleanTest :samples:04.0-file-transfer:integration-tests:test --tests org.eclipse.dataspaceconnector.samples.FileTransferSystemTest ``` This setup is stable and straightforward and also resembles a real deployment scenario in which provider and consumer connectors are running as a separate independent java process, but the inner loop is not very efficient for local development as to reload every new code change connector jars needs to rebuild and restart docker process to use latest jars. Debugging a remote process is possible but adds complexity. For faster local development/debugging one efficient approach could be to run connectors code in debug mode within IDE and necessary IDE run configurations can be provided as xml config files. These run configurations also can be committed in the repository to reduce local development environment setup effort. ### Class Loader with Gradle Classpath The integration tests module only contains the classpath of the Provider module: ```kotlin implementation(project(":samples:04.0-file-transfer:provider")) ``` The JUnit integration test runs the Provider using the preexisting `EdcExtension`, and the Consumer using a newly developed extension: ```java // EDC Consumer runtime @RegisterExtension static EdcRuntimeExtension consumer = new EdcRuntimeExtension( ":samples:04.0-file-transfer:consumer", // Gradle module of the runtime to be started "consumer", // prefix for console log output Map.of( // settings "web.http.port", String.valueOf(CONSUMER_CONNECTOR_PORT), "edc.api.control.auth.apikey.value", API_KEY_CONTROL_AUTH, "ids.webhook.address", CONSUMER_CONNECTOR_HOST)); // EDC Provider runtime @RegisterExtension static EdcRuntimeExtension provider = new EdcRuntimeExtension( ":samples:04.0-file-transfer:provider", "provider", Map.of(...)); ``` The extension determines the class path by running a custom Gradle task: ```java // EdcRuntimeExtension.java Runtime.getRuntime().exec("/gradlew -q " + moduleName + ":printClasspath"); // ... process classpath (see below) ... // run a thread with a custom class loader var classLoader = URLClassLoader.newInstance(classPathEntries, ClassLoader.getSystemClassLoader()); var mainClassName = BaseRuntime.class.getCanonicalName(); var mainClass = classLoader.loadClass(mainClassName); var mainMethod = mainClass.getMethod("main", String[].class); runtimeThread = new Thread(() -> { Thread.currentThread().setContextClassLoader(classLoader); mainMethod.invoke(null, new Object[]{new String[0]}); }) ``` ```kotlin // build.gradle.kts (under allprojects) tasks.register("printClasspath") { doLast { println("${sourceSets["main"].runtimeClasspath.asPath}"); } } ``` The Gradle classpath is composed of JARs within the Connector project (e.g. `/path/to/EclipseDataSpaceConnector/extensions/api/control/build/libs/control-0.0.1-SNAPSHOT.jar`) and JARs from the Gradle cache (e.g. `/home/user/.gradle/caches/modules-2/files-2.1/org.eclipse.jetty/jetty-util/11.0.6/292fa5d7b2cef3483da8a7fa9dd608bfc9896564/jetty-util-11.0.6.jar`). The Extension code replaces JAR en