Git Stub Downloader (#596)

when stubsMode is set to LOCAL or REMOTE, and repositoryRoot starts with git:// we can clone the provided git repository, and search for the folder with stubs for the given artifact. So if the git repo has a folder structure of groupid/artifactid/version (where group id is either dot or slash separated), then we will provide a path to that repository for the stub runner to harvest the stubs

- externalized versions
- added more debugging messages
- added ContractProjectUpdater that updates the project containing contracts from SCM. ATM supports only git
- added ResourceResolver that retrieves the ProtocolResolvers. It does it via spring.factories entries containing StubDownloaderBuilder. SDP extends ProtocolResovler.
added StubRunner.properties map, that will contain any properties that will be later used by any StubDownloader implementations
- added PUBLISH_STUBS_TO_SCM env var for Docker, so that publishStubsToScm task gets called
- updated docs

Breaking:

- StubDownloaderBuilder extends ProtocolResovler. By default the ProtocolResolver methods return null.
- stubRunnerOptions.stubRepositoryRoot is a Resource not a String
- generateWireMockClientStubs Gradle task got removed
- if folder with contracts has a subfolder called contracts, we will pick contracts from the subfolder

fixes #580
This commit is contained in:
Marcin Grzejszczak
2018-03-31 09:35:00 +02:00
committed by GitHub
parent 663e1929d8
commit 0fe31ce3a5
220 changed files with 4111 additions and 4707 deletions

View File

@@ -13,6 +13,7 @@ have Docker installed.
Here you can find the Spring Cloud Contract folder structure
```
├── config
├── docker
├── samples
├── scripts
@@ -26,6 +27,7 @@ Here you can find the Spring Cloud Contract folder structure
└── tests
```
- `config` - folder contains setup for Spring Cloud Release Tools automated release process
- `docker` - folder contains docker images
- `samples` - folder contains test samples together with standalone ones used also to build documentation
- `scripts` - contains scripts to build and test `Spring Cloud Contract` with Maven, Gradle and standalone projects
@@ -33,6 +35,7 @@ Here you can find the Spring Cloud Contract folder structure
- `spring-cloud-contract-starters` - contains Spring Cloud Contract Starters
- `spring-cloud-contract-spec` - contains specification modules (contains concept of a Contract)
- `spring-cloud-contract-stub-runner` - contains Stub Runner related modules
- `spring-cloud-contract-stub-runner-boot` - contains Stub Runner Boot app
- `spring-cloud-contract-tools` - Gradle and Maven plugin for `Spring Cloud Contract Verifier`
- `spring-cloud-contract-verifier` - core of the `Spring Cloud Contract Verifier` functionality
- `spring-cloud-contract-wiremock` - all WireMock related functionality

View File

@@ -4,6 +4,9 @@
== Migrations
TIP: For up to date migration guides please visit
the project's https://github.com/spring-cloud/spring-cloud-contract/wiki/[wiki page].
This section covers migrating from one version of Spring Cloud Contract Verifier to the
next version. It covers the following versions upgrade paths:

View File

@@ -5,12 +5,12 @@
:numbered:
:icons: font
:sectlinks: true
:branch: 1.2.x
:branch: master
= Spring Cloud Contract
_Documentation Authors: Adam Dudczak, Mathias Düsterhöft, Marcin Grzejszczak, Dennis Kieselhorst, Jakub Kubryński, Karol Lassak,
Olga Maciaszek-Sharma, Mariusz Smykuła, Dave Syer, Jay Bryant
Olga Maciaszek-Sharma, Mariusz Smykuła, Dave Syer, Jay Bryant_
{spring-cloud-version}

View File

@@ -1606,4 +1606,56 @@ Now you can pick a folder with the source of your stubs.
IMPORTANT: If you do not provide any implementation, then the default is used (scan classpath).
If you provide the `stubsMode = StubRunnerProperties.StubsMode.LOCAL` or
`, stubsMode = StubRunnerProperties.StubsMode.REMOTE` then the Aether implementation will be used
If you provide more than one, then the first one on the list is used.
If you provide more than one, then the first one on the list is used.
[[scm-stub-downloader]]
=== Using the SCM Stub Downloader
Whenever the `repositoryRoot` starts with a SCM protocol
(currently we support only `git://`), the stub downloader will try
to clone the repository and use it as a source of contracts
to generate tests or stubs.
Either via environment variables, system properties, properties set
inside the plugin or contracts repository configuration you can
tweak the downloader's behaviour. Below you can find the list of
properties
.SCM Stub Downloader properties
|===
|Type of a property |Name of the property | Description
|
* `git.branch` (plugin prop)
* `stubrunner.properties.git.branch` (system prop)
* `STUBRUNNER_PROPERTIES_GIT_BRANCH` (env prop)
|master
|Which branch to checkout
|
* `git.username` (plugin prop)
* `stubrunner.properties.git.username` (system prop)
* `STUBRUNNER_PROPERTIES_GIT_USERNAME` (env prop)
|
|Git clone username
|
* `git.password` (plugin prop)
* `stubrunner.properties.git.password` (system prop)
* `STUBRUNNER_PROPERTIES_GIT_PASSWORD` (env prop)
|
|Git clone password
|
* `git.no-of-attempts` (plugin prop)
* `stubrunner.properties.git.no-of-attempts` (system prop)
* `STUBRUNNER_PROPERTIES_GIT_NO_OF_ATTEMPTS` (env prop)
|10
|Number of attempts to push the commits to `origin`
|
* `git.wait-between-attempts` (Plugin prop)
* `stubrunner.properties.git.wait-between-attempts` (system prop)
* `STUBRUNNER_PROPERTIES_GIT_WAIT_BETWEEN_ATTEMPTS` (env prop)
|1000
|Number of millis to wait between attempts to push the commits to `origin`
|===

View File

@@ -1,10 +1,12 @@
:introduction_url: https://raw.githubusercontent.com/spring-cloud/spring-cloud-contract/{branch}
:samples_branch: 2.0.x
:samples_branch: 2.0.x
== Spring Cloud Contract FAQ
=== Why use Spring Cloud Contract Verifier and not X ?
For the time being Spring Cloud Contract Verifier is a JVM based tool. So it could be your first pick when you're already creating
For the time being Spring Cloud Contract is a JVM based tool. So it could be your first pick when you're already creating
software for the JVM. This project has a lot of really interesting features but especially quite a few of them definitely make
Spring Cloud Contract Verifier stand out on the "market" of Consumer Driven Contract (CDC) tooling. Out of many the most interesting are:
@@ -14,6 +16,8 @@ Spring Cloud Contract Verifier stand out on the "market" of Consumer Driven Cont
- Automatic generation of tests from the defined Contract
- Stub Runner functionality - the stubs are automatically downloaded at runtime from Nexus / Artifactory
- Spring Cloud integration - no discovery service is needed for integration tests
- Spring Cloud Contract integrates with Pact out of the box and provides easy hooks to extend its functionality
- Via Docker adds support for any language & framework used
=== I don't want to write a contract in Groovy!
@@ -249,7 +253,7 @@ consumer will you break with your local changes.
Let's assume that we have a producer with coordinates `com.example:server` and 3 consumers: `client1`,
`client2`, `client3`. Then in the repository with common contracts you would have the following setup
(which you can checkout https://github.com/spring-cloud/spring-cloud-contract/tree/1.0.x/samples/standalone/contracts[here]:
(which you can checkout https://github.com/spring-cloud/spring-cloud-contract/tree/{branch}/samples/standalone/contracts[here]):
[source,bash,indent=0]
----
@@ -501,8 +505,8 @@ task deleteUnwantedContracts(type: Delete) {
include: "**/*",
excludes: [
"**/${project.name}/**"",
**/${first-topic}/**",
**/${second-topic}/**])
"**/${first-topic}/**",
"**/${second-topic}/**"])
}
----
@@ -515,20 +519,193 @@ deleteUnwantedContracts.dependsOn("unzipContracts")
build.dependsOn("deleteUnwantedContracts")
----
- Configure plugin by specifying the directory containing contracts using ```contractsDslDir``` property
- Configure plugin by specifying the directory containing contracts using `contractsDslDir` property
[source,groovy,indent=0]
----
contracts {
contractsDslDir = new File("${buildDir}/unpackedContracts")
}
----
=== Can I have multiple base classes for tests?
=== Do I need a Binary Storage? Can't I use Git?
Yes! Check out the https://cloud.spring.io/spring-cloud-contract/spring-cloud-contract.html#_different_base_classes_for_contracts[Different base classes for contracts] sections
of either Gradle or Maven plugins.
In the polyglot world, there are languages that don't use binary storages like
Artifactory or Nexus. Starting from Spring Cloud Contract version 2.0.0 we provide
mechanisms to store contracts and stubs in a SCM repository. Currently the
only supported SCM is Git.
The repository would have to the following setup
(which you can checkout https://github.com/spring-cloud-samples/spring-cloud-contract-samples/tree/{samples_branch}/contracts_git/[here]):
[source,indent=0]
----
.
└── META-INF
└── com.example
└── beer-api-producer-git
└── 0.0.1-SNAPSHOT
├── contracts
│   └── beer-api-consumer
│   ├── messaging
│   │   ├── shouldSendAcceptedVerification.groovy
│   │   └── shouldSendRejectedVerification.groovy
│   └── rest
│   ├── shouldGrantABeerIfOldEnough.groovy
│   └── shouldRejectABeerIfTooYoung.groovy
└── mappings
└── beer-api-consumer
└── rest
├── shouldGrantABeerIfOldEnough.json
└── shouldRejectABeerIfTooYoung.json
----
Under `META-INF` folder:
* we group applications via `groupId` (e.g. `com.example`)
* then each application is represented via the `artifactId` (e.g. `beer-api-producer-git`)
* next, the version of the application. The version is mandatory! (e.g. `0.0.1-SNAPSHOT`)
* finally, there are two folders:
** `contracts` - the good practice is to store the contracts required by each
consumer in the folder with the consumer name (e.g. `beer-api-consumer`). That way you
can use the `stubs-per-consumer` feature. Further directory structure is arbitrary.
** `mappings` - in this folder the Maven / Gradle Spring Cloud Contract plugins will push
the stub server mappings. On the consumer side, Stub Runner will scan this folder
to start stub servers with stub definitions. The folder structure will be a copy
of the one created in the `contracts` subfolder.
==== Protocol convention
In order to control the type and location of the source of contracts (whether it's
a binary storage or an SCM repository), you can use the protocol in the URL of
the repository. Spring Cloud Contract iterates over registered protocol resolvers
and tries to fetch the contracts (via a plugin) or stubs (via Stub Runner).
For the SCM functionality, currently, we support the Git repository. To use it,
in the property, where the repository URL needs to be placed you just have to prefix
the connection URL with `git://`. Here you can find a couple of examples:
[source,indent=0]
----
git://file:///foo/bar
git://https://github.com/spring-cloud-samples/spring-cloud-contract-nodejs-contracts-git.git
git://git@github.com:spring-cloud-samples/spring-cloud-contract-nodejs-contracts-git.git
----
==== Producer
For the producer, to use the SCM approach, we can reuse the
same mechanism we use for external contracts. We route Spring Cloud Contract
to use the SCM implementation via the URL that contains
the `git://` protocol.
IMPORTANT: You have to manually add the `pushStubsToScm`
goal in Maven or execute (bind) the `pushStubsToScm` task in
Gradle. We don't push stubs to `origin` of your git
repository out of the box.
.Maven
[source,xml,indent=0]
----
<plugin>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-contract-maven-plugin</artifactId>
<version>${spring-cloud-contract.version}</version>
<extensions>true</extensions>
<configuration>
<!-- Base class mappings etc. -->
<!-- We want to pick contracts from a Git repository -->
<contractsRepositoryUrl>git://https://github.com/spring-cloud-samples/spring-cloud-contract-nodejs-contracts-git.git</contractsRepositoryUrl>
<!-- We reuse the contract dependency section to set up the path
to the folder that contains the contract definitions. In our case the
path will be /groupId/artifactId/version/contracts -->
<contractDependency>
<groupId>${project.groupId}</groupId>
<artifactId>${project.artifactId}</artifactId>
<version>${project.version}</version>
</contractDependency>
<!-- The contracts mode can't be classpath -->
<contractsMode>REMOTE</contractsMode>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<!-- By default we will not push the stubs back to SCM,
you have to explicitly add it as a goal -->
<goal>pushStubsToScm</goal>
</goals>
</execution>
</executions>
</plugin>
----
.Gradle
[source,gradle,indent=0]
----
contracts {
// We want to pick contracts from a Git repository
contractDependency {
stringNotation = "${project.group}:${project.name}:${project.version}"
}
/*
We reuse the contract dependency section to set up the path
to the folder that contains the contract definitions. In our case the
path will be /groupId/artifactId/version/contracts
*/
contractRepository {
repositoryUrl = "git://https://github.com/spring-cloud-samples/spring-cloud-contract-nodejs-contracts-git.git"
}
// The mode can't be classpath
contractsMode = "REMOTE"
// Base class mappings etc.
}
/*
In this scenario we want to publish stubs to SCM whenever
the `publish` task is executed
*/
publish.dependsOn("publishStubsToScm")
----
With such a setup:
* Git project will be cloned to a temporary directory
* The SCM stub downloader will go to `META-INF/groupId/artifactId/version/contracts` folder
to find contracts. E.g. for `com.example:foo:1.0.0` the path would be
`META-INF/com.example/foo/1.0.0/contracts`
* Tests will be generated from the contracts
* Stubs will be created from the contracts
* Once the tests pass, the stubs will be committed in the cloned repository
* Finally, a push will be done to that repo's `origin`
==== Consumer
On the consumer side when passing the `repositoryRoot` parameter,
either from the `@AutoConfigureStubRunner` annotation, the
JUnit rule or properties, it's enough to pass the URL of the
SCM repository, prefixed with the protocol. For example
[source,java,indent=0]
----
@AutoConfigureStubRunner(
stubsMode="REMOTE",
repositoryRoot="git://https://github.com/spring-cloud-samples/spring-cloud-contract-nodejs-contracts-git.git",
ids="com.example:bookstore:0.0.1.RELEASE"
)
----
With such a setup:
* Git project will be cloned to a temporary directory
* The SCM stub downloader will go to `META-INF/groupId/artifactId/version/` folder
to find stub definitions and contracts. E.g. for `com.example:foo:1.0.0` the path would be
`META-INF/com.example/foo/1.0.0/`
* Stub servers will be started and fed with mappings
* Messaging definitions will be read and used in the messaging tests
=== How can I debug the request/response being sent by the generated tests client?
@@ -558,11 +735,6 @@ You can use the `mappingsOutputFolder` property on `@AutoConfigureStubRunner` or
to dump all mappings per artifact id. Also the port at which the given stub server was
started will be attached.
==== Can I reference the request from the response?
Yes! With version 1.1.0 we've added such a possibility. On the HTTP stub server side we're providing support
for this for WireMock. In case of other HTTP server stubs you'll have to implement the approach yourself.
==== Can I reference text from file?
Yes! With version 1.2.0 we've added such a possibility. It's enough to call `file(...)` method in the

View File

@@ -497,7 +497,6 @@ to the following in the build logs:
2016-07-19 14:22:27.737 INFO 41050 --- [ main] o.s.c.c.stubrunner.StubRunnerExecutor : All stubs are now running RunningStubs [namesAndPorts={com.example:http-server:0.0.1-SNAPSHOT:stubs=8080}]
----
==== Defining the Contract
As consumers of services, we need to define what exactly we want to achieve. We need to

View File

@@ -23,6 +23,7 @@ following sections:
* <<gradle-single-base-class>>
* <<gradle-different-base-classes>>
* <<gradle-invoking-generated-tests>>
* <<gradle-pushing-stubs-to-scm>>
* <<gradle-consumer>>
[[gradle-prerequisites]]
@@ -246,6 +247,8 @@ from the Groovy DSL should be placed. By default its value is
the Groovy DSL should be placed.
* *targetFramework*: Specifies the target test framework to be used. Currently, Spock and
JUnit are supported with JUnit being the default framework.
* *contractsProperties*: a map containing properties to be passed to Spring Cloud Contract
components. Those properties might be used by e.g. inbuilt or custom Stub Downloaders.
The following properties are used when you want to specify the location of the JAR
containing the contracts:
@@ -334,6 +337,25 @@ To ensure that the provider side is compliant with defined contracts, you need t
./gradlew generateContractTests test
----
[[gradle-pushing-stubs-to-scm]]
==== Pushing stubs to SCM
If you're using the SCM repository to keep the contracts and
stubs, you might want to automate the step of pushing stubs to
the repository. To do that, it's enough to call the `pushStubsToScm`
task. Example:
[source,bash,indent=0]
----
$ ./gradlew pushStubsToScm
----
Under <<scm-stub-downloader>> you can find all possible
configuration options that you can pass either via
the `contractsProperties` field e.g. `contracts { contractsProperties = [foo:"bar"] }`,
via `contractsProperties` method e.g. `contracts { contractsProperties([foo:"bar"]) }`,
a system property or an environment variable.
[[gradle-consumer]]
==== Spring Cloud Contract Verifier on the Consumer Side
@@ -395,6 +417,7 @@ following sections:
* <<maven-single-base>>
* <<maven-different-base>>
* <<maven-invoking-generated-tests>>
* <<maven-pushing-stubs-to-scm>>
* <<maven-sts>>
[[maven-add-plugin]]
@@ -576,6 +599,8 @@ the matched contract. For example, if you have a contract under
`.* -> com.example.base.BaseClass`, then the test class generated from these contracts
extends `com.example.base.BaseClass`. This setting takes precedence over
*packageWithBaseClasses* and *baseClassForTests*.
* *contractsProperties*: a map containing properties to be passed to Spring Cloud Contract
components. Those properties might be used by e.g. inbuilt or custom Stub Downloaders.
If you want to download your contract definitions from a Maven repository, you can use
the following options:
@@ -766,6 +791,57 @@ For Groovy Spock code, use the following:
To ensure that provider side is compliant with defined contracts, you need to invoke
`mvn generateTest test`.
[[maven-pushing-stubs-to-scm]]
==== Pushing stubs to SCM
If you're using the SCM repository to keep the contracts and
stubs, you might want to automate the step of pushing stubs to
the repository. To do that, it's enough to add the `pushStubsToScm`
goal. Example:
[source,xml,indent=0]
----
<plugin>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-contract-maven-plugin</artifactId>
<version>${spring-cloud-contract.version}</version>
<extensions>true</extensions>
<configuration>
<!-- Base class mappings etc. -->
<!-- We want to pick contracts from a Git repository -->
<contractsRepositoryUrl>git://https://github.com/spring-cloud-samples/spring-cloud-contract-nodejs-contracts-git.git</contractsRepositoryUrl>
<!-- We reuse the contract dependency section to set up the path
to the folder that contains the contract definitions. In our case the
path will be /groupId/artifactId/version/contracts -->
<contractDependency>
<groupId>${project.groupId}</groupId>
<artifactId>${project.artifactId}</artifactId>
<version>${project.version}</version>
</contractDependency>
<!-- The contracts mode can't be classpath -->
<contractsMode>REMOTE</contractsMode>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<!-- By default we will not push the stubs back to SCM,
you have to explicitly add it as a goal -->
<goal>pushStubsToScm</goal>
</goals>
</execution>
</executions>
</plugin>
----
Under <<scm-stub-downloader>> you can find all possible
configuration options that you can pass either via
the `<configuration><contractProperties>` map, a system property
or an environment variable.
[[maven-sts]]
==== Maven Plugin and STS