spring-data-build

Files

Greg Turnquist 8c58eab3fd #752 - Reduce CI notifications on slack.

Instead of letting Spring HATEOAS trigger a build on Commons, and subsequently a build on every data store, culminating in n-builds of REST, instead use this strategy:

* HATEOAS artifactory triggers Commons test, but not build.
* HATEOAS artifactory triggers REST test and build, since it's a direct impact.
* REST is NOT triggered by individual data store builds, since it doesn't directly depend upon them.

Slack notification policy is:

* Signal test failure.
* Signal build failure.
* Signal build success.

By NOT sending a notification for every test success, it should reduce volume of messages. By reducing the number of total builds happening, will also reduce traffic to only chronic failing tests, which we probably need to see anyway. And with number of total builds reduced, should still give us the occasional warm fuzzy when a build DOES complete.

2019-03-07 08:05:33 -06:00

build.sh

#714 - Introduce Concourse.

2019-03-01 11:58:12 -06:00

build.yml

#714 - Introduce Concourse.

2019-03-01 11:58:12 -06:00

pipeline-master.yml

#752 - Reduce CI notifications on slack.

2019-03-07 08:05:33 -06:00

README.adoc

#714 - Introduce Concourse.

2019-03-01 11:58:12 -06:00

test.sh

#714 - Introduce Concourse.

2019-03-01 11:58:12 -06:00

test.yml

#714 - Introduce Concourse.

2019-03-01 11:58:12 -06:00

README.adoc

== Spring Data CI

Spring Data uses Concourse as it's CI tool of choice. This provides support for:

* Pipeline against the `master` branch
* Pull requests

=== Creating a pipeline

Using the `fly` command, you can execute a series of commands to create multiple pipelines to manage everything. But
first, some critical credentials are needed.

Create a `credentials.yml` file like this:

[source,yml]
----
github-access-token: <your Personal Access Token from github>
docker-email: <your docker hub email address>
docker-username: <your docker hub username>
docker-password: <your docker hub password>
artifactory-username: <your artifactory username>
artifactory-password: <your artifactory encoded password>
----

WARNING: Do NOT check this file into source control! If you'll check, `credentials.yml` is listed in `.gitignore` to prevent this.

With this in place, run the following `fly` commands to create pipelines:

----
% fly -t spring-data sp -p spring-data -c ci/pipeline-master.yml -l credentials.yml
----

With this pipeline in place, you can now activate and expose it:

----
% fly -t spring-data unpause-pipeline -p spring-hateoas
% fly -t spring-data expose-pipeline -p spring-hateoas
----

=== Running CI tasks locally

Since Concourse is built on top of Docker, it's easy to:

* Debug what went wrong on your local machine.
* Test out a a tweak to your `test.sh` script before sending it out.
* Experiment against a new image before submitting your pull request.

All of these use cases are great reasons to essentially run what Concourse does on your local machine.

IMPORTANT: To do this you must have Docker installed on your machine.

1. `docker run -it --mount type=bind,source="$(pwd)",target=/spring-data-build-github openjdk:8-jdk /bin/bash`
+
This will launch the Docker image and mount your source code at `spring-data-build-github`.
+
Next, run the `test.sh` script from inside the container:
+
2. `PROFILE=none spring-data-build-github/ci/test.sh`

Since the container is binding to your source, you can make edits from your IDE and continue to run build jobs.

If you need to test the `build.sh` script, then do this:

1. `mkdir /tmp/spring-data-build-artifactory`
2. `docker run -it --mount type=bind,source="$(pwd)",target=/spring-data-build-github --mount type=bind,source="/tmp/spring-data-build-artifactory",target=/spring-data-build-artifactory openjdk:8-jdk /bin/bash`
+
This will launch the Docker image and mount your source code at `spring-data-build-github` and the temporary
artifactory output directory at `spring-data-build-artifactory`.
+
Next, run the `build.sh` script from inside the container:
+
3. `spring-data-build-github/ci/build.sh`

IMPORTANT: `build.sh` doesn't actually push to Artifactory so don't worry about accidentally deploying anything.
It just deploys to a local folder. That way, the `artifactory-resource` later in the pipeline can pick up these artifacts
and deliver them to artifactory.

NOTE: Docker containers can eat up disk space fast! From time to time, run `docker system prune` to clean out old images.