8. Jenkins Pipeline (Cloud Foundry)
Prev |  | Next

# 8. Jenkins Pipeline (Cloud Foundry)

| Important
In this chapter we assume that you perform deployment of your application to Cloud Foundry PaaS

The Spring Cloud Pipelines repository contains job definitions and the opinionated setup pipeline using Jenkins Job DSL plugin. Those jobs will form an empty pipeline and a sample, opinionated one that you can use in your company.

All in all there are the following projects taking part in the whole `microservice setup` for this demo.

- Github Analytics - the app that has a REST endpoint and uses messaging. Our business application.
- Github Webhook - project that emits messages that are used by Github Analytics. Our business application.
- Eureka - simple Eureka Server. This is an infrastructure application.
- Github Analytics Stub Runner Boot - Stub Runner Boot server to be used for tests with Github Analytics. Uses Eureka and Messaging. This is an infrastructure application.

## 8.1 Step by step

This is a guide for Jenkins Job DSL based pipeline.

If you want to just run the demo as far as possible using PCF Dev and Docker Compose

- Fork repos
- Start Jenkins and Artifactory
- Deploy infra to Artifactory
- Start PCF Dev (if you don’t want to use an existing one)
- Run the seed job
- Run the `github-webhook` pipeline

### 8.1.1 Fork repos

There are 4 apps that are composing the pipeline

- Github Webhook
- Github Analytics
- Github Eureka
- Github Stub Runner Boot

You need to fork only these. That’s because only then will your user be able to tag and push the tag to repo.

- Github Webhook
- Github Analytics

### 8.1.2 Start Jenkins and Artifactory

Jenkins + Artifactory can be ran locally. To do that just execute the `start.sh` script from this repo.

    git clone https://github.com/spring-cloud/spring-cloud-pipelines
    cd spring-cloud-pipelines/jenkins
    ./start.sh yourGitUsername yourGitPassword yourForkedGithubOrg

Then Jenkins will be running on port `8080` and Artifactory `8081`. The provided parameters will be passed as env variables to Jenkins VM and credentials will be set in your set. That way you don’t have to do any manual work on the Jenkins side. In the above parameters, the third parameter could be yourForkedGithubOrg or yourGithubUsername. Also the `REPOS` env variable will contain your GitHub org in which you have the forked repos.

#### Deploy the infra JARs to Artifactory

When Artifactory is running, just execute the `tools/deploy-infra.sh` script from this repo.

    git clone https://github.com/spring-cloud/spring-cloud-pipelines
    cd spring-cloud-pipelines/
    ./tools/deploy-infra.sh

As a result both `eureka` and `stub runner` repos will be cloned, built and uploaded to Artifactory.

### 8.1.3 Start PCF Dev

| Tip
You can skip this step if you have CF installed and don’t want to use PCF Dev The only thing you have to do is to set up spaces.

| Warning
It’s more than likely that you’ll run out of resources when you reach stage step. Don’t worry! Keep calm and clear some apps from PCF Dev and continue.

You have to download and start PCF Dev. A link how to do it is available here.

The default credentials when using PCF Dev are:

    username: user
    password: pass
    email: user
    org: pcfdev-org
    space: pcfdev-space
    api: api.local.pcfdev.io

You can start the PCF Dev like this:

    cf dev start

You’ll have to create 3 separate spaces (email admin, pass admin)

    cf login -a https://api.local.pcfdev.io --skip-ssl-validation -u admin -p admin -o pcfdev-org

    cf create-space pcfdev-test
    cf set-space-role user pcfdev-org pcfdev-test SpaceDeveloper
    cf create-space pcfdev-stage
    cf set-space-role user pcfdev-org pcfdev-stage SpaceDeveloper
    cf create-space pcfdev-prod
    cf set-space-role user pcfdev-org pcfdev-prod SpaceDeveloper

You can also execute the `./tools/cf-helper.sh setup-spaces` to do this.

### 8.1.4 Run the seed job

We already create the seed job for you but you’ll have to run it. When you do run it you have to provide some properties. By default we create a seed that has all the properties options, but you can delete most of it. If you set the properties as global env variables you have to remove them from the seed.

Anyways, to run the demo just provide in the `REPOS` var the comma separated list of URLs of the 2 aforementioned forks of `github-webhook` and `github-analytics'.

Figure 8.1. Click the 'jenkins-pipeline-seed-cf' job for Cloud Foundry and `jenkins-pipeline-seed-k8s` for Kubernetes

Figure 8.2. Click the 'Build with parameters'

Figure 8.3. The `REPOS` parameter should already contain your forked repos (you’ll have more properties than the ones in the screenshot)

Figure 8.4. This is how the results of seed should look like

### 8.1.5 Run the `github-webhook` pipeline

We already create the seed job for you but you’ll have to run it. When you do run it you have to provide some properties. By default we create a seed that has all the properties options, but you can delete most of it. If you set the properties as global env variables you have to remove them from the seed.

Anyways, to run the demo just provide in the `REPOS` var the comma separated list of URLs of the 2 aforementioned forks of `github-webhook` and `github-analytics`.

Figure 8.5. Click the 'github-webhook' view

Figure 8.6. Run the pipeline

| Important
If your build fails on the deploy previous version to stage due to missing jar, that means that you’ve forgotten to clear the tags in your repo. Typically that’s due to the fact that you’ve removed the Artifactory volume with deployed JAR whereas a tag in the repo is still pointing there. Check out this section on how to remove the tag.

Figure 8.7. Click the manual step to go to stage (remember about killing the apps on test env). To do this click the ARROW next to the job name

| Important
Most likely you will run out of memory so when reaching the stage environment it’s good to kill all apps on test. Check out the FAQ section for more details!

Figure 8.8. The full pipeline should look like this

## 8.2 Declarative pipeline & Blue Ocean

You can also use the declarative pipeline approach with the Blue Ocean UI. Here is a step by step guide to run a pipeline via this approach.

The Blue Ocean UI is available under the `blue/` URL. E.g. for Docker Machine based setup `http://192.168.99.100:8080/blue`.

Figure 8.9. Open Blue Ocean UI and click on `github-webhook-declarative-pipeline`

Figure 8.10. Your first run will look like this. Click `Run` button

Figure 8.11. Enter parameters required for the build and click `run`

Figure 8.12. A list of pipelines will be shown. Click your first run.

Figure 8.13. State if you want to go to production or not and click `Proceed`

Figure 8.14. The build is in progress…​

Figure 8.15. The pipeline is done!

| Important
There is no possibility of restarting pipeline from specific stage, after failure. Please check out this issue for more information

| Warning
Currently there is no way to introduce manual steps in a performant way. Jenkins is blocking an executor when manual step is required. That means that you’ll run out of executors pretty fast. You can check out this issue for and this StackOverflow question for more information.

## 8.3 Jenkins Cloud Foundry customization

     All the steps below are not necessary to run the demo. They are needed only
    when you want to do some custom changes.

### 8.3.1 All env vars

The env vars that are used in all of the jobs are as follows:

Property Name | Property Description | Default value
--- | --- | ---
| BINARY_EXTENSION | Extension of the binary uploaded to Artifactory / Nexus. Example: change this to `war` for WAR artifacts
PAAS_TEST_API_URL | The URL to the CF Api for TEST env | api.local.pcfdev.io
PAAS_STAGE_API_URL | The URL to the CF Api for STAGE env | api.local.pcfdev.io
PAAS_PROD_API_URL | The URL to the CF Api for PROD env | api.local.pcfdev.io
PAAS_TEST_ORG | Name of the org for the test env | pcfdev-org
PAAS_TEST_SPACE | Name of the space for the test env | pcfdev-space
PAAS_STAGE_ORG | Name of the org for the stage env | pcfdev-org
PAAS_STAGE_SPACE | Name of the space for the stage env | pcfdev-space
PAAS_PROD_ORG | Name of the org for the prod env | pcfdev-org
PAAS_PROD_SPACE | Name of the space for the prod env | pcfdev-space
REPO_WITH_BINARIES | URL to repo with the deployed jars | http://artifactory:8081/artifactory/libs-release-local
M2_SETTINGS_REPO_ID | The id of server from Maven settings.xml | artifactory-local
JDK_VERSION | The name of the JDK installation | jdk8
PIPELINE_VERSION | What should be the version of the pipeline (ultimately also version of the jar) | 1.0.0.M1-${GROOVY,script ="new Date().format('yyMMdd_HHmmss')"}-VERSION
GIT_EMAIL | The email used by Git to tag repo | email@example.com
GIT_NAME | The name used by Git to tag repo | Pivo Tal
PAAS_HOSTNAME_UUID | Additional suffix for the route. In a shared environment the default routes can be already taken |
AUTO_DEPLOY_TO_STAGE | Should deployment to stage be automatic | false
AUTO_DEPLOY_TO_PROD | Should deployment to prod be automatic | false
API_COMPATIBILITY_STEP_REQUIRED | Should api compatibility step be required | true
DB_ROLLBACK_STEP_REQUIRED | Should DB rollback step be present | true
DEPLOY_TO_STAGE_STEP_REQUIRED | Should deploy to stage step be present | true
APP_MEMORY_LIMIT | How much memory should be used by the infra apps (Eureka, Stub Runner etc.) | 256m
JAVA_BUILDPACK_URL | The URL to the Java buildpack to be used by CF | https://github.com/cloudfoundry/java-buildpack.git#v3.8.1
| BUILD_OPTIONS | Additional options you would like to pass to the Maven / Gradle build

### 8.3.2 Jenkins Credentials

In your scripts we reference the credentials via IDs. These are the defaults for credentials

Property Name | Property Description | Default value
--- | --- | ---
PAAS_PROD_CREDENTIAL_ID | Credential ID for CF Prod env access | cf-prod
GIT_CREDENTIAL_ID | Credential ID used to tag a git repo | git
GIT_SSH_CREDENTIAL_ID | SSH credential ID used to tag a git repo | gitSsh
GIT_USE_SSH_KEY | if `true` will pick to use the SSH credential id | false
REPO_WITH_BINARIES_CREDENTIAL_ID | Credential ID used for the repo with jars | repo-with-binaries
PAAS_TEST_CREDENTIAL_ID | Credential ID for CF Test env access | cf-test
PAAS_STAGE_CREDENTIAL_ID | Credential ID for CF Stage env access | cf-stage

If you already have in your system a credential to for example tag a repo you can use it by passing the value of the property `GIT_CREDENTIAL_ID`

| Tip
Check out the `cf-helper` script for all the configuration options!

Prev |  | Next
7. Jenkins Pipeline (Common) | Home | 9. Jenkins Pipeline (Kubernetes)