diff --git a/README.md b/README.md index 24aaf597..0f3585b5 100644 --- a/README.md +++ b/README.md @@ -28,6 +28,10 @@ This project provides an implementation of [Discovery Client](https://github.com/spring-cloud/spring-cloud-commons/blob/master/spring-cloud-commons/src/main/java/org/springframework/cloud/client/discovery/DiscoveryClient.java) for [Kubernetes](http://kubernetes.io). This allows you to query Kubernetes endpoints *(see [services](http://kubernetes.io/docs/user-guide/services/))* by name. +A service is typically exposed by the Kubernetes API server as a collection of endpoints which represent `http`, `https` addresses that a client can +access from a Spring Boot application running as a pod. This discovery feature is also used by the Spring Cloud Kubernetes Ribbon or Zipkin projects +to fetch respectively the list of the endpoints defined for an application to be load balanced or the Zipkin servers available to send the traces or spans. + This is something that you get for free just by adding the following dependency inside your project: ```xml @@ -38,43 +42,59 @@ This is something that you get for free just by adding the following dependency ``` -Then you can inject the client in your cloud simply by: +Then you can inject the client in your code simply by: ```java @Autowire private DiscoveryClient discoveryClient; ``` -If for any reason you need to disable the `DiscoveryClient` you can simply set the following property: +If for any reason you need to disable the `DiscoveryClient` you can simply set the following property in `application +.properties`: ``` spring.cloud.kubernetes.discovery.enabled=false ``` -Some spring cloud components use the `DiscoveryClient` in order obtain info about the local service instance. For this to work you need to align the service name with `spring.application.name`. +[//]: # "TODO: make clearer with an example and details on how to align service and application name" + +Some Spring Cloud components use the `DiscoveryClient` in order to obtain info about the local service instance. For +this to work you need to align the service name with the `spring.application.name` property. ### Kubernetes PropertySource -The most common approach to configure your spring boot application is to edit the `application.yaml` file. Often the user may override properties by specifying system properties or env variables. +The most common approach to configure your Spring Boot application is to create an `application.properties|yaml` or +an `application-profile.properties|yaml` file containing key-value pairs providing customization values to your +application or Spring Boot starters. Users may override these properties by specifying system properties or environment +variables. #### ConfigMap PropertySource -Kubernetes has the notion of [ConfigMap](http://kubernetes.io/docs/user-guide/configmap/) for passing configuration to the application. This project provides integration with `ConfigMap` to make config maps accessible by spring boot. +Kubernetes provides a resource named [ConfigMap](http://kubernetes.io/docs/user-guide/configmap/) to externalize the +parameters to pass to your application in the form of key-value pairs or embedded `application.properties|yaml` files. +The [Spring Cloud Kubernetes Config](./spring-cloud-kubernetes-config) project makes Kubernetes `ConfigMap`s available +during application bootstrapping and triggers hot reloading of beans or Spring context when changes are detected on +observed `ConfigMap`s. -The `ConfigMap` `PropertySource` when enabled will lookup Kubernetes for a `ConfigMap` named after the application (see `spring.application.name`). If the map is found it will read its data and do the following: +`ConfigMapPropertySource` will search for a Kubernetes `ConfigMap` which `metadata.name` is either the name of +your Spring application (as defined by its `spring.application.name` property) or a custom name defined within the +`bootstrap.properties` file under the following key `spring.cloud.kubernetes.config.name`. + +If such a `ConfigMap` is found, it will be processed as follows: - apply individual configuration properties. -- apply as yaml the content of any property named `application.yaml` +- apply as `yaml` the content of any property named `application.yaml` - apply as properties file the content of any property named `application.properties` Example: -Let's assume that we have a spring boot application named ``demo`` that uses properties to read its thread pool configuration. +Let's assume that we have a Spring Boot application named ``demo`` that uses properties to read its thread pool +configuration. - `pool.size.core` - `pool.size.maximum` -This can be externalized to config map in yaml format: +This can be externalized to config map in `yaml` format: ```yaml kind: ConfigMap @@ -86,7 +106,8 @@ data: pool.size.max: 16 ``` -Individual properties work fine for most cases but sometimes yaml is more convinient. In this case we will use a single property named `application.yaml` and embed our yaml inside it: +Individual properties work fine for most cases but sometimes embedded `yaml` is more convenient. In this case we will +use a single property named `application.yaml` to embed our `yaml`: ```yaml kind: ConfigMap @@ -101,68 +122,140 @@ data: max:16 ``` +Spring Boot applications can also be configured differently depending on active profiles and it is possible to +provide different property values for different profiles using an `application.properties|yaml` property, specifying +profile-specific values each in their own document (indicated by the `---` sequence) as follows: + +```yaml +kind: ConfigMap +apiVersion: v1 +metadata: + name: demo +data: + application.yml: |- + greeting: + message: Say Hello to the World + --- + spring: + profiles: development + greeting: + message: Say Hello to the Developers + --- + spring: + profiles: production + greeting: + message: Say Hello to the Ops +``` + +To tell to Spring Boot which `profile` should be enabled at bootstrap, a system property can be passed to the Java +command launching your Spring Boot application using an env variable that you will define with the OpenShift +`DeploymentConfig` or Kubernetes `ReplicationConfig` resource file as follows: + +```yaml +apiVersion: v1 +kind: DeploymentConfig +spec: + replicas: 1 + ... + spec: + containers: + - env: + - name: JAVA_APP_DIR + value: /deployments + - name: JAVA_OPTIONS + value: -Dspring.profile.active=developer +``` + **Notes:** -- To access ConfigMaps on OpenShift the service account needs at least view permissions i.e.: +- To access `ConfigMap`s on OpenShift the service account needs at least view permissions i.e.: ```oc policy add-role-to-user view system:serviceaccount:$(oc project -q):default -n $(oc project -q)``` + +**Properties:** + +| Name | Type | Default | Description +| --- | --- | --- | --- +| spring.cloud.kubernetes.config.enabled | Boolean | true | Enable Secrets PropertySource +| spring.cloud.kubernetes.config.name | String | ${spring.application.name} | Sets the name of ConfigMap to lookup +| spring.cloud.kubernetes.config.namespace | String | Client namespace | Sets the Kubernetes namespace where to lookup + #### Secrets PropertySource -Kubernetes has the notion of [Secrets](http://kubernetes.io/docs/user-guide/secrets/) for storing sensitive data such as password, OAuth tokens, etc. This project provides integration with `Secrets` to make secrets accessible by spring boot. +Kubernetes has the notion of [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) for storing +sensitive data such as password, OAuth tokens, etc. This project provides integration with `Secrets` to make secrets +accessible by Spring Boot applications. This feature can be explicitly enabled/disabled using the `spring.cloud +.kubernetes.secrets.enabled` property. -The `Secrets` `PropertySource` when enabled will lookup Kubernetes for `Secrets` from the following sources: +The `SecretsPropertySource` when enabled will lookup Kubernetes for `Secrets` from the following sources: 1. reading recursively from secrets mounts -2. named after the application (see `spring.application.name`) +2. named after the application (as defined by `spring.application.name`) 3. matching some labels -Please note that by default, consuming Secrets via API (points 2 and 3 above) **is not enabled**. +Please note that by default, consuming Secrets via API (points 2 and 3 above) **is not enabled** for security reasons + and it is recommend that containers share secrets via mounted volumes. If the secrets are found theirs data is made available to the application. **Example:** -Let's assume that we have a spring boot application named ``demo`` that uses properties to read its ActiveMQ and PostreSQL configuration. +Let's assume that we have a spring boot application named ``demo`` that uses properties to read its database +configuration. We can create a Kubernetes secret using the following command: -- `amq.username` -- `amq.password` -- `pg.username` -- `pg.password` +``` +oc create secret generic db-secret --from-literal=username=user --from-literal=password=p455w0rd +``` -This can be externalized to Secrets in yaml format: +This would create the following secret (shown using `oc get secrets db-secret -o yaml`): -- **ActiveMQ** - ```yaml - apiVersion: v1 - kind: Secret - metadata: - name: activemq-secrets - labels: - broker: activemq - type: Opaque - data: - amq.username: bXl1c2VyCg== - amq.password: MWYyZDFlMmU2N2Rm - ``` +```yaml +apiVersion: v1 +data: + password: cDQ1NXcwcmQ= + username: dXNlcg== +kind: Secret +metadata: + creationTimestamp: 2017-07-04T09:15:57Z + name: db-secret + namespace: default + resourceVersion: "357496" + selfLink: /api/v1/namespaces/default/secrets/db-secret + uid: 63c89263-6099-11e7-b3da-76d6186905a8 +type: Opaque +``` -- **PostgreSQL** - ```yaml - apiVersion: v1 - kind: Secret - metadata: - name: postgres-secrets - labels: - db: postgres - type: Opaque - data: - pg.username: dXNlcgo= - pg.password: cGdhZG1pbgo= - ``` + +Note that the data contains Base64-encoded versions of the literal provided by the create command. + +This secret can then be used by your application for example by exporting the secret's value as environment variables: + +```yaml +apiVersion: v1 +kind: Deployment +metadata: + name: ${project.artifactId} +spec: + template: + spec: + containers: + - env: + - name: DB_USERNAME + valueFrom: + secretKeyRef: + name: db-secret + key: username + - name: DB_PASSWORD + valueFrom: + secretKeyRef: + name: db-secret + key: password +``` You can select the Secrets to consume in a number of ways: 1. By listing the directories were secrets are mapped: ``` - -Dspring.cloud.kubernetes.secrets.paths=/etc/secrets/activemq,etc/secrets/postgres + -Dspring.cloud.kubernetes.secrets.paths=/etc/secrets/db-secret,etc/secrets/postgresql ``` If you have all the secrets mapped to a common root, you can set them like: @@ -173,13 +266,13 @@ You can select the Secrets to consume in a number of ways: 2. By setting a named secret: ``` - -Dspring.cloud.kubernetes.secrets.name=postgres-secrets + -Dspring.cloud.kubernetes.secrets.name=db-secret ``` 3. By defining a list of labels: ``` -Dspring.cloud.kubernetes.secrets.labels.broker=activemq - -Dspring.cloud.kubernetes.secrets.labels.db=postgres + -Dspring.cloud.kubernetes.secrets.labels.db=postgresql ``` **Properties:** @@ -188,34 +281,41 @@ You can select the Secrets to consume in a number of ways: | --- | --- | --- | --- | spring.cloud.kubernetes.secrets.enabled | Boolean | true | Enable Secrets PropertySource | spring.cloud.kubernetes.secrets.name | String | ${spring.application.name} | Sets the name of the secret to lookup +| spring.cloud.kubernetes.secrets.namespace | String | Client namespace | Sets the Kubernetes namespace where to lookup | spring.cloud.kubernetes.secrets.labels | Map | null | Sets the labels used to lookup secrets -| spring.cloud.kubernetes.secrets.paths | List | null | Sets the paths were secrets are mounted /example 1) +| spring.cloud.kubernetes.secrets.paths | List | null | Sets the paths were secrets are mounted (example 1) | spring.cloud.kubernetes.secrets.enableApi | Boolean | false | Enable/Disable consuming secrets via APIs (examples 2 and 3) **Notes:** -- The property spring.cloud.kubernetes.secrets.labels behave as defined by [Map-based binding](https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-Configuration-Binding#map-based-binding). -- The property spring.cloud.kubernetes.secrets.paths behave as defined by [Collection-based binding](https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-Configuration-Binding#collection-based-binding). +- The property `spring.cloud.kubernetes.secrets.labels` behaves as defined by +[Map-based binding](https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-Configuration-Binding#map-based-binding). +- The property `spring.cloud.kubernetes.secrets.paths` behaves as defined by +[Collection-based binding](https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-Configuration-Binding#collection-based-binding). - Access to secrets via API may be restricted for security reasons, the preferred way is to mount secret to the POD. +Example of application using secrets (though it hasn't been updated to use the new `spring-cloud-kubernetes` project): +[spring-boot-camel-config](https://github.com/fabric8-quickstarts/spring-boot-camel-config) + #### PropertySource Reload Some applications may need to detect changes on external property sources and update their internal status to reflect the new configuration. -The reload feature of Spring Cloud Kubernetes is able to trigger an application reload when a related ConfigMap or Secret change. +The reload feature of Spring Cloud Kubernetes is able to trigger an application reload when a related `ConfigMap` or +`Secret` changes. This feature is disabled by default and can be enabled using the configuration property `spring.cloud.kubernetes.reload.enabled=true` (eg. in the *application.properties* file). The following levels of reload are supported (property `spring.cloud.kubernetes.reload.strategy`): -- **refresh (default)**: only configuration beans annotated with `@ConfigurationProperties` or `@RefreshScope` are reloaded. +- **`refresh` (default)**: only configuration beans annotated with `@ConfigurationProperties` or `@RefreshScope` are reloaded. This reload level leverages the refresh feature of Spring Cloud Context. -- **restart_context**: the whole Spring _ApplicationContext_ is gracefully restarted. Beans are recreated with the new configuration. -- **shutdown**: the Spring _ApplicationContext_ is shut down to activate a restart of the container. +- **`restart_context`**: the whole Spring _ApplicationContext_ is gracefully restarted. Beans are recreated with the new configuration. +- **`shutdown`**: the Spring _ApplicationContext_ is shut down to activate a restart of the container. When using this level, make sure that the lifecycle of all non-daemon threads is bound to the ApplicationContext and that a replication controller or replica set is configured to restart the pod. Example: -Assuming that the reload feature is enabled with default settings (*refresh* mode), the following bean will be refreshed when the config map changes: +Assuming that the reload feature is enabled with default settings (*`refresh`* mode), the following bean will be refreshed when the config map changes: ```java @Configuration @@ -245,9 +345,9 @@ public class MyBean { } ``` -The message printed by the application can be changed using a config map like the following one: +The message printed by the application can be changed using a `ConfigMap` as follows: -```yml +```yaml apiVersion: v1 kind: ConfigMap metadata: @@ -257,8 +357,10 @@ data: bean.message=Hello World! ``` -Any change to the property named `bean.message` in the Config Map associated to the pod will be reflected in the output of the program -(more details [here](#configmap-propertysource) about how to associate a Config Map to a pod). +Any change to the property named `bean.message` in the `ConfigMap` associated to the pod will be reflected in the +output. More generally speaking, changes associated to properties prefixed with the value defined by the `prefix` +field of the `@ConfigurationProperties` annotation will be detected and reflected in the application. +[Associating a `ConfigMap` to a pod](#configmap-propertysource) is explained above. The full example is available in [spring-cloud-kubernetes-reload-example](spring-cloud-kubernetes-examples/kubernetes-reload-example). @@ -284,7 +386,7 @@ Properties: | spring.cloud.kubernetes.reload.period | Long | 15000 | The period in milliseconds for verifying changes when using the *polling* strategy Notes: -- Properties under *spring.cloud.kubernetes.reload.** should not be used in config maps or secrets: changing such properties at runtime may lead to unexpected results; +- Properties under *spring.cloud.kubernetes.reload.* should not be used in config maps or secrets: changing such properties at runtime may lead to unexpected results; - Deleting a property or the whole config map does not restore the original state of the beans when using the *refresh* level. @@ -293,20 +395,26 @@ Notes: Spring Boot uses [HealthIndicator](https://github.com/spring-projects/spring-boot/blob/master/spring-boot-actuator/src/main/java/org/springframework/boot/actuate/health/HealthIndicator.java) to expose info about the health of an application. That makes it really useful for exposing health related information to the user and are also a good fit for use as [readiness probes](http://kubernetes.io/docs/user-guide/production-pods/#liveness-and-readiness-probes-aka-health-checks). -The Kubernetes health indicator which is part of the core modules exposes the following info: +The Kubernetes health indicator which is part of the core module exposes the following info: -- pod name -- visible services -- flag that indicates if app is internal or external to Kubernetes +- pod name, ip address, namespace, service account, node name amd its ip address +- flag that indicates if the Spring Boot application is internal or external to Kubernetes ### Transparency -All of the features described above will work equally fine regardless of wether our application is inside Kubernetes or not. This is really helpful for development and troubleshooting. +All of the features described above will work equally well regardless of whether your application is running inside +Kubernetes or not. This is really helpful for development and troubleshooting. +From a development point of view, this is really helpful as you can start your Spring Boot application and debug one +of the modules part of this project. It is not required to deploy it in Kubernetes +as the code of the project relies on the +[Fabric8 Kubernetes Java client](https://github.com/fabric8io/kubernetes-client) which is a fluent DSL able to +communicate using `http` protocol to the REST API of Kubernetes Server. ### Kubernetes Profile Autoconfiguration -When the application is run inside Kubernetes a profile named `kubernetes` will automatically get activated. -This allows the user to customize the configuration that will be applied in and out of kubernetes *(e.g. different dev and prod configuration)*. +When the application runs as a pod inside Kubernetes a Spring profile named `kubernetes` will automatically get activated. +This allows the developer to customize the configuration, to define beans that will be applied when the Spring Boot application is deployed +within the Kubernetes platform *(e.g. different dev and prod configuration)*. ### Ribbon discovery in Kubernetes @@ -314,7 +422,13 @@ This allows the user to customize the configuration that will be applied in and [![Javadocs](http://www.javadoc.io/badge/org.springframework.cloud/spring-cloud-starter-kubernetes-netflix.svg?color=blue)](http://www.javadoc.io/doc/org.springframework.cloud/spring-cloud-starter-kubernetes-netflix) [![Dependency Status](https://www.versioneye.com/java/org.springframework.cloud:spring-cloud-starter-kubernetes-netflix/badge?style=flat)](https://www.versioneye.com/java/org.springframework.cloud:spring-cloud-starter-kubernetes-netflix/) -A Kubernetes based `ServerList` for Ribbon has been implemented. The implementation is part of the [spring-cloud-kubernetes-ribbon](spring-cloud-kubernetes-ribbon/pom.xml) module and you can use it by adding: +Spring Cloud client applications calling a microservice should be interested on relying on a client load-balancing +feature in order to automatically discover at which endpoint(s) it can reach a given service. This mechanism has been +implemented within the [spring-cloud-kubernetes-ribbon](spring-cloud-kubernetes-ribbon/pom.xml) project where a +Kubernetes client will populate a [Ribbon](https://github.com/Netflix/ribbon) `ServerList` containing information +about such endpoints. + +The implementation is part of the following starter that you can use by adding its dependency to your pom file: ```xml @@ -324,19 +438,37 @@ A Kubernetes based `ServerList` for Ribbon has been implemented. The implementat ``` -The ribbon discovery client can be disabled by setting `spring.cloud.kubernetes.ribbon.enabled=false`. +When the list of the endpoints is populated, the Kubernetes client will search the registered endpoints living in +the current namespace/project matching the service name defined using the Ribbon Client annotation: -By default the client will detect all endpoints with the configured *client name* that lives in the current namespace. -If the endpoint contains multiple ports, the first port will be used. To fine tune the name of the desired port (if the service is a multiport service) or fine tune the namespace you can use one of the following properties. +```java +@RibbonClient(name = "name-service") +``` -- `KubernetesNamespace` -- `PortName` +You can configure Ribbon's behavior by providing properties in your `application.properties` (via your application's +dedicated `ConfigMap`) using the following format: `.ribbon.` where: + +- `` corresponds to the service name you're accessing over Ribbon, as configured using the +`@RibbonClient` annotation (e.g. `name-service` in the example above) +- ` @@ -356,10 +492,20 @@ Discovery of the services required by Zipkin (e.g. `zipkin-query`) is provided b ``` -This works as an extension of [spring-cloud-sleuth-zipkin](https://github.com/spring-cloud/spring-cloud-sleuth/tree/master/spring-cloud-sleuth-zipkin). +This works as an extension of the [spring-cloud-sleuth-zipkin](https://github.com/spring-cloud/spring-cloud-sleuth/tree/master/spring-cloud-sleuth-zipkin) project. +The name of the Zipkin service and the target Kubernetes namespace/project where the service runs can be specified +using the following `application.properties` properties: + +```bash +spring.cloud.kubernetes.zipkin.discovery.serviceName=my-zipkin +spring.cloud.kubernetes.zipkin.discovery.serviceNamespace=tracing +``` + +By default, the discovery client will look for a Zipkin service named `zipkin` within the current namespace. Examples of application that are using Zipkin discovery in Kubernetes: +- [Spring Cloud Kubernetes and Zipkin](kubernetes-zipkin) - [fabric8-quickstarts - Spring Boot - Ribbon](https://github.com/fabric8-quickstarts/spring-boot-ribbon) - [Kubeflix - LoanBroker - Bank](https://github.com/fabric8io/kubeflix/tree/master/examples/loanbroker/bank) @@ -369,11 +515,14 @@ Examples of application that are using Zipkin discovery in Kubernetes: [![Javadocs](http://www.javadoc.io/badge/org.springframework.cloud/spring-cloud-kubernetes-archaius.svg?color=blue)](http://www.javadoc.io/doc/org.springframework.cloud/spring-cloud-kubernetes-archaius) [![Dependency Status](https://www.versioneye.com/java/org.springframework.cloud:spring-cloud-kubernetes-archaius/badge?style=flat)](https://www.versioneye.com/java/org.springframework.cloud:spring-cloud-kubernetes-archaius/) -Section [ConfigMap PropertySource](#configmap-propertysource) provides a brief explanation on how to configure spring boot application via ConfigMap. -This approach will aid in creating the configuration properties objects that will be passed in our application. If our application is using Archaius it will indirectly benefit by it. -An alternative approach that provides more direct Archaius support without getting in the way of spring configuration properties by using [spring-cloud-kubernetes-archaius](spring-cloud-kubernetes-archaius/pom.xml) that is part of the Netflix starter. +The section [ConfigMap PropertySource](#configmap-propertysource) introduced how to configure a spring boot application via `Kubernetes ConfigMap` containing your configuration properties file. -This module allows you to annotate your application with the `@ArchaiusConfigMapSource` and archaius will automatically use the configmap as a watched source *(get notification on changes)*. +If you prefer to use the configuration management library [NetFlix Archaius](https://github.com/Netflix/archaius/wiki) instead of using the Spring application.properties|"yaml file, +then you can also leverage the `ConfigMap feature` by using the [spring-cloud-kubernetes-archaius](spring-cloud-kubernetes-archaius/pom.xml) project. + +To use it, add the following starter `spring-cloud-starter-kubernetes-all` to yiur pom file definition. + +This module allows you to annotate your application with the `@ArchaiusConfigMapSource` and archaius will automatically use the `Kubernetes configmap` as a watched source *(get notification on changes)*. --- ### Troubleshooting @@ -396,7 +545,9 @@ For earlier version it needs to be specified as an env var to the pod. A quick w #### Service Account -For distros of Kubernetes that support more fine-grained role-based access within the cluster, you need to make sure a pod that runs with spring-cloud-kubernetes has access to the Kubernetes API. For example, OpenShift has very comprehensive security measures that are on by default (typically) in a shared cluster. For any service accounts you assign to a deployment/pod, you need to make sure it has the correct roles. For example, you can add `cluster-reader` permissions to your `default` service account depending on the project you're in: +For distros of Kubernetes that support more fine-grained role-based access within the cluster, you need to make sure a pod that runs with spring-cloud-kubernetes has access to the Kubernetes API. +For example, OpenShift has very comprehensive security measures that are on by default (typically) in a shared cluster. +For any service accounts you assign to a deployment/pod, you need to make sure it has the correct roles. For example, you can add `cluster-reader` permissions to your `default` service account depending on the project you're in: ``` oc policy add-role-to-user cluster-reader system:serviceaccount::default