diff --git a/docs/src/main/asciidoc/discovery-client.adoc b/docs/src/main/asciidoc/discovery-client.adoc index 99d6a182..00d893f3 100644 --- a/docs/src/main/asciidoc/discovery-client.adoc +++ b/docs/src/main/asciidoc/discovery-client.adoc @@ -6,9 +6,36 @@ This client lets you query Kubernetes endpoints (see https://kubernetes.io/docs/ A service is typically exposed by the Kubernetes API server as a collection of endpoints that represent `http` and `https` addresses and that a client can access from a Spring Boot application running as a pod. -DiscoveryClient can also find services of type `ExternalName` (see https://kubernetes.io/docs/concepts/services-networking/service/#externalname[ExternalName services]). At the moment, external name support type of services is only available if the following property `spring.cloud.kubernetes.discovery.include-external-name-services` is set to `true` and only in the `fabric8` implementation. In a later release, support will be added for the kubernetes native client also. +DiscoveryClient can also find services of type `ExternalName` (see https://kubernetes.io/docs/concepts/services-networking/service/#externalname[ExternalName services]). At the moment, external name support type of services is only available if the following property `spring.cloud.kubernetes.discovery.include-external-name-services` is set to `true` (it is `false` by default). -This is something that you get for free by adding the following dependency inside your project: +There are 3 types of discovery clients that we support: + +1. +==== +Fabric8 Kubernetes Client +[source,xml] +---- + + org.springframework.cloud + spring-cloud-starter-kubernetes-fabric8 + +---- +==== + +2. + +==== +Kubernetes Java Client +[source,xml] +---- + + org.springframework.cloud + spring-cloud-starter-kubernetes-client + +---- +==== + +3. ==== HTTP Based `DiscoveryClient` @@ -24,28 +51,6 @@ HTTP Based `DiscoveryClient` NOTE: `spring-cloud-starter-kubernetes-discoveryclient` is designed to be used with the <>. -==== -Fabric8 Kubernetes Client -[source,xml] ----- - - org.springframework.cloud - spring-cloud-starter-kubernetes-fabric8 - ----- -==== - -==== -Kubernetes Java Client -[source,xml] ----- - - org.springframework.cloud - spring-cloud-starter-kubernetes-client - ----- -==== - To enable loading of the `DiscoveryClient`, add `@EnableDiscoveryClient` to the according configuration or application class, as the following example shows: ==== @@ -71,7 +76,19 @@ private DiscoveryClient discoveryClient; ---- ==== -You can choose to enable `DiscoveryClient` from all namespaces by setting the following property in `application.properties`: +The first question you should ask yourself is _where_ a `DiscoveryClient` supposed to discover services. In the kubernetes world, this means what namespace(s). There are 3 options here: + + - `selective namespaces`. For example: + +[source] +---- +spring.cloud.kubernetes.discovery.namespaces[0]=ns1 +spring.cloud.kubernetes.discovery.namespaces[1]=ns2 +---- + +Such a configuration makes discovery client only search for services in two namespaces `ns1` and `ns2`. + + - `all-namespaces`. ==== [source] @@ -80,15 +97,108 @@ spring.cloud.kubernetes.discovery.all-namespaces=true ---- ==== -To discover services and endpoints only from specified namespaces you should set property `all-namespaces` to `false` and set the following property in `application.properties` (in this example namespaces are: `ns1` and `ns2`). +While such an option exists, this can be a burden on both kube-api and your application. It is rare to need such a setting. + + - `one namespace`. This is the default setting, if you do not specify any of the above. It works on the rules outlined in xref:property-source-config.adoc#namespace-resolution[Namespace Resolution]. + ==== +NOTE: The above options work exactly as written for fabric8 and k8s clients. For the HTTP based client, you need to enable those options on the _server_. That can be achieved by setting them in `deployment.yaml` used to deploy the image in the cluster, using env variable(s). +==== + +For example: + [source] ---- -spring.cloud.kubernetes.discovery.namespaces[0]=ns1 -spring.cloud.kubernetes.discovery.namespaces[1]=ns2 + containers: + - name: discovery-server + image: springcloud/spring-cloud-kubernetes-discoveryserver:3.0.5-SNAPSHOT + env: + - name: SPRING_CLOUD_KUBERNETES_DISCOVERY_NAMESPACES_0 + value: "namespace-a" ---- -==== + +Once namespaces have been configured, the next question to answer is what services to discover. Think about it as what filter to apply. By default, no filtering is applied at all and all services are discovered. If you need to narrow what discovery client can find, you have two options: + +- Only take services that match certain service labels. This property is specified with: `spring.cloud.kubernetes.discovery.service-labels`. It accepts a `Map` and only those services that have such labels (as seen in `metadata.labels` in the service definition) will be taken into account. + +- The other option is to use https://docs.spring.io/spring-framework/reference/core/expressions.html[SpEL expression]. This is denoted by the `spring.cloud.kubernetes.discovery.filter` property, and its value depends on the client that you chose. If you use the fabric8 client, this SpEL expression must be created against `io.fabric8.kubernetes.api.model.Service` class. One such example could be: + +[source] +---- +spring.cloud.kubernetes.discovery.filter='#root.metadata.namespace matches "^.+A$"' +---- + +which tells discovery client to only get services that have the `metadata.namespace` that ends in upper case `A`. + +If your discovery client is based on k8s-native client, then the SpEL expression must be based on `io.kubernetes.client.openapi.models.V1Service` class. The same filter showed above would work here. + +If your discovery client is the http based one, then the SeEL expression has to be based on the same `io.kubernetes.client.openapi.models.V1Service` class, with the only distinction that this needs to be set as an env variable in the deployment yaml: + + +---- + containers: + - name: discovery-server + image: springcloud/spring-cloud-kubernetes-discoveryserver:3.0.5-SNAPSHOT + env: + - name: SPRING_CLOUD_KUBERNETES_DISCOVERY_FILTER + value: '#root.metadata.namespace matches "^.+A$"' +---- + +It's now time to think what discovery client is supposed to return back. In general, there are two methods that `DiscoveryClient` has: `getServices` and `getInstances`. + +`getServices` will return the service _names_ as seen in the `metadata.name`. + + +NOTE: This method will return unique service names, even if there are duplicates across different namespaces (that you chose for the search). + + +`getInstances` returns a `List`. Besides the usual fields that a `ServiceInstance` has, we also add some data, like namespace or pod metadata (more explanation about these will follow in the document). Here is the data that we return at the moment: + +. `instanceId` - unique id of the service instance +. `serviceId` - the name of the service (it is the same as the one reported by calling `getServices`) +. `host` - IP of the instance (or name in case of the `ExternalName` type of service) +. `port` - port number of the instance. This requires a bit more explanation, as choosing the port number has its rules: + + .. service has no port defined, 0 (zero) will be returned. + .. service has a single port defined, that one will be returned. + .. If the service has a label `primary-port-name`, we will use the port number that has the name specified in the label's value. + .. If the above label is not present, then we will use the port name specified in `spring.cloud.kubernetes.discovery.primary-port-name` to find the port number. + .. If neither of the above are specified, we will use the port named `https` or `http` to compute the port number. + .. As a last resort we wil pick the first port in the list of ports. This last option may result in non-deterministic behaviour. + +. `uri` of the service instance + +. `scheme` either `http` or `https` (depending on the `secure` result) + +. `metadata` of the service: + +.. `labels` (if requested via `spring.cloud.kubernetes.discovery.metadata.add-labels=true`). Label keys can be "prefixed" with the value of `spring.cloud.kubernetes.discovery.metadata.labels-prefix` if it is set. + +.. `annotations` (if requested via `spring.cloud.kubernetes.discovery.metadata.add-annotations=true`). Annotations keys can be "prefixed" with the value of `spring.cloud.kubernetes.discovery.metadata.annotations-prefix` if it is set. + +.. `ports` (if requested via `spring.cloud.kubernetes.discovery.metadata.add-ports=true`). Port keys can be "prefixed" with the value of `spring.cloud.kubernetes.discovery.metadata.ports-prefix` if it is set. + +.. `k8s_namespace` with the value of the namespace where instance resides. + +.. `type` that holds the service type, for example `ClusterIP` or `ExternalName` + +. `secure` if the port that was discovered should be treated as secure. We will use the same rules outlined above to find the port name and number, and then: + +.. If this service has a label called `secured` with any of the values : `["true", "on", "yes", "1"]`, then treat the port that was found as secure. + +.. If such a label is not found, search for an annotation called `secured` and apply the same above rules. + +.. If this port number is part of `spring.cloud.kubernetes.discovery.known-secure-ports` (by default this value holds `[443, 8443]`), treat port number as secured. + +.. Last resort is to see if port name matches `https`; if it does treat this port as secured. + +. `namespace` - the namespace of the found instance. + +. `pod-metadata` labels and annotations of the service instance (pod), in the form of `Map>`. This support needs to be enabled via `spring.cloud.kubernetes.discovery.metadata.add-pod-labels=true` and/or `spring.cloud.kubernetes.discovery.metadata.add-pod-annotaations=true` + +''' + To discover service endpoint addresses that are not marked as "ready" by the kubernetes api server, you can set the following property in `application.properties` (default: false): @@ -98,42 +208,27 @@ To discover service endpoint addresses that are not marked as "ready" by the kub spring.cloud.kubernetes.discovery.include-not-ready-addresses=true ---- NOTE: This might be useful when discovering services for monitoring purposes, and would enable inspecting the `/health` endpoint of not-ready service instances. -==== -If your service exposes multiple ports, you will need to specify which port the `DiscoveryClient` should use. -The `DiscoveryClient` will choose the port using the following logic. -1. If the service has a label `primary-port-name` it will use the port with the name specified in the label's value. -2. If no label is present, then the port name specified in `spring.cloud.kubernetes.discovery.primary-port-name` will be used. -3. If neither of the above are specified it will use the port named `https`. -4. If none of the above conditions are met it will use the port named `http`. -5. As a last resort it wil pick the first port in the list of ports. - -WARNING: The last option may result in non-deterministic behaviour. -Please make sure to configure your service and/or application accordingly. - -By default all of the ports and their names will be added to the metadata of the `ServiceInstance`. - -As said before, if you want to get the list of `ServiceInstance` to also include the `ExternalName` type services, you need to enable that support via: `spring.cloud.kubernetes.discovery.include-external-name-services=true`. As such, when calling `DiscoveryClient::getInstances` those will be returned also. You can distinguish between `ExternalName` and any other types by inspecting `ServiceInstance::getMetadata` and lookup for a field called `type`. This will be the type of the service returned : `ExternalName`/`ClusterIP`, etc. - -`ServiceInstance` can include the labels and annotations of specific pods from the underlying service instance. To obtain such information, you need to also enable: - -`spring.cloud.kubernetes.discovery.metadata.add-pod-labels=true` and/or `spring.cloud.kubernetes.discovery.metadata.add-pod-annotations=true`. At the moment, such functionality is present only in the fabric8 client implementation, but will be added to the kubernetes native client in a later release. +If you want to get the list of `ServiceInstance` to also include the `ExternalName` type services, you need to enable that support via: `spring.cloud.kubernetes.discovery.include-external-name-services=true`. As such, when calling `DiscoveryClient::getInstances` those will be returned also. You can distinguish between `ExternalName` and any other types by inspecting `ServiceInstance::getMetadata` and lookup for a field called `type`. This will be the type of the service returned : `ExternalName`/`ClusterIP`, etc. If, for any reason, you need to disable the `DiscoveryClient`, you can set the following property in `application.properties`: ==== [source] ---- -spring.cloud.kubernetes.discovery.enabled=false +spring.main.cloud-platform=NONE ---- -==== + +Note that the support of discovery client is _automatic_, depending on where you run the application. So the above setting might not be needed. Some Spring Cloud components use the `DiscoveryClient` in order to obtain information about the local service instance. For this to work, you need to align the Kubernetes service name with the `spring.application.name` property. NOTE: `spring.application.name` has no effect as far as the name registered for the application within Kubernetes +''' + Spring Cloud Kubernetes can also watch the Kubernetes service catalog for changes and update the `DiscoveryClient` implementation accordingly. By "watch" we mean that we will publish a heartbeat event every `spring.cloud.kubernetes.discovery.catalog-services-watch-delay` milliseconds (by default it is `30000`). The heartbeat event will contain the target references (and their namespaces of the addresses of all endpoints @@ -145,7 +240,8 @@ The endpoints will be queried in either : - specific namespaces (enabled via `spring.cloud.kubernetes.discovery.namespaces`), for example: -``` +[source] +---- spring: cloud: kubernetes: @@ -153,7 +249,7 @@ spring: namespaces: - namespace-a - namespace-b -``` +---- - we will use: xref:property-source-config.adoc#namespace-resolution[Namespace Resolution] if the above two paths are not taken. @@ -162,7 +258,8 @@ In order to enable this functionality you need to add By default, we use the `Endpoints`(see https://kubernetes.io/docs/concepts/services-networking/service/#endpoints) API to find out the current state of services. There is another way though, via `EndpointSlices` (https://kubernetes.io/docs/concepts/services-networking/endpoint-slices/). Such support can be enabled via a property: `spring.cloud.kubernetes.discovery.use-endpoint-slices=true` (by default it is `false`). Of course, your cluster has to support it also. As a matter of fact, if you enable this property, but your cluster does not support it, we will fail starting the application. If you decide to enable such support, you also need proper Role/ClusterRole set-up. For example: -``` +[source] +---- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: @@ -172,4 +269,4 @@ rules: - apiGroups: ["discovery.k8s.io"] resources: ["endpointslices"] verbs: ["get", "list", "watch"] -``` +----