Remove *.stream.* fragment from property prefixes

- SchemaRegistryClientProperties replace prefix from spring.cloud.cloud.schema-registry-client to spring.cloud.schema-registry-client.
 - AvroMessageConverterProperties replace prefix from spring.cloud.stream.schema.avro to spring.cloud.schema.avro.
 - SchemaServerProperties replace prefix from spring.cloud.stream.schema.server to spring.cloud.schema.server.
 - Inject EnvironmentPostProcessors to convert old properties into new format.
 - Add new tests and Parametrize existing tests to ensure backward compatibility.

 - proper empyt collection check

 Resolves #13
This commit is contained in:
Christian Tzolov
2019-10-29 17:13:44 +01:00
committed by Soby Chacko
parent fc15acbef2
commit 2ffe170280
22 changed files with 578 additions and 64 deletions

View File

@@ -17,7 +17,7 @@ When organizations have a messaging based pub/sub architecture and multiple prod
When such a schema needs to evolve to accommodate new business requirements, the existing components are still required to continue to work.
This project provides support for a standalone schema registry server using which aforementioned schema can be registered and used by the applications.
It also contains support for avro based schema registry clients, which essentially provide message converters that communicates with the schema registry for reconciling schema during message conversion.
The schema evolution support provided by this project works both with the aforementioned standalone schema registry as well as the scheam registry provided by Confluent that specifically works with Apache Kafka.
The schema evolution support provided by this project works both with the aforementioned standalone schema registry as well as the schema registry provided by Confluent that specifically works with Apache Kafka.
==== Spring Cloud Schema Registry overview
@@ -76,17 +76,17 @@ A client for the Spring Cloud Stream schema registry can be configured by using
NOTE: The default converter is optimized to cache not only the schemas from the remote server but also the `parse()` and `toString()` methods, which are quite expensive.
Because of this, it uses a `DefaultSchemaRegistryClient` that does not cache responses.
If you intend to change the default behavior, you can use the client directly on your code and override it to the desired outcome.
To do so, you have to add the property `spring.cloud.stream.schemaRegistryClient.cached=true` to your application properties.
To do so, you have to add the property `spring.cloud.schemaRegistryClient.cached=true` to your application properties.
==== Schema Registry Client Properties
The Schema Registry Client supports the following properties:
`spring.cloud.stream.schemaRegistryClient.endpoint`:: The location of the schema-server.
`spring.cloud.schemaRegistryClient.endpoint`:: The location of the schema-server.
When setting this, use a full URL, including protocol (`http` or `https`) , port, and context path.
+
Default:: `http://localhost:8990/`
`spring.cloud.stream.schemaRegistryClient.cached`:: Whether the client should cache schema server responses.
`spring.cloud.schemaRegistryClient.cached`:: Whether the client should cache schema server responses.
Normally set to `false`, as the caching happens in the message converter.
Clients using the schema registry client should set this to `true`.
+
@@ -117,21 +117,21 @@ When receiving messages, the converter infers the schema reference from the head
If you have enabled Avro based schema registry client by setting `spring.cloud.stream.bindings.output.contentType=application/*+avro`, you can customize the behavior of the registration by setting the following properties.
spring.cloud.stream.schema.avro.dynamicSchemaGenerationEnabled:: Enable if you want the converter to use reflection to infer a Schema from a POJO.
spring.cloud.schema.avro.dynamicSchemaGenerationEnabled:: Enable if you want the converter to use reflection to infer a Schema from a POJO.
+
Default: `false`
+
spring.cloud.stream.schema.avro.readerSchema:: Avro compares schema versions by looking at a writer schema (origin payload) and a reader schema (your application payload). See the https://avro.apache.org/docs/1.7.6/spec.html[Avro documentation] for more information. If set, this overrides any lookups at the schema server and uses the local schema as the reader schema.
spring.cloud.schema.avro.readerSchema:: Avro compares schema versions by looking at a writer schema (origin payload) and a reader schema (your application payload). See the https://avro.apache.org/docs/1.7.6/spec.html[Avro documentation] for more information. If set, this overrides any lookups at the schema server and uses the local schema as the reader schema.
Default: `null`
+
spring.cloud.stream.schema.avro.schemaLocations:: Registers any `.avsc` files listed in this property with the Schema Server.
spring.cloud.schema.avro.schemaLocations:: Registers any `.avsc` files listed in this property with the Schema Server.
+
Default: `empty`
+
spring.cloud.stream.schema.avro.prefix:: The prefix to be used on the Content-Type header.
spring.cloud.schema.avro.prefix:: The prefix to be used on the Content-Type header.
+
Default: `vnd`
spring.cloud.stream.schema.avro.subjectNamingStrategy:: Determines the subject name used to register the Avro schema in the schema registry. Two implementations are available, `org.springframework.cloud.stream.schema.avro.DefaultSubjectNamingStrategy`,
spring.cloud.schema.avro.subjectNamingStrategy:: Determines the subject name used to register the Avro schema in the schema registry. Two implementations are available, `org.springframework.cloud.stream.schema.avro.DefaultSubjectNamingStrategy`,
where the subject is the schema name, and `org.springframework.cloud.stream.schema.avro.QualifiedSubjectNamingStrategy`, which returns a fully qualified subject using the Avro schema namespace and name. Custom strategies can be created by implementing `org.springframework.cloud.stream.schema.avro.SubjectNamingStrategy`.
+
Default: `org.springframework.cloud.stream.schema.avro.DefaultSubjectNamingStrategy`
@@ -196,10 +196,10 @@ public static class SinkApplication {
=== Schema Registry Server
Spring Cloud Stream provides a schema registry server implementation.
To use it, you can add the `spring-cloud-stream-schema-server` artifact to your project and use the `@EnableSchemaRegistryServer` annotation, which adds the schema registry server REST controller to your application.
To use it, you can add the `spring-cloud-schema-server` artifact to your project and use the `@EnableSchemaRegistryServer` annotation, which adds the schema registry server REST controller to your application.
This annotation is intended to be used with Spring Boot web applications, and the listening port of the server is controlled by the `server.port` property.
The `spring.cloud.stream.schema.server.path` property can be used to control the root path of the schema server (especially when it is embedded in other applications).
The `spring.cloud.stream.schema.server.allowSchemaDeletion` boolean property enables the deletion of a schema. By default, this is disabled.
The `spring.cloud.schema.server.path` property can be used to control the root path of the schema server (especially when it is embedded in other applications).
The `spring.cloud.schema.server.allowSchemaDeletion` boolean property enables the deletion of a schema. By default, this is disabled.
The schema registry server uses a relational database to store the schemas.
By default, it uses an embedded database.
@@ -317,7 +317,7 @@ If you want to use the Confluent schema registry, you need to create a bean of t
[source,java]
----
@Bean
public SchemaRegistryClient schemaRegistryClient(@Value("${spring.cloud.stream.schemaRegistryClient.endpoint}") String endpoint){
public SchemaRegistryClient schemaRegistryClient(@Value("${spring.cloud.schemaRegistryClient.endpoint}") String endpoint){
ConfluentSchemaRegistryClient client = new ConfluentSchemaRegistryClient();
client.setEndpoint(endpoint);
return client;
@@ -337,7 +337,7 @@ To better understand how Spring Cloud Stream registers and resolves new schemas
The first part of the registration process is extracting a schema from the payload that is being sent over a channel.
Avro types such as `SpecificRecord` or `GenericRecord` already contain a schema, which can be retrieved immediately from the instance.
In the case of POJOs, a schema is inferred if the `spring.cloud.stream.schema.avro.dynamicSchemaGenerationEnabled` property is set to `true` (the default).
In the case of POJOs, a schema is inferred if the `spring.cloud.schema.avro.dynamicSchemaGenerationEnabled` property is set to `true` (the default).
.Schema Writer Resolution Process
image::{github-raw}/docs/src/main/asciidoc/images/schema_resolution.png[width=800,scaledwidth="75%",align="center"]
@@ -363,4 +363,4 @@ image::{github-raw}/docs/src/main/asciidoc/images/schema_reading.png[width=800,s
NOTE: You should understand the difference between a writer schema (the application that wrote the message) and a reader schema (the receiving application).
We suggest taking a moment to read https://avro.apache.org/docs/1.7.6/spec.html[the Avro terminology] and understand the process.
Spring Cloud Stream always fetches the writer schema to determine how to read a message.
If you want to get Avro's schema evolution support working, you need to make sure that a `readerSchema` was properly set for your application.
If you want to get Avro's schema evolution support working, you need to make sure that a `readerSchema` was properly set for your application.

View File

@@ -0,0 +1,68 @@
/*
* Copyright 2019-2019 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.schema.registry;
import java.util.Map;
import java.util.Properties;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.env.EnvironmentPostProcessor;
import org.springframework.core.env.ConfigurableEnvironment;
import org.springframework.core.env.PropertiesPropertySource;
import org.springframework.util.CollectionUtils;
/**
* Override deprecated properties for backward compatibility.
* @author Christian Tzolov
*/
public abstract class AbstractBackwardCompatibilityEnvironmentPostProcessor implements EnvironmentPostProcessor {
private final Log logger = LogFactory.getLog(AbstractBackwardCompatibilityEnvironmentPostProcessor.class);
private final String propertyKeyName;
public AbstractBackwardCompatibilityEnvironmentPostProcessor(String propertyKeyName) {
this.propertyKeyName = propertyKeyName;
}
@Override
public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) {
Properties properties = new Properties();
for (Map.Entry<String, String> e : doGetPropertyMappings().entrySet()) {
if (environment.containsProperty(e.getKey())) {
properties.setProperty(e.getValue(), environment.getProperty(e.getKey()));
}
}
// This post-processor is called multiple times but sets the properties only once.
if (!CollectionUtils.isEmpty(properties)) {
logger.info(" 'spring.cloud.stream.schemaXXX' property prefix detected! " +
"Use the 'spring.schemaXXX' prefix instead!");
PropertiesPropertySource propertiesPropertySource =
new PropertiesPropertySource(propertyKeyName, properties);
environment.getPropertySources().addLast(propertiesPropertySource);
}
}
protected abstract Map<String, String> doGetPropertyMappings();
}

View File

@@ -37,20 +37,16 @@ import org.springframework.util.ReflectionUtils;
* @author Vinicius Carvalho
* @author Sercan Karaoglu
* @author Ish Mahajan
* @author Christian Tzolov
*/
@Configuration
@ConditionalOnClass(name = "org.apache.avro.Schema")
@ConditionalOnProperty(value = "spring.cloud.stream.schemaRegistryClient.enabled", matchIfMissing = true)
@ConditionalOnProperty(value = "spring.cloud.schemaRegistryClient.enabled", matchIfMissing = true)
@ConditionalOnBean(type = "org.springframework.cloud.schema.registry.client.SchemaRegistryClient")
@EnableConfigurationProperties({ AvroMessageConverterProperties.class })
@Import(AvroSchemaServiceManagerImpl.class)
public class AvroMessageConverterAutoConfiguration {
// @Autowired
// private AvroMessageConverterProperties avroMessageConverterProperties;
//// @Autowired
// private AvroSchemaServiceManager avroSchemaServiceManager;
@Bean
@ConditionalOnMissingBean(AvroSchemaRegistryClientMessageConverter.class)
public AvroSchemaRegistryClientMessageConverter avroSchemaMessageConverter(

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2016-2018 the original author or authors.
* Copyright 2016-2019 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -23,8 +23,9 @@ import org.springframework.util.Assert;
/**
* @author Vinicius Carvalho
* @author Sercan Karaoglu
* @author Christian Tzolov
*/
@ConfigurationProperties(prefix = "spring.cloud.stream.schema.avro")
@ConfigurationProperties(prefix = "spring.cloud.schema.avro")
public class AvroMessageConverterProperties {
private boolean dynamicSchemaGenerationEnabled;

View File

@@ -0,0 +1,52 @@
/*
* Copyright 2019-2019 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.schema.registry.avro;
import java.util.HashMap;
import java.util.Map;
import org.springframework.cloud.schema.registry.AbstractBackwardCompatibilityEnvironmentPostProcessor;
/**
* Processor overrides deprecated AvroMessageConverterProperties for backward compatibility.
*
* @author Christian Tzolov
*/
public class AvroMessageConverterPropertiesBackwardCompatibilityEnvironmentPostProcessor extends AbstractBackwardCompatibilityEnvironmentPostProcessor {
public AvroMessageConverterPropertiesBackwardCompatibilityEnvironmentPostProcessor() {
super(AvroMessageConverterPropertiesBackwardCompatibilityEnvironmentPostProcessor.class.getName());
}
@Override
protected Map<String, String> doGetPropertyMappings() {
Map<String, String> propertyMapping = new HashMap<>();
propertyMapping.put("spring.cloud.stream.schema.avro.dynamicSchemaGenerationEnabled", "spring.cloud.schema.avro.dynamicSchemaGenerationEnabled");
propertyMapping.put("spring.cloud.stream.schema.avro.dynamic-schema-generation-enabled", "spring.cloud.schema.avro.dynamic-schema-generation-enabled");
propertyMapping.put("spring.cloud.stream.schema.avro.readerSchema", "spring.cloud.schema.avro.readerSchema");
propertyMapping.put("spring.cloud.stream.schema.avro.reader-schema", "spring.cloud.schema.avro.reader-schema");
propertyMapping.put("spring.cloud.stream.schema.avro.schemaLocations", "spring.cloud.schema.avro.schemaLocations");
propertyMapping.put("spring.cloud.stream.schema.avro.schema-locations", "spring.cloud.schema.avro.schema-locations");
propertyMapping.put("spring.cloud.stream.schema.avro.schemaImports", "spring.cloud.schema.avro.schemaImports");
propertyMapping.put("spring.cloud.stream.schema.avro.schema-imports", "spring.cloud.schema.avro.schema-imports");
propertyMapping.put("spring.cloud.stream.schema.avro.prefix", "spring.cloud.schema.avro.prefix");
propertyMapping.put("spring.cloud.stream.schema.avro.subjectNamingStrategy", "spring.cloud.schema.avro.subjectNamingStrategy");
propertyMapping.put("spring.cloud.stream.schema.avro.subject-naming-strategy", "spring.cloud.schema.avro.subject-naming-strategy");
return propertyMapping;
}
}

View File

@@ -34,9 +34,6 @@ import org.springframework.util.StringUtils;
@EnableConfigurationProperties(SchemaRegistryClientProperties.class)
public class SchemaRegistryClientConfiguration {
// @Autowired
// private SchemaRegistryClientProperties schemaRegistryClientProperties;
@Bean
@ConditionalOnMissingBean
public SchemaRegistryClient schemaRegistryClient(SchemaRegistryClientProperties schemaRegistryClientProperties) {

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2016-2017 the original author or authors.
* Copyright 2016-2019 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -21,8 +21,9 @@ import org.springframework.boot.context.properties.ConfigurationProperties;
/**
* @author Marius Bogoevici
* @author Vinicius Carvalho
* @author Christian Tzolov
*/
@ConfigurationProperties(prefix = "spring.cloud.stream.schema-registry-client")
@ConfigurationProperties(prefix = "spring.cloud.schema-registry-client")
public class SchemaRegistryClientProperties {
private String endpoint;

View File

@@ -0,0 +1,44 @@
/*
* Copyright 2019-2019 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.schema.registry.client.config;
import java.util.HashMap;
import java.util.Map;
import org.springframework.cloud.schema.registry.AbstractBackwardCompatibilityEnvironmentPostProcessor;
/**
* Processor overrides deprecated SchemaRegistryClientProperties for backward compatibility.
*
* @author Christian Tzolov
*/
public class SchemaRegistryClientPropertiesBackwardCompatibilityEnvironmentPostProcessor extends AbstractBackwardCompatibilityEnvironmentPostProcessor {
public SchemaRegistryClientPropertiesBackwardCompatibilityEnvironmentPostProcessor() {
super(SchemaRegistryClientPropertiesBackwardCompatibilityEnvironmentPostProcessor.class.getName());
}
@Override
protected Map<String, String> doGetPropertyMappings() {
Map<String, String> propertyMapping = new HashMap<>();
propertyMapping.put("spring.cloud.stream.schema-registry-client.endpoint", "spring.cloud.schema-registry-client.endpoint");
propertyMapping.put("spring.cloud.stream.schemaRegistryClient.endpoint", "spring.cloud.schemaRegistryClient.endpoint");
propertyMapping.put("spring.cloud.stream.schema-registry-client.cached", "spring.cloud.schema-registry-client.cached");
propertyMapping.put("spring.cloud.stream.schemaRegistryClient.cached", "spring.cloud.schemaRegistryClient.cached");
return propertyMapping;
}
}

View File

@@ -1,2 +1,6 @@
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
org.springframework.cloud.schema.registry.avro.AvroMessageConverterAutoConfiguration
org.springframework.boot.env.EnvironmentPostProcessor=\
org.springframework.cloud.schema.registry.client.config.SchemaRegistryClientPropertiesBackwardCompatibilityEnvironmentPostProcessor,\
org.springframework.cloud.schema.registry.avro.AvroMessageConverterPropertiesBackwardCompatibilityEnvironmentPostProcessor

View File

@@ -59,7 +59,6 @@ public class AvroSchemaMessageConverterTests {
AvroSourceApplication.class, "--server.port=0",
"--spring.jmx.enabled=false",
"--schemaLocation=classpath:schemas/users_v1.schema",
"--spring.cloud.stream.schemaRegistryClient.enabled=false",
"--spring.cloud.stream.bindings.output.contentType=avro/bytes");
Source source = sourceContext.getBean(Source.class);
User1 firstOutboundFoo = new User1();
@@ -75,7 +74,6 @@ public class AvroSchemaMessageConverterTests {
AvroSourceApplication.class, "--server.port=0",
"--spring.jmx.enabled=false",
"--schemaLocation=classpath:schemas/users_v1.schema",
"--spring.cloud.stream.schemaRegistryClient.enabled=false",
"--spring.cloud.stream.bindings.output.contentType=avro/bytes");
Source barSource = barSourceContext.getBean(Source.class);
User2 firstOutboundUser2 = new User2();
@@ -101,7 +99,6 @@ public class AvroSchemaMessageConverterTests {
ConfigurableApplicationContext sinkContext = SpringApplication.run(
AvroSinkApplication.class, "--server.port=0",
"--spring.jmx.enabled=false",
"--spring.cloud.stream.schemaRegistryClient.enabled=false",
"--schemaLocation=classpath:schemas/users_v1.schema");
Sink sink = sinkContext.getBean(Sink.class);
sink.input().send(outboundMessage);
@@ -135,7 +132,6 @@ public class AvroSchemaMessageConverterTests {
ConfigurableApplicationContext sourceContext = SpringApplication.run(
AvroSourceApplication.class, "--server.port=0",
"--spring.jmx.enabled=false",
"--spring.cloud.stream.schemaRegistryClient.enabled=false",
"--spring.cloud.stream.bindings.output.contentType=avro/bytes");
Source source = sourceContext.getBean(Source.class);
User1 firstOutboundFoo = new User1();
@@ -150,7 +146,6 @@ public class AvroSchemaMessageConverterTests {
ConfigurableApplicationContext barSourceContext = SpringApplication.run(
AvroSourceApplication.class, "--server.port=0",
"--spring.jmx.enabled=false",
"--spring.cloud.stream.schemaRegistryClient.enabled=false",
"--spring.cloud.stream.bindings.output.contentType=avro/bytes");
Source barSource = barSourceContext.getBean(Source.class);
User2 firstOutboundUser2 = new User2();
@@ -175,8 +170,7 @@ public class AvroSchemaMessageConverterTests {
ConfigurableApplicationContext sinkContext = SpringApplication.run(
AvroSinkApplication.class, "--server.port=0",
"--spring.jmx.enabled=false",
"--spring.cloud.stream.schemaRegistryClient.enabled=false");
"--spring.jmx.enabled=false");
Sink sink = sinkContext.getBean(Sink.class);
sink.input().send(outboundMessage);
sink.input().send(barOutboundMessage);

View File

@@ -17,11 +17,15 @@
package org.springframework.cloud.schema.avro;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.List;
import java.util.UUID;
import java.util.concurrent.TimeUnit;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
@@ -40,18 +44,33 @@ import static org.assertj.core.api.Assertions.assertThat;
/**
* @author Marius Bogoevici
* @author Christian Tzolov
*/
@RunWith(Parameterized.class)
public class AvroStubSchemaRegistryClientMessageConverterTests {
private String propertyPrefix;
static SchemaRegistryClient stubSchemaRegistryClient = new StubSchemaRegistryClient();
public AvroStubSchemaRegistryClientMessageConverterTests(String propertyPrefix) {
this.propertyPrefix = propertyPrefix;
}
// Use parametrization to test the deprecated prefix (spring.cloud.stream) is handled as the new (spring.cloud)
// prefix.
@Parameterized.Parameters
public static Collection primeNumbers() {
return Arrays.asList("spring.cloud.stream", "spring.cloud");
}
@Test
public void testSendMessage() throws Exception {
ConfigurableApplicationContext sourceContext = SpringApplication.run(
AvroSourceApplication.class, "--server.port=0", "--debug",
"--spring.jmx.enabled=false",
"--spring.cloud.stream.bindings.output.contentType=application/*+avro",
"--spring.cloud.stream.schema.avro.dynamicSchemaGenerationEnabled=true");
"--" + propertyPrefix + ".schema.avro.dynamicSchemaGenerationEnabled=true");
Source source = sourceContext.getBean(Source.class);
User1 firstOutboundFoo = new User1();
firstOutboundFoo.setFavoriteColor("foo" + UUID.randomUUID().toString());
@@ -66,7 +85,7 @@ public class AvroStubSchemaRegistryClientMessageConverterTests {
AvroSourceApplication.class, "--server.port=0",
"--spring.jmx.enabled=false",
"--spring.cloud.stream.bindings.output.contentType=application/vnd.user1.v1+avro",
"--spring.cloud.stream.schema.avro.dynamicSchemaGenerationEnabled=true");
"--" + propertyPrefix + ".schema.avro.dynamicSchemaGenerationEnabled=true");
Source barSource = barSourceContext.getBean(Source.class);
User2 firstOutboundUser2 = new User2();
firstOutboundUser2.setFavoriteColor("foo" + UUID.randomUUID().toString());

View File

@@ -16,10 +16,14 @@
package org.springframework.cloud.schema.avro;
import java.util.Arrays;
import java.util.Collection;
import java.util.UUID;
import java.util.concurrent.TimeUnit;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
@@ -38,20 +42,35 @@ import static org.assertj.core.api.Assertions.assertThat;
/**
* @author David Kalosi
* @author José A. Íñigo
* @author Christian Tzolov
*/
@RunWith(Parameterized.class)
public class SubjectNamingStrategyTest {
private String propertyPrefix;
private static StubSchemaRegistryClient stubSchemaRegistryClient = new StubSchemaRegistryClient();
public SubjectNamingStrategyTest(String propertyPrefix) {
this.propertyPrefix = propertyPrefix;
}
// Use parametrization to test the deprecated prefix (spring.cloud.stream) is handled as the new (spring.cloud)
// prefix.
@Parameterized.Parameters
public static Collection primeNumbers() {
return Arrays.asList("spring.cloud.stream", "spring.cloud");
}
@Test
public void testQualifiedSubjectNamingStrategy() throws Exception {
ConfigurableApplicationContext sourceContext = SpringApplication.run(
AvroSourceApplication.class, "--server.port=0", "--debug",
"--spring.jmx.enabled=false",
"--spring.cloud.stream.bindings.output.contentType=application/*+avro",
"--spring.cloud.stream.schema.avro.subjectNamingStrategy="
+ "org.springframework.cloud.schema.registry.avro.QualifiedSubjectNamingStrategy",
"--spring.cloud.stream.schema.avro.dynamicSchemaGenerationEnabled=true");
AvroSourceApplication.class, "--server.port=0", "--debug",
"--spring.jmx.enabled=false",
"--spring.cloud.stream.bindings.output.contentType=application/*+avro",
"--" + propertyPrefix + ".schema.avro.subjectNamingStrategy="
+ "org.springframework.cloud.schema.registry.avro.QualifiedSubjectNamingStrategy",
"--" + propertyPrefix + ".schema.avro.dynamicSchemaGenerationEnabled=true");
Source source = sourceContext.getBean(Source.class);
User1 user1 = new User1();
@@ -60,12 +79,12 @@ public class SubjectNamingStrategyTest {
source.output().send(MessageBuilder.withPayload(user1).build());
MessageCollector barSourceMessageCollector = sourceContext
.getBean(MessageCollector.class);
.getBean(MessageCollector.class);
Message<?> message = barSourceMessageCollector.forChannel(source.output())
.poll(1000, TimeUnit.MILLISECONDS);
.poll(1000, TimeUnit.MILLISECONDS);
assertThat(message.getHeaders().get("contentType")).isEqualTo(MimeType.valueOf(
"application/vnd.org.springframework.cloud.schema.avro.User1.v1+avro"));
"application/vnd.org.springframework.cloud.schema.avro.User1.v1+avro"));
}
@EnableBinding(Source.class)

View File

@@ -17,6 +17,8 @@
package org.springframework.cloud.schema.serialization;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.List;
import java.util.UUID;
import java.util.concurrent.TimeUnit;
@@ -25,6 +27,8 @@ import example.avro.Command;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.mockito.Mockito;
import org.springframework.beans.DirectFieldAccessor;
@@ -36,7 +40,6 @@ import org.springframework.boot.web.servlet.server.ServletWebServerFactory;
import org.springframework.cache.CacheManager;
import org.springframework.cache.support.NoOpCache;
import org.springframework.cache.support.NoOpCacheManager;
import org.springframework.cloud.schema.avro.StubSchemaRegistryClient;
import org.springframework.cloud.schema.avro.User1;
import org.springframework.cloud.schema.avro.User2;
import org.springframework.cloud.schema.registry.EnableSchemaRegistryServer;
@@ -45,7 +48,6 @@ import org.springframework.cloud.schema.registry.avro.AvroSchemaServiceManager;
import org.springframework.cloud.schema.registry.avro.AvroSchemaServiceManagerImpl;
import org.springframework.cloud.schema.registry.client.DefaultSchemaRegistryClient;
import org.springframework.cloud.schema.registry.client.EnableSchemaRegistryClient;
import org.springframework.cloud.schema.registry.client.SchemaRegistryClient;
import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.cloud.stream.messaging.Sink;
@@ -69,13 +71,26 @@ import static org.springframework.cloud.schema.serialization.AvroMessageConverte
* @author Oleg Zhurakousky
* @author Sercan Karaoglu
* @author James Gee
* @author Christian Tzolov
*/
@RunWith(Parameterized.class)
public class AvroSchemaRegistryClientMessageConverterTests {
static SchemaRegistryClient stubSchemaRegistryClient = new StubSchemaRegistryClient();
private String propertyPrefix;
private ConfigurableApplicationContext schemaRegistryServerContext;
public AvroSchemaRegistryClientMessageConverterTests(String propertyPrefix) {
this.propertyPrefix = propertyPrefix;
}
// Use parametrization to test the deprecated prefix (spring.cloud.stream) is handled as the new (spring.cloud)
// prefix.
@Parameterized.Parameters
public static Collection primeNumbers() {
return Arrays.asList("spring.cloud.stream", "spring.cloud");
}
@Before
public void setup() {
this.schemaRegistryServerContext = SpringApplication.run(
@@ -95,7 +110,7 @@ public class AvroSchemaRegistryClientMessageConverterTests {
AvroSourceApplication.class, "--server.port=0",
"--spring.jmx.enabled=false",
"--spring.cloud.stream.bindings.output.contentType=application/*+avro",
"--spring.cloud.stream.schema.avro.dynamicSchemaGenerationEnabled=true");
"--" + propertyPrefix + ".schema.avro.dynamicSchemaGenerationEnabled=true");
Source source = sourceContext.getBean(Source.class);
User1 firstOutboundFoo = new User1();
firstOutboundFoo.setFavoriteColor("foo" + UUID.randomUUID().toString());
@@ -110,7 +125,7 @@ public class AvroSchemaRegistryClientMessageConverterTests {
AvroSourceApplication.class, "--server.port=0",
"--spring.jmx.enabled=false",
"--spring.cloud.stream.bindings.output.contentType=application/vnd.user1.v1+avro",
"--spring.cloud.stream.schema.avro.dynamicSchemaGenerationEnabled=true");
"--" + propertyPrefix + ".schema.avro.dynamicSchemaGenerationEnabled=true");
Source barSource = barSourceContext.getBean(Source.class);
User2 firstOutboundUser2 = new User2();
firstOutboundUser2.setFavoriteColor("foo" + UUID.randomUUID().toString());
@@ -170,12 +185,12 @@ public class AvroSchemaRegistryClientMessageConverterTests {
@Test
public void testSchemaImportConfiguration() throws Exception {
final String[] args = { "--server.port=0", "--spring.jmx.enabled=false",
"--spring.cloud.stream.schema.avro.dynamicSchemaGenerationEnabled=true",
"--" + propertyPrefix + ".schema.avro.dynamicSchemaGenerationEnabled=true",
"--spring.cloud.stream.bindings.output.contentType=application/*+avro",
"--spring.cloud.stream.bindings.output.destination=test",
"--spring.cloud.stream.bindings.schema-registry-client.endpoint=http://localhost:8990",
"--spring.cloud.stream.schema.avro.schema-locations=classpath:schemas/Command.avsc",
"--spring.cloud.stream.schema.avro.schema-imports=classpath:schemas/imports/Sms.avsc,"
"--" + propertyPrefix + ".schema.avro.schema-locations=classpath:schemas/Command.avsc",
"--" + propertyPrefix + ".schema.avro.schema-imports=classpath:schemas/imports/Sms.avsc,"
+ " classpath:schemas/imports/Email.avsc, classpath:schemas/imports/PushNotification.avsc" };
final ConfigurableApplicationContext sourceContext = SpringApplication
@@ -202,13 +217,15 @@ public class AvroSchemaRegistryClientMessageConverterTests {
@Test
public void testNoCacheConfiguration() {
ConfigurableApplicationContext sourceContext = SpringApplication
.run(NoCacheConfiguration.class, "--spring.main.web-environment=false");
AvroSchemaRegistryClientMessageConverter converter = sourceContext
.getBean(AvroSchemaRegistryClientMessageConverter.class);
DirectFieldAccessor accessor = new DirectFieldAccessor(converter);
assertThat(accessor.getPropertyValue("cacheManager"))
.isInstanceOf(NoOpCacheManager.class);
if (propertyPrefix.equalsIgnoreCase("spring.cloud.stream")) {
ConfigurableApplicationContext sourceContext = SpringApplication
.run(NoCacheConfiguration.class, "--spring.main.web-environment=false");
AvroSchemaRegistryClientMessageConverter converter = sourceContext
.getBean(AvroSchemaRegistryClientMessageConverter.class);
DirectFieldAccessor accessor = new DirectFieldAccessor(converter);
assertThat(accessor.getPropertyValue("cacheManager"))
.isInstanceOf(NoOpCacheManager.class);
}
}
@Test

View File

@@ -0,0 +1,68 @@
/*
* Copyright 2019-2019 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.schema.registry.config;
import java.util.Map;
import java.util.Properties;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.env.EnvironmentPostProcessor;
import org.springframework.core.env.ConfigurableEnvironment;
import org.springframework.core.env.PropertiesPropertySource;
import org.springframework.util.CollectionUtils;
/**
* Override deprecated properties for backward compatibility.
* @author Christian Tzolov
*/
public abstract class AbstractBackwardCompatibilityEnvironmentPostProcessor implements EnvironmentPostProcessor {
private final Log logger = LogFactory.getLog(AbstractBackwardCompatibilityEnvironmentPostProcessor.class);
private final String propertyKeyName;
public AbstractBackwardCompatibilityEnvironmentPostProcessor(String propertyKeyName) {
this.propertyKeyName = propertyKeyName;
}
@Override
public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) {
Properties properties = new Properties();
for (Map.Entry<String, String> e : doGetPropertyMappings().entrySet()) {
if (environment.containsProperty(e.getKey())) {
properties.setProperty(e.getValue(), environment.getProperty(e.getKey()));
}
}
// This post-processor is called multiple times but sets the properties only once.
if (!CollectionUtils.isEmpty(properties)) {
logger.info(" 'spring.cloud.stream.schemaXXX' property prefix detected! " +
"Use the 'spring.schemaXXX' prefix instead!");
PropertiesPropertySource propertiesPropertySource =
new PropertiesPropertySource(propertyKeyName, properties);
environment.getPropertySources().addLast(propertiesPropertySource);
}
}
protected abstract Map<String, String> doGetPropertyMappings();
}

View File

@@ -21,8 +21,9 @@ import org.springframework.boot.context.properties.ConfigurationProperties;
/**
* @author Vinicius Carvalho
* @author Ilayaperumal Gopinathan
* @author Christian Tzolov
*/
@ConfigurationProperties("spring.cloud.stream.schema.server")
@ConfigurationProperties("spring.cloud.schema.server")
public class SchemaServerProperties {
/**

View File

@@ -0,0 +1,42 @@
/*
* Copyright 2019-2019 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.schema.registry.config;
import java.util.HashMap;
import java.util.Map;
/**
* Processor overrides deprecated SchemaServerProperties for backward compatibility.
*
* @author Christian Tzolov
*/
public class SchemaServerPropertiesBackwardCompatibilityEnvironmentPostProcessor
extends AbstractBackwardCompatibilityEnvironmentPostProcessor {
public SchemaServerPropertiesBackwardCompatibilityEnvironmentPostProcessor() {
super(SchemaServerPropertiesBackwardCompatibilityEnvironmentPostProcessor.class.getName());
}
@Override
protected Map<String, String> doGetPropertyMappings() {
Map<String, String> propertyMapping = new HashMap<>();
propertyMapping.put("spring.cloud.stream.schema.server.path", "spring.cloud.schema.server.path");
propertyMapping.put("spring.cloud.stream.schema.server.allowSchemaDeletion", "spring.cloud.schema.server.allowSchemaDeletion");
propertyMapping.put("spring.cloud.stream.schema.server.allow-schema-deletion", "spring.cloud.schema.server.allow-schema-deletion");
return propertyMapping;
}
}

View File

@@ -50,9 +50,10 @@ import static org.springframework.http.MediaType.APPLICATION_JSON_VALUE;
* @author Vinicius Carvalho
* @author Ilayaperumal Gopinathan
* @author Jeff Maxwell
* @author Christian Tzolov
*/
@RestController
@RequestMapping(path = "${spring.cloud.stream.schema.server.path:}")
@RequestMapping(path = "${spring.cloud.schema.server.path:}")
public class ServerController {
private final SchemaRepository repository;

View File

@@ -1 +1,2 @@
org.springframework.boot.env.EnvironmentPostProcessor=\
org.springframework.cloud.schema.registry.config.SchemaServerPropertiesBackwardCompatibilityEnvironmentPostProcessor

View File

@@ -0,0 +1,59 @@
/*
* Copyright 2019-2019 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.schema.registry.entityScanning;
import org.junit.Before;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.schema.registry.config.SchemaServerProperties;
import org.springframework.cloud.schema.registry.model.Schema;
import org.springframework.cloud.schema.registry.repository.SchemaRepository;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.setup.MockMvcBuilders;
import org.springframework.web.context.WebApplicationContext;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
/**
* @author Christian Tzolov
*/
public abstract class AbstractServerControllerTest {
protected MockMvc mockMvc;
@Autowired
protected SchemaRepository schemaRepository;
@Autowired
protected SchemaServerProperties schemaServerProperties;
@Autowired
private WebApplicationContext wac;
@Before
public void setupMocks() {
this.mockMvc = MockMvcBuilders.webAppContextSetup(wac)
.defaultRequest(get("/").accept(MediaType.APPLICATION_JSON)).build();
Schema schema = new Schema();
schema.setSubject("test667");
schema.setVersion(667);
schema.setFormat("format");
schema.setDefinition("Test Schema Definition");
schemaRepository.save(schema);
}
}

View File

@@ -0,0 +1,65 @@
/*
* Copyright 2019-2019 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.schema.registry.entityScanning;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.test.autoconfigure.jdbc.AutoConfigureTestDatabase;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.cloud.schema.registry.config.SchemaServerConfiguration;
import org.springframework.http.MediaType;
import org.springframework.test.annotation.DirtiesContext;
import org.springframework.test.context.TestPropertySource;
import org.springframework.test.context.junit4.SpringRunner;
import static org.assertj.core.api.Assertions.assertThat;
import static org.hamcrest.CoreMatchers.containsString;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
/**
* @author Christian Tzolov
*/
@RunWith(SpringRunner.class)
@SpringBootTest(classes = { SchemaServerConfiguration.class })
@DirtiesContext(classMode = DirtiesContext.ClassMode.BEFORE_EACH_TEST_METHOD)
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.ANY)
@EnableAutoConfiguration
@TestPropertySource(properties = {
"spring.cloud.stream.schema.server.path=/testpath2",
"spring.cloud.stream.schema.server.allowSchemaDeletion=true"
})
public class ServerControllerPropertiesBackwardCompatibilityTest extends AbstractServerControllerTest {
@Test
public void propertiesTest() {
assertThat(schemaServerProperties.getPath()).isEqualTo("/testpath2");
assertThat(schemaServerProperties.isAllowSchemaDeletion()).isEqualTo(true);
}
@Test
public void findSchema() throws Exception {
mockMvc.perform(get(schemaServerProperties.getPath() + "/test667/format/v667")
.accept(MediaType.APPLICATION_JSON))
.andExpect(status().isOk())
.andExpect(content().string(containsString("Test Schema Definition")));
}
}

View File

@@ -0,0 +1,65 @@
/*
* Copyright 2019-2019 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.schema.registry.entityScanning;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.test.autoconfigure.jdbc.AutoConfigureTestDatabase;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.cloud.schema.registry.config.SchemaServerConfiguration;
import org.springframework.http.MediaType;
import org.springframework.test.annotation.DirtiesContext;
import org.springframework.test.context.TestPropertySource;
import org.springframework.test.context.junit4.SpringRunner;
import static org.assertj.core.api.Assertions.assertThat;
import static org.hamcrest.CoreMatchers.containsString;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
/**
* @author Christian Tzolov
*/
@RunWith(SpringRunner.class)
@SpringBootTest(classes = { SchemaServerConfiguration.class })
@DirtiesContext(classMode = DirtiesContext.ClassMode.BEFORE_EACH_TEST_METHOD)
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.ANY)
@EnableAutoConfiguration
@TestPropertySource(properties = {
"spring.cloud.schema.server.path=/testpath",
"spring.cloud.schema.server.allowSchemaDeletion=false"
})
public class ServerControllerTest extends AbstractServerControllerTest {
@Test
public void propertiesTest() {
assertThat(schemaServerProperties.getPath()).isEqualTo("/testpath");
assertThat(schemaServerProperties.isAllowSchemaDeletion()).isEqualTo(false);
}
@Test
public void findSchema() throws Exception {
mockMvc.perform(get(schemaServerProperties.getPath() + "/test667/format/v667")
.accept(MediaType.APPLICATION_JSON))
.andExpect(status().isOk())
.andExpect(content().string(containsString("Test Schema Definition")));
}
}