spring-kafka - 0.9 Clients, @KafkaListener

from container factory to container.

Propagate Remaining Container Properties

More @KafkaListener Tests

Use the other options to select topics

KafkaTemplate Javadocs; Polishing

Reference Manual - Initial Commit

@KafkaListener Tests Improvements

Polishing for the top-level repo
This commit is contained in:
Gary Russell
2016-02-25 16:20:52 -05:00
committed by Artem Bilan
commit c0fe2c2fb8
81 changed files with 8113 additions and 0 deletions

12
.gitignore vendored Normal file
View File

@@ -0,0 +1,12 @@
*.iml
*.ipr
*.iws
.classpath
.gradle
.idea
.project
.settings
bin
build
out
target

44
CODE_OF_CONDUCT.adoc Normal file
View File

@@ -0,0 +1,44 @@
= Contributor Code of Conduct
As contributors and maintainers of this project, and in the interest of fostering an open
and welcoming community, we pledge to respect all people who contribute through reporting
issues, posting feature requests, updating documentation, submitting pull requests or
patches, and other activities.
We are committed to making participation in this project a harassment-free experience for
everyone, regardless of level of experience, gender, gender identity and expression,
sexual orientation, disability, personal appearance, body size, race, ethnicity, age,
religion, or nationality.
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery
* Personal attacks
* Trolling or insulting/derogatory comments
* Public or private harassment
* Publishing other's private information, such as physical or electronic addresses,
without explicit permission
* Other unethical or unprofessional conduct
Project maintainers have the right and responsibility to remove, edit, or reject comments,
commits, code, wiki edits, issues, and other contributions that are not aligned to this
Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors
that they deem inappropriate, threatening, offensive, or harmful.
By adopting this Code of Conduct, project maintainers commit themselves to fairly and
consistently applying these principles to every aspect of managing this project. Project
maintainers who do not follow or enforce the Code of Conduct may be permanently removed
from the project team.
This Code of Conduct applies both within project spaces and in public spaces when an
individual is representing the project or its community.
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by
contacting a project maintainer at spring-code-of-conduct@pivotal.io . All complaints will
be reviewed and investigated and will result in a response that is deemed necessary and
appropriate to the circumstances. Maintainers are obligated to maintain confidentiality
with regard to the reporter of an incident.
This Code of Conduct is adapted from the
http://contributor-covenant.org[Contributor Covenant], version 1.3.0, available at
http://contributor-covenant.org/version/1/3/0/[contributor-covenant.org/version/1/3/0/]

85
README.md Normal file
View File

@@ -0,0 +1,85 @@
Spring Kafka <img src="https://build.spring.io/plugins/servlet/buildStatusImage/SK-MAS">
==================
# Checking out and Building
To check out the project and build from source, do the following:
git clone git://github.com/spring-projects/spring-kafka.git
cd spring-kafka
./gradlew build
The Java SE 7 or higher is recommended to build the project.
If you encounter out of memory errors during the build, increase available heap and permgen for Gradle:
GRADLE_OPTS='-XX:MaxPermSize=1024m -Xmx1024m'
To build and install jars into your local Maven cache:
./gradlew install
To build api Javadoc (results will be in `build/api`):
./gradlew api
To build reference documentation (results will be in `build/reference`):
./gradlew reference
To build complete distribution including `-dist`, `-docs`, and `-schema` zip files (results will be in `build/distributions`)
./gradlew dist
# Using Eclipse
To generate Eclipse metadata (.classpath and .project files), do the following:
./gradlew eclipse
Once complete, you may then import the projects into Eclipse as usual:
*File -> Import -> Existing projects into workspace*
Browse to the *'spring-kafka'* root directory. All projects should import
free of errors.
# Using IntelliJ IDEA
To generate IDEA metadata (.iml and .ipr files), do the following:
./gradlew idea
# Resources
For more information, please visit the Spring Kafka website at:
[Reference Manual](http://docs.spring.io/spring-kafka/docs/1.0.0.BUILD-SNAPSHOT/reference/html/)
# Contributing to Spring Kafka
Here are some ways for you to get involved in the community:
* Get involved with the Spring community on the Spring Community Forums. Please help out on the [StackOverflow](http://stackoverflow.com/questions/tagged/spring-kafka) by responding to questions and joining the debate.
* Create [GitHub issues](https://github.com/spring-projects/spring-kafka/issues) for bugs and new features and comment and vote on the ones that you are interested in.
* Github is for social coding: if you want to write code, we encourage contributions through pull requests from [forks of this repository](http://help.github.com/forking/). If you want to contribute code this way, please reference a JIRA ticket as well covering the specific issue you are addressing.
* Watch for upcoming articles on Spring by [subscribing](http://www.springsource.org/node/feed) to springframework.org
Before we accept a non-trivial patch or pull request we will need you to sign the [contributor's agreement](https://support.springsource.com/spring_committer_signup).
Signing the contributor's agreement does not grant anyone commit rights to the main repository, but it does mean that we can accept your contributions, and you will get an author credit if we do.
Active contributors might be asked to join the core team, and given the ability to merge pull requests.
## Code Conventions and Housekeeping
None of these is essential for a pull request, but they will all help.
They can also be added after the original pull request but before a merge.
* Use the Spring Framework code format conventions (import `eclipse-code-formatter.xml` from the root of the project if you are using Eclipse).
* Make sure all new .java files to have a simple Javadoc class comment with at least an @author tag identifying you, and preferably at least a paragraph on what the class is for.
* Add the ASF license header comment to all new .java files (copy from existing files in the project)
* Add yourself as an @author to the .java files that you modify substantially (more than cosmetic changes).
* Add some Javadocs and, if you change the namespace, some XSD doc elements.
* A few unit tests would help a lot as well - someone has to do it.
* If no-one else is using your branch, please rebase it against the current master (or other target branch in the main project).
# License
Spring Kafka is released under the terms of the Apache Software License Version 2.0 (see license.txt).

372
build.gradle Normal file
View File

@@ -0,0 +1,372 @@
description = "Spring Kafka"
apply plugin: 'base'
apply plugin: 'project-report'
apply plugin: 'idea'
buildscript {
repositories {
maven { url 'https://repo.spring.io/plugins-release' }
}
dependencies {
classpath 'io.spring.gradle:spring-io-plugin:0.0.4.RELEASE'
classpath 'io.spring.gradle:docbook-reference-plugin:0.3.1'
classpath 'org.asciidoctor:asciidoctor-gradle-plugin:1.5.0'
}
}
def docsDir = 'src/reference/asciidoc' // Will be default with newer asciidoctor plugin
ext {
linkHomepage = 'https://github.com/spring-projects/spring-kafka'
linkCi = 'https://build.spring.io/browse/SK'
linkIssue = 'https://github.com/spring-projects/spring-kafka/issues'
linkScmUrl = 'https://github.com/spring-projects/spring-kafka'
linkScmConnection = 'https://github.com/spring-projects/spring-kafka.git'
linkScmDevConnection = 'git@github.com:spring-projects/spring-kafka.git'
}
allprojects {
group = 'org.springframework.kafka'
repositories {
maven { url 'https://repo.spring.io/libs-release' }
maven { url 'https://repo.spring.io/libs-milestone' }
if (project.hasProperty('platformVersion')) {
maven { url 'https://repo.spring.io/snapshot' }
}
}
}
subprojects { subproject ->
apply plugin: 'java'
apply from: "${rootProject.projectDir}/publish-maven.gradle"
apply plugin: 'eclipse'
apply plugin: 'idea'
apply plugin: 'jacoco'
if (project.hasProperty('platformVersion')) {
apply plugin: 'spring-io'
dependencyManagement {
springIoTestRuntime {
imports {
mavenBom "io.spring.platform:platform-bom:${platformVersion}"
}
}
}
}
sourceCompatibility = 1.7
targetCompatibility = 1.7
ext {
avroVersion = '1.7.6'
gsCollectionsVersion = '5.0.0'
hamcrestVersion = '1.3'
junitVersion = '4.11'
kafkaVersion = '0.9.0.1'
log4jVersion = '1.2.17'
mockitoVersion = '1.9.5'
// metricsVersion = '2.2.0'
scalaVersion = '2.11'
reactor2Version = '2.0.6.RELEASE'
springRetryVersion = '1.1.2.RELEASE'
springVersion = '4.2.5.RELEASE'
idPrefix = 'kafka'
}
eclipse.project.natures += 'org.springframework.ide.eclipse.core.springnature'
jacoco {
toolVersion = "0.7.2.201409121644"
}
// enable all compiler warnings; individual projects may customize further
[compileJava, compileTestJava]*.options*.compilerArgs = ['-Xlint:all,-options']
test {
// suppress all console output during testing unless running `gradle -i`
logging.captureStandardOutput(LogLevel.INFO)
maxHeapSize = "1024m"
jacoco {
append = false
destinationFile = file("$buildDir/jacoco.exec")
}
}
jacocoTestReport {
reports {
xml.enabled false
csv.enabled false
html.destination "${buildDir}/reports/jacoco/html"
}
}
build.dependsOn jacocoTestReport
task sourcesJar(type: Jar) {
classifier = 'sources'
from sourceSets.main.allJava
}
task javadocJar(type: Jar) {
classifier = 'javadoc'
from javadoc
}
artifacts {
archives sourcesJar
archives javadocJar
}
}
project ('spring-kafka') {
description = 'Spring Kafka Support'
dependencies {
compile "org.springframework:spring-messaging:$springVersion"
compile ("org.apache.avro:avro:$avroVersion", optional)
compile ("org.apache.avro:avro-compiler:$avroVersion", optional)
compile "com.goldmansachs:gs-collections:$gsCollectionsVersion"
compile "io.projectreactor:reactor-core:$reactor2Version"
compile "org.apache.kafka:kafka-clients:$kafkaVersion"
testCompile project (":spring-kafka-test")
}
}
project ('spring-kafka-test') {
description = 'Spring Kafka Test Support'
dependencies {
// compile "org.springframework:spring-context:$springVersion"
// compile ("org.apache.avro:avro:$avroVersion", optional)
// compile ("org.apache.avro:avro-compiler:$avroVersion", optional)
compile "com.goldmansachs:gs-collections:$gsCollectionsVersion"
// compile "io.projectreactor:reactor-core:$reactor2Version"
//
// compile "org.apache.kafka:kafka-clients:$kafkaVersion"
// runtime "com.yammer.metrics:metrics-core:$metricsVersion"
// runtime "com.yammer.metrics:metrics-annotation:$metricsVersion"
compile "org.springframework:spring-test:$springVersion"
compile "org.springframework.retry:spring-retry:$springRetryVersion"
compile "org.apache.kafka:kafka_$scalaVersion:$kafkaVersion"
compile "org.apache.kafka:kafka_$scalaVersion:$kafkaVersion:test"
compile ("junit:junit:$junitVersion") {
exclude group: 'org.hamcrest'
}
compile "log4j:log4j:$log4jVersion"
compile ("org.mockito:mockito-core:$mockitoVersion") {
exclude group: 'org.hamcrest'
}
compile "org.hamcrest:hamcrest-all:$hamcrestVersion"
}
}
apply plugin: org.asciidoctor.gradle.AsciidoctorPlugin
asciidoctor {
sourceDir file("$docsDir")
sourceDocumentNames = files("$docsDir/index.adoc") // Change in >= 1.5.1
outputDir file("$buildDir/html")
backends = ['html5', 'docbook']
logDocuments = true
options = [
doctype: 'book',
attributes: [
docinfo: '',
toc2: '',
'compat-mode': '',
imagesdir: '',
stylesdir: "stylesheets/",
stylesheet: 'golo.css',
'spring-kafka-version': "$version",
'source-highlighter': 'highlightjs'
]
]
}
apply plugin: DocbookReferencePlugin
reference {
sourceFileName = 'index.xml'
sourceDir = file("$buildDir/html")
pdfFilename = 'spring-kafka-reference.pdf'
expandPlaceholders = ''
}
reference.dependsOn asciidoctor
[asciidoctor, reference, referenceEpub, referenceHtmlMulti, referenceHtmlSingle, referencePdf].each {
it.onlyIf { "$System.env.NO_REFERENCE_TASK" != 'true' || project.hasProperty('ignoreEnvToStopReference') }
}
apply plugin: 'sonar-runner'
sonarRunner {
sonarProperties {
property "sonar.jacoco.reportPath", "${buildDir.name}/jacoco.exec"
property "sonar.links.homepage", linkHomepage
property "sonar.links.ci", linkCi
property "sonar.links.issue", linkIssue
property "sonar.links.scm", linkScmUrl
property "sonar.links.scm_dev", linkScmDevConnection
property "sonar.java.coveragePlugin", "jacoco"
}
}
task api(type: Javadoc) {
group = 'Documentation'
description = 'Generates aggregated Javadoc API documentation.'
title = "${rootProject.description} ${version} API"
options.memberLevel = org.gradle.external.javadoc.JavadocMemberLevel.PROTECTED
options.author = true
options.header = rootProject.description
options.overview = 'src/api/overview.html'
source subprojects.collect { project ->
project.sourceSets.main.allJava
}
classpath = files(subprojects.collect { project ->
project.sourceSets.main.compileClasspath
})
destinationDir = new File(buildDir, "api")
}
/*
task schemaZip(type: Zip) {
group = 'Distribution'
classifier = 'schema'
description = "Builds -${classifier} archive containing all " +
"XSDs for deployment at static.springframework.org/schema."
def Properties schemas = new Properties();
def shortName = idPrefix.replaceFirst("${idPrefix}-", '')
project.sourceSets.main.resources.find {
it.path.endsWith("META-INF${File.separator}spring.schemas")
}?.withInputStream { schemas.load(it) }
// for (def key : schemas.keySet()) {
// File xsdFile = project.sourceSets.main.resources.find {
// it.path.replaceAll('\\\\', '/').endsWith(schemas.get(key))
// }
// assert xsdFile != null
// into ("kafka/${shortName}") {
// from xsdFile.path
// }
// }
}
*/
task docsZip(type: Zip) {
group = 'Distribution'
classifier = 'docs'
description = "Builds -${classifier} archive containing api " +
"for deployment at static.spring.io/spring-kafka/docs."
from('src/dist') {
include 'changelog.txt'
}
from(api) {
into 'api'
}
}
task distZip(type: Zip, dependsOn: [docsZip]) { //, schemaZip]) {
group = 'Distribution'
classifier = 'dist'
description = "Builds -${classifier} archive, containing all jars and docs, " +
"suitable for community download page."
ext.baseDir = "${project.name}-${project.version}";
from('src/dist') {
include 'readme.txt'
include 'license.txt'
include 'notice.txt'
into "${baseDir}"
}
from(zipTree(docsZip.archivePath)) {
into "${baseDir}/docs"
}
/*
from(zipTree(schemaZip.archivePath)) {
into "${baseDir}/schema"
}
*/
subprojects.each { subproject ->
into ("${baseDir}/libs") {
from subproject.jar
from subproject.sourcesJar
from subproject.javadocJar
}
}
}
/*
// Create an optional "with dependencies" distribution.
// Not published by default; only for use when building from source.
task depsZip(type: Zip, dependsOn: distZip) { zipTask ->
group = 'Distribution'
classifier = 'dist-with-deps'
description = "Builds -${classifier} archive, containing everything " +
"in the -${distZip.classifier} archive plus all dependencies."
from zipTree(distZip.archivePath)
gradle.taskGraph.whenReady { taskGraph ->
if (taskGraph.hasTask(":${zipTask.name}")) {
def projectName = rootProject.name
def artifacts = new HashSet()
rootProject.configurations.runtime.resolvedConfiguration.resolvedArtifacts.each { artifact ->
def dependency = artifact.moduleVersion.id
if (!projectName.equals(dependency.name)) {
artifacts << artifact.file
}
}
zipTask.from(artifacts) {
into "${distZip.baseDir}/deps"
}
}
}
}
*/
artifacts {
archives distZip
archives docsZip
// archives schemaZip
}
task dist(dependsOn: assemble) {
group = 'Distribution'
description = 'Builds -dist, -docs distribution archives.' // and -schema
}
task wrapper(type: Wrapper) {
description = 'Generates gradlew[.bat] scripts'
gradleVersion = '2.5'
distributionUrl = "http://services.gradle.org/distributions/gradle-${gradleVersion}-all.zip"
}

1
gradle.properties Normal file
View File

@@ -0,0 +1 @@
version=1.0.0.BUILD-SNAPSHOT

BIN
gradle/wrapper/gradle-wrapper.jar vendored Normal file

Binary file not shown.

View File

@@ -0,0 +1,6 @@
#Wed Sep 02 11:48:49 EDT 2015
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
distributionUrl=http\://services.gradle.org/distributions/gradle-2.5-all.zip

164
gradlew vendored Executable file
View File

@@ -0,0 +1,164 @@
#!/usr/bin/env bash
##############################################################################
##
## Gradle start up script for UN*X
##
##############################################################################
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
DEFAULT_JVM_OPTS=""
APP_NAME="Gradle"
APP_BASE_NAME=`basename "$0"`
# Use the maximum available, or set MAX_FD != -1 to use that value.
MAX_FD="maximum"
warn ( ) {
echo "$*"
}
die ( ) {
echo
echo "$*"
echo
exit 1
}
# OS specific support (must be 'true' or 'false').
cygwin=false
msys=false
darwin=false
case "`uname`" in
CYGWIN* )
cygwin=true
;;
Darwin* )
darwin=true
;;
MINGW* )
msys=true
;;
esac
# For Cygwin, ensure paths are in UNIX format before anything is touched.
if $cygwin ; then
[ -n "$JAVA_HOME" ] && JAVA_HOME=`cygpath --unix "$JAVA_HOME"`
fi
# Attempt to set APP_HOME
# Resolve links: $0 may be a link
PRG="$0"
# Need this for relative symlinks.
while [ -h "$PRG" ] ; do
ls=`ls -ld "$PRG"`
link=`expr "$ls" : '.*-> \(.*\)$'`
if expr "$link" : '/.*' > /dev/null; then
PRG="$link"
else
PRG=`dirname "$PRG"`"/$link"
fi
done
SAVED="`pwd`"
cd "`dirname \"$PRG\"`/" >&-
APP_HOME="`pwd -P`"
cd "$SAVED" >&-
CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
# Determine the Java command to use to start the JVM.
if [ -n "$JAVA_HOME" ] ; then
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
# IBM's JDK on AIX uses strange locations for the executables
JAVACMD="$JAVA_HOME/jre/sh/java"
else
JAVACMD="$JAVA_HOME/bin/java"
fi
if [ ! -x "$JAVACMD" ] ; then
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
else
JAVACMD="java"
which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
# Increase the maximum file descriptors if we can.
if [ "$cygwin" = "false" -a "$darwin" = "false" ] ; then
MAX_FD_LIMIT=`ulimit -H -n`
if [ $? -eq 0 ] ; then
if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then
MAX_FD="$MAX_FD_LIMIT"
fi
ulimit -n $MAX_FD
if [ $? -ne 0 ] ; then
warn "Could not set maximum file descriptor limit: $MAX_FD"
fi
else
warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT"
fi
fi
# For Darwin, add options to specify how the application appears in the dock
if $darwin; then
GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\""
fi
# For Cygwin, switch paths to Windows format before running java
if $cygwin ; then
APP_HOME=`cygpath --path --mixed "$APP_HOME"`
CLASSPATH=`cygpath --path --mixed "$CLASSPATH"`
# We build the pattern for arguments to be converted via cygpath
ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null`
SEP=""
for dir in $ROOTDIRSRAW ; do
ROOTDIRS="$ROOTDIRS$SEP$dir"
SEP="|"
done
OURCYGPATTERN="(^($ROOTDIRS))"
# Add a user-defined pattern to the cygpath arguments
if [ "$GRADLE_CYGPATTERN" != "" ] ; then
OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)"
fi
# Now convert the arguments - kludge to limit ourselves to /bin/sh
i=0
for arg in "$@" ; do
CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -`
CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option
if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition
eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"`
else
eval `echo args$i`="\"$arg\""
fi
i=$((i+1))
done
case $i in
(0) set -- ;;
(1) set -- "$args0" ;;
(2) set -- "$args0" "$args1" ;;
(3) set -- "$args0" "$args1" "$args2" ;;
(4) set -- "$args0" "$args1" "$args2" "$args3" ;;
(5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;;
(6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;;
(7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;;
(8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;;
(9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;;
esac
fi
# Split up the JVM_OPTS And GRADLE_OPTS values into an array, following the shell quoting and substitution rules
function splitJvmOpts() {
JVM_OPTS=("$@")
}
eval splitJvmOpts $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS
JVM_OPTS[${#JVM_OPTS[*]}]="-Dorg.gradle.appname=$APP_BASE_NAME"
exec "$JAVACMD" "${JVM_OPTS[@]}" -classpath "$CLASSPATH" org.gradle.wrapper.GradleWrapperMain "$@"

90
gradlew.bat vendored Normal file
View File

@@ -0,0 +1,90 @@
@if "%DEBUG%" == "" @echo off
@rem ##########################################################################
@rem
@rem Gradle startup script for Windows
@rem
@rem ##########################################################################
@rem Set local scope for the variables with windows NT shell
if "%OS%"=="Windows_NT" setlocal
@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
set DEFAULT_JVM_OPTS=
set DIRNAME=%~dp0
if "%DIRNAME%" == "" set DIRNAME=.
set APP_BASE_NAME=%~n0
set APP_HOME=%DIRNAME%
@rem Find java.exe
if defined JAVA_HOME goto findJavaFromJavaHome
set JAVA_EXE=java.exe
%JAVA_EXE% -version >NUL 2>&1
if "%ERRORLEVEL%" == "0" goto init
echo.
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
echo.
echo Please set the JAVA_HOME variable in your environment to match the
echo location of your Java installation.
goto fail
:findJavaFromJavaHome
set JAVA_HOME=%JAVA_HOME:"=%
set JAVA_EXE=%JAVA_HOME%/bin/java.exe
if exist "%JAVA_EXE%" goto init
echo.
echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME%
echo.
echo Please set the JAVA_HOME variable in your environment to match the
echo location of your Java installation.
goto fail
:init
@rem Get command-line arguments, handling Windowz variants
if not "%OS%" == "Windows_NT" goto win9xME_args
if "%@eval[2+2]" == "4" goto 4NT_args
:win9xME_args
@rem Slurp the command line arguments.
set CMD_LINE_ARGS=
set _SKIP=2
:win9xME_args_slurp
if "x%~1" == "x" goto execute
set CMD_LINE_ARGS=%*
goto execute
:4NT_args
@rem Get arguments from the 4NT Shell from JP Software
set CMD_LINE_ARGS=%$
:execute
@rem Setup the command line
set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar
@rem Execute Gradle
"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %CMD_LINE_ARGS%
:end
@rem End local scope for the variables with windows NT shell
if "%ERRORLEVEL%"=="0" goto mainEnd
:fail
rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
rem the _cmd.exe /c_ return code!
if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1
exit /b 1
:mainEnd
if "%OS%"=="Windows_NT" endlocal
:omega

62
publish-maven.gradle Normal file
View File

@@ -0,0 +1,62 @@
apply plugin: 'maven'
ext.optionalDeps = []
ext.providedDeps = []
ext.optional = { optionalDeps << it }
ext.provided = { providedDeps << it }
install {
repositories.mavenInstaller {
customizePom(pom, project)
}
}
def customizePom(pom, gradleProject) {
pom.whenConfigured { generatedPom ->
// respect 'optional' and 'provided' dependencies
gradleProject.optionalDeps.each { dep ->
generatedPom.dependencies.find { it.artifactId == dep.name }?.optional = true
}
gradleProject.providedDeps.each { dep ->
generatedPom.dependencies.find { it.artifactId == dep.name }?.scope = 'provided'
}
// eliminate test-scoped dependencies (no need in maven central poms)
generatedPom.dependencies.removeAll { dep ->
dep.scope == 'test'
}
// add all items necessary for maven central publication
generatedPom.project {
name = gradleProject.description
description = gradleProject.description
url = linkHomepage
organization {
name = 'SpringIO'
url = 'http://spring.io'
}
licenses {
license {
name 'The Apache Software License, Version 2.0'
url 'http://www.apache.org/licenses/LICENSE-2.0.txt'
distribution 'repo'
}
}
scm {
url = linkScmUrl
connection = 'scm:git:' + linkScmConnection
developerConnection = 'scm:git:' + linkScmDevConnection
}
developers {
developer {
id = 'grussell'
name = 'Gary Russell'
email = 'grussell@pivotal.io'
}
}
}
}
}

4
settings.gradle Normal file
View File

@@ -0,0 +1,4 @@
rootProject.name = 'spring-kafka-dist'
include 'spring-kafka'
include 'spring-kafka-test'

View File

@@ -0,0 +1,101 @@
/*
* Copyright 2015-2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.core;
import org.springframework.util.Assert;
import kafka.cluster.BrokerEndPoint;
/**
* Encapsulates the address of a Kafka broker.
*
* @author Marius Bogoevici
* @author Gary Russell
*/
public class BrokerAddress {
public static final int DEFAULT_PORT = 9092;
private final String host;
private final int port;
public BrokerAddress(String host, int port) {
Assert.hasText(host, "Host cannot be empty");
this.host = host;
this.port = port;
}
public BrokerAddress(String host) {
this(host, DEFAULT_PORT);
}
public BrokerAddress(BrokerEndPoint broker) {
Assert.notNull(broker, "Broker cannot be null");
this.host = broker.host();
this.port = broker.port();
}
public static BrokerAddress fromAddress(String address) {
String[] split = address.split(":");
if (split.length == 0 || split.length > 2) {
throw new IllegalArgumentException("Expected format <host>[:<port>]");
}
if (split.length == 2) {
return new BrokerAddress(split[0], Integer.parseInt(split[1]));
}
else {
return new BrokerAddress(split[0]);
}
}
public String getHost() {
return this.host;
}
public int getPort() {
return this.port;
}
@Override
public int hashCode() {
return 31 * this.host.hashCode() + this.port;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
BrokerAddress brokerAddress = (BrokerAddress) o;
return this.port == brokerAddress.port && this.host.equals(brokerAddress.host);
}
@Override
public String toString() {
return this.host + ":" + this.port;
}
}

View File

@@ -0,0 +1,355 @@
/*
* Copyright 2015-2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.rule;
import java.io.File;
import java.net.ServerSocket;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
import java.util.Properties;
import javax.net.ServerSocketFactory;
import org.I0Itec.zkclient.ZkClient;
import org.I0Itec.zkclient.exception.ZkInterruptedException;
import org.apache.kafka.common.protocol.Errors;
import org.apache.kafka.common.protocol.SecurityProtocol;
import org.junit.rules.ExternalResource;
import org.springframework.kafka.core.BrokerAddress;
import org.springframework.retry.RetryCallback;
import org.springframework.retry.RetryContext;
import org.springframework.retry.backoff.ExponentialBackOffPolicy;
import org.springframework.retry.policy.SimpleRetryPolicy;
import org.springframework.retry.support.RetryTemplate;
import com.gs.collections.api.block.function.Function;
import com.gs.collections.impl.list.mutable.FastList;
import com.gs.collections.impl.utility.ListIterate;
import kafka.admin.AdminUtils;
import kafka.admin.AdminUtils$;
import kafka.api.PartitionMetadata;
import kafka.api.TopicMetadata;
import kafka.cluster.BrokerEndPoint;
import kafka.server.KafkaConfig;
import kafka.server.KafkaServer;
import kafka.server.NotRunning;
import kafka.utils.CoreUtils;
import kafka.utils.SystemTime$;
import kafka.utils.TestUtils;
import kafka.utils.ZKStringSerializer$;
import kafka.utils.ZkUtils;
import kafka.zk.EmbeddedZookeeper;
import scala.collection.JavaConversions;
import scala.collection.Map;
import scala.collection.Set;
/**
* @author Marius Bogoevici
* @author Artem Bilan
* @author Gary Russell
*/
@SuppressWarnings("serial")
public class KafkaEmbedded extends ExternalResource implements KafkaRule {
public static final long METADATA_PROPAGATION_TIMEOUT = 10000L;
private final int count;
private final boolean controlledShutdown;
private final String[] topics;
private final int partitionsPerTopic;
private List<KafkaServer> kafkaServers;
private EmbeddedZookeeper zookeeper;
private ZkClient zookeeperClient;
private String zkConnect;
public KafkaEmbedded(int count) {
this(count, false);
}
public KafkaEmbedded(int count, boolean controlledShutdown, String... topics) {
this(count, controlledShutdown, 2, topics);
}
public KafkaEmbedded(int count, boolean controlledShutdown, int partitions, String... topics) {
this.count = count;
this.controlledShutdown = controlledShutdown;
if (topics != null) {
this.topics = topics;
}
else {
this.topics = new String[0];
}
this.partitionsPerTopic = partitions;
}
@Override
protected void before() throws Throwable {
startZookeeper();
int zkConnectionTimeout = 6000;
int zkSessionTimeout = 6000;
this.zkConnect = "127.0.0.1:" + this.zookeeper.port();
zookeeperClient = new ZkClient(zkConnect, zkSessionTimeout, zkConnectionTimeout,
ZKStringSerializer$.MODULE$);
kafkaServers = new ArrayList<KafkaServer>();
for (int i = 0; i < count; i++) {
ServerSocket ss = ServerSocketFactory.getDefault().createServerSocket(0);
int randomPort = ss.getLocalPort();
ss.close();
Properties brokerConfigProperties = TestUtils.createBrokerConfig(i, this.zkConnect, this.controlledShutdown,
true, randomPort,
scala.Option.<SecurityProtocol>apply(null),
scala.Option.<File>apply(null),
true, false, 0, false, 0, false, 0);
brokerConfigProperties.setProperty("replica.socket.timeout.ms","1000");
brokerConfigProperties.setProperty("controller.socket.timeout.ms","1000");
brokerConfigProperties.setProperty("offsets.topic.replication.factor","1");
KafkaServer server = TestUtils.createServer(new KafkaConfig(brokerConfigProperties), SystemTime$.MODULE$);
kafkaServers.add(server);
}
ZkUtils zkUtils = new ZkUtils(getZkClient(), null, false);
Properties props = new Properties();
for (String topic : topics) {
AdminUtils.createTopic(zkUtils, topic, this.partitionsPerTopic, this.count, props);
}
}
@Override
protected void after() {
for (KafkaServer kafkaServer : kafkaServers) {
try {
if (kafkaServer.brokerState().currentState() != (NotRunning.state())) {
kafkaServer.shutdown();
kafkaServer.awaitShutdown();
}
}
catch (Exception e) {
// do nothing
}
try {
CoreUtils.rm(kafkaServer.config().logDirs());
}
catch (Exception e) {
// do nothing
}
}
try {
zookeeperClient.close();
}
catch (ZkInterruptedException e) {
// do nothing
}
try {
zookeeper.shutdown();
}
catch (Exception e) {
// do nothing
}
}
@Override
public List<KafkaServer> getKafkaServers() {
return kafkaServers;
}
public KafkaServer getKafkaServer(int id) {
return kafkaServers.get(id);
}
public EmbeddedZookeeper getZookeeper() {
return zookeeper;
}
@Override
public ZkClient getZkClient() {
return zookeeperClient;
}
@Override
public String getZookeeperConnectionString() {
return zkConnect;
}
public BrokerAddress getBrokerAddress(int i) {
KafkaServer kafkaServer = this.kafkaServers.get(i);
return new BrokerAddress(kafkaServer.config().hostName(), kafkaServer.config().port());
}
@Override
public BrokerAddress[] getBrokerAddresses() {
return ListIterate.collect(this.kafkaServers,
new Function<KafkaServer, BrokerAddress>() {
@Override
public BrokerAddress valueOf(KafkaServer kafkaServer) {
return new BrokerAddress("127.0.0.1", kafkaServer.config().port());
}
})
.toArray(new BrokerAddress[this.kafkaServers.size()]);
}
@Override
public int getPartitionsPerTopic() {
return this.partitionsPerTopic;
}
public void bounce(BrokerAddress brokerAddress) {
for (KafkaServer kafkaServer : getKafkaServers()) {
if (brokerAddress.equals(new BrokerAddress(kafkaServer.config().hostName(), kafkaServer.config().port()))) {
kafkaServer.shutdown();
kafkaServer.awaitShutdown();
}
}
}
public void startZookeeper() {
zookeeper = new EmbeddedZookeeper();
}
public void bounce(int index, boolean waitForPropagation) {
kafkaServers.get(index).shutdown();
if (waitForPropagation) {
long initialTime = System.currentTimeMillis();
boolean canExit = false;
do {
try {
Thread.sleep(100);
}
catch (InterruptedException e) {
break;
}
canExit = true;
ZkUtils zkUtils = new ZkUtils(getZkClient(), null, false);
Map<String, Properties> topicProperties = AdminUtils$.MODULE$.fetchAllTopicConfigs(zkUtils);
Set<TopicMetadata> topicMetadatas =
AdminUtils$.MODULE$.fetchTopicMetadataFromZk(topicProperties.keySet(), zkUtils);
for (TopicMetadata topicMetadata : JavaConversions.asJavaCollection(topicMetadatas)) {
if (Errors.forCode(topicMetadata.errorCode()).exception() == null) {
for (PartitionMetadata partitionMetadata :
JavaConversions.asJavaCollection(topicMetadata.partitionsMetadata())) {
Collection<BrokerEndPoint> inSyncReplicas = JavaConversions.asJavaCollection(partitionMetadata.isr());
for (BrokerEndPoint broker : inSyncReplicas) {
if (broker.id() == index) {
canExit = false;
}
}
}
}
}
}
while (!canExit && (System.currentTimeMillis() - initialTime < METADATA_PROPAGATION_TIMEOUT));
}
}
public void bounce(int index) {
bounce(index, true);
}
public void restart(final int index) throws Exception {
// retry restarting repeatedly, first attempts may fail
SimpleRetryPolicy retryPolicy = new SimpleRetryPolicy(10,
Collections.<Class<? extends Throwable>,Boolean>singletonMap(Exception.class, true));
ExponentialBackOffPolicy backOffPolicy = new ExponentialBackOffPolicy();
backOffPolicy.setInitialInterval(100);
backOffPolicy.setMaxInterval(1000);
backOffPolicy.setMultiplier(2);
RetryTemplate retryTemplate = new RetryTemplate();
retryTemplate.setRetryPolicy(retryPolicy);
retryTemplate.setBackOffPolicy(backOffPolicy);
retryTemplate.execute(new RetryCallback<Void, Exception>() {
@Override
public Void doWithRetry(RetryContext context) throws Exception {
kafkaServers.get(index).startup();
return null;
}
});
}
public void waitUntilSynced(String topic, int brokerId) {
long initialTime = System.currentTimeMillis();
boolean canExit = false;
do {
try {
Thread.sleep(100);
}
catch (InterruptedException e) {
break;
}
canExit = true;
ZkUtils zkUtils = new ZkUtils(getZkClient(), null, false);
TopicMetadata topicMetadata = AdminUtils$.MODULE$.fetchTopicMetadataFromZk(topic, zkUtils);
if (Errors.forCode(topicMetadata.errorCode()).exception() == null) {
for (PartitionMetadata partitionMetadata :
JavaConversions.asJavaCollection(topicMetadata.partitionsMetadata())) {
Collection<BrokerEndPoint> isr = JavaConversions.asJavaCollection(partitionMetadata.isr());
boolean containsIndex = false;
for (BrokerEndPoint broker : isr) {
if (broker.id() == brokerId) {
containsIndex = true;
}
}
if (!containsIndex) {
canExit = false;
}
}
}
}
while (!canExit && (System.currentTimeMillis() - initialTime < METADATA_PROPAGATION_TIMEOUT));
}
@Override
public String getBrokersAsString() {
return FastList.newList(Arrays.asList(getBrokerAddresses()))
.collect(new Function<BrokerAddress, String>() {
@Override
public String valueOf(BrokerAddress object) {
return object.getHost() + ":" + object.getPort();
}
})
.makeString(",");
}
@Override
public boolean isEmbedded() {
return true;
}
}

View File

@@ -0,0 +1,50 @@
/*
* Copyright 2015-2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.rule;
import java.util.List;
import org.I0Itec.zkclient.ZkClient;
import org.junit.rules.TestRule;
import org.springframework.kafka.core.BrokerAddress;
import kafka.server.KafkaServer;
/**
* Common functionality for the Kafka JUnit rules
*
* @author Marius Bogoevici
* @author Gary Russell
*/
public interface KafkaRule extends TestRule {
ZkClient getZkClient();
String getZookeeperConnectionString();
BrokerAddress[] getBrokerAddresses();
String getBrokersAsString();
boolean isEmbedded();
List<KafkaServer> getKafkaServers();
int getPartitionsPerTopic();
}

View File

@@ -0,0 +1,10 @@
log4j.rootCategory=WARN, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{HH:mm:ss.SSS} %-5p [%t][%c] %m%n
log4j.category.org.springframework.kafka=TRACE
log4j.category.org.apache.kafka.clients=WARN
log4j.category.org.apache.kafka.common.network.Selector=ERROR
log4j.category.kafka.server.ReplicaFetcherThread=ERROR

View File

@@ -0,0 +1,253 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.annotation;
import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import org.springframework.context.annotation.Import;
/**
* Enable Kafka listener annotated endpoints that are created under the covers by a
* {@link org.springframework.kafka.config.AbstractKafkaListenerContainerFactory
* AbstractListenerContainerFactory}. To be used on
* {@link org.springframework.context.annotation.Configuration Configuration} classes as
* follows:
*
* <pre class="code">
* &#064;Configuration
* &#064;EnableKafka
* public class AppConfig {
* &#064;Bean
* public SimpleKafkaListenerContainerFactory myKafkaListenerContainerFactory() {
* SimpleKafkaListenerContainerFactory factory = new SimpleKafkaListenerContainerFactory();
* factory.setConsumerFactory(consumerFactory());
* factory.setConcurrency(4);
* return factory;
* }
* // other &#064;Bean definitions
* }
* </pre>
*
* The {@code KafkaListenerContainerFactory} is responsible to create the listener
* container for a particular endpoint. Typical implementations, as the
* {@link org.springframework.kafka.config.SimpleKafkaListenerContainerFactory
* SimpleKafkaListenerContainerFactory} used in the sample above, provides the necessary
* configuration options that are supported by the underlying
* {@link org.springframework.kafka.listener.MessageListenerContainer
* MessageListenerContainer}.
*
* <p>
* {@code @EnableKafka} enables detection of {@link KafkaListener} annotations on any
* Spring-managed bean in the container. For example, given a class {@code MyService}:
*
* <pre class="code">
* package com.acme.foo;
*
* public class MyService {
* &#064;KafkaListener(containerFactory = "myKafkaListenerContainerFactory", topics = "myTopic")
* public void process(String msg) {
* // process incoming message
* }
* }
* </pre>
*
* The container factory to use is identified by the
* {@link KafkaListener#containerFactory() containerFactory} attribute defining the name
* of the {@code KafkaListenerContainerFactory} bean to use. When none is set a
* {@code KafkaListenerContainerFactory} bean with name
* {@code kafkaListenerContainerFactory} is assumed to be present.
*
* <p>
* the following configuration would ensure that every time a message is receied from
* topic "myQueue", {@code MyService.process()} is called with the content of the message:
*
* <pre class="code">
* &#064;Configuration
* &#064;EnableKafka
* public class AppConfig {
* &#064;Bean
* public MyService myService() {
* return new MyService();
* }
*
* // Kafka infrastructure setup
* }
* </pre>
*
* Alternatively, if {@code MyService} were annotated with {@code @Component}, the
* following configuration would ensure that its {@code @KafkaListener} annotated method
* is invoked with a matching incoming message:
*
* <pre class="code">
* &#064;Configuration
* &#064;EnableKafka
* &#064;ComponentScan(basePackages = "com.acme.foo")
* public class AppConfig {
* }
* </pre>
*
* Note that the created containers are not registered with the application context but
* can be easily located for management purposes using the
* {@link org.springframework.kafka.listener.KafkaListenerEndpointRegistry
* KafkaListenerEndpointRegistry}.
*
* <p>
* Annotated methods can use a flexible signature; in particular, it is possible to use
* the {@link org.springframework.messaging.Message Message} abstraction and related
* annotations, see {@link KafkaListener} Javadoc for more details. For instance, the
* following would inject the content of the message and a the kafka partition
* header:
*
* <pre class="code">
* &#064;KafkaListener(containerFactory = "myKafkaListenerContainerFactory", topics = "myTopic")
* public void process(String msg, @Header("kafka_partition") int partition) {
* // process incoming message
* }
* </pre>
*
* These features are abstracted by the
* {@link org.springframework.messaging.handler.annotation.support.MessageHandlerMethodFactory
* MessageHandlerMethodFactory} that is responsible to build the necessary invoker to
* process the annotated method. By default,
* {@link org.springframework.messaging.handler.annotation.support.DefaultMessageHandlerMethodFactory
* DefaultMessageHandlerMethodFactory} is used.
*
* <p>
* When more control is desired, a {@code @Configuration} class may implement
* {@link KafkaListenerConfigurer}. This allows access to the underlying
* {@link org.springframework.kafka.listener.KafkaListenerEndpointRegistrar
* KafkaListenerEndpointRegistrar} instance. The following example demonstrates how to
* specify an explicit default {@code KafkaListenerContainerFactory}
*
* <pre class="code">
* {
* &#64;code
* &#064;Configuration
* &#064;EnableKafka
* public class AppConfig implements KafkaListenerConfigurer {
* &#064;Override
* public void configureKafkaListeners(KafkaListenerEndpointRegistrar registrar) {
* registrar.setContainerFactory(myKafkaListenerContainerFactory());
* }
*
* &#064;Bean
* public KafkaListenerContainerFactory&lt;?, ?&gt; myKafkaListenerContainerFactory() {
* // factory settings
* }
*
* &#064;Bean
* public MyService myService() {
* return new MyService();
* }
* }
* }
* </pre>
*
* It is also possible to specify a custom
* {@link org.springframework.kafka.listener.KafkaListenerEndpointRegistry
* KafkaListenerEndpointRegistry} in case you need more control on the way the containers
* are created and managed. The example below also demonstrates how to customize the
* {@code KafkaHandlerMethodFactory} to use with a custom
* {@link org.springframework.validation.Validator Validator} so that payloads annotated
* with {@link org.springframework.validation.annotation.Validated Validated} are first
* validated against a custom {@code Validator}.
*
* <pre class="code">
* {
* &#64;code
* &#064;Configuration
* &#064;EnableKafka
* public class AppConfig implements KafkaListenerConfigurer {
* &#064;Override
* public void configureKafkaListeners(KafkaListenerEndpointRegistrar registrar) {
* registrar.setEndpointRegistry(myKafkaListenerEndpointRegistry());
* registrar.setMessageHandlerMethodFactory(myMessageHandlerMethodFactory);
* }
*
* &#064;Bean
* public KafkaListenerEndpointRegistry myKafkaListenerEndpointRegistry() {
* // registry configuration
* }
*
* &#064;Bean
* public KafkaHandlerMethodFactory myMessageHandlerMethodFactory() {
* DefaultKafkaHandlerMethodFactory factory = new DefaultKafkaHandlerMethodFactory();
* factory.setValidator(new MyValidator());
* return factory;
* }
*
* &#064;Bean
* public MyService myService() {
* return new MyService();
* }
* }
* }
* </pre>
*
* Implementing {@code KafkaListenerConfigurer} also allows for fine-grained control over
* endpoints registration via the {@code KafkaListenerEndpointRegistrar}. For example, the
* following configures an extra endpoint:
*
* <pre class="code">
* {
* &#64;code
* &#064;Configuration
* &#064;EnableKafka
* public class AppConfig implements KafkaListenerConfigurer {
* &#064;Override
* public void configureKafkaListeners(KafkaListenerEndpointRegistrar registrar) {
* SimpleKafkaListenerEndpoint myEndpoint = new SimpleKafkaListenerEndpoint();
* // ... configure the endpoint
* registrar.registerEndpoint(endpoint, anotherKafkaListenerContainerFactory());
* }
*
* &#064;Bean
* public MyService myService() {
* return new MyService();
* }
*
* &#064;Bean
* public KafkaListenerContainerFactory&lt;?, ?&gt; anotherKafkaListenerContainerFactory() {
* // ...
* }
*
* // Kafka infrastructure setup
* }
* }
* </pre>
*
* Note that all beans implementing {@code KafkaListenerConfigurer} will be detected and
* invoked in a similar fashion. The example above can be translated in a regular bean
* definition registered in the context in case you use the XML configuration.
*
* @author Stephane Nicoll
* @author Gary Russell
* @see KafkaListener
* @see KafkaListenerAnnotationBeanPostProcessor
* @see org.springframework.kafka.listener.KafkaListenerEndpointRegistrar
* @see org.springframework.kafka.listener.KafkaListenerEndpointRegistry
*/
@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Import(KafkaBootstrapConfiguration.class)
public @interface EnableKafka {
}

View File

@@ -0,0 +1,55 @@
/*
* Copyright 2002-2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.annotation;
import org.springframework.beans.factory.config.BeanDefinition;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Role;
import org.springframework.kafka.config.KafkaListenerConfigUtils;
import org.springframework.kafka.listener.KafkaListenerEndpointRegistry;
/**
* {@code @Configuration} class that registers a {@link KafkaListenerAnnotationBeanPostProcessor}
* bean capable of processing Spring's @{@link KafkaListener} annotation. Also register
* a default {@link KafkaListenerEndpointRegistry}.
*
* <p>This configuration class is automatically imported when using the @{@link EnableKafka}
* annotation. See {@link EnableKafka} Javadoc for complete usage.
*
* @author Stephane Nicoll
* @author Gary Russell
* @see KafkaListenerAnnotationBeanPostProcessor
* @see KafkaListenerEndpointRegistry
* @see EnableKafka
*/
@Configuration
public class KafkaBootstrapConfiguration {
@SuppressWarnings("rawtypes")
@Bean(name = KafkaListenerConfigUtils.KAFKA_LISTENER_ANNOTATION_PROCESSOR_BEAN_NAME)
@Role(BeanDefinition.ROLE_INFRASTRUCTURE)
public KafkaListenerAnnotationBeanPostProcessor kafkaListenerAnnotationProcessor() {
return new KafkaListenerAnnotationBeanPostProcessor();
}
@Bean(name = KafkaListenerConfigUtils.KAFKA_LISTENER_ENDPOINT_REGISTRY_BEAN_NAME)
public KafkaListenerEndpointRegistry defaultKafkaListenerEndpointRegistry() {
return new KafkaListenerEndpointRegistry();
}
}

View File

@@ -0,0 +1,50 @@
/*
* Copyright 2015 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.annotation;
import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import org.springframework.messaging.handler.annotation.MessageMapping;
/**
* Annotation that marks a method to be the target of a Kafka message
* listener within a class that is annotated with {@link KafkaListener}.
*
* <p>See the {@link KafkaListener} for information about permitted method signatures
* and available parameters.
* <p><b>It is important to understand that when a message arrives, the method selection
* depends on the payload type. The type is matched with a single non-annotated parameter,
* or one that is annotated with {@code @Payload}.
* There must be no ambiguity - the system
* must be able to select exactly one method based on the payload type.</b>
*
* @author Gary Russell
* @see EnableKafka
* @see KafkaListener
* @see KafkaListenerAnnotationBeanPostProcessor
*/
@Target({ ElementType.METHOD, ElementType.ANNOTATION_TYPE })
@Retention(RetentionPolicy.RUNTIME)
@MessageMapping
@Documented
public @interface KafkaHandler {
}

View File

@@ -0,0 +1,137 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.annotation;
import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Repeatable;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import org.springframework.kafka.listener.MessageListener;
import org.springframework.kafka.listener.adapter.MessagingMessageListenerAdapter;
import org.springframework.messaging.handler.annotation.MessageMapping;
/**
* Annotation that marks a method to be the target of a Kafka message
* listener on the specified topics.
*
* The {@link #containerFactory()}
* identifies the {@link org.springframework.kafka.listener.KafkaListenerContainerFactory
* KafkaListenerContainerFactory} to use to build the Kafka listener container. If not
* set, a <em>default</em> container factory is assumed to be available with a bean
* name of {@code kafkaListenerContainerFactory} unless an explicit default has been
* provided through configuration.
*
* <p>Processing of {@code @KafkaListener} annotations is performed by
* registering a {@link KafkaListenerAnnotationBeanPostProcessor}. This can be
* done manually or, more conveniently, through {@link EnableKafka} annotation.
*
* <p>Annotated methods are allowed to have flexible signatures similar to what
* {@link MessageMapping} provides, that is
* <ul>
* <li>{@link org.apache.kafka.clients.consumer.ConsumerRecord} to
* access to the raw Kafka message</li>
* <li>{@link org.springframework.kafka.listener.Acknowledgment} to manually ack</li>
* <li>{@link org.springframework.messaging.handler.annotation.Payload @Payload}-annotated method
* arguments including the support of validation</li>
* <li>{@link org.springframework.messaging.handler.annotation.Header @Header}-annotated method
* arguments to extract a specific header value, defined by
* {@link org.springframework.kafka.support.KafkaHeaders KafkaHeaders}</li>
* <li>{@link org.springframework.messaging.handler.annotation.Headers @Headers}-annotated
* argument that must also be assignable to {@link java.util.Map} for getting access to all
* headers.</li>
* <li>{@link org.springframework.messaging.MessageHeaders MessageHeaders} arguments for
* getting access to all headers.</li>
* <li>{@link org.springframework.messaging.support.MessageHeaderAccessor MessageHeaderAccessor}
* for convenient access to all method arguments.</li>
* </ul>
*
* <p>When defined at the method level, a listener container is created for each method. The
* {@link MessageListener} is a {@link MessagingMessageListenerAdapter}, configured with a
* {@link org.springframework.kafka.listener.MethodKafkaListenerEndpoint}.
*
* <p>When defined at the class level, a single message listener container is used to service
* all methods annotated with {@code @KafkaHandler}. Method signatures of such annotated
* methods must not cause any ambiguity such that a single method can be resolved for a
* particular inbound message. The {@link MessagingMessageListenerAdapter} is configured with
* a {@link org.springframework.kafka.listener.MultiMethodKafkaListenerEndpoint}.
* @author Gary Russell
* @see EnableKafka
* @see KafkaListenerAnnotationBeanPostProcessor
* @see KafkaListeners
*/
@Target({ ElementType.TYPE, ElementType.METHOD, ElementType.ANNOTATION_TYPE })
@Retention(RetentionPolicy.RUNTIME)
@MessageMapping
@Documented
@Repeatable(KafkaListeners.class)
public @interface KafkaListener {
/**
* The unique identifier of the container managing for this endpoint.
* <p>If none is specified an auto-generated one is provided.
* @return the {@code id} for the container managing for this endpoint.
* @see org.springframework.kafka.listener.KafkaListenerEndpointRegistry#getListenerContainer(String)
*/
String id() default "";
/**
* The bean name of the {@link org.springframework.kafka.listener.KafkaListenerContainerFactory}
* to use to create the message listener container responsible to serve this endpoint.
* <p>If not specified, the default container factory is used, if any.
* @return the container factory bean name.
*/
String containerFactory() default "";
/**
* The topics for this listener.
* The entries can be 'topic name', 'property-placeholder keys' or 'expressions'.
* Expression must be resolved to the topic name.
* Mutually exclusive with {@link #topicPattern()} and {@link #topicPartitions()}.
* @return the topic names or expressions (SpEL) to listen to.
*/
String[] topics() default {};
/**
* The topic pattern for this listener.
* The entries can be 'topic name', 'property-placeholder keys' or 'expressions'.
* Expression must be resolved to the topic pattern.
* Mutually exclusive with {@link #topics()} and {@link #topicPartitions()}.
* @return the topic pattern or expression (SpEL).
*/
String topicPattern() default "";
/**
* The topicPartitions for this listener.
* Mutually exclusive with {@link #topicPattern()} and {@link #topics()}.
* @return the topic names or expressions (SpEL) to listen to.
*/
TopicPartition[] topicPartitions() default {};
/**
* If provided, the listener container for this listener will be added to a bean
* with this value as its name, of type {@code Collection<MessageListenerContainer>}.
* This allows, for example, iteration over the collection to start/stop a subset
* of containers.
* @return the bean name for the group.
* @since 1.5
*/
String group() default "";
}

View File

@@ -0,0 +1,494 @@
/*
* Copyright 2014-2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.annotation;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.regex.Pattern;
import org.springframework.aop.framework.Advised;
import org.springframework.aop.support.AopUtils;
import org.springframework.beans.BeansException;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.beans.factory.BeanFactoryAware;
import org.springframework.beans.factory.BeanInitializationException;
import org.springframework.beans.factory.ListableBeanFactory;
import org.springframework.beans.factory.NoSuchBeanDefinitionException;
import org.springframework.beans.factory.SmartInitializingSingleton;
import org.springframework.beans.factory.config.BeanExpressionContext;
import org.springframework.beans.factory.config.BeanExpressionResolver;
import org.springframework.beans.factory.config.BeanPostProcessor;
import org.springframework.beans.factory.config.ConfigurableBeanFactory;
import org.springframework.beans.factory.config.ConfigurableListableBeanFactory;
import org.springframework.context.expression.StandardBeanExpressionResolver;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.AnnotationUtils;
import org.springframework.kafka.config.KafkaListenerConfigUtils;
import org.springframework.kafka.listener.KafkaListenerContainerFactory;
import org.springframework.kafka.listener.KafkaListenerEndpointRegistrar;
import org.springframework.kafka.listener.KafkaListenerEndpointRegistry;
import org.springframework.kafka.listener.MethodKafkaListenerEndpoint;
import org.springframework.kafka.listener.MultiMethodKafkaListenerEndpoint;
import org.springframework.messaging.handler.annotation.support.DefaultMessageHandlerMethodFactory;
import org.springframework.messaging.handler.annotation.support.MessageHandlerMethodFactory;
import org.springframework.messaging.handler.invocation.InvocableHandlerMethod;
import org.springframework.util.Assert;
import org.springframework.util.ReflectionUtils;
import org.springframework.util.StringUtils;
/**
* Bean post-processor that registers methods annotated with {@link KafkaListener}
* to be invoked by a Kafka message listener container created under the covers
* by a {@link org.springframework.kafka.listener.KafkaListenerContainerFactory}
* according to the parameters of the annotation.
*
* <p>Annotated methods can use flexible arguments as defined by {@link KafkaListener}.
*
* <p>This post-processor is automatically registered by Spring's {@link EnableKafka}
* annotation.
*
* <p>Auto-detect any {@link KafkaListenerConfigurer} instances in the container,
* allowing for customization of the registry to be used, the default container
* factory or for fine-grained control over endpoints registration. See
* {@link EnableKafka} Javadoc for complete usage details.
*
* @author Stephane Nicoll
* @author Juergen Hoeller
* @author Gary Russell
* @since 1.4
* @see KafkaListener
* @see EnableKafka
* @see KafkaListenerConfigurer
* @see KafkaListenerEndpointRegistrar
* @see KafkaListenerEndpointRegistry
* @see org.springframework.kafka.listener.KafkaListenerEndpoint
* @see MethodKafkaListenerEndpoint
*/
public class KafkaListenerAnnotationBeanPostProcessor<K, V>
implements BeanPostProcessor, Ordered, BeanFactoryAware, SmartInitializingSingleton {
/**
* The bean name of the default {@link org.springframework.kafka.listener.KafkaListenerContainerFactory}.
*/
static final String DEFAULT_KAFKA_LISTENER_CONTAINER_FACTORY_BEAN_NAME = "kafkaListenerContainerFactory";
private KafkaListenerEndpointRegistry endpointRegistry;
private String containerFactoryBeanName = DEFAULT_KAFKA_LISTENER_CONTAINER_FACTORY_BEAN_NAME;
private BeanFactory beanFactory;
private final KafkaHandlerMethodFactoryAdapter messageHandlerMethodFactory =
new KafkaHandlerMethodFactoryAdapter();
private final KafkaListenerEndpointRegistrar registrar = new KafkaListenerEndpointRegistrar();
private final AtomicInteger counter = new AtomicInteger();
private BeanExpressionResolver resolver = new StandardBeanExpressionResolver();
private BeanExpressionContext expressionContext;
@Override
public int getOrder() {
return LOWEST_PRECEDENCE;
}
/**
* Set the {@link KafkaListenerEndpointRegistry} that will hold the created
* endpoint and manage the lifecycle of the related listener container.
* @param endpointRegistry the {@link KafkaListenerEndpointRegistry} to set.
*/
public void setEndpointRegistry(KafkaListenerEndpointRegistry endpointRegistry) {
this.endpointRegistry = endpointRegistry;
}
/**
* Set the name of the {@link KafkaListenerContainerFactory} to use by default.
* <p>If none is specified, "KafkaListenerContainerFactory" is assumed to be defined.
* @param containerFactoryBeanName the {@link KafkaListenerContainerFactory} bean name.
*/
public void setContainerFactoryBeanName(String containerFactoryBeanName) {
this.containerFactoryBeanName = containerFactoryBeanName;
}
/**
* Set the {@link MessageHandlerMethodFactory} to use to configure the message
* listener responsible to serve an endpoint detected by this processor.
* <p>By default, {@link DefaultMessageHandlerMethodFactory} is used and it
* can be configured further to support additional method arguments
* or to customize conversion and validation support. See
* {@link DefaultMessageHandlerMethodFactory} Javadoc for more details.
* @param messageHandlerMethodFactory the {@link MessageHandlerMethodFactory} instance.
*/
public void setMessageHandlerMethodFactory(MessageHandlerMethodFactory messageHandlerMethodFactory) {
this.messageHandlerMethodFactory.setMessageHandlerMethodFactory(messageHandlerMethodFactory);
}
/**
* Making a {@link BeanFactory} available is optional; if not set,
* {@link KafkaListenerConfigurer} beans won't get autodetected and an
* {@link #setEndpointRegistry endpoint registry} has to be explicitly configured.
* @param beanFactory the {@link BeanFactory} to be used.
*/
@Override
public void setBeanFactory(BeanFactory beanFactory) {
this.beanFactory = beanFactory;
if (beanFactory instanceof ConfigurableListableBeanFactory) {
this.resolver = ((ConfigurableListableBeanFactory) beanFactory).getBeanExpressionResolver();
this.expressionContext = new BeanExpressionContext((ConfigurableListableBeanFactory) beanFactory, null);
}
}
@Override
public void afterSingletonsInstantiated() {
this.registrar.setBeanFactory(this.beanFactory);
if (this.beanFactory instanceof ListableBeanFactory) {
Map<String, KafkaListenerConfigurer> instances =
((ListableBeanFactory) this.beanFactory).getBeansOfType(KafkaListenerConfigurer.class);
for (KafkaListenerConfigurer configurer : instances.values()) {
configurer.configureKafkaListeners(this.registrar);
}
}
if (this.registrar.getEndpointRegistry() == null) {
if (this.endpointRegistry == null) {
Assert.state(this.beanFactory != null,
"BeanFactory must be set to find endpoint registry by bean name");
this.endpointRegistry = this.beanFactory.getBean(
KafkaListenerConfigUtils.KAFKA_LISTENER_ENDPOINT_REGISTRY_BEAN_NAME,
KafkaListenerEndpointRegistry.class);
}
this.registrar.setEndpointRegistry(this.endpointRegistry);
}
if (this.containerFactoryBeanName != null) {
this.registrar.setContainerFactoryBeanName(this.containerFactoryBeanName);
}
// Set the custom handler method factory once resolved by the configurer
MessageHandlerMethodFactory handlerMethodFactory = this.registrar.getMessageHandlerMethodFactory();
if (handlerMethodFactory != null) {
this.messageHandlerMethodFactory.setMessageHandlerMethodFactory(handlerMethodFactory);
}
// Actually register all listeners
this.registrar.afterPropertiesSet();
}
@Override
public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
return bean;
}
@Override
public Object postProcessAfterInitialization(final Object bean, final String beanName) throws BeansException {
Class<?> targetClass = AopUtils.getTargetClass(bean);
Collection<KafkaListener> classLevelListeners = findListenerAnnotations(targetClass);
final boolean hasClassLevelListeners = classLevelListeners.size() > 0;
final List<Method> multiMethods = new ArrayList<Method>();
ReflectionUtils.doWithMethods(targetClass, new ReflectionUtils.MethodCallback() {
@Override
public void doWith(Method method) throws IllegalArgumentException, IllegalAccessException {
for (KafkaListener KafkaListener : findListenerAnnotations(method)) {
processKafkaListener(KafkaListener, method, bean, beanName);
}
if (hasClassLevelListeners) {
KafkaHandler KafkaHandler = AnnotationUtils.findAnnotation(method, KafkaHandler.class);
if (KafkaHandler != null) {
multiMethods.add(method);
}
}
}
});
if (hasClassLevelListeners) {
processMultiMethodListeners(classLevelListeners, multiMethods, bean, beanName);
}
return bean;
}
/*
* AnnotationUtils.getRepeatableAnnotations does not look at interfaces
*/
private Collection<KafkaListener> findListenerAnnotations(Class<?> clazz) {
Set<KafkaListener> listeners = new HashSet<KafkaListener>();
KafkaListener ann = AnnotationUtils.findAnnotation(clazz, KafkaListener.class);
if (ann != null) {
listeners.add(ann);
}
KafkaListeners anns = AnnotationUtils.findAnnotation(clazz, KafkaListeners.class);
if (anns != null) {
listeners.addAll(Arrays.asList(anns.value()));
}
return listeners;
}
/*
* AnnotationUtils.getRepeatableAnnotations does not look at interfaces
*/
private Collection<KafkaListener> findListenerAnnotations(Method method) {
Set<KafkaListener> listeners = new HashSet<KafkaListener>();
KafkaListener ann = AnnotationUtils.findAnnotation(method, KafkaListener.class);
if (ann != null) {
listeners.add(ann);
}
KafkaListeners anns = AnnotationUtils.findAnnotation(method, KafkaListeners.class);
if (anns != null) {
listeners.addAll(Arrays.asList(anns.value()));
}
return listeners;
}
private void processMultiMethodListeners(Collection<KafkaListener> classLevelListeners, List<Method> multiMethods,
Object bean, String beanName) {
List<Method> checkedMethods = new ArrayList<Method>();
for (Method method : multiMethods) {
checkedMethods.add(checkProxy(method, bean));
}
for (KafkaListener classLevelListener : classLevelListeners) {
MultiMethodKafkaListenerEndpoint<K, V> endpoint = new MultiMethodKafkaListenerEndpoint<K, V>(checkedMethods,
bean);
endpoint.setBeanFactory(this.beanFactory);
processListener(endpoint, classLevelListener, bean, bean.getClass(), beanName);
}
}
protected void processKafkaListener(KafkaListener KafkaListener, Method method, Object bean, String beanName) {
Method methodToUse = checkProxy(method, bean);
MethodKafkaListenerEndpoint<K, V> endpoint = new MethodKafkaListenerEndpoint<K, V>();
endpoint.setMethod(methodToUse);
endpoint.setBeanFactory(this.beanFactory);
processListener(endpoint, KafkaListener, bean, methodToUse, beanName);
}
private Method checkProxy(Method method, Object bean) {
if (AopUtils.isJdkDynamicProxy(bean)) {
try {
// Found a @KafkaListener method on the target class for this JDK proxy ->
// is it also present on the proxy itself?
method = bean.getClass().getMethod(method.getName(), method.getParameterTypes());
Class<?>[] proxiedInterfaces = ((Advised) bean).getProxiedInterfaces();
for (Class<?> iface : proxiedInterfaces) {
try {
method = iface.getMethod(method.getName(), method.getParameterTypes());
break;
}
catch (NoSuchMethodException noMethod) {
}
}
}
catch (SecurityException ex) {
ReflectionUtils.handleReflectionException(ex);
}
catch (NoSuchMethodException ex) {
throw new IllegalStateException(String.format(
"@KafkaListener method '%s' found on bean target class '%s', " +
"but not found in any interface(s) for bean JDK proxy. Either " +
"pull the method up to an interface or switch to subclass (CGLIB) " +
"proxies by setting proxy-target-class/proxyTargetClass " +
"attribute to 'true'", method.getName(), method.getDeclaringClass().getSimpleName()));
}
}
return method;
}
protected void processListener(MethodKafkaListenerEndpoint<?, ?> endpoint, KafkaListener kafkaListener, Object bean,
Object adminTarget, String beanName) {
endpoint.setBean(bean);
endpoint.setMessageHandlerMethodFactory(this.messageHandlerMethodFactory);
endpoint.setId(getEndpointId(kafkaListener));
endpoint.setTopicPartitions(resolveTopicPartitions(kafkaListener));
endpoint.setTopics(resolveTopics(kafkaListener));
endpoint.setTopicPattern(resolvePattern(kafkaListener));
String group = kafkaListener.group();
if (StringUtils.hasText(group)) {
Object resolvedGroup = resolveExpression(group);
if (resolvedGroup instanceof String) {
endpoint.setGroup((String) resolvedGroup);
}
}
KafkaListenerContainerFactory<?> factory = null;
String containerFactoryBeanName = resolve(kafkaListener.containerFactory());
if (StringUtils.hasText(containerFactoryBeanName)) {
Assert.state(this.beanFactory != null, "BeanFactory must be set to obtain container factory by bean name");
try {
factory = this.beanFactory.getBean(containerFactoryBeanName, KafkaListenerContainerFactory.class);
}
catch (NoSuchBeanDefinitionException ex) {
throw new BeanInitializationException("Could not register Kafka listener endpoint on [" +
adminTarget + "] for bean " + beanName + ", no " + KafkaListenerContainerFactory.class.getSimpleName() + " with id '" +
containerFactoryBeanName + "' was found in the application context", ex);
}
}
this.registrar.registerEndpoint(endpoint, factory);
}
private String getEndpointId(KafkaListener KafkaListener) {
if (StringUtils.hasText(KafkaListener.id())) {
return resolve(KafkaListener.id());
}
else {
return "org.springframework.kafka.KafkaListenerEndpointContainer#" + counter.getAndIncrement();
}
}
private org.apache.kafka.common.TopicPartition[] resolveTopicPartitions(KafkaListener kafkaListener) {
TopicPartition[] partitions = kafkaListener.topicPartitions();
List<org.apache.kafka.common.TopicPartition> result = new ArrayList<>();
if (partitions.length > 0) {
for (int i = 0; i < partitions.length; i++) {
Object topic = resolveExpression(partitions[i].topic());
Assert.state(topic instanceof String, "topic in @TopicPartition must resolve to a String, not"
+ topic.getClass());
Object partition = resolveExpression(partitions[i].partition());
Assert.state(partition instanceof Integer || partition instanceof String,
"partition in @TopicPartition must resolve to an Integer or String, not a "
+ partition.getClass());
if (partition instanceof String) {
partition = Integer.valueOf((String) partition);
}
result.add(new org.apache.kafka.common.TopicPartition((String) topic, (Integer) partition));
}
}
return result.toArray(new org.apache.kafka.common.TopicPartition[result.size()]);
}
private String[] resolveTopics(KafkaListener kafkaListener) {
String[] topics = kafkaListener.topics();
List<String> result = new ArrayList<>();
if (topics.length > 0) {
for (int i = 0; i < topics.length; i++) {
Object topic = resolveExpression(topics[i]);
resolveAsString(topic, result);
}
}
return result.toArray(new String[result.size()]);
}
private Pattern resolvePattern(KafkaListener kafkaListener) {
Pattern pattern = null;
String text = kafkaListener.topicPattern();
if (StringUtils.hasText(text)) {
Object resolved = resolveExpression(text);
if (resolved instanceof Pattern) {
pattern = (Pattern) resolved;
}
else if (resolved instanceof String) {
pattern = Pattern.compile((String) resolved);
}
else {
throw new IllegalStateException("topicPattern must resolve to a Pattern or String, not "
+ resolved.getClass());
}
}
return pattern;
}
@SuppressWarnings("unchecked")
private void resolveAsString(Object resolvedValue, List<String> result) {
Object resolvedValueToUse = resolvedValue;
if (resolvedValue instanceof String[]) {
for (Object object : (String[]) resolvedValue) {
resolveAsString(object, result);
}
}
if (resolvedValueToUse instanceof String) {
result.add((String) resolvedValueToUse);
}
else if (resolvedValueToUse instanceof Iterable) {
for (Object object : (Iterable<Object>) resolvedValueToUse) {
resolveAsString(object, result);
}
}
else {
throw new IllegalArgumentException(String.format(
"@KafKaListener can't resolve '%s' as a String", resolvedValue));
}
}
private Object resolveExpression(String value) {
String resolvedValue = resolve(value);
if (!(resolvedValue.startsWith("#{") && value.endsWith("}"))) {
return resolvedValue;
}
return this.resolver.evaluate(resolvedValue, this.expressionContext);
}
/**
* Resolve the specified value if possible.
*
* @see ConfigurableBeanFactory#resolveEmbeddedValue
*/
private String resolve(String value) {
if (this.beanFactory != null && this.beanFactory instanceof ConfigurableBeanFactory) {
return ((ConfigurableBeanFactory) this.beanFactory).resolveEmbeddedValue(value);
}
return value;
}
/**
* An {@link MessageHandlerMethodFactory} adapter that offers a configurable underlying
* instance to use. Useful if the factory to use is determined once the endpoints
* have been registered but not created yet.
* @see KafkaListenerEndpointRegistrar#setMessageHandlerMethodFactory
*/
private class KafkaHandlerMethodFactoryAdapter implements MessageHandlerMethodFactory {
private MessageHandlerMethodFactory messageHandlerMethodFactory;
public void setMessageHandlerMethodFactory(MessageHandlerMethodFactory KafkaHandlerMethodFactory1) {
this.messageHandlerMethodFactory = KafkaHandlerMethodFactory1;
}
@Override
public InvocableHandlerMethod createInvocableHandlerMethod(Object bean, Method method) {
return getMessageHandlerMethodFactory().createInvocableHandlerMethod(bean, method);
}
private MessageHandlerMethodFactory getMessageHandlerMethodFactory() {
if (this.messageHandlerMethodFactory == null) {
this.messageHandlerMethodFactory = createDefaultMessageHandlerMethodFactory();
}
return this.messageHandlerMethodFactory;
}
private MessageHandlerMethodFactory createDefaultMessageHandlerMethodFactory() {
DefaultMessageHandlerMethodFactory defaultFactory = new DefaultMessageHandlerMethodFactory();
defaultFactory.setBeanFactory(beanFactory);
defaultFactory.afterPropertiesSet();
return defaultFactory;
}
}
}

View File

@@ -0,0 +1,51 @@
/*
* Copyright 2002-2014 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.annotation;
import org.springframework.kafka.listener.KafkaListenerEndpointRegistrar;
/**
* Optional interface to be implemented by Spring managed bean willing
* to customize how Kafka listener endpoints are configured. Typically
* used to defined the default
* {@link org.springframework.kafka.listener.KafkaListenerContainerFactory
* KafkaListenerContainerFactory} to use or for registering Kafka endpoints
* in a <em>programmatic</em> fashion as opposed to the <em>declarative</em>
* approach of using the @{@link KafkaListener} annotation.
*
* <p>See @{@link EnableKafka} for detailed usage examples.
*
* @author Stephane Nicoll
* @since 1.4
* @see EnableKafka
* @see org.springframework.kafka.listener.KafkaListenerEndpointRegistrar
*/
public interface KafkaListenerConfigurer {
/**
* Callback allowing a {@link org.springframework.kafka.listener.KafkaListenerEndpointRegistry
* KafkaListenerEndpointRegistry} and specific {@link org.springframework.kafka.listener.KafkaListenerEndpoint
* KafkaListenerEndpoint} instances to be registered against the given
* {@link KafkaListenerEndpointRegistrar}. The default
* {@link org.springframework.kafka.listener.KafkaListenerContainerFactory KafkaListenerContainerFactory}
* can also be customized.
* @param registrar the registrar to be configured
*/
void configureKafkaListeners(KafkaListenerEndpointRegistrar registrar);
}

View File

@@ -0,0 +1,42 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.annotation;
import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
/**
* Container annotation that aggregates several {@link KafkaListener} annotations.
* <p>
* Can be used natively, declaring several nested {@link KafkaListener} annotations.
* Can also be used in conjunction with Java 8's support for repeatable annotations,
* where {@link KafkaListener} can simply be declared several times on the same method
* (or class), implicitly generating this container annotation.
*
* @author Gary Russell
* @see KafkaListener
*/
@Target({ ElementType.TYPE, ElementType.METHOD, ElementType.ANNOTATION_TYPE })
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface KafkaListeners {
KafkaListener[] value();
}

View File

@@ -0,0 +1,28 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.annotation;
/**
* @author Gary Russell
*
*/
public @interface TopicPartition {
String topic() default "";
String partition() default "";
}

View File

@@ -0,0 +1,4 @@
/**
* Package for kafka annotations
*/
package org.springframework.kafka.annotation;

View File

@@ -0,0 +1,180 @@
/*
* Copyright 2014-2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.config;
import java.util.concurrent.Executor;
import java.util.concurrent.atomic.AtomicInteger;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.kafka.core.ConsumerFactory;
import org.springframework.kafka.listener.AbstractMessageListenerContainer;
import org.springframework.kafka.listener.AbstractMessageListenerContainer.AckMode;
import org.springframework.kafka.listener.ErrorHandler;
import org.springframework.kafka.listener.KafkaListenerContainerFactory;
import org.springframework.kafka.listener.KafkaListenerEndpoint;
/**
* Base {@link KafkaListenerContainerFactory} for Spring's base container implementation.
*
* @author Stephane Nicoll
* @see AbstractMessageListenerContainer
*/
public abstract class AbstractKafkaListenerContainerFactory<C extends AbstractMessageListenerContainer<K, V>, K, V>
implements KafkaListenerContainerFactory<C> {
protected final Log logger = LogFactory.getLog(getClass());
private ConsumerFactory<K, V> consumerFactory;
private ErrorHandler errorHandler;
private Boolean autoStartup;
private Integer phase;
protected final AtomicInteger counter = new AtomicInteger();
private Executor taskExecutor;
private Integer ackCount;
private AckMode ackMode;
private Long pollTimeout;
/**
* @param consumerFactory The consumer factory.
*/
public void setConsumerFactory(ConsumerFactory<K, V> consumerFactory) {
this.consumerFactory = consumerFactory;
}
public ConsumerFactory<K, V> getConsumerFactory() {
return consumerFactory;
}
/**
* @param errorHandler The error handler.
* @see AbstractMessageListenerContainer#setErrorHandler(ErrorHandler)
*/
public void setErrorHandler(ErrorHandler errorHandler) {
this.errorHandler = errorHandler;
}
/**
* @param taskExecutor the {@link Executor} to use.
* @see AbstractKafkaListenerContainerFactory#setTaskExecutor
*/
public void setTaskExecutor(Executor taskExecutor) {
this.taskExecutor = taskExecutor;
}
/**
* @param autoStartup true for auto startup.
* @see AbstractMessageListenerContainer#setAutoStartup(boolean)
*/
public void setAutoStartup(Boolean autoStartup) {
this.autoStartup = autoStartup;
}
/**
* @param phase The phase.
* @see AbstractMessageListenerContainer#setPhase(int)
*/
public void setPhase(int phase) {
this.phase = phase;
}
/**
* @param ackCount the ack count.
* @see AbstractMessageListenerContainer#setAckCount(int)
*/
public void setAckCount(Integer ackCount) {
this.ackCount = ackCount;
}
/**
* @param ackMode the ack mode.
* @see AbstractMessageListenerContainer#setAckMode(AckMode)
*/
public void setAckMode(AckMode ackMode) {
this.ackMode = ackMode;
}
/**
* @param pollTimeout the poll timeout
* @see AbstractMessageListenerContainer#setPollTimeout(long)
*/
public void setPollTimeout(Long pollTimeout) {
this.pollTimeout = pollTimeout;
}
@Override
public C createListenerContainer(KafkaListenerEndpoint endpoint) {
C instance = createContainerInstance(endpoint);
if (this.taskExecutor != null) {
instance.setTaskExecutor(this.taskExecutor);
}
if (this.errorHandler != null) {
instance.setErrorHandler(this.errorHandler);
}
if (this.autoStartup != null) {
instance.setAutoStartup(this.autoStartup);
}
if (this.phase != null) {
instance.setPhase(this.phase);
}
if (this.ackCount != null) {
instance.setAckCount(this.ackCount);
}
if (this.ackMode != null) {
instance.setAckMode(this.ackMode);
}
if (endpoint.getId() != null) {
instance.setBeanName(endpoint.getId());
}
if (this.pollTimeout != null) {
instance.setPollTimeout(this.pollTimeout);
}
endpoint.setupListenerContainer(instance);
initializeContainer(instance);
return instance;
}
/**
* Create an empty container instance.
* @param endpoint the endpoint.
* @return the new container instance.
*/
protected abstract C createContainerInstance(KafkaListenerEndpoint endpoint);
/**
* Further initialize the specified container.
* <p>Subclasses can inherit from this method to apply extra
* configuration if necessary.
* @param instance the containe instance to configure.
*/
protected void initializeContainer(C instance) {
}
}

View File

@@ -0,0 +1,40 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.config;
/**
* Configuration constants for internal sharing across subpackages.
*
* @author Juergen Hoeller
* @author Gary Russell
* @since 1.4
*/
public abstract class KafkaListenerConfigUtils {
/**
* The bean name of the internally managed Kafka listener annotation processor.
*/
public static final String KAFKA_LISTENER_ANNOTATION_PROCESSOR_BEAN_NAME =
"org.springframework.kafka.config.internalKafkaListenerAnnotationProcessor";
/**
* The bean name of the internally managed Kafka listener endpoint registry.
*/
public static final String KAFKA_LISTENER_ENDPOINT_REGISTRY_BEAN_NAME =
"org.springframework.kafka.config.internalKafkaListenerEndpointRegistry";
}

View File

@@ -0,0 +1,106 @@
/*
* Copyright 2014-2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.config;
import java.util.Collection;
import org.apache.kafka.common.TopicPartition;
import org.springframework.kafka.listener.ConcurrentMessageListenerContainer;
import org.springframework.kafka.listener.ConcurrentMessageListenerContainer.ContainerOffsetResetStrategy;
import org.springframework.kafka.listener.KafkaListenerContainerFactory;
import org.springframework.kafka.listener.KafkaListenerEndpoint;
/**
* A {@link KafkaListenerContainerFactory} implementation to build a regular
* {@link ConcurrentMessageListenerContainer}.
*
* <p>This should be the default for most users and a good transition paths
* for those that are used to build such container definition manually.
*
* @author Stephane Nicoll
* @author Gary Russell
* @author Artem Bilan
*/
public class SimpleKafkaListenerContainerFactory<K, V>
extends AbstractKafkaListenerContainerFactory<ConcurrentMessageListenerContainer<K, V>, K, V> {
private Integer concurrency;
private Long receentOffset;
private ContainerOffsetResetStrategy resetStrategy;
/**
* @param concurrency the number of consumers to create.
* @see ConcurrentMessageListenerContainer#setConcurrency(int)
*/
public void setConcurrency(Integer concurrency) {
this.concurrency = concurrency;
}
/**
* @param receentOffset the recent offset.
* @see ConcurrentMessageListenerContainer#setRecentOffset(long)
*/
public void setReceentOffset(Long receentOffset) {
this.receentOffset = receentOffset;
}
/**
* @param resetStrategy the reset strategy
* @see ConcurrentMessageListenerContainer#setResetStrategy(ContainerOffsetResetStrategy)
*/
public void setResetStrategy(ContainerOffsetResetStrategy resetStrategy) {
this.resetStrategy = resetStrategy;
}
@Override
protected ConcurrentMessageListenerContainer<K, V> createContainerInstance(KafkaListenerEndpoint endpoint) {
Collection<TopicPartition> topicPartitions = endpoint.getTopicPartitions();
if (!topicPartitions.isEmpty()) {
return new ConcurrentMessageListenerContainer<K, V>(getConsumerFactory(),
topicPartitions.toArray(new TopicPartition[topicPartitions.size()]));
}
else {
Collection<String> topics = endpoint.getTopics();
if (!topics.isEmpty()) {
return new ConcurrentMessageListenerContainer<K, V>(getConsumerFactory(),
topics.toArray(new String[topics.size()]));
}
else {
return new ConcurrentMessageListenerContainer<K, V>(getConsumerFactory(), endpoint.getTopicPattern());
}
}
}
@Override
protected void initializeContainer(ConcurrentMessageListenerContainer<K, V> instance) {
super.initializeContainer(instance);
if (this.concurrency != null) {
instance.setConcurrency(this.concurrency);
}
if (this.receentOffset != null) {
instance.setRecentOffset(this.receentOffset);
}
if (this.resetStrategy != null) {
instance.setResetStrategy(this.resetStrategy);
}
}
}

View File

@@ -0,0 +1,66 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.config;
import org.springframework.kafka.listener.AbstractKafkaListenerEndpoint;
import org.springframework.kafka.listener.KafkaListenerEndpoint;
import org.springframework.kafka.listener.MessageListener;
import org.springframework.kafka.listener.MessageListenerContainer;
/**
* A {@link KafkaListenerEndpoint} simply providing the {@link MessageListener} to
* invoke to process an incoming message for this endpoint.
*
* @author Stephane Nicoll
* @author Gary Russell
*/
public class SimpleKafkaListenerEndpoint<K, V> extends AbstractKafkaListenerEndpoint<K, V> {
private MessageListener<K, V> messageListener;
/**
* Set the {@link MessageListener} to invoke when a message matching
* the endpoint is received.
* @param messageListener the {@link MessageListener} instance.
*/
public void setMessageListener(MessageListener<K, V> messageListener) {
this.messageListener = messageListener;
}
/**
* @return the {@link MessageListener} to invoke when a message matching
* the endpoint is received.
*/
public MessageListener<K, V> getMessageListener() {
return this.messageListener;
}
@Override
protected MessageListener<K, V> createMessageListener(MessageListenerContainer container) {
return getMessageListener();
}
@Override
protected StringBuilder getEndpointDescription() {
return super.getEndpointDescription()
.append(" | messageListener='").append(this.messageListener).append("'");
}
}

View File

@@ -0,0 +1,4 @@
/**
* Package for kafka configuration
*/
package org.springframework.kafka.config;

View File

@@ -0,0 +1,30 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.core;
import org.apache.kafka.clients.consumer.Consumer;
/**
* @author Gary Russell
*
*/
public interface ConsumerFactory<K, V> {
Consumer<K, V> createConsumer();
boolean isAutoCommit();
}

View File

@@ -0,0 +1,48 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.core;
import java.util.HashMap;
import java.util.Map;
import org.apache.kafka.clients.consumer.Consumer;
import org.apache.kafka.clients.consumer.KafkaConsumer;
/**
* @author Gary Russell
*
*/
public class DefaultKafkaConsumerFactory<K, V> implements ConsumerFactory<K, V> {
private final Map<String, Object> configs;
public DefaultKafkaConsumerFactory(Map<String, Object> configs) {
this.configs = new HashMap<>(configs);
}
@Override
public Consumer<K, V> createConsumer() {
return new KafkaConsumer<>(this.configs);
}
@Override
public boolean isAutoCommit() {
Object auto = this.configs.get("enable.auto.commit");
return auto instanceof Boolean ? (Boolean) auto
: auto instanceof String ? Boolean.valueOf((String) auto) : false;
}
}

View File

@@ -0,0 +1,41 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.core;
import java.util.HashMap;
import java.util.Map;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.Producer;
/**
* @author Gary Russell
*
*/
public class DefaultKafkaProducerFactory<K, V> implements ProducerFactory<K, V> {
private final Map<String, Object> configs;
public DefaultKafkaProducerFactory(Map<String, Object> configs) {
this.configs = new HashMap<>(configs);
}
@Override
public Producer<K, V> createProducer() {
return new KafkaProducer<>(this.configs);
}
}

View File

@@ -0,0 +1,33 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.core;
/**
* @author Gary Russell
*
*/
@SuppressWarnings("serial")
public class KafkaException extends RuntimeException {
public KafkaException(String message) {
super(message);
}
public KafkaException(String message, Throwable cause) {
super(message, cause);
}
}

View File

@@ -0,0 +1,88 @@
/*
* Copyright 2015-2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.core;
import java.util.concurrent.Future;
import org.apache.kafka.clients.producer.RecordMetadata;
/**
* @author Marius Bogoevici
* @author Gary Russell
*
* @param <K> the key type.
* @param <V> the value type.
*/
public interface KafkaOperations<K, V> {
/**
* Send the data to the default topic with no key or partition.
* @param data The data.
* @return a Future for the {@link RecordMetadata}.
*/
Future<RecordMetadata> convertAndSend(V data);
/**
* Send the data to the default topic with the provided key and no partition.
* @param key the key.
* @param data The data.
* @return a Future for the {@link RecordMetadata}.
*/
Future<RecordMetadata> convertAndSend(K key, V data);
/**
* Send the data to the default topic with the provided key and partition.
* @param partition the partition.
* @param key the key.
* @param data the data.
* @return a Future for the {@link RecordMetadata}.
*/
Future<RecordMetadata> convertAndSend(int partition, K key, V data);
/**
* Send the data to the provided topic with no key or partition.
* @param topic the topic.
* @param data The data.
* @return a Future for the {@link RecordMetadata}.
*/
Future<RecordMetadata> convertAndSend(String topic, V data);
/**
* Send the data to the provided topic with the provided key and no partition.
* @param topic the topic.
* @param key the key.
* @param data The data.
* @return a Future for the {@link RecordMetadata}.
*/
Future<RecordMetadata> convertAndSend(String topic, K key, V data);
/**
* Send the data to the provided topic with the provided key and partition.
* @param topic the topic.
* @param partition the partition.
* @param key the key.
* @param data the data.
* @return a Future for the {@link RecordMetadata}.
*/
Future<RecordMetadata> convertAndSend(String topic, int partition, K key, V data);
/**
* Flush the producer.
*/
void flush();
}

View File

@@ -0,0 +1,131 @@
/*
* Copyright 2015-2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.core;
import java.util.concurrent.Future;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.kafka.clients.producer.Producer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.clients.producer.RecordMetadata;
/**
* A template for executing high-level operations.
*
* @author Marius Bogoevici
* @author Gary Russell
*/
public class KafkaTemplate<K, V> implements KafkaOperations<K, V> {
protected final Log logger = LogFactory.getLog(this.getClass());
private final ProducerFactory<K, V> producerFactory;
private volatile Producer<K, V> producer;
private volatile String defaultTopic;
/**
* Create an instance using the supplied producer factory.
* @param producerFactory the producer factory.
*/
public KafkaTemplate(ProducerFactory<K, V> producerFactory) {
this.producerFactory = producerFactory;
}
/**
* The default topic for send methods where a topic is not
* providing.
* @return the topic.
*/
public String getDefaultTopic() {
return defaultTopic;
}
/**
* Set the default topic for send methods where a topic is not
* providing.
* @param defaultTopic the topic.
*/
public void setDefaultTopic(String defaultTopic) {
this.defaultTopic = defaultTopic;
}
@Override
public Future<RecordMetadata> convertAndSend(V data) {
return convertAndSend(this.defaultTopic, data);
}
@Override
public Future<RecordMetadata> convertAndSend(K key, V data) {
return convertAndSend(this.defaultTopic, key, data);
}
@Override
public Future<RecordMetadata> convertAndSend(int partition, K key, V data) {
return convertAndSend(this.defaultTopic, partition, key, data);
}
@Override
public Future<RecordMetadata> convertAndSend(String topic, V data) {
ProducerRecord<K, V> producerRecord = new ProducerRecord<>(topic, data);
return doSend(producerRecord);
}
@Override
public Future<RecordMetadata> convertAndSend(String topic, K key, V data) {
ProducerRecord<K, V> producerRecord = new ProducerRecord<>(topic, key, data);
return doSend(producerRecord);
}
@Override
public Future<RecordMetadata> convertAndSend(String topic, int partition, K key, V data) {
ProducerRecord<K, V> producerRecord = new ProducerRecord<>(topic, partition, key, data);
return doSend(producerRecord);
}
/**
* Send the producer record.
* @param producerRecord the producer record.
* @return a Future for the {@link RecordMetadata}.
*/
protected Future<RecordMetadata> doSend(ProducerRecord<K, V> producerRecord) {
if (this.producer == null) {
synchronized (this) {
if (this.producer == null) {
this.producer = this.producerFactory.createProducer();
}
}
}
if (logger.isTraceEnabled()) {
logger.trace("Sending: " + producerRecord);
}
Future<RecordMetadata> future = this.producer.send(producerRecord);
if (logger.isTraceEnabled()) {
logger.trace("Sent: " + producerRecord);
}
return future;
}
@Override
public void flush() {
this.producer.flush();
}
}

View File

@@ -0,0 +1,28 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.core;
import org.apache.kafka.clients.producer.Producer;
/**
* @author Gary Russell
*
*/
public interface ProducerFactory<K, V> {
Producer<K, V> createProducer();
}

View File

@@ -0,0 +1,4 @@
/**
* Package for kafka core components
*/
package org.springframework.kafka.core;

View File

@@ -0,0 +1,224 @@
/*
* Copyright 2014-2015 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.regex.Pattern;
import org.apache.kafka.common.TopicPartition;
import org.springframework.beans.BeansException;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.beans.factory.BeanFactoryAware;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.beans.factory.config.BeanExpressionContext;
import org.springframework.beans.factory.config.BeanExpressionResolver;
import org.springframework.beans.factory.config.ConfigurableListableBeanFactory;
import org.springframework.util.Assert;
/**
* Base model for a Kafka listener endpoint
*
* @author Stephane Nicoll
* @author Gary Russell
* @see MethodKafkaListenerEndpoint
* @see org.springframework.kafka.config.SimpleKafkaListenerEndpoint
*/
public abstract class AbstractKafkaListenerEndpoint<K, V>
implements KafkaListenerEndpoint, BeanFactoryAware, InitializingBean {
private String id;
private final Collection<String> topics = new ArrayList<>();
private Pattern topicPattern;
private final Collection<TopicPartition> topicPartitions = new ArrayList<>();
private BeanFactory beanFactory;
private BeanExpressionResolver resolver;
private BeanExpressionContext expressionContext;
private String group;
@Override
public void setBeanFactory(BeanFactory beanFactory) throws BeansException {
this.beanFactory = beanFactory;
if (beanFactory instanceof ConfigurableListableBeanFactory) {
this.resolver = ((ConfigurableListableBeanFactory) beanFactory).getBeanExpressionResolver();
this.expressionContext = new BeanExpressionContext((ConfigurableListableBeanFactory) beanFactory, null);
}
}
protected BeanFactory getBeanFactory() {
return beanFactory;
}
protected BeanExpressionResolver getResolver() {
return resolver;
}
protected BeanExpressionContext getBeanExpressionContext() {
return expressionContext;
}
public void setId(String id) {
this.id = id;
}
@Override
public String getId() {
return this.id;
}
/**
* Set the topics to use. Either these or 'topicPattern'
* or 'topicPartitions'
* should be provided, but not a mixture.
* @param topics to set.
* @see #setTopicPartitions(TopicPartition...)
* @see #setTopicPattern(Pattern)
*/
public void setTopics(String... topics) {
Assert.notNull(topics, "'topics' must not be null");
this.topics.clear();
this.topics.addAll(Arrays.asList(topics));
}
/**
* @return the topics for this endpoint.
*/
@Override
public Collection<String> getTopics() {
return Collections.unmodifiableCollection(this.topics);
}
/**
* Set the topics to use. Either these or 'topicPattern'
* or 'topicPartitions'
* should be provided, but not a mixture.
* @param topicPartitions to set.
* @see #setTopics(String...)
* @see #setTopicPattern(Pattern)
*/
public void setTopicPartitions(TopicPartition... topicPartitions) {
Assert.notNull(topicPartitions, "'topics' must not be null");
this.topicPartitions.clear();
this.topicPartitions.addAll(Arrays.asList(topicPartitions));
}
/**
* @return the topicPartitions for this endpoint.
*/
@Override
public Collection<TopicPartition> getTopicPartitions() {
return Collections.unmodifiableCollection(this.topicPartitions);
}
/**
* Set the topic pattern to use. Cannot be used with
* topics or topicPartitions.
* @param topicPattern the pattern
* @see #setTopicPartitions(TopicPartition...)
* @see #setTopics(String...)
*/
public void setTopicPattern(Pattern topicPattern) {
this.topicPattern = topicPattern;
}
/**
* @return the topicPattern for this endpoint.
*/
@Override
public Pattern getTopicPattern() {
return this.topicPattern;
}
@Override
public String getGroup() {
return this.group;
}
/**
* Set the group for the corresponding listener container.
* @param group the group.
* @since 1.5
*/
public void setGroup(String group) {
this.group = group;
}
@Override
public void afterPropertiesSet() {
boolean topicsEmpty = getTopics().isEmpty();
boolean topicPartitionsEmpty = getTopicPartitions().isEmpty();
if (!topicsEmpty && !topicPartitionsEmpty) {
throw new IllegalStateException("Topics or topicPartitions must be provided but not both for " + this);
}
if (this.topicPattern != null && (!topicsEmpty || !topicPartitionsEmpty)) {
throw new IllegalStateException("Only one of topics, topicPartitions or topicPattern must are allowed for "
+ this);
}
if (this.topicPattern == null && topicsEmpty && topicPartitionsEmpty) {
throw new IllegalStateException("At least one of topics, topicPartitions or topicPattern must be provided "
+ "for " + this);
}
}
@Override
public void setupListenerContainer(MessageListenerContainer listenerContainer) {
setupMessageListener(listenerContainer);
}
/**
* Create a {@link MessageListener} that is able to serve this endpoint for the
* specified container.
* @param container the {@link MessageListenerContainer} to create a {@link MessageListener}.
* @return a a {@link MessageListener} instance.
*/
protected abstract MessageListener<K, V> createMessageListener(MessageListenerContainer container);
private void setupMessageListener(MessageListenerContainer container) {
MessageListener<K, V> messageListener = createMessageListener(container);
Assert.state(messageListener != null, "Endpoint [" + this + "] must provide a non null message listener");
container.setupMessageListener(messageListener);
}
/**
* @return a description for this endpoint.
* <p>Available to subclasses, for inclusion in their {@code toString()} result.
*/
protected StringBuilder getEndpointDescription() {
StringBuilder result = new StringBuilder();
return result.append(getClass().getSimpleName()).append("[").append(this.id).
append("] topics=").append(this.topics).
append("' | topicPartitions='").append(this.topicPartitions).
append("' | topicPattern='").append(this.topicPattern).append("'");
}
@Override
public String toString() {
return getEndpointDescription().toString();
}
}

View File

@@ -0,0 +1,285 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
import java.util.concurrent.Executor;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.kafka.clients.consumer.Consumer;
import org.springframework.beans.factory.BeanNameAware;
import org.springframework.context.SmartLifecycle;
import org.springframework.util.Assert;
/**
*
* @author Gary Russell
*/
public abstract class AbstractMessageListenerContainer<K, V>
implements MessageListenerContainer, BeanNameAware, SmartLifecycle {
protected final Log logger = LogFactory.getLog(this.getClass());
public enum AckMode {
/**
* Call {@link Consumer#commitAsync()} after each record is passed to the listener.
*/
RECORD,
/**
* Call {@link Consumer#commitAsync()} after the results of each poll have been
* passed to the listener.
*/
BATCH,
/**
* Call {@link Consumer#commitAsync()} for pending updates after
* {@link AbstractMessageListenerContainer#setAckTime(long) ackTime} has elapsed.
*/
TIME,
/**
* Call {@link Consumer#commitAsync()} for pending updates after
* {@link AbstractMessageListenerContainer#setAckCount(int) ackCount} has been
* exceeded.
*/
COUNT,
/**
* Call {@link Consumer#commitAsync()} for pending updates after
* {@link AbstractMessageListenerContainer#setAckCount(int) ackCount} has been
* exceeded or after {@link AbstractMessageListenerContainer#setAckTime(long)
* ackTime} has elapsed.
*/
COUNT_TIME,
/**
* Same as {@link #COUNT_TIME} except for pending manual acks.
*/
MANUAL,
/**
* Call {@link Consumer#commitAsync()} immediately for pending acks.
*/
MANUAL_IMMEDIATE
}
private final Object lifecycleMonitor = new Object();
private String beanName;
private AckMode ackMode = AckMode.BATCH;
private int ackCount;
private long ackTime;
private Object messageListener;
private volatile long pollTimeout = 1000;
private boolean autoStartup = true;
private int phase = 0;
private volatile boolean running = false;
private Executor taskExecutor;
private ErrorHandler errorHandler = new LoggingErrorHandler();
@Override
public void setBeanName(String name) {
this.beanName = name;
}
public String getBeanName() {
return this.beanName;
}
/**
* Set the message listener; must be a {@link MessageListener} or
* {@link AcknowledgingMessageListener}.
* @param messageListener the listener.
*/
public void setMessageListener(Object messageListener) {
Assert.isTrue(
messageListener instanceof MessageListener || messageListener instanceof AcknowledgingMessageListener,
"Either a " + MessageListener.class.getName() + " or a " + AcknowledgingMessageListener.class.getName()
+ " must be provided");
this.messageListener = messageListener;
}
public Object getMessageListener() {
return messageListener;
}
@Override
public void setupMessageListener(Object messageListener) {
setMessageListener(messageListener);
}
/**
* The ack mode to use when auto ack (in the configuration properties) is
* false.
* <ul>
* <li>RECORD: Ack after each record has been passed to the listener.</li>
* <li>BATCH: Ack after each batch of records received from the consumer has
* been passed to the listener</li>
* <li>TIME: Ack after this number of milliseconds;
* (should be greater than {@code #setPollTimeout(long) pollTimeout}.</li>
* <li>COUNT: Ack after at least this number of records have been received</li>
* <li>MANUAL: Listener is responsible for acking - use a
* {@link AcknowledgingMessageListener}.
* </ul>
* @param ackMode the {@link AckMode}; default BATCH.
*/
public void setAckMode(AckMode ackMode) {
this.ackMode = ackMode;
}
/**
* @return the {@link AckMode}
* @see #setAckMode(AckMode)
*/
public AckMode getAckMode() {
return ackMode;
}
/**
* The max time to block in the consumer waiting for records.
* @param pollTimeout the timeout in ms; default 1000.
*/
public void setPollTimeout(long pollTimeout) {
this.pollTimeout = pollTimeout;
}
/**
* @return the poll timeout.
* @see #setPollTimeout(long)
*/
public long getPollTimeout() {
return pollTimeout;
}
/**
* Set the number of outstanding record count after which offsets should be committed
* when {@link AckMode#COUNT} or {@link AckMode#COUNT_TIME} is being used.
* @param count the count
*/
public void setAckCount(int count) {
this.ackCount = count;
}
/**
* @return the count.
* @see #setAckCount(int)
*/
public int getAckCount() {
return this.ackCount;
}
/**
* Set the time (ms) after which outstanding offsets should be committed
* when {@link AckMode#TIME} or {@link AckMode#COUNT_TIME} is being used. Should
* be larger than
* @param millis the time
*/
public void setAckTime(long millis) {
this.ackTime = millis;
}
/**
* @return the time.
* @see AbstractMessageListenerContainer#setAckTime(long)
*/
public long getAckTime() {
return this.ackTime;
}
@Override
public boolean isAutoStartup() {
return this.autoStartup;
}
public void setAutoStartup(boolean autoStartup) {
this.autoStartup = autoStartup;
}
@Override
public final void start() {
synchronized (this.lifecycleMonitor) {
doStart();
}
}
protected abstract void doStart();
@Override
public final void stop() {
stop(null);
}
@Override
public void stop(Runnable callback) {
synchronized (this.lifecycleMonitor) {
doStop();
}
if (callback != null) {
callback.run();
}
}
protected abstract void doStop();
protected void setRunning(boolean running) {
this.running = running;
}
@Override
public boolean isRunning() {
return this.running;
}
public void setPhase(int phase) {
this.phase = phase;
}
@Override
public int getPhase() {
return this.phase;
}
public ErrorHandler getErrorHandler() {
return this.errorHandler;
}
public void setErrorHandler(ErrorHandler errorHandler) {
this.errorHandler = errorHandler;
}
public Executor getTaskExecutor() {
return this.taskExecutor;
}
public void setTaskExecutor(Executor fetchTaskExecutor) {
this.taskExecutor = fetchTaskExecutor;
}
}

View File

@@ -0,0 +1,39 @@
/*
* Copyright 2015 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
import org.apache.kafka.clients.consumer.ConsumerRecord;
/**
* Listener for handling incoming Kafka messages, propagating an acknowledgment handle that recipients
* can invoke when the message has been processed.
*
* @author Marius Bogoevici
* @author Gary Russell
* @since 1.0.1
*/
public interface AcknowledgingMessageListener<K, V> {
/**
* Executes when a Kafka message is received
*
* @param record the Kafka message to be processed
* @param acknowledgment a handle for acknowledging the message processing
*/
void onMessage(ConsumerRecord<K, V> record, Acknowledgment acknowledgment);
}

View File

@@ -0,0 +1,37 @@
/*
* Copyright 2015-2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
/**
* Handle for acknowledging the processing of a
* {@link org.apache.kafka.clients.consumer.ConsumerRecord}. Recipients can store the
* reference in asynchronous scenarios, but the internal state should be assumed transient
* (i.e. it cannot be serialized and deserialized later)
*
* @author Marius Bogoevici
* @author Gary Russell
* @since 1.0.1
*/
public interface Acknowledgment {
/**
* Invoked when the message for which the acknowledgment has been created has been processed.
* Calling this method implies that all the previous messages in the partition have been processed already.
*/
void acknowledge();
}

View File

@@ -0,0 +1,256 @@
/*
* Copyright 2015-2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.regex.Pattern;
import org.apache.kafka.common.TopicPartition;
import org.springframework.kafka.core.ConsumerFactory;
import org.springframework.util.Assert;
/**
* Creates 1 or more {@link KafkaMessageListenerContainer}s based on
* {@link #setConcurrency(int) concurrency}. If the
* {@link #ConcurrentMessageListenerContainer(ConsumerFactory, TopicPartition...)}
* constructor is used, the {@link TopicPartition}s are distributed evenly across the
* instances.
*
* @author Marius Bogoevici
* @author Gary Russell
*/
public class ConcurrentMessageListenerContainer<K, V> extends AbstractMessageListenerContainer<K, V> {
private static final int DEFAULT_STOP_TIMEOUT = 1000;
private final ConsumerFactory<K, V> consumerFactory;
private final String[] topics;
private final Pattern topicPattern;
final List<KafkaMessageListenerContainer<K, V>> containers = new ArrayList<>();
private ContainerOffsetResetStrategy resetStrategy = ContainerOffsetResetStrategy.NONE;
private long recentOffset = 1;
TopicPartition[] partitions;
int concurrency = 1;
private int stopTimeout = DEFAULT_STOP_TIMEOUT;
/**
* Construct an instance with the supplied configuration properties and specific
* topics/partitions - when using this constructor, a
* {@link #setResetStrategy(ContainerOffsetResetStrategy)} can be used.
* The topic partitions are distributed evenly across the delegate
* {@link KafkaMessageListenerContainer}s.
* @param consumerFactory the consumer factory.
* @param topicPartitions the topics/partitions; duplicates are eliminated.
*/
public ConcurrentMessageListenerContainer(ConsumerFactory<K, V> consumerFactory, TopicPartition... topicPartitions) {
Assert.notNull(consumerFactory, "A ConsumerFactory must be provided");
Assert.notEmpty(topicPartitions, "A list of partitions must be provided");
Assert.noNullElements(topicPartitions, "The list of partitions cannot contain null elements");
this.consumerFactory = consumerFactory;
this.partitions = new LinkedHashSet<>(Arrays.asList(topicPartitions)).toArray(new TopicPartition[0]);
this.topics = null;
this.topicPattern = null;
}
/**
* Construct an instance with the supplied configuration properties and topics.
* When using this constructor, a
* {@link #setResetStrategy(ContainerOffsetResetStrategy)} cannot be used.
* @param consumerFactory the consumer factory.
* @param topics the topics.
*/
public ConcurrentMessageListenerContainer(ConsumerFactory<K, V> consumerFactory, String... topics) {
Assert.notNull(consumerFactory, "A ConsumerFactory must be provided");
Assert.notNull(topics, "A list of topics must be provided");
Assert.noNullElements(topics, "The list of topics cannot contain null elements");
this.consumerFactory = consumerFactory;
this.topics = topics;
this.topicPattern = null;
}
/**
* Construct an instance with the supplied configuration properties and topic
* pattern. When using this constructor, a
* {@link #setResetStrategy(ContainerOffsetResetStrategy)} cannot be used.
* @param consumerFactory the consumer factory.
* @param topicPattern the topic pattern.
*/
public ConcurrentMessageListenerContainer(ConsumerFactory<K, V> consumerFactory, Pattern topicPattern) {
Assert.notNull(consumerFactory, "A ConsumerFactory must be provided");
Assert.notNull(topicPattern, "A topic pattern must be provided");
this.consumerFactory = consumerFactory;
this.topics = null;
this.topicPattern = topicPattern;
}
/**
* The initial offset reset strategy, when explicit partitions are provided.
* <ul>
* <li>NONE: No reset</li>
* <li>EARLIEST: Set to the earliest message</li>
* <li>LATEST: Set to the last message; receive new messages only</li>
* <li>RECENT: Set to a recent message based on {@link #setRecentOffset(long) recentOffset}</li>
* </ul>
*
* @param resetStrategy the {@link ContainerOffsetResetStrategy}
*/
public void setResetStrategy(ContainerOffsetResetStrategy resetStrategy) {
this.resetStrategy = resetStrategy;
}
/**
* Set the number of records back from the latest when using
* {@link ContainerOffsetResetStrategy#RECENT}.
* @param recentOffset the offset from the latest; default 1.
*/
public void setRecentOffset(long recentOffset) {
this.recentOffset = recentOffset;
}
public int getConcurrency() {
return concurrency;
}
/**
* The maximum number of concurrent {@link KafkaMessageListenerContainer}s running.
* Messages from within the same partition will be processed sequentially.
* @param concurrency the concurrency.
*/
public void setConcurrency(int concurrency) {
Assert.isTrue(concurrency > 0, "concurrency must be greater than 0");
this.concurrency = concurrency;
}
/**
* The timeout for waiting for each concurrent {@link MessageListener} to finish on
* stopping.
* @param stopTimeout timeout in milliseconds
* @since 1.1
*/
public void setStopTimeout(int stopTimeout) {
this.stopTimeout = stopTimeout;
}
public int getStopTimeout() {
return stopTimeout;
}
/**
* @return the list of {@link KafkaMessageListenerContainer}s created by
* this container.
*/
public List<KafkaMessageListenerContainer<K, V>> getContainers() {
return Collections.unmodifiableList(this.containers);
}
/*
* Under lifecycle lock.
*/
@Override
protected void doStart() {
if (!isRunning()) {
if (this.partitions != null && this.concurrency > this.partitions.length) {
logger.warn("When specific partitions are provided, the concurrency must be less than or "
+ "equal to the number of partitions; reduced from " + this.concurrency
+ " to " + this.partitions.length);
this.concurrency = this.partitions.length;
}
setRunning(true);
for (int i = 0; i < this.concurrency; i++) {
KafkaMessageListenerContainer<K, V> container;
if (this.partitions == null) {
container = new KafkaMessageListenerContainer<>(this.consumerFactory, this.topics,
this.topicPattern, this.partitions);
}
else {
container = new KafkaMessageListenerContainer<>(this.consumerFactory, this.topics,
this.topicPattern, partitionSubset(i));
}
container.setAckMode(getAckMode());
container.setAckCount(getAckCount());
container.setAckTime(getAckTime());
container.setResetStrategy(this.resetStrategy);
container.setRecentOffset(this.recentOffset);
container.setAutoStartup(false);
container.setMessageListener(getMessageListener());
if (getTaskExecutor() != null) {
container.setTaskExecutor(getTaskExecutor());
}
if (getBeanName() != null) {
container.setBeanName(getBeanName() + "-" + i);
}
container.start();
this.containers.add(container);
}
}
}
private TopicPartition[] partitionSubset(int i) {
if (this.concurrency == 1) {
return this.partitions;
}
else {
int numPartitions = this.partitions.length;
if (numPartitions == this.concurrency) {
return new TopicPartition[] { this.partitions[i] };
}
else {
int perContainer = numPartitions / this.concurrency;
TopicPartition[] subset;
if (i == this.concurrency - 1) {
subset = Arrays.copyOfRange(this.partitions, i * perContainer, partitions.length);
}
else {
subset = Arrays.copyOfRange(this.partitions, i * perContainer, (i + 1) * perContainer);
}
return subset;
}
}
}
/*
* Under lifecycle lock.
*/
@Override
protected void doStop() {
if (isRunning()) {
setRunning(false);
for (KafkaMessageListenerContainer<K, V> container : this.containers) {
container.stop();
}
this.containers.clear();
}
}
public enum ContainerOffsetResetStrategy {
LATEST, EARLIEST, NONE, RECENT
}
}

View File

@@ -0,0 +1,32 @@
/*
* Copyright 2015-2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
import org.apache.kafka.clients.consumer.ConsumerRecord;
/**
* Handles errors thrown during the execution of a {@link MessageListener}
*
* @author Marius Bogoevici
* @author Gary Russell
*/
public interface ErrorHandler {
void handle(Exception thrownException, ConsumerRecord<?, ?> record);
}

View File

@@ -0,0 +1,37 @@
/*
* Copyright 2002-2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
/**
* Factory of {@link MessageListenerContainer} based on a
* {@link KafkaListenerEndpoint} definition.
*
* @author Stephane Nicoll
* @see KafkaListenerEndpoint
*/
public interface KafkaListenerContainerFactory<C extends MessageListenerContainer> {
/**
* Create a {@link MessageListenerContainer} for the given {@link KafkaListenerEndpoint}.
* @param endpoint the endpoint to configure
* @return the created container
*/
C createListenerContainer(KafkaListenerEndpoint endpoint);
}

View File

@@ -0,0 +1,75 @@
/*
* Copyright 2002-2015 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
import java.util.Collection;
import java.util.regex.Pattern;
import org.apache.kafka.common.TopicPartition;
/**
* Model for a Kafka listener endpoint. Can be used against a
* {@link org.springframework.kafka.annotation.KafkaListenerConfigurer
* KafkaListenerConfigurer} to register endpoints programmatically.
*
* @author Stephane Nicoll
* @author Gary Russell
* @since 1.4
*/
public interface KafkaListenerEndpoint {
/**
* @return the id of this endpoint. The id can be further qualified
* when the endpoint is resolved against its actual listener
* container.
* @see KafkaListenerContainerFactory#createListenerContainer
*/
String getId();
/**
* @return the group of this endpoint or null if not in a group.
*/
String getGroup();
/**
* @return the topics for this endpoint.
*/
public Collection<String> getTopics();
/**
* @return the topicPartitions for this endpoint.
*/
public Collection<TopicPartition> getTopicPartitions();
/**
* @return the topicPattern for this endpoint.
*/
public Pattern getTopicPattern();
/**
* Setup the specified message listener container with the model
* defined by this endpoint.
* <p>This endpoint must provide the requested missing option(s) of
* the specified container to make it usable. Usually, this is about
* setting the {@code queues} and the {@code messageListener} to
* use but an implementation may override any default setting that
* was already set.
* @param listenerContainer the listener container to configure
*/
void setupListenerContainer(MessageListenerContainer listenerContainer);
}

View File

@@ -0,0 +1,210 @@
/*
* Copyright 2014-2015 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
import java.util.ArrayList;
import java.util.List;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.beans.factory.BeanFactoryAware;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.messaging.handler.annotation.support.DefaultMessageHandlerMethodFactory;
import org.springframework.messaging.handler.annotation.support.MessageHandlerMethodFactory;
import org.springframework.util.Assert;
/**
* Helper bean for registering {@link KafkaListenerEndpoint} with
* a {@link KafkaListenerEndpointRegistry}.
*
* @author Stephane Nicoll
* @author Juergen Hoeller
* @author Artem Bilan
* @since 1.4
* @see org.springframework.kafka.annotation.KafkaListenerConfigurer
*/
public class KafkaListenerEndpointRegistrar implements BeanFactoryAware, InitializingBean {
private final List<KafkaListenerEndpointDescriptor> endpointDescriptors =
new ArrayList<KafkaListenerEndpointDescriptor>();
private KafkaListenerEndpointRegistry endpointRegistry;
private MessageHandlerMethodFactory messageHandlerMethodFactory;
private KafkaListenerContainerFactory<?> containerFactory;
private String containerFactoryBeanName;
private BeanFactory beanFactory;
private boolean startImmediately;
/**
* Set the {@link KafkaListenerEndpointRegistry} instance to use.
* @param endpointRegistry the {@link KafkaListenerEndpointRegistry} instance to use.
*/
public void setEndpointRegistry(KafkaListenerEndpointRegistry endpointRegistry) {
this.endpointRegistry = endpointRegistry;
}
/**
* @return the {@link KafkaListenerEndpointRegistry} instance for this
* registrar, may be {@code null}.
*/
public KafkaListenerEndpointRegistry getEndpointRegistry() {
return this.endpointRegistry;
}
/**
* Set the {@link MessageHandlerMethodFactory} to use to configure the message
* listener responsible to serve an endpoint detected by this processor.
* <p>By default, {@link DefaultMessageHandlerMethodFactory} is used and it
* can be configured further to support additional method arguments
* or to customize conversion and validation support. See
* {@link DefaultMessageHandlerMethodFactory} javadoc for more details.
* @param kafkaHandlerMethodFactory the {@link MessageHandlerMethodFactory} instance.
*/
public void setMessageHandlerMethodFactory(MessageHandlerMethodFactory kafkaHandlerMethodFactory) {
this.messageHandlerMethodFactory = kafkaHandlerMethodFactory;
}
/**
* @return the custom {@link MessageHandlerMethodFactory} to use, if any.
*/
public MessageHandlerMethodFactory getMessageHandlerMethodFactory() {
return this.messageHandlerMethodFactory;
}
/**
* Set the {@link KafkaListenerContainerFactory} to use in case a {@link KafkaListenerEndpoint}
* is registered with a {@code null} container factory.
* <p>Alternatively, the bean name of the {@link KafkaListenerContainerFactory} to use
* can be specified for a lazy lookup, see {@link #setContainerFactoryBeanName}.
* @param containerFactory the {@link KafkaListenerContainerFactory} instance.
*/
public void setContainerFactory(KafkaListenerContainerFactory<?> containerFactory) {
this.containerFactory = containerFactory;
}
/**
* Set the bean name of the {@link KafkaListenerContainerFactory} to use in case
* a {@link KafkaListenerEndpoint} is registered with a {@code null} container factory.
* Alternatively, the container factory instance can be registered directly:
* see {@link #setContainerFactory(KafkaListenerContainerFactory)}.
* @param containerFactoryBeanName the {@link KafkaListenerContainerFactory} bean name.
* @see #setBeanFactory
*/
public void setContainerFactoryBeanName(String containerFactoryBeanName) {
this.containerFactoryBeanName = containerFactoryBeanName;
}
/**
* A {@link BeanFactory} only needs to be available in conjunction with
* {@link #setContainerFactoryBeanName}.
* @param beanFactory the {@link BeanFactory} instance.
*/
@Override
public void setBeanFactory(BeanFactory beanFactory) {
this.beanFactory = beanFactory;
}
@Override
public void afterPropertiesSet() {
registerAllEndpoints();
}
protected void registerAllEndpoints() {
synchronized (this.endpointDescriptors) {
for (KafkaListenerEndpointDescriptor descriptor : this.endpointDescriptors) {
this.endpointRegistry.registerListenerContainer(
descriptor.endpoint, resolveContainerFactory(descriptor));
}
this.startImmediately = true; // trigger immediate startup
}
}
private KafkaListenerContainerFactory<?> resolveContainerFactory(KafkaListenerEndpointDescriptor descriptor) {
if (descriptor.containerFactory != null) {
return descriptor.containerFactory;
}
else if (this.containerFactory != null) {
return this.containerFactory;
}
else if (this.containerFactoryBeanName != null) {
Assert.state(this.beanFactory != null, "BeanFactory must be set to obtain container factory by bean name");
this.containerFactory = this.beanFactory.getBean(
this.containerFactoryBeanName, KafkaListenerContainerFactory.class);
return this.containerFactory; // Consider changing this if live change of the factory is required
}
else {
throw new IllegalStateException("Could not resolve the " +
KafkaListenerContainerFactory.class.getSimpleName() + " to use for [" +
descriptor.endpoint + "] no factory was given and no default is set.");
}
}
/**
* Register a new {@link KafkaListenerEndpoint} alongside the
* {@link KafkaListenerContainerFactory} to use to create the underlying container.
* <p>The {@code factory} may be {@code null} if the default factory has to be
* used for that endpoint.
* @param endpoint the {@link KafkaListenerEndpoint} instance to register.
* @param factory the {@link KafkaListenerContainerFactory} to use.
*/
public void registerEndpoint(KafkaListenerEndpoint endpoint, KafkaListenerContainerFactory<?> factory) {
Assert.notNull(endpoint, "Endpoint must be set");
Assert.hasText(endpoint.getId(), "Endpoint id must be set");
// Factory may be null, we defer the resolution right before actually creating the container
KafkaListenerEndpointDescriptor descriptor = new KafkaListenerEndpointDescriptor(endpoint, factory);
synchronized (this.endpointDescriptors) {
if (this.startImmediately) { // Register and start immediately
this.endpointRegistry.registerListenerContainer(descriptor.endpoint,
resolveContainerFactory(descriptor), true);
}
else {
this.endpointDescriptors.add(descriptor);
}
}
}
/**
* Register a new {@link KafkaListenerEndpoint} using the default
* {@link KafkaListenerContainerFactory} to create the underlying container.
* @param endpoint the {@link KafkaListenerEndpoint} instance to register.
* @see #setContainerFactory(KafkaListenerContainerFactory)
* @see #registerEndpoint(KafkaListenerEndpoint, KafkaListenerContainerFactory)
*/
public void registerEndpoint(KafkaListenerEndpoint endpoint) {
registerEndpoint(endpoint, null);
}
private static class KafkaListenerEndpointDescriptor {
private final KafkaListenerEndpoint endpoint;
private final KafkaListenerContainerFactory<?> containerFactory;
private KafkaListenerEndpointDescriptor(KafkaListenerEndpoint endpoint, KafkaListenerContainerFactory<?> containerFactory) {
this.endpoint = endpoint;
this.containerFactory = containerFactory;
}
}
}

View File

@@ -0,0 +1,294 @@
/*
* Copyright 2014-2015 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicInteger;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.beans.BeansException;
import org.springframework.beans.factory.BeanInitializationException;
import org.springframework.beans.factory.DisposableBean;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.context.SmartLifecycle;
import org.springframework.util.Assert;
import org.springframework.util.StringUtils;
/**
* Creates the necessary {@link MessageListenerContainer} instances for the
* registered {@linkplain KafkaListenerEndpoint endpoints}. Also manages the
* lifecycle of the listener containers, in particular within the lifecycle
* of the application context.
*
* <p>Contrary to {@link MessageListenerContainer}s created manually, listener
* containers managed by registry are not beans in the application context and
* are not candidates for autowiring. Use {@link #getListenerContainers()} if
* you need to access this registry's listener containers for management purposes.
* If you need to access to a specific message listener container, use
* {@link #getListenerContainer(String)} with the id of the endpoint.
*
* @author Stephane Nicoll
* @author Juergen Hoeller
* @author Artem Bilan
* @author Gary Russell
* @since 1.4
* @see KafkaListenerEndpoint
* @see MessageListenerContainer
* @see KafkaListenerContainerFactory
*/
public class KafkaListenerEndpointRegistry implements DisposableBean, SmartLifecycle, ApplicationContextAware {
protected final Log logger = LogFactory.getLog(getClass());
private final Map<String, MessageListenerContainer> listenerContainers =
new ConcurrentHashMap<String, MessageListenerContainer>();
private int phase = Integer.MAX_VALUE;
private ConfigurableApplicationContext applicationContext;
@Override
public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
if (applicationContext instanceof ConfigurableApplicationContext) {
this.applicationContext = (ConfigurableApplicationContext) applicationContext;
}
}
/**
* Return the {@link MessageListenerContainer} with the specified id or
* {@code null} if no such container exists.
* @param id the id of the container
* @return the container or {@code null} if no container with that id exists
* @see KafkaListenerEndpoint#getId()
* @see #getListenerContainerIds()
*/
public MessageListenerContainer getListenerContainer(String id) {
Assert.hasText(id, "Container identifier must not be empty");
return this.listenerContainers.get(id);
}
/**
* Return the ids of the managed {@link MessageListenerContainer} instance(s).
* @return the ids.
* @see #getListenerContainer(String)
* @since 1.5.2
*/
public Set<String> getListenerContainerIds() {
return Collections.unmodifiableSet(this.listenerContainers.keySet());
}
/**
* @return the managed {@link MessageListenerContainer} instance(s).
*/
public Collection<MessageListenerContainer> getListenerContainers() {
return Collections.unmodifiableCollection(this.listenerContainers.values());
}
/**
* Create a message listener container for the given {@link KafkaListenerEndpoint}.
* <p>This create the necessary infrastructure to honor that endpoint
* with regards to its configuration.
* @param endpoint the endpoint to add
* @param factory the listener factory to use
* @see #registerListenerContainer(KafkaListenerEndpoint, KafkaListenerContainerFactory, boolean)
*/
public void registerListenerContainer(KafkaListenerEndpoint endpoint, KafkaListenerContainerFactory<?> factory) {
registerListenerContainer(endpoint, factory, false);
}
/**
* Create a message listener container for the given {@link KafkaListenerEndpoint}.
* <p>This create the necessary infrastructure to honor that endpoint
* with regards to its configuration.
* <p>The {@code startImmediately} flag determines if the container should be
* started immediately.
* @param endpoint the endpoint to add.
* @param factory the {@link KafkaListenerContainerFactory} to use.
* @param startImmediately start the container immediately if necessary
* @see #getListenerContainers()
* @see #getListenerContainer(String)
*/
@SuppressWarnings("unchecked")
public void registerListenerContainer(KafkaListenerEndpoint endpoint, KafkaListenerContainerFactory<?> factory,
boolean startImmediately) {
Assert.notNull(endpoint, "Endpoint must not be null");
Assert.notNull(factory, "Factory must not be null");
String id = endpoint.getId();
Assert.hasText(id, "Endpoint id must not be empty");
synchronized (this.listenerContainers) {
Assert.state(!this.listenerContainers.containsKey(id),
"Another endpoint is already registered with id '" + id + "'");
MessageListenerContainer container = createListenerContainer(endpoint, factory);
this.listenerContainers.put(id, container);
if (StringUtils.hasText(endpoint.getGroup()) && this.applicationContext != null) {
List<MessageListenerContainer> containerGroup;
if (this.applicationContext.containsBean(endpoint.getGroup())) {
containerGroup = this.applicationContext.getBean(endpoint.getGroup(), List.class);
}
else {
containerGroup = new ArrayList<MessageListenerContainer>();
this.applicationContext.getBeanFactory().registerSingleton(endpoint.getGroup(), containerGroup);
}
containerGroup.add(container);
}
if (startImmediately) {
startIfNecessary(container);
}
}
}
/**
* Create and start a new {@link MessageListenerContainer} using the specified factory.
* @param endpoint the endpoint to create a {@link MessageListenerContainer}.
* @param factory the {@link KafkaListenerContainerFactory} to use.
* @return the {@link MessageListenerContainer}.
*/
protected MessageListenerContainer createListenerContainer(KafkaListenerEndpoint endpoint,
KafkaListenerContainerFactory<?> factory) {
MessageListenerContainer listenerContainer = factory.createListenerContainer(endpoint);
if (listenerContainer instanceof InitializingBean) {
try {
((InitializingBean) listenerContainer).afterPropertiesSet();
}
catch (Exception ex) {
throw new BeanInitializationException("Failed to initialize message listener container", ex);
}
}
int containerPhase = listenerContainer.getPhase();
if (containerPhase < Integer.MAX_VALUE) { // a custom phase value
if (this.phase < Integer.MAX_VALUE && this.phase != containerPhase) {
throw new IllegalStateException("Encountered phase mismatch between container factory definitions: " +
this.phase + " vs " + containerPhase);
}
this.phase = listenerContainer.getPhase();
}
return listenerContainer;
}
@Override
public void destroy() {
for (MessageListenerContainer listenerContainer : getListenerContainers()) {
if (listenerContainer instanceof DisposableBean) {
try {
((DisposableBean) listenerContainer).destroy();
}
catch (Exception ex) {
logger.warn("Failed to destroy message listener container", ex);
}
}
}
}
// Delegating implementation of SmartLifecycle
@Override
public int getPhase() {
return this.phase;
}
@Override
public boolean isAutoStartup() {
return true;
}
@Override
public void start() {
for (MessageListenerContainer listenerContainer : getListenerContainers()) {
if (listenerContainer.isAutoStartup()) {
startIfNecessary(listenerContainer);
}
}
}
@Override
public void stop() {
for (MessageListenerContainer listenerContainer : getListenerContainers()) {
listenerContainer.stop();
}
}
@Override
public void stop(Runnable callback) {
Collection<MessageListenerContainer> listenerContainers = getListenerContainers();
AggregatingCallback aggregatingCallback = new AggregatingCallback(listenerContainers.size(), callback);
for (MessageListenerContainer listenerContainer : listenerContainers) {
listenerContainer.stop(aggregatingCallback);
}
}
@Override
public boolean isRunning() {
for (MessageListenerContainer listenerContainer : getListenerContainers()) {
if (listenerContainer.isRunning()) {
return true;
}
}
return false;
}
/**
* Start the specified {@link MessageListenerContainer} if it should be started
* on startup.
* @see MessageListenerContainer#isAutoStartup()
*/
private static void startIfNecessary(MessageListenerContainer listenerContainer) {
if (listenerContainer.isAutoStartup()) {
listenerContainer.start();
}
}
private static class AggregatingCallback implements Runnable {
private final AtomicInteger count;
private final Runnable finishCallback;
private AggregatingCallback(int count, Runnable finishCallback) {
this.count = new AtomicInteger(count);
this.finishCallback = finishCallback;
}
@Override
public void run() {
if (this.count.decrementAndGet() == 0) {
this.finishCallback.run();
}
}
}
}

View File

@@ -0,0 +1,470 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.Iterator;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
import java.util.regex.Pattern;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.kafka.clients.consumer.Consumer;
import org.apache.kafka.clients.consumer.ConsumerRebalanceListener;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.OffsetAndMetadata;
import org.apache.kafka.clients.consumer.OffsetCommitCallback;
import org.apache.kafka.common.TopicPartition;
import org.apache.kafka.common.errors.WakeupException;
import org.springframework.core.task.SimpleAsyncTaskExecutor;
import org.springframework.kafka.core.ConsumerFactory;
import org.springframework.kafka.listener.ConcurrentMessageListenerContainer.ContainerOffsetResetStrategy;
import org.springframework.scheduling.SchedulingAwareRunnable;
import org.springframework.util.Assert;
/**
* Single-threaded Message listener container using the Java {@link Consumer} supporting
* auto-partition assignment or user-configured assignment.
* <p>
* With the latter, initial partition offsets can be provided.
*
* @author Gary Russell
*
*/
public class KafkaMessageListenerContainer<K, V> extends AbstractMessageListenerContainer<K, V> implements SchedulingAwareRunnable {
private final ConsumerFactory<K, V> consumerFactory;
private final String[] topics;
private final Pattern topicPattern;
private final TopicPartition[] partitions;
private final boolean autoCommit;
private final Map<String, Map<Integer, Long>> offsets = new HashMap<String, Map<Integer, Long>>();
private final ConcurrentMap<String, ConcurrentMap<Integer, Long>> manualOffsets = new ConcurrentHashMap<>();
private final CommitCallback callback = new CommitCallback();
private Consumer<K, V> consumer;
private ContainerOffsetResetStrategy resetStrategy = ContainerOffsetResetStrategy.NONE;
private long recentOffset = 1;
private MessageListener<K, V> listener;
private AcknowledgingMessageListener<K, V> acknowledgingMessageListener;
private volatile Collection<TopicPartition> definedPartitions;
private volatile Collection<TopicPartition> assignedPartitions;
/**
* Construct an instance with the supplied configuration properties and specific
* topics/partitions - when using this constructor, a
* {@code #setResetStrategy(ContainerOffsetResetStrategy)} can be used.
* @param consumerFactory the consumer factory.
* @param topicPartitions the topics/partitions; duplicates are eliminated.
*/
public KafkaMessageListenerContainer(ConsumerFactory<K, V> consumerFactory, TopicPartition... topicPartitions) {
this(consumerFactory, null, null, topicPartitions);
}
/**
* Construct an instance with the supplied configuration properties and topics.
* When using this constructor, a
* {@code #setResetStrategy(ContainerOffsetResetStrategy)} cannot be used.
* @param consumerFactory the consumer factory.
* @param topics the topics.
*/
public KafkaMessageListenerContainer(ConsumerFactory<K, V> consumerFactory, String... topics) {
this(consumerFactory, topics, null, null);
}
/**
* Construct an instance with the supplied configuration properties and topic
* pattern. When using this constructor, a
* {@code #setResetStrategy(ContainerOffsetResetStrategy)} cannot be used.
* @param consumerFactory the consumer factory.
* @param topicPattern the topic pattern.
*/
public KafkaMessageListenerContainer(ConsumerFactory<K, V> consumerFactory, Pattern topicPattern) {
this(consumerFactory, null, topicPattern, null);
}
/**
* Construct an instance with the supplied configuration properties and topic
* pattern. Note: package protected - used by the ConcurrentMessageListenerContainer.
* @param consumerFactory the consumer factory.
* @param topics the topics.
* @param topicPattern the topic pattern.
* @param topicPartitions the topics/partitions; duplicates are eliminated.
*/
KafkaMessageListenerContainer(ConsumerFactory<K, V> consumerFactory, String[] topics, Pattern topicPattern,
TopicPartition[] topicPartitions) {
this.consumerFactory = consumerFactory;
this.topics = topics;
this.topicPattern = topicPattern;
this.partitions = topicPartitions == null ? null
: new LinkedHashSet<>(Arrays.asList(topicPartitions)).toArray(new TopicPartition[0]);
this.autoCommit = consumerFactory.isAutoCommit();
}
/**
* The initial offset reset strategy, when explicit partitions are provided.
* <ul>
* <li>NONE: No reset</li>
* <li>EARLIEST: Set to the earliest message</li>
* <li>LATEST: Set to the last message; receive new messages only</li>
* <li>RECENT: Set to a recent message based on {@link #setRecentOffset(long) recentOffset}</li>
* </ul>
*
* @param resetStrategy the {@link ContainerOffsetResetStrategy}
*/
public void setResetStrategy(ContainerOffsetResetStrategy resetStrategy) {
this.resetStrategy = resetStrategy;
}
/**
* Set the number of records back from the latest when using
* {@link ContainerOffsetResetStrategy#RECENT}.
* @param recentOffset the offset from the latest; default 1.
*/
public void setRecentOffset(long recentOffset) {
this.recentOffset = recentOffset;
}
/**
* @return the {@link TopicPartition}s currently assigned to this container,
* either explicitly or by Kafka; may be null if not assigned yet.
*/
public Collection<TopicPartition> getAssignedPartitions() {
if (this.definedPartitions != null) {
return Collections.unmodifiableCollection(this.definedPartitions);
}
else if (this.assignedPartitions != null) {
return Collections.unmodifiableCollection(this.assignedPartitions);
}
else {
return null;
}
}
@SuppressWarnings("unchecked")
@Override
protected void doStart() {
if (isRunning()) {
return;
}
Assert.state(getAckMode().equals(AckMode.MANUAL) || getAckMode().equals(AckMode.MANUAL_IMMEDIATE)
? !this.autoCommit : true,
"Consumer cannot be configured for auto commit for ackMode " + getAckMode());
setRunning(true);
Object messageListener = getMessageListener();
Assert.state(messageListener != null, "A MessageListener is required");
if (messageListener instanceof AcknowledgingMessageListener) {
this.acknowledgingMessageListener = (AcknowledgingMessageListener<K, V>) messageListener;
}
else if (messageListener instanceof MessageListener) {
this.listener = (MessageListener<K, V>) messageListener;
}
else {
throw new IllegalStateException("messageListener must be 'MessageListener' "
+ "or 'AcknowledgingMessageListener', not " + messageListener.getClass().getName());
}
Consumer<K, V> consumer = this.consumerFactory.createConsumer();
ConsumerRebalanceListener rebalanceListener = new ConsumerRebalanceListener() {
@Override
public void onPartitionsRevoked(Collection<TopicPartition> partitions) {
logger.info("partitions revoked:" + partitions);
}
@Override
public void onPartitionsAssigned(Collection<TopicPartition> partitions) {
assignedPartitions = partitions;
logger.info("partitions assigned:" + partitions);
}
};
if (this.partitions == null) {
if (topicPattern != null) {
consumer.subscribe(topicPattern, rebalanceListener);
}
else {
consumer.subscribe(Arrays.asList(topics), rebalanceListener);
}
}
else {
List<TopicPartition> topicPartitions = Arrays.asList(partitions);
this.definedPartitions = topicPartitions;
consumer.assign(topicPartitions);
}
this.consumer = consumer;
if (getTaskExecutor() == null) {
setTaskExecutor(
new SimpleAsyncTaskExecutor(getBeanName() == null ? "kafka-" : (getBeanName() + "-kafka-")));
}
getTaskExecutor().execute(this);
}
@Override
protected void doStop() {
if (isRunning()) {
setRunning(false);
this.consumer.wakeup();
}
}
@Override
public void run() {
int count = 0;
long last = System.currentTimeMillis();
long now;
if (isRunning() && this.definedPartitions != null) {
initPartitionsIfNeeded();
}
final AckMode ackMode = getAckMode();
while (isRunning()) {
try {
if (logger.isTraceEnabled()) {
logger.trace("Polling...");
}
ConsumerRecords<K, V> records = consumer.poll(getPollTimeout());
if (records != null) {
count += records.count();
if (logger.isDebugEnabled()) {
logger.debug("Received: " + records.count() + " records");
}
Iterator<ConsumerRecord<K, V>> iterator = records.iterator();
while (iterator.hasNext()) {
final ConsumerRecord<K, V> record = iterator.next();
invokeListener(record);
if (!this.autoCommit && ackMode.equals(AckMode.RECORD)) {
this.consumer.commitAsync(
Collections.singletonMap(new TopicPartition(record.topic(), record.partition()),
new OffsetAndMetadata(record.offset())), callback);
}
}
if (!this.autoCommit) {
if (ackMode.equals(AckMode.BATCH)) {
if (!records.isEmpty()) {
this.consumer.commitAsync(callback);
}
}
else {
if (!ackMode.equals(AckMode.MANUAL)) {
updatePendingOffsets(records);
}
boolean countExceeded = count >= getAckCount();
if (ackMode.equals(AckMode.COUNT) && countExceeded) {
commitIfNecessary();
count = 0;
}
else {
now = System.currentTimeMillis();
boolean elapsed = now - last > getAckTime();
if (ackMode.equals(AckMode.TIME) && elapsed) {
commitIfNecessary();
last = now;
}
else if ((ackMode.equals(AckMode.COUNT_TIME) || ackMode.equals(AckMode.MANUAL))
&& (elapsed || countExceeded)) {
commitIfNecessary();
last = now;
count = 0;
}
}
}
}
}
else {
if (logger.isDebugEnabled()) {
logger.debug("No records");
}
}
}
catch (WakeupException e) {
;
}
catch (Exception e) {
if (getErrorHandler() != null) {
getErrorHandler().handle(e, null);
}
else {
logger.error("Container exception", e);
}
}
}
if (this.offsets.size() > 0) {
commitIfNecessary();
}
try {
this.consumer.unsubscribe();
}
catch (WakeupException e) {
;
}
this.consumer.close();
if (logger.isInfoEnabled()) {
logger.info("Consumer stopped");
}
}
private void invokeListener(final ConsumerRecord<K, V> record) {
try {
if (this.acknowledgingMessageListener != null) {
this.acknowledgingMessageListener.onMessage(record, new Acknowledgment() {
@Override
public void acknowledge() {
if (getAckMode().equals(AckMode.MANUAL)) {
updateManualOffset(record);
}
else if (getAckMode().equals(AckMode.MANUAL_IMMEDIATE)) {
updateManualOffset(record);
consumer.wakeup();
}
else {
throw new IllegalStateException("AckMode must be MANUAL or MANUAL_IMMEDIATE "
+ "for manual acks");
}
}
});
}
else {
this.listener.onMessage(record);
}
}
catch (Exception e) {
if (getErrorHandler() != null) {
getErrorHandler().handle(e, record);
}
else {
logger.error("Listener threw an exception and no error handler for " + record, e);
}
}
}
private void initPartitionsIfNeeded() {
/*
* Note: initial position setting is only supported with explicit topic assignment.
* When using auto assignment (subscribe), the ConsumerRebalanceListener is not
* called until we poll() the consumer.
*/
if (this.resetStrategy.equals(ContainerOffsetResetStrategy.EARLIEST)) {
this.consumer.seekToBeginning(
this.definedPartitions.toArray(new TopicPartition[this.definedPartitions.size()]));
}
else if (this.resetStrategy.equals(ContainerOffsetResetStrategy.LATEST)) {
this.consumer.seekToEnd(
this.definedPartitions.toArray(new TopicPartition[this.definedPartitions.size()]));
}
else if (this.resetStrategy.equals(ContainerOffsetResetStrategy.RECENT)) {
this.consumer.seekToEnd(
this.definedPartitions.toArray(new TopicPartition[this.definedPartitions.size()]));
for (TopicPartition topicPartition : this.definedPartitions) {
long newOffset = this.consumer.position(topicPartition) - this.recentOffset;
this.consumer.seek(topicPartition, newOffset);
if (logger.isDebugEnabled()) {
logger.debug("Reset " + topicPartition + " to offset " + newOffset);
}
}
}
}
private void updatePendingOffsets(ConsumerRecords<K, V> records) {
for (ConsumerRecord<K, V> record : records) {
if (!this.offsets.containsKey(record.topic())) {
this.offsets.put(record.topic(), new HashMap<Integer, Long>());
}
this.offsets.get(record.topic()).put(record.partition(), record.offset());
}
}
private void updateManualOffset(ConsumerRecord<K, V> record) {
if (!this.manualOffsets.containsKey(record.topic())) {
this.manualOffsets.putIfAbsent(record.topic(), new ConcurrentHashMap<Integer, Long>());
}
this.manualOffsets.get(record.topic()).put(record.partition(), record.offset());
}
private void commitIfNecessary() {
Map<TopicPartition, OffsetAndMetadata> commits = new HashMap<>();
if (AckMode.MANUAL.equals(getAckMode())) {
for (Entry<String, ConcurrentMap<Integer, Long>> entry : this.manualOffsets.entrySet()) {
Iterator<Entry<Integer, Long>> iterator = entry.getValue().entrySet().iterator();
while (iterator.hasNext()) {
Entry<Integer, Long> offset = iterator.next();
commits.put(new TopicPartition(entry.getKey(), offset.getKey()),
new OffsetAndMetadata(offset.getValue()));
}
}
}
else {
for (Entry<String, Map<Integer, Long>> entry : this.offsets.entrySet()) {
for (Entry<Integer, Long> offset : entry.getValue().entrySet()) {
commits.put(new TopicPartition(entry.getKey(), offset.getKey()),
new OffsetAndMetadata(offset.getValue()));
}
}
}
this.offsets.clear();
if (logger.isDebugEnabled()) {
logger.debug("Committing: " + commits);
}
if (!commits.isEmpty()) {
this.consumer.commitAsync(commits, this.callback);
}
}
@Override
public boolean isLongLived() {
return true;
}
private static final class CommitCallback implements OffsetCommitCallback {
private final Log logger = LogFactory.getLog(OffsetCommitCallback.class);
@Override
public void onComplete(Map<TopicPartition, OffsetAndMetadata> offsets, Exception exception) {
if (exception != null) {
logger.error("Commit failed for " + offsets, exception);
}
else if (logger.isDebugEnabled()) {
logger.debug("Commits for " + offsets + " completed");
}
}
}
}

View File

@@ -0,0 +1,35 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
import org.springframework.kafka.core.KafkaException;
/**
* @author Gary Russell
*
*/
@SuppressWarnings("serial")
public class ListenerExecutionFailedException extends KafkaException {
public ListenerExecutionFailedException(String message) {
super(message);
}
public ListenerExecutionFailedException(String message, Throwable cause) {
super(message, cause);
}
}

View File

@@ -0,0 +1,39 @@
/*
* Copyright 2015-2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.springframework.util.ObjectUtils;
/**
* @author Marius Bogoevici
* @author Gary Russell
*/
public class LoggingErrorHandler implements ErrorHandler {
private static final Log log = LogFactory.getLog(LoggingErrorHandler.class);
@Override
public void handle(Exception thrownException, ConsumerRecord<?, ?> record) {
log.error("Error while processing: " + ObjectUtils.nullSafeToString(record), thrownException);
}
}

View File

@@ -0,0 +1,36 @@
/*
* Copyright 2015-2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
import org.apache.kafka.clients.consumer.ConsumerRecord;
/**
* Listener for handling incoming Kafka messages
*
* @author Marius Bogoevici
* @author Gary Russell
*/
public interface MessageListener<K, V> {
/**
* Executes when a {@link ConsumerRecord} is received.
* @param record the ConsumerRecord to be processed.
*/
void onMessage(ConsumerRecord<K, V> record);
}

View File

@@ -0,0 +1,38 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
import org.springframework.context.SmartLifecycle;
/**
* Internal abstraction used by the framework representing a message
* listener container. Not meant to be implemented externally.
*
* @author Stephane Nicoll
* @author Gary Russell
* @since 1.4
*/
public interface MessageListenerContainer extends SmartLifecycle {
/**
* Setup the message listener to use. Throws an {@link IllegalArgumentException}
* if that message listener type is not supported.
* @param messageListener the {@code object} to wrapped to the {@code MessageListener}.
*/
void setupMessageListener(Object messageListener);
}

View File

@@ -0,0 +1,120 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
import java.lang.reflect.Method;
import org.springframework.kafka.listener.adapter.HandlerAdapter;
import org.springframework.kafka.listener.adapter.MessagingMessageListenerAdapter;
import org.springframework.messaging.handler.annotation.support.MessageHandlerMethodFactory;
import org.springframework.messaging.handler.invocation.InvocableHandlerMethod;
import org.springframework.util.Assert;
/**
* A {@link KafkaListenerEndpoint} providing the method to invoke to process
* an incoming message for this endpoint.
*
* @author Stephane Nicoll
* @author Artem Bilan
* @since 1.4
*/
public class MethodKafkaListenerEndpoint<K, V> extends AbstractKafkaListenerEndpoint<K, V> {
private Object bean;
private Method method;
private MessageHandlerMethodFactory messageHandlerMethodFactory;
/**
* Set the object instance that should manage this endpoint.
* @param bean the target bean instance.
*/
public void setBean(Object bean) {
this.bean = bean;
}
public Object getBean() {
return this.bean;
}
/**
* Set the method to invoke to process a message managed by this endpoint.
* @param method the target method for the {@link #bean}.
*/
public void setMethod(Method method) {
this.method = method;
}
public Method getMethod() {
return this.method;
}
/**
* Set the {@link MessageHandlerMethodFactory} to use to build the
* {@link InvocableHandlerMethod} responsible to manage the invocation
* of this endpoint.
* @param messageHandlerMethodFactory the {@link MessageHandlerMethodFactory} instance.
*/
public void setMessageHandlerMethodFactory(MessageHandlerMethodFactory messageHandlerMethodFactory) {
this.messageHandlerMethodFactory = messageHandlerMethodFactory;
}
/**
* @return the messageHandlerMethodFactory
*/
protected MessageHandlerMethodFactory getMessageHandlerMethodFactory() {
return messageHandlerMethodFactory;
}
@Override
protected MessagingMessageListenerAdapter<K, V> createMessageListener(MessageListenerContainer container) {
Assert.state(this.messageHandlerMethodFactory != null,
"Could not create message listener - MessageHandlerMethodFactory not set");
MessagingMessageListenerAdapter<K, V> messageListener = createMessageListenerInstance();
messageListener.setHandlerMethod(configureListenerAdapter(messageListener));
return messageListener;
}
/**
* Create a {@link HandlerAdapter} for this listener adapter.
* @param messageListener the listener adapter.
* @return the handler adapter.
*/
protected HandlerAdapter configureListenerAdapter(MessagingMessageListenerAdapter<K, V> messageListener) {
InvocableHandlerMethod invocableHandlerMethod =
this.messageHandlerMethodFactory.createInvocableHandlerMethod(getBean(), getMethod());
return new HandlerAdapter(invocableHandlerMethod);
}
/**
* Create an empty {@link MessagingMessageListenerAdapter} instance.
* @return the {@link MessagingMessageListenerAdapter} instance.
*/
protected MessagingMessageListenerAdapter<K, V> createMessageListenerInstance() {
return new MessagingMessageListenerAdapter<K, V>();
}
@Override
protected StringBuilder getEndpointDescription() {
return super.getEndpointDescription()
.append(" | bean='").append(this.bean).append("'")
.append(" | method='").append(this.method).append("'");
}
}

View File

@@ -0,0 +1,53 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.List;
import org.springframework.kafka.listener.adapter.DelegatingInvocableHandler;
import org.springframework.kafka.listener.adapter.HandlerAdapter;
import org.springframework.kafka.listener.adapter.MessagingMessageListenerAdapter;
import org.springframework.messaging.handler.invocation.InvocableHandlerMethod;
/**
* @author Gary Russell
*
*/
public class MultiMethodKafkaListenerEndpoint<K, V> extends MethodKafkaListenerEndpoint<K, V> {
private final List<Method> methods;
private DelegatingInvocableHandler delegatingHandler;
public MultiMethodKafkaListenerEndpoint(List<Method> methods, Object bean) {
this.methods = methods;
setBean(bean);
}
@Override
protected HandlerAdapter configureListenerAdapter(MessagingMessageListenerAdapter<K, V> messageListener) {
List<InvocableHandlerMethod> invocableHandlerMethods = new ArrayList<InvocableHandlerMethod>();
for (Method method : this.methods) {
invocableHandlerMethods.add(getMessageHandlerMethodFactory()
.createInvocableHandlerMethod(getBean(), method));
}
this.delegatingHandler = new DelegatingInvocableHandler(invocableHandlerMethods, getBean());
return new HandlerAdapter(this.delegatingHandler);
}
}

View File

@@ -0,0 +1,85 @@
/*
* Copyright 2014 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener.adapter;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.springframework.kafka.listener.AcknowledgingMessageListener;
import org.springframework.kafka.listener.MessageListener;
/**
* An abstract {@link MessageListener} adapter providing the necessary infrastructure
* to extract the payload of a {@link org.springframework.messaging.Message}.
*
* @author Stephane Nicoll
* @author Gary Russell
* @see MessageListener
* @see AcknowledgingMessageListener
*/
public abstract class AbstractAdaptableMessageListener<K, V> implements MessageListener<K, V>,
AcknowledgingMessageListener<K, V> {
/** Logger available to subclasses */
protected final Log logger = LogFactory.getLog(getClass());
/**
* Kafka {@link MessageListener} entry point.
* <p>
* Delegates the message to the target listener method, with appropriate conversion of the message argument. In case
* of an exception, the {@link #handleListenerException(Throwable)} method will be invoked.
* <p>
* @param record the incoming Kafka {@link ConsumerRecord}.
* @see #handleListenerException
* @see #onMessage(ConsumerRecord, org.springframework.kafka.listener.Acknowledgment)
*/
@Override
public void onMessage(ConsumerRecord<K, V> record) {
try {
onMessage(record, null);
}
catch (Exception ex) {
handleListenerException(ex);
}
}
/**
* Handle the given exception that arose during listener execution.
* The default implementation logs the exception at error level.
* <p>
* This method only applies when using a Kafka {@link MessageListener}. With
* {@link AcknowledgingMessageListener}, exceptions get handled by the
* caller instead.
* @param ex the exception to handle
* @see #onMessage(ConsumerRecord)
*/
protected void handleListenerException(Throwable ex) {
logger.error("Listener execution failed", ex);
}
/**
* Extract the message body from the given Kafka message.
* @param record the Kafka <code>Message</code>
* @return the content of the message, to be passed into the listener method as argument
*/
protected Object extractMessage(ConsumerRecord<K, V> record) {
return record.value();
}
}

View File

@@ -0,0 +1,147 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener.adapter;
import java.lang.annotation.Annotation;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
import org.springframework.core.MethodParameter;
import org.springframework.kafka.core.KafkaException;
import org.springframework.messaging.Message;
import org.springframework.messaging.handler.annotation.Payload;
import org.springframework.messaging.handler.invocation.InvocableHandlerMethod;
/**
* Delegates to an {@link InvocableHandlerMethod} based on the message payload type.
* Matches a single, non-annotated parameter or one that is annotated with {@link Payload}.
* Matches must be unambiguous.
*
* @author Gary Russell
*
*/
public class DelegatingInvocableHandler {
private final List<InvocableHandlerMethod> handlers;
private final ConcurrentMap<Class<?>, InvocableHandlerMethod> cachedHandlers =
new ConcurrentHashMap<Class<?>, InvocableHandlerMethod>();
private final Object bean;
/**
* Construct an instance with the supplied handlers for the bean.
* @param handlers the handlers.
* @param bean the bean.
*/
public DelegatingInvocableHandler(List<InvocableHandlerMethod> handlers, Object bean) {
this.handlers = new ArrayList<InvocableHandlerMethod>(handlers);
this.bean = bean;
}
/**
* @return the bean
*/
public Object getBean() {
return bean;
}
/**
* Invoke the method with the given message.
* @param message the message.
* @param providedArgs additional arguments.
* @throws Exception raised if no suitable argument resolver can be found,
* or the method raised an exception.
* @return the result of the invocation.
*/
public Object invoke(Message<?> message, Object... providedArgs) throws Exception {
Class<? extends Object> payloadClass = message.getPayload().getClass();
InvocableHandlerMethod handler = getHandlerForPayload(payloadClass);
return handler.invoke(message, providedArgs);
}
/**
* @param payloadClass the payload class.
* @return the handler.
*/
protected InvocableHandlerMethod getHandlerForPayload(Class<? extends Object> payloadClass) {
InvocableHandlerMethod handler = this.cachedHandlers.get(payloadClass);
if (handler == null) {
handler = findHandlerForPayload(payloadClass);
if (handler == null) {
throw new KafkaException("No method found for " + payloadClass);
}
this.cachedHandlers.putIfAbsent(payloadClass, handler);//NOSONAR
}
return handler;
}
protected InvocableHandlerMethod findHandlerForPayload(Class<? extends Object> payloadClass) {
InvocableHandlerMethod result = null;
for (InvocableHandlerMethod handler : this.handlers) {
if (matchHandlerMethod(payloadClass, handler)) {
if (result != null) {
throw new KafkaException("Ambiguous methods for payload type: " + payloadClass + ": " +
result.getMethod().getName() + " and " + handler.getMethod().getName());
}
result = handler;
}
}
return result;
}
protected boolean matchHandlerMethod(Class<? extends Object> payloadClass, InvocableHandlerMethod handler) {
Method method = handler.getMethod();
Annotation[][] parameterAnnotations = method.getParameterAnnotations();
// Single param; no annotation or @Payload
if (parameterAnnotations.length == 1) {
MethodParameter methodParameter = new MethodParameter(method, 0);
if (methodParameter.getParameterAnnotations().length == 0 || methodParameter.hasParameterAnnotation(Payload.class)) {
if (methodParameter.getParameterType().isAssignableFrom(payloadClass)) {
return true;
}
}
}
boolean foundCandidate = false;
for (int i = 0; i < parameterAnnotations.length; i++) {
MethodParameter methodParameter = new MethodParameter(method, i);
if (methodParameter.getParameterAnnotations().length == 0 || methodParameter.hasParameterAnnotation(Payload.class)) {
if (methodParameter.getParameterType().isAssignableFrom(payloadClass)) {
if (foundCandidate) {
throw new KafkaException("Ambiguous payload parameter for " + method.toGenericString());
}
foundCandidate = true;
}
}
}
return foundCandidate;
}
/**
* Return a string representation of the method that will be invoked for this payload.
* @param payload the payload.
* @return the method name.
*/
public String getMethodNameFor(Object payload) {
InvocableHandlerMethod handlerForPayload = getHandlerForPayload(payload.getClass());
return handlerForPayload == null ? "no match" : handlerForPayload.getMethod().toGenericString();//NOSONAR
}
}

View File

@@ -0,0 +1,74 @@
/*
* Copyright 2015 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener.adapter;
import org.springframework.messaging.Message;
import org.springframework.messaging.handler.invocation.InvocableHandlerMethod;
/**
* A wrapper for either an {@link InvocableHandlerMethod} or
* {@link DelegatingInvocableHandler}. All methods delegate to the
* underlying handler.
*
* @author Gary Russell
* @since 1.5
*
*/
public class HandlerAdapter {
private final InvocableHandlerMethod invokerHandlerMethod;
private final DelegatingInvocableHandler delegatingHandler;
public HandlerAdapter(InvocableHandlerMethod invokerHandlerMethod) {
this.invokerHandlerMethod = invokerHandlerMethod;
this.delegatingHandler = null;
}
public HandlerAdapter(DelegatingInvocableHandler delegatingHandler) {
this.invokerHandlerMethod = null;
this.delegatingHandler = delegatingHandler;
}
public Object invoke(Message<?> message, Object... providedArgs) throws Exception {
if (this.invokerHandlerMethod != null) {
return this.invokerHandlerMethod.invoke(message, providedArgs);
}
else {
return this.delegatingHandler.invoke(message, providedArgs);
}
}
public String getMethodAsString(Object payload) {
if (this.invokerHandlerMethod != null) {
return this.invokerHandlerMethod.getMethod().toGenericString();
}
else {
return this.delegatingHandler.getMethodNameFor(payload);
}
}
public Object getBean() {
if (this.invokerHandlerMethod != null) {
return this.invokerHandlerMethod.getBean();
}
else {
return this.delegatingHandler.getBean();
}
}
}

View File

@@ -0,0 +1,119 @@
/*
* Copyright 2002-2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener.adapter;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.springframework.kafka.listener.Acknowledgment;
import org.springframework.kafka.listener.ListenerExecutionFailedException;
import org.springframework.kafka.support.converter.MessageConverter;
import org.springframework.kafka.support.converter.MessagingMessageConverter;
import org.springframework.messaging.Message;
import org.springframework.messaging.MessagingException;
import org.springframework.messaging.converter.MessageConversionException;
/**
* A {@link org.springframework.kafka.listener.MessageListener MessageListener}
* adapter that invokes a configurable {@link HandlerAdapter}.
*
* <p>Wraps the incoming Kafka Message to Spring's {@link Message} abstraction.
*
* <p>The original {@link ConsumerRecord} and
* the {@link Acknowledgment} are provided as additional arguments so that these can
* be injected as method arguments if necessary.
*
* @author Stephane Nicoll
* @author Gary Russell
* @author Artem Bilan
* @since 1.4
*/
public class MessagingMessageListenerAdapter<K, V> extends AbstractAdaptableMessageListener<K, V> {
private HandlerAdapter handlerMethod;
private MessageConverter<K, V> messageConverter = new MessagingMessageConverter<>();
/**
* Set the {@link HandlerAdapter} to use to invoke the method
* processing an incoming {@link ConsumerRecord}.
* @param handlerMethod {@link HandlerAdapter} instance.
*/
public void setHandlerMethod(HandlerAdapter handlerMethod) {
this.handlerMethod = handlerMethod;
}
/**
* Set the MessageConverter
* @param messageConverter the converter.
*/
public void setMessageConverter(MessageConverter<K, V> messageConverter) {
this.messageConverter = messageConverter;
}
/**
* @return the {@link MessagingMessageConverter} for this listener,
* being able to convert {@link org.springframework.messaging.Message}.
*/
protected final MessageConverter<K, V> getMessageConverter() {
return this.messageConverter;
}
@Override
public void onMessage(ConsumerRecord<K, V> record, Acknowledgment acknowledgment) {
Message<?> message = toMessagingMessage(record, acknowledgment);
if (logger.isDebugEnabled()) {
logger.debug("Processing [" + message + "]");
}
invokeHandler(record, acknowledgment, message);
}
protected Message<?> toMessagingMessage(ConsumerRecord<K, V> record, Acknowledgment acknowledgment) {
return getMessageConverter().toMessage(record, acknowledgment);
}
/**
* Invoke the handler, wrapping any exception to a {@link ListenerExecutionFailedException}
* with a dedicated error message.
*/
private Object invokeHandler(ConsumerRecord<K, V> record, Acknowledgment acknowledgment, Message<?> message) {
try {
return this.handlerMethod.invoke(message, record, acknowledgment);
}
catch (org.springframework.messaging.converter.MessageConversionException ex) {
throw new ListenerExecutionFailedException(createMessagingErrorMessage("Listener method could not " +
"be invoked with the incoming message", message.getPayload()),
new MessageConversionException("Cannot handle message", ex));
}
catch (MessagingException ex) {
throw new ListenerExecutionFailedException(createMessagingErrorMessage("Listener method could not " +
"be invoked with the incoming message", message.getPayload()), ex);
}
catch (Exception ex) {
throw new ListenerExecutionFailedException("Listener method '" +
this.handlerMethod.getMethodAsString(message.getPayload()) + "' threw exception", ex);
}
}
private String createMessagingErrorMessage(String description, Object payload) {
return description + "\n"
+ "Endpoint handler details:\n"
+ "Method [" + this.handlerMethod.getMethodAsString(payload) + "]\n"
+ "Bean [" + this.handlerMethod.getBean() + "]";
}
}

View File

@@ -0,0 +1,4 @@
/**
* Provides classes for adapting listeners.
*/
package org.springframework.kafka.listener.adapter;

View File

@@ -0,0 +1,4 @@
/**
* Package for kafka listeners
*/
package org.springframework.kafka.listener;

View File

@@ -0,0 +1,4 @@
/**
* Base package for kafka
*/
package org.springframework.kafka;

View File

@@ -0,0 +1,38 @@
/*
* Copyright 2014-2015 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.support;
/**
* @author Artem Bilan
* @author Marius Bogoevici
* @since 1.0
*/
public abstract class KafkaHeaders {
private static final String PREFIX = "kafka_";
public static final String TOPIC = PREFIX + "topic";
public static final String MESSAGE_KEY = PREFIX + "messageKey";
public static final String PARTITION_ID = PREFIX + "partitionId";
public static final String OFFSET = PREFIX + "offset";
public static final String ACKNOWLEDGMENT = PREFIX + "acknowledgment";
}

View File

@@ -0,0 +1,31 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.support.converter;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.springframework.kafka.listener.Acknowledgment;
import org.springframework.messaging.Message;
/**
* @author Gary Russell
*
*/
public interface MessageConverter<K, V> {
Message<?> toMessage(ConsumerRecord<K, V> record, Acknowledgment acknowledgment);
}

View File

@@ -0,0 +1,98 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.support.converter;
import java.util.Map;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.springframework.kafka.listener.Acknowledgment;
import org.springframework.kafka.support.KafkaHeaders;
import org.springframework.messaging.Message;
import org.springframework.messaging.MessageHeaders;
import org.springframework.messaging.support.MessageBuilder;
/**
* @author Marius Bogoevici
* @author Gary Russell
*/
public class MessagingMessageConverter<K, V> implements MessageConverter<K, V> {
private boolean generateMessageId = false;
private boolean generateTimestamp = false;
/**
* Generate {@link Message} {@code ids} for produced messages. If set to {@code false},
* will try to use a default value. By default set to {@code false}.
* @param generateMessageId true if a message id should be generated
*/
public void setGenerateMessageId(boolean generateMessageId) {
this.generateMessageId = generateMessageId;
}
/**
* Generate {@code timestamp} for produced messages. If set to {@code false}, -1 is
* used instead. By default set to {@code false}.
* @param generateTimestamp true if a timestamp should be generated
* @since 1.1
*/
public void setGenerateTimestamp(boolean generateTimestamp) {
this.generateTimestamp = generateTimestamp;
}
@Override
public Message<?> toMessage(ConsumerRecord<K, V> record, Acknowledgment acknowledgment) {
KafkaMessageHeaders kafkaMessageHeaders = new KafkaMessageHeaders(generateMessageId, generateTimestamp);
Map<String, Object> rawHeaders = kafkaMessageHeaders.getRawHeaders();
rawHeaders.put(KafkaHeaders.MESSAGE_KEY, record.key());
rawHeaders.put(KafkaHeaders.TOPIC, record.topic());
rawHeaders.put(KafkaHeaders.PARTITION_ID, record.partition());
rawHeaders.put(KafkaHeaders.OFFSET, record.offset());
if (acknowledgment != null) {
rawHeaders.put(KafkaHeaders.ACKNOWLEDGMENT, acknowledgment);
}
return MessageBuilder.createMessage(extractAndConvertValue(record), kafkaMessageHeaders);
}
/**
* Subclasses can convert the value; by default, it's returned as provided by Kafka.
* @param record the record.
* @return the value.
*/
protected V extractAndConvertValue(ConsumerRecord<K, V> record) {
return record.value();
}
@SuppressWarnings("serial")
private static class KafkaMessageHeaders extends MessageHeaders {
public KafkaMessageHeaders(boolean generateId, boolean generateTimestamp) {
super(null, generateId ? null : ID_VALUE_NONE, generateTimestamp ? null : -1L);
}
@Override
public Map<String, Object> getRawHeaders() {
return super.getRawHeaders();
}
}
}

View File

@@ -0,0 +1,4 @@
/**
* Package for kafka converters
*/
package org.springframework.kafka.support.converter;

View File

@@ -0,0 +1,4 @@
/**
* Package for kafka support
*/
package org.springframework.kafka.support;

View File

@@ -0,0 +1,233 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.annotation;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertTrue;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.junit.ClassRule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.config.SimpleKafkaListenerContainerFactory;
import org.springframework.kafka.core.ConsumerFactory;
import org.springframework.kafka.core.DefaultKafkaConsumerFactory;
import org.springframework.kafka.core.DefaultKafkaProducerFactory;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.core.ProducerFactory;
import org.springframework.kafka.listener.AbstractMessageListenerContainer.AckMode;
import org.springframework.kafka.listener.Acknowledgment;
import org.springframework.kafka.listener.ConcurrentMessageListenerContainer;
import org.springframework.kafka.listener.KafkaListenerContainerFactory;
import org.springframework.kafka.listener.KafkaListenerEndpointRegistry;
import org.springframework.kafka.listener.KafkaMessageListenerContainer;
import org.springframework.kafka.listener.MessageListenerContainer;
import org.springframework.kafka.rule.KafkaEmbedded;
import org.springframework.kafka.support.KafkaHeaders;
import org.springframework.messaging.handler.annotation.Header;
import org.springframework.messaging.handler.annotation.Payload;
import org.springframework.test.annotation.DirtiesContext;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
/**
* @author Gary Russell
*
*/
@ContextConfiguration
@RunWith(SpringJUnit4ClassRunner.class)
@DirtiesContext
public class EnableKafkaIntegrationTests {
@ClassRule
public static KafkaEmbedded embeddedKafka = new KafkaEmbedded(1, true, "annotated1", "annotated2", "annotated3",
"annotated4");
@Autowired
public Listener listener;
@Autowired
public KafkaTemplate<Integer, String> template;
@Autowired
public KafkaListenerEndpointRegistry registry;
@Test
public void testSimple() throws Exception {
waitListening("foo");
template.convertAndSend("annotated1", 0, "foo");
assertTrue(this.listener.latch1.await(10, TimeUnit.SECONDS));
waitListening("bar");
template.convertAndSend("annotated2", 0, "foo");
assertTrue(this.listener.latch2.await(10, TimeUnit.SECONDS));
assertNotNull(this.listener.partition);
waitListening("baz");
template.convertAndSend("annotated3", 0, "foo");
assertTrue(this.listener.latch3.await(10, TimeUnit.SECONDS));
assertEquals("foo", this.listener.record.value());
waitListening("qux");
template.convertAndSend("annotated4", 0, "foo");
assertTrue(this.listener.latch4.await(10, TimeUnit.SECONDS));
assertEquals("foo", this.listener.record.value());
assertNotNull(this.listener.ack);
this.listener.ack.acknowledge();
}
private void waitListening(String id) throws InterruptedException {
MessageListenerContainer container = registry.getListenerContainer(id);
@SuppressWarnings("unchecked")
KafkaMessageListenerContainer<Integer, String> kmlc =
((ConcurrentMessageListenerContainer<Integer, String>) container).getContainers().get(0);
int n = 0;
while (n++ < 6000 && (kmlc.getAssignedPartitions() == null || kmlc.getAssignedPartitions().size() == 0)) {
Thread.sleep(100);
}
assertTrue(kmlc.getAssignedPartitions().size() > 0);
}
@Configuration
@EnableKafka
public static class Config {
@Bean
public KafkaListenerContainerFactory<ConcurrentMessageListenerContainer<Integer, String>>
kafkaListenerContainerFactory() {
SimpleKafkaListenerContainerFactory<Integer, String> factory = new SimpleKafkaListenerContainerFactory<>();
factory.setConsumerFactory(consumerFactory());
return factory;
}
@Bean
public KafkaListenerContainerFactory<ConcurrentMessageListenerContainer<Integer, String>>
kafkaManualAckListenerContainerFactory() {
SimpleKafkaListenerContainerFactory<Integer, String> factory = new SimpleKafkaListenerContainerFactory<>();
factory.setConsumerFactory(manualConsumerFactory());
factory.setAckMode(AckMode.MANUAL_IMMEDIATE);
return factory;
}
@Bean
public ConsumerFactory<Integer, String> consumerFactory() {
return new DefaultKafkaConsumerFactory<>(consumerConfigs());
}
@Bean
public ConsumerFactory<Integer, String> manualConsumerFactory() {
Map<String, Object> configs = consumerConfigs();
configs.put("enable.auto.commit", "false");
return new DefaultKafkaConsumerFactory<>(configs);
}
@Bean
public Map<String, Object> consumerConfigs() {
Map<String, Object> props = new HashMap<>();
props.put("bootstrap.servers", embeddedKafka.getBrokersAsString());
// props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "testAnnot");
props.put("enable.auto.commit", true);
props.put("auto.commit.interval.ms", "100");
props.put("session.timeout.ms", "15000");
props.put("key.deserializer", "org.apache.kafka.common.serialization.IntegerDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
return props;
}
@Bean
public Listener listener() {
return new Listener();
}
@Bean
public ProducerFactory<Integer, String> producerFactory() {
return new DefaultKafkaProducerFactory<>(producerConfigs());
}
@Bean
public Map<String, Object> producerConfigs() {
Map<String, Object> props = new HashMap<>();
props.put("bootstrap.servers", embeddedKafka.getBrokersAsString());
// props.put("bootstrap.servers", "localhost:9092");
props.put("retries", 0);
props.put("batch.size", 16384);
props.put("linger.ms", 1);
props.put("buffer.memory", 33554432);
props.put("key.serializer", "org.apache.kafka.common.serialization.IntegerSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
return props;
}
@Bean
public KafkaTemplate<Integer, String> kafkaTemplate() {
return new KafkaTemplate<Integer, String>(producerFactory());
}
}
public static class Listener {
private final CountDownLatch latch1 = new CountDownLatch(1);
private final CountDownLatch latch2 = new CountDownLatch(1);
private final CountDownLatch latch3 = new CountDownLatch(1);
private final CountDownLatch latch4 = new CountDownLatch(1);
private volatile Integer partition;
private volatile ConsumerRecord<?, ?> record;
private volatile Acknowledgment ack;
@KafkaListener(id="foo", topics = "annotated1")
public void listen1(String foo) {
this.latch1.countDown();
}
@KafkaListener(id="bar", topicPattern = "annotated2")
public void listen2(@Payload String foo, @Header(KafkaHeaders.PARTITION_ID) int partitionHeader) {
this.partition = partitionHeader;
this.latch2.countDown();
}
@KafkaListener(id="baz", topicPartitions = @TopicPartition(topic = "annotated3", partition="0"))
public void listen3(ConsumerRecord<?, ?> record) {
this.record = record;
this.latch3.countDown();
}
@KafkaListener(id="qux", topics = "annotated4", containerFactory = "kafkaManualAckListenerContainerFactory")
public void listen4(@Payload String foo, Acknowledgment ack) {
this.ack = ack;
this.latch4.countDown();
}
}
}

View File

@@ -0,0 +1,101 @@
/*
* Copyright 2015-2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.core;
import org.springframework.util.Assert;
import kafka.cluster.BrokerEndPoint;
/**
* Encapsulates the address of a Kafka broker.
*
* @author Marius Bogoevici
* @author Gary Russell
*/
public class BrokerAddress {
public static final int DEFAULT_PORT = 9092;
private final String host;
private final int port;
public BrokerAddress(String host, int port) {
Assert.hasText(host, "Host cannot be empty");
this.host = host;
this.port = port;
}
public BrokerAddress(String host) {
this(host, DEFAULT_PORT);
}
public BrokerAddress(BrokerEndPoint broker) {
Assert.notNull(broker, "Broker cannot be null");
this.host = broker.host();
this.port = broker.port();
}
public static BrokerAddress fromAddress(String address) {
String[] split = address.split(":");
if (split.length == 0 || split.length > 2) {
throw new IllegalArgumentException("Expected format <host>[:<port>]");
}
if (split.length == 2) {
return new BrokerAddress(split[0], Integer.parseInt(split[1]));
}
else {
return new BrokerAddress(split[0]);
}
}
public String getHost() {
return this.host;
}
public int getPort() {
return this.port;
}
@Override
public int hashCode() {
return 31 * this.host.hashCode() + this.port;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
BrokerAddress brokerAddress = (BrokerAddress) o;
return this.port == brokerAddress.port && this.host.equals(brokerAddress.host);
}
@Override
public String toString() {
return this.host + ":" + this.port;
}
}

View File

@@ -0,0 +1,410 @@
/*
* Copyright 2016 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.kafka.listener;
import static org.hamcrest.Matchers.anyOf;
import static org.hamcrest.Matchers.equalTo;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertThat;
import static org.junit.Assert.assertTrue;
import static org.mockito.Matchers.anyLong;
import static org.mockito.Mockito.doAnswer;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicReference;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.kafka.clients.consumer.Consumer;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.common.TopicPartition;
import org.junit.ClassRule;
import org.junit.Test;
import org.mockito.invocation.InvocationOnMock;
import org.mockito.stubbing.Answer;
import org.springframework.beans.DirectFieldAccessor;
import org.springframework.kafka.core.ConsumerFactory;
import org.springframework.kafka.core.DefaultKafkaConsumerFactory;
import org.springframework.kafka.core.DefaultKafkaProducerFactory;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.core.ProducerFactory;
import org.springframework.kafka.listener.AbstractMessageListenerContainer.AckMode;
import org.springframework.kafka.listener.ConcurrentMessageListenerContainer.ContainerOffsetResetStrategy;
import org.springframework.kafka.rule.KafkaEmbedded;
/**
* @author Gary Russell
* @since 2.0
*
*/
public class ConcurrentMessageListenerContainerTests {
private final Log logger = LogFactory.getLog(this.getClass());
private static String topic1 = "testTopic1";
private static String topic2 = "testTopic2";
private static String topic3 = "testTopic3";
private static String topic4 = "testTopic4";
private static String topic5 = "testTopic5";
private static String topic6 = "testTopic6";
@ClassRule
public static KafkaEmbedded embeddedKafka = new KafkaEmbedded(1, true, topic1, topic2, topic3, topic4, topic5,
topic6);
@Test
public void testAutoCommit() throws Exception {
logger.info("Start auto");
Map<String, Object> props = consumerProps("test1", "true");
DefaultKafkaConsumerFactory<Integer, String> cf = new DefaultKafkaConsumerFactory<Integer, String>(props);
ConcurrentMessageListenerContainer<Integer, String> container =
new ConcurrentMessageListenerContainer<>(cf, topic1);
final CountDownLatch latch = new CountDownLatch(4);
container.setMessageListener(new MessageListener<Integer, String>() {
@Override
public void onMessage(ConsumerRecord<Integer, String> message) {
logger.info("auto: " + message);
latch.countDown();
}
});
container.setConcurrency(2);
container.setBeanName("testAuto");
container.start();
waitForAssignment(container, embeddedKafka.getPartitionsPerTopic());
Map<String, Object> senderProps = senderProps();
ProducerFactory<Integer, String> pf = new DefaultKafkaProducerFactory<Integer, String>(senderProps);
KafkaTemplate<Integer, String> template = new KafkaTemplate<>(pf);
template.setDefaultTopic(topic1);
template.convertAndSend(0, "foo");
template.convertAndSend(2, "bar");
template.convertAndSend(0, "baz");
template.convertAndSend(2, "qux");
template.flush();
assertTrue(latch.await(60, TimeUnit.SECONDS));
container.stop();
logger.info("Stop auto");
}
@Test
public void testAfterListenCommit() throws Exception {
logger.info("Start manual");
Map<String, Object> props = consumerProps("test2", "false");
DefaultKafkaConsumerFactory<Integer, String> cf = new DefaultKafkaConsumerFactory<Integer, String>(props);
ConcurrentMessageListenerContainer<Integer, String> container =
new ConcurrentMessageListenerContainer<>(cf, topic2);
final CountDownLatch latch = new CountDownLatch(4);
container.setMessageListener(new MessageListener<Integer, String>() {
@Override
public void onMessage(ConsumerRecord<Integer, String> message) {
logger.info("manual: " + message);
latch.countDown();
}
});
container.setConcurrency(2);
container.setBeanName("testBatch");
container.start();
waitForAssignment(container, embeddedKafka.getPartitionsPerTopic());
Map<String, Object> senderProps = senderProps();
ProducerFactory<Integer, String> pf = new DefaultKafkaProducerFactory<Integer, String>(senderProps);
KafkaTemplate<Integer, String> template = new KafkaTemplate<>(pf);
template.setDefaultTopic(topic2);
template.convertAndSend(0, "foo");
template.convertAndSend(2, "bar");
template.convertAndSend(0, "baz");
template.convertAndSend(2, "qux");
template.flush();
assertTrue(latch.await(60, TimeUnit.SECONDS));
container.stop();
logger.info("Stop manual");
}
@Test
public void testDefinedPartitions() throws Exception {
logger.info("Start auto parts");
Map<String, Object> props = consumerProps("test3", "true");
TopicPartition topic1Partition0 = new TopicPartition(topic3, 0);
DefaultKafkaConsumerFactory<Integer, String> cf = new DefaultKafkaConsumerFactory<Integer, String>(props);
ConcurrentMessageListenerContainer<Integer, String> container1 =
new ConcurrentMessageListenerContainer<>(cf, topic1Partition0);
final CountDownLatch latch1 = new CountDownLatch(2);
container1.setMessageListener(new MessageListener<Integer, String>() {
@Override
public void onMessage(ConsumerRecord<Integer, String> message) {
logger.info("auto part: " + message);
latch1.countDown();
}
});
container1.setBeanName("b1");
container1.start();
Thread.sleep(1000);
TopicPartition topic1Partition1 = new TopicPartition(topic3, 1);
ConcurrentMessageListenerContainer<Integer, String> container2 =
new ConcurrentMessageListenerContainer<>(cf, topic1Partition1);
final CountDownLatch latch2 = new CountDownLatch(2);
container2.setMessageListener(new MessageListener<Integer, String>() {
@Override
public void onMessage(ConsumerRecord<Integer, String> message) {
logger.info("auto part: " + message);
latch2.countDown();
}
});
container2.setBeanName("b2");
container2.start();
Thread.sleep(1000);
Map<String, Object> senderProps = senderProps();
ProducerFactory<Integer, String> pf = new DefaultKafkaProducerFactory<Integer, String>(senderProps);
KafkaTemplate<Integer, String> template = new KafkaTemplate<>(pf);
template.setDefaultTopic(topic3);
template.convertAndSend(0, "foo");
template.convertAndSend(2, "bar");
template.convertAndSend(0, "baz");
template.convertAndSend(2, "qux");
template.flush();
assertTrue(latch1.await(60, TimeUnit.SECONDS));
container1.stop();
container2.stop();
// reset earliest
ConcurrentMessageListenerContainer<Integer, String> resettingContainer =
new ConcurrentMessageListenerContainer<>(cf, topic1Partition0, topic1Partition1);
resettingContainer.setBeanName("b3");
final CountDownLatch latch3 = new CountDownLatch(4);
resettingContainer.setMessageListener(new MessageListener<Integer, String>() {
@Override
public void onMessage(ConsumerRecord<Integer, String> message) {
logger.info("auto part e: " + message);
latch3.countDown();
}
});
resettingContainer.setResetStrategy(ContainerOffsetResetStrategy.EARLIEST);
resettingContainer.start();
assertTrue(latch3.await(60, TimeUnit.SECONDS));
resettingContainer.stop();
assertThat(latch3.getCount(), equalTo(0L));
// reset minusone
resettingContainer = new ConcurrentMessageListenerContainer<>(cf, topic1Partition0, topic1Partition1);
resettingContainer.setBeanName("b4");
final CountDownLatch latch4 = new CountDownLatch(2);
final AtomicReference<String> receivedMessage = new AtomicReference<>();
resettingContainer.setMessageListener(new MessageListener<Integer, String>() {
@Override
public void onMessage(ConsumerRecord<Integer, String> message) {
logger.info("auto part -1: " + message);
receivedMessage.set(message.value());
latch4.countDown();
}
});
resettingContainer.setResetStrategy(ContainerOffsetResetStrategy.RECENT);
resettingContainer.start();
assertTrue(latch4.await(60, TimeUnit.SECONDS));
resettingContainer.stop();
assertThat(receivedMessage.get(), anyOf(equalTo("baz"), equalTo("qux")));
assertThat(latch4.getCount(), equalTo(0L));
logger.info("Stop auto parts");
}
@Test
public void testManualCommit() throws Exception {
testManualCommitGuts(AckMode.MANUAL, topic4);
testManualCommitGuts(AckMode.MANUAL_IMMEDIATE, topic5);
}
private void testManualCommitGuts(AckMode ackMode, String topic) throws Exception {
logger.info("Start " + ackMode);
Map<String, Object> props = consumerProps("test4", "false");
DefaultKafkaConsumerFactory<Integer, String> cf = new DefaultKafkaConsumerFactory<Integer, String>(props);
ConcurrentMessageListenerContainer<Integer, String> container =
new ConcurrentMessageListenerContainer<>(cf, topic);
final CountDownLatch latch = new CountDownLatch(4);
container.setMessageListener(new AcknowledgingMessageListener<Integer, String>() {
@Override
public void onMessage(ConsumerRecord<Integer, String> message, Acknowledgment ack) {
logger.info("manual: " + message);
ack.acknowledge();
latch.countDown();
}
});
container.setConcurrency(2);
container.setAckMode(ackMode);
container.setBeanName("test" + ackMode);
container.start();
waitForAssignment(container, embeddedKafka.getPartitionsPerTopic());
Map<String, Object> senderProps = senderProps();
ProducerFactory<Integer, String> pf = new DefaultKafkaProducerFactory<Integer, String>(senderProps);
KafkaTemplate<Integer, String> template = new KafkaTemplate<>(pf);
template.setDefaultTopic(topic);
template.convertAndSend(0, "foo");
template.convertAndSend(2, "bar");
template.convertAndSend(0, "baz");
template.convertAndSend(2, "qux");
template.flush();
assertTrue(latch.await(60, TimeUnit.SECONDS));
container.stop();
logger.info("Stop " + ackMode);
}
@SuppressWarnings("unchecked")
@Test
public void testConcurrencyWithPartitions() {
TopicPartition[] topic1PartitionS = new TopicPartition[] {
new TopicPartition(topic1, 0),
new TopicPartition(topic1, 1),
new TopicPartition(topic1, 2),
new TopicPartition(topic1, 3),
new TopicPartition(topic1, 4),
new TopicPartition(topic1, 5),
new TopicPartition(topic1, 6)
};
ConsumerFactory<Integer, String> cf = mock(ConsumerFactory.class);
Consumer<Integer, String> consumer = mock(Consumer.class);
when(cf.createConsumer()).thenReturn(consumer);
doAnswer(new Answer<ConsumerRecords<Integer, String>>() {
@Override
public ConsumerRecords<Integer, String> answer(InvocationOnMock invocation) throws Throwable {
Thread.sleep(100);
return null;
}
}).when(consumer).poll(anyLong());
ConcurrentMessageListenerContainer<Integer, String> container =
new ConcurrentMessageListenerContainer<>(cf, topic1PartitionS);
container.setMessageListener(new MessageListener<Integer, String>() {
@Override
public void onMessage(ConsumerRecord<Integer, String> message) {
}
});
container.setConcurrency(3);
container.start();
List<KafkaMessageListenerContainer<Integer, String>> containers = (List<KafkaMessageListenerContainer<Integer, String>>) new DirectFieldAccessor(
container).getPropertyValue("containers");
assertEquals(3, containers.size());
for (int i = 0; i < 3; i++) {
assertEquals(i < 2 ? 2 : 3, ((TopicPartition[]) new DirectFieldAccessor(containers.get(i))
.getPropertyValue("partitions")).length);
}
container.stop();
}
@Test
public void testListenerException() throws Exception {
logger.info("Start exception");
Map<String, Object> props = consumerProps("test1", "true");
DefaultKafkaConsumerFactory<Integer, String> cf = new DefaultKafkaConsumerFactory<Integer, String>(props);
ConcurrentMessageListenerContainer<Integer, String> container =
new ConcurrentMessageListenerContainer<>(cf, topic6);
final CountDownLatch latch = new CountDownLatch(4);
container.setMessageListener(new MessageListener<Integer, String>() {
@Override
public void onMessage(ConsumerRecord<Integer, String> message) {
logger.info("auto: " + message);
latch.countDown();
throw new RuntimeException("intended");
}
});
container.setConcurrency(2);
container.setBeanName("testException");
container.start();
waitForAssignment(container, embeddedKafka.getPartitionsPerTopic());
Map<String, Object> senderProps = senderProps();
ProducerFactory<Integer, String> pf = new DefaultKafkaProducerFactory<Integer, String>(senderProps);
KafkaTemplate<Integer, String> template = new KafkaTemplate<>(pf);
template.setDefaultTopic(topic6);
template.convertAndSend(0, "foo");
template.convertAndSend(2, "bar");
template.convertAndSend(0, "baz");
template.convertAndSend(2, "qux");
template.flush();
assertTrue(latch.await(60, TimeUnit.SECONDS));
container.stop();
logger.info("Stop exception");
}
private Map<String, Object> consumerProps(String group, String autoCommit) {
Map<String, Object> props = new HashMap<>();
props.put("bootstrap.servers", embeddedKafka.getBrokersAsString());
// props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", group);
props.put("enable.auto.commit", autoCommit);
props.put("auto.commit.interval.ms", "100");
props.put("session.timeout.ms", "15000");
props.put("key.deserializer", "org.apache.kafka.common.serialization.IntegerDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
return props;
}
private Map<String, Object> senderProps() {
Map<String, Object> props = new HashMap<>();
props.put("bootstrap.servers", embeddedKafka.getBrokersAsString());
// props.put("bootstrap.servers", "localhost:9092");
props.put("retries", 0);
props.put("batch.size", 16384);
props.put("linger.ms", 1);
props.put("buffer.memory", 33554432);
props.put("key.serializer", "org.apache.kafka.common.serialization.IntegerSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
return props;
}
private void waitForAssignment(ConcurrentMessageListenerContainer<Integer, String> container, int partitions)
throws Exception {
List<KafkaMessageListenerContainer<Integer, String>> containers = container.getContainers();
int n = 0;
int count = 0;
while (n++ < 600 && count < partitions) {
count = 0;
for (KafkaMessageListenerContainer<Integer, String> aContainer : containers) {
if (aContainer.getAssignedPartitions() != null) {
count += aContainer.getAssignedPartitions().size();
}
}
if (count < partitions) {
Thread.sleep(100);
}
}
assertThat(count, equalTo(partitions));
}
}

22
src/api/overview.html Normal file
View File

@@ -0,0 +1,22 @@
<html>
<body>
This document is the API specification for Spring for Apache Kafka project
<hr/>
<div id="overviewBody">
<p>
For further API reference and developer documentation, see the
<a href="http://static.springsource.org/spring-kafka/reference" target="_top">Spring
Kafka reference documentation</a>.
That documentation contains more detailed, developer-targeted
descriptions, with conceptual overviews, definitions of terms,
workarounds, and working code examples.
</p>
<p>
If you are interested in commercial training, consultancy, and
support for Spring Kafka, please visit <a href="http://www.springsource.com" target="_top">
http://www.springsource.com</a>
</p>
</div>
</body>
</html>

201
src/dist/license.txt vendored Normal file
View File

@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by testData1) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class testData1 and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [testData1 of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

21
src/dist/notice.txt vendored Normal file
View File

@@ -0,0 +1,21 @@
========================================================================
== NOTICE file corresponding to section 4 d of the Apache License, ==
== Version 2.0, in this case for the Spring Kafka distribution. ==
========================================================================
This product includes software developed by
the Apache Software Foundation (http://www.apache.org).
The end-user documentation included with a redistribution, if any,
must include the following acknowledgement:
"This product includes software developed by the Spring Framework
Project (http://www.springframework.org)."
Alternatively, this acknowledgement may appear in the software itself,
if and wherever such third-party acknowledgements normally appear.
The names "Spring", "Spring Framework", and "Spring Kafka" must
not be used to endorse or promote products derived from this software
without prior written permission. For written permission, please contact
enquiries@springsource.com.

View File

@@ -0,0 +1 @@
== Change History

View File

@@ -0,0 +1,20 @@
<productname>Spring Kafka</productname>
<releaseinfo>{spring-kafka-version}</releaseinfo>
<copyright>
<year>2016</year>
<holder>Pivotal Software Inc.</holder>
</copyright>
<authorgroup>
<author>
<firstname>Gary</firstname>
<surname>Russell</surname>
</author>
</authorgroup>
<legalnotice>
<para>
Copies of this document may be made for your own use and for distribution to
others, provided that you do not charge any fee for such copies and further
provided that each copy contains this Copyright Notice, whether distributed in
print or electronically.
</para>
</legalnotice>

View File

@@ -0,0 +1,42 @@
[[spring-amqp-reference]]
= Spring for Apache Kafka
:toc:
== Preface
include::preface.adoc[]
== Introduction
This first part of the reference documentation is a high-level overview of Spring for Apache Kafka and the underlying
concepts and some code snippets that will get you up and running as quickly as possible.
include::quick-tour.adoc[]
// include::whats-new.adoc[]
== Reference
This part of the reference documentation details the various components that comprise Spring for Apache Kafka.
The <<kafka,main chapter>> covers the core classes to develop a Kafka application with Spring.
include::kafka.adoc[]
include::testing.adoc[]
== Spring Integration
This part of the reference shows how to use the `spring-integration-kafka` module of Spring Integration.
include::si-kafka.adoc[]
[[resources]]
== Other Resources
In addition to this reference documentation, there exist a number of other resources that may help you learn about AMQP.
[appendix]
include::appendix.adoc[]

View File

@@ -0,0 +1,10 @@
[[kafka]]
=== Using Spring for Apache Kafka
==== Sending Messages with the KafkaTemplate
==== Receiving Messages
===== Message Listener Containers
===== @KafkaListener Annotation

View File

@@ -0,0 +1,7 @@
[[preface]]
The Spring for Apache Kafka project applies core Spring concepts to the development of Kafka-based messaging solutions.
We provide a "template" as a high-level abstraction for sending messages.
We also provide support for Message-driven POJOs.
// TODO: spring-kafka project page?
For other project-related information visit the Spring Integration project http://projects.spring.io/spring-integration/[homepage].

View File

@@ -0,0 +1,205 @@
[[quick-tour]]
=== Quick Tour for the impatient
==== Introduction
This is the 5 minute tour to get started with Spring AMQP.
Prerequisites: install and run Apache Kafka
Then grab the spring-kafka JAR and all of its dependencies - the easiest way to do that is to declare a dependency in
your build tool, e.g. for Maven:
[source,xml,subs="+attributes"]
----
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
<version>{spring-kafka-version}</version>
</dependency>
----
And for gradle:
[source,groovy,subs="+attributes"]
----
compile 'org.springframework.kafka:spring-kafka:{spring-kafka-version}'
----
[[compatibility]]
===== Compatibility
- Apache Kafka 0.9.0.1
- Tested with Spring Framework version dependency is 4.2.5 but it is expected that the framework will work with earlier
versions of Spring.
- Annotation-based listeners require Spring Framework 4.1 or higher, however.
- Minimum Java version: 7.
===== Very, Very Quick
Using plain, imperative Java to send and receive a message:
[source,java]
----
@Test
public void testAutoCommit() throws Exception {
logger.info("Start auto");
KafkaMessageListenerContainer<Integer, String> container = createContainer();
final CountDownLatch latch = new CountDownLatch(4);
container.setMessageListener(new MessageListener<Integer, String>() {
@Override
public void onMessage(ConsumerRecord<Integer, String> message) {
logger.info("auto: " + message);
latch.countDown();
}
});
container.setBeanName("testAuto");
container.start();
Thread.sleep(1000); // wait a bit for the container to start
KafkaTemplate<Integer, String> template = createTemplate();
template.setDefaultTopic(topic1);
template.convertAndSend(0, "foo");
template.convertAndSend(2, "bar");
template.convertAndSend(0, "baz");
template.convertAndSend(2, "qux");
template.flush();
assertTrue(latch.await(60, TimeUnit.SECONDS));
container.stop();
logger.info("Stop auto");
}
private KafkaMessageListenerContainer<Integer, String> createContainer() {
Map<String, Object> props = consumerProps();
DefaultKafkaConsumerFactory<Integer, String> cf = new DefaultKafkaConsumerFactory<Integer, String>(props);
KafkaMessageListenerContainer<Integer, String> container = new KafkaMessageListenerContainer<>(cf, topic1);
return container;
}
private KafkaTemplate<Integer, String> createTemplate() {
Map<String, Object> senderProps = senderProps();
ProducerFactory<Integer, String> pf = new DefaultKafkaProducerFactory<Integer, String>(senderProps);
KafkaTemplate<Integer, String> template = new KafkaTemplate<>(pf);
return template;
}
private Map<String, Object> consumerProps() {
Map<String, Object> props = new HashMap<>();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "myGroup");
props.put("enable.auto.commit", "true");
props.put("auto.commit.interval.ms", "100");
props.put("session.timeout.ms", "15000");
props.put("key.deserializer", "org.apache.kafka.common.serialization.IntegerDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
return props;
}
private Map<String, Object> senderProps() {
Map<String, Object> props = new HashMap<>();
props.put("bootstrap.servers", "localhost:9092");
props.put("retries", 0);
props.put("batch.size", 16384);
props.put("linger.ms", 1);
props.put("buffer.memory", 33554432);
props.put("key.serializer", "org.apache.kafka.common.serialization.IntegerSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
return props;
}
----
===== With Java Configuration
A similar example but with Spring configuration in Java:
[source,java]
----
@Autowired
public Listener listener;
@Autowired
public KafkaTemplate<Integer, String> template;
@Autowired
public KafkaListenerEndpointRegistry registry;
@Test
public void testSimple() throws Exception {
waitListening("foo");
template.convertAndSend("annotated1", 0, "foo");
assertTrue(this.listener.latch1.await(10, TimeUnit.SECONDS));
}
@Configuration
@EnableKafka
public class Config {
@Bean
public KafkaListenerContainerFactory<ConcurrentMessageListenerContainer<Integer, String>>
kafkaListenerContainerFactory() {
SimpleKafkaListenerContainerFactory<Integer, String> factory = new SimpleKafkaListenerContainerFactory<>();
factory.setConsumerFactory(consumerFactory());
return factory;
}
@Bean
public ConsumerFactory<Integer, String> consumerFactory() {
return new DefaultKafkaConsumerFactory<>(consumerConfigs());
}
@Bean
public Map<String, Object> consumerConfigs() {
Map<String, Object> props = new HashMap<>();
props.put("bootstrap.servers", embeddedKafka.getBrokersAsString());
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "myGroup");
props.put("enable.auto.commit", true);
props.put("auto.commit.interval.ms", "100");
props.put("session.timeout.ms", "15000");
props.put("key.deserializer", "org.apache.kafka.common.serialization.IntegerDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
return props;
}
@Bean
public Listener listener() {
return new Listener();
}
@Bean
public ProducerFactory<Integer, String> producerFactory() {
return new DefaultKafkaProducerFactory<>(producerConfigs());
}
@Bean
public Map<String, Object> producerConfigs() {
Map<String, Object> props = new HashMap<>();
props.put("bootstrap.servers", embeddedKafka.getBrokersAsString());
props.put("bootstrap.servers", "localhost:9092");
props.put("retries", 0);
props.put("batch.size", 16384);
props.put("linger.ms", 1);
props.put("buffer.memory", 33554432);
props.put("key.serializer", "org.apache.kafka.common.serialization.IntegerSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
return props;
}
@Bean
public KafkaTemplate<Integer, String> kafkaTemplate() {
return new KafkaTemplate<Integer, String>(producerFactory());
}
}
public class Listener {
private final CountDownLatch latch1 = new CountDownLatch(1);
@KafkaListener(id="foo", topics = "annotated1")
public void listen1(String foo) {
this.latch1.countDown();
}
}
----

View File

@@ -0,0 +1,2 @@
[[si-kafka]]
=== Spring Integration Kafka

View File

@@ -0,0 +1,677 @@
@import url(http://fonts.googleapis.com/css?family=Varela+Round|Open+Sans:400italic,700italic,400,700);
/*! normalize.css v2.1.2 | MIT License | git.io/normalize */
/* ========================================================================== HTML5 display definitions ========================================================================== */
/** Correct `block` display not defined in IE 8/9. */
@import url(http://cdnjs.cloudflare.com/ajax/libs/font-awesome/3.2.1/css/font-awesome.css);
article, aside, details, figcaption, figure, footer, header, hgroup, main, nav, section, summary { display: block; }
/** Correct `inline-block` display not defined in IE 8/9. */
audio, canvas, video { display: inline-block; }
/** Prevent modern browsers from displaying `audio` without controls. Remove excess height in iOS 5 devices. */
audio:not([controls]) { display: none; height: 0; }
/** Address `[hidden]` styling not present in IE 8/9. Hide the `template` element in IE, Safari, and Firefox < 22. */
[hidden], template { display: none; }
script { display: none !important; }
/* ========================================================================== Base ========================================================================== */
/** 1. Set default font family to sans-serif. 2. Prevent iOS text size adjust after orientation change, without disabling user zoom. */
html { font-family: sans-serif; /* 1 */ -ms-text-size-adjust: 100%; /* 2 */ -webkit-text-size-adjust: 100%; /* 2 */ }
/** Remove default margin. */
body { margin: 0; }
/* ========================================================================== Links ========================================================================== */
/** Remove the gray background color from active links in IE 10. */
a { background: transparent; }
/** Address `outline` inconsistency between Chrome and other browsers. */
a:focus { outline: thin dotted; }
/** Improve readability when focused and also mouse hovered in all browsers. */
a:active, a:hover { outline: 0; }
/* ========================================================================== Typography ========================================================================== */
/** Address variable `h1` font-size and margin within `section` and `article` contexts in Firefox 4+, Safari 5, and Chrome. */
h1 { font-size: 2em; margin: 0.67em 0; }
/** Address styling not present in IE 8/9, Safari 5, and Chrome. */
abbr[title] { border-bottom: 1px dotted; }
/** Address style set to `bolder` in Firefox 4+, Safari 5, and Chrome. */
b, strong { font-weight: bold; }
/** Address styling not present in Safari 5 and Chrome. */
dfn { font-style: italic; }
/** Address differences between Firefox and other browsers. */
hr { -moz-box-sizing: content-box; box-sizing: content-box; height: 0; }
/** Address styling not present in IE 8/9. */
mark { background: #ff0; color: #000; }
/** Correct font family set oddly in Safari 5 and Chrome. */
code, kbd, pre, samp { font-family: monospace, serif; font-size: 1em; }
/** Improve readability of pre-formatted text in all browsers. */
pre { white-space: pre-wrap; }
/** Set consistent quote types. */
q { quotes: "\201C" "\201D" "\2018" "\2019"; }
/** Address inconsistent and variable font size in all browsers. */
small { font-size: 80%; }
/** Prevent `sub` and `sup` affecting `line-height` in all browsers. */
sub, sup { font-size: 75%; line-height: 0; position: relative; vertical-align: baseline; }
sup { top: -0.5em; }
sub { bottom: -0.25em; }
/* ========================================================================== Embedded content ========================================================================== */
/** Remove border when inside `a` element in IE 8/9. */
img { border: 0; }
/** Correct overflow displayed oddly in IE 9. */
svg:not(:root) { overflow: hidden; }
/* ========================================================================== Figures ========================================================================== */
/** Address margin not present in IE 8/9 and Safari 5. */
figure { margin: 0; }
/* ========================================================================== Forms ========================================================================== */
/** Define consistent border, margin, and padding. */
fieldset { border: 1px solid #c0c0c0; margin: 0 2px; padding: 0.35em 0.625em 0.75em; }
/** 1. Correct `color` not being inherited in IE 8/9. 2. Remove padding so people aren't caught out if they zero out fieldsets. */
legend { border: 0; /* 1 */ padding: 0; /* 2 */ }
/** 1. Correct font family not being inherited in all browsers. 2. Correct font size not being inherited in all browsers. 3. Address margins set differently in Firefox 4+, Safari 5, and Chrome. */
button, input, select, textarea { font-family: inherit; /* 1 */ font-size: 100%; /* 2 */ margin: 0; /* 3 */ }
/** Address Firefox 4+ setting `line-height` on `input` using `!important` in the UA stylesheet. */
button, input { line-height: normal; }
/** Address inconsistent `text-transform` inheritance for `button` and `select`. All other form control elements do not inherit `text-transform` values. Correct `button` style inheritance in Chrome, Safari 5+, and IE 8+. Correct `select` style inheritance in Firefox 4+ and Opera. */
button, select { text-transform: none; }
/** 1. Avoid the WebKit bug in Android 4.0.* where (2) destroys native `audio` and `video` controls. 2. Correct inability to style clickable `input` types in iOS. 3. Improve usability and consistency of cursor style between image-type `input` and others. */
button, html input[type="button"], input[type="reset"], input[type="submit"] { -webkit-appearance: button; /* 2 */ cursor: pointer; /* 3 */ }
/** Re-set default cursor for disabled elements. */
button[disabled], html input[disabled] { cursor: default; }
/** 1. Address box sizing set to `content-box` in IE 8/9. 2. Remove excess padding in IE 8/9. */
input[type="checkbox"], input[type="radio"] { box-sizing: border-box; /* 1 */ padding: 0; /* 2 */ }
/** 1. Address `appearance` set to `searchfield` in Safari 5 and Chrome. 2. Address `box-sizing` set to `border-box` in Safari 5 and Chrome (include `-moz` to future-proof). */
input[type="search"] { -webkit-appearance: textfield; /* 1 */ -moz-box-sizing: content-box; -webkit-box-sizing: content-box; /* 2 */ box-sizing: content-box; }
/** Remove inner padding and search cancel button in Safari 5 and Chrome on OS X. */
input[type="search"]::-webkit-search-cancel-button, input[type="search"]::-webkit-search-decoration { -webkit-appearance: none; }
/** Remove inner padding and border in Firefox 4+. */
button::-moz-focus-inner, input::-moz-focus-inner { border: 0; padding: 0; }
/** 1. Remove default vertical scrollbar in IE 8/9. 2. Improve readability and alignment in all browsers. */
textarea { overflow: auto; /* 1 */ vertical-align: top; /* 2 */ }
/* ========================================================================== Tables ========================================================================== */
/** Remove most spacing between table cells. */
table { border-collapse: collapse; border-spacing: 0; }
meta.foundation-mq-small { font-family: "only screen and (min-width: 768px)"; width: 768px; }
meta.foundation-mq-medium { font-family: "only screen and (min-width:1280px)"; width: 1280px; }
meta.foundation-mq-large { font-family: "only screen and (min-width:1440px)"; width: 1440px; }
*, *:before, *:after { -moz-box-sizing: border-box; -webkit-box-sizing: border-box; box-sizing: border-box; }
html, body { font-size: 100%; }
body { background: white; color: #222222; padding: 0; margin: 0; font-family: "Helvetica Neue", "Helvetica", Helvetica, Arial, sans-serif; font-weight: normal; font-style: normal; line-height: 1; position: relative; cursor: auto; }
a:hover { cursor: pointer; }
img, object, embed { max-width: 100%; height: auto; }
object, embed { height: 100%; }
img { -ms-interpolation-mode: bicubic; }
#map_canvas img, #map_canvas embed, #map_canvas object, .map_canvas img, .map_canvas embed, .map_canvas object { max-width: none !important; }
.left { float: left !important; }
.right { float: right !important; }
.text-left { text-align: left !important; }
.text-right { text-align: right !important; }
.text-center { text-align: center !important; }
.text-justify { text-align: justify !important; }
.hide { display: none; }
.antialiased, body { -webkit-font-smoothing: antialiased; }
img { display: inline-block; vertical-align: middle; }
textarea { height: auto; min-height: 50px; }
select { width: 100%; }
p.lead, .paragraph.lead > p, #preamble > .sectionbody > .paragraph:first-of-type p { font-size: 1.21875em; line-height: 1.6; }
.subheader, #content #toctitle, .admonitionblock td.content > .title, .exampleblock > .title, .imageblock > .title, .listingblock > .title, .literalblock > .title, .mathblock > .title, .openblock > .title, .paragraph > .title, .quoteblock > .title, .sidebarblock > .title, .tableblock > .title, .verseblock > .title, .videoblock > .title, .dlist > .title, .olist > .title, .ulist > .title, .qlist > .title, .hdlist > .title, .tableblock > caption { line-height: 1.4; color: #385dbd; font-weight: 300; margin-top: 0.2em; margin-bottom: 0.5em; }
/* Typography resets */
div, dl, dt, dd, ul, ol, li, h1, h2, h3, #toctitle, .sidebarblock > .content > .title, h4, h5, h6, pre, form, p, blockquote, th, td { margin: 0; padding: 0; direction: ltr; }
/* Default Link Styles */
a { color: #095557; text-decoration: underline; line-height: inherit; }
a:hover, a:focus { color: #042829; }
a img { border: none; }
/* Default paragraph styles */
p { font-family: inherit; font-weight: normal; font-size: 1em; line-height: 1.6; margin-bottom: 1.25em; text-rendering: optimizeLegibility; }
p aside { font-size: 0.875em; line-height: 1.35; font-style: italic; }
/* Default header styles */
h1, h2, h3, #toctitle, .sidebarblock > .content > .title, h4, h5, h6 { font-family: "Varela Round", Arial, sans-serif; font-weight: normal; font-style: normal; color: #152347; text-rendering: optimizeLegibility; margin-top: 0.8em; margin-bottom: 0.5em; line-height: 1.2125em; }
h1 small, h2 small, h3 small, #toctitle small, .sidebarblock > .content > .title small, h4 small, h5 small, h6 small { font-size: 60%; color: #385dbd; line-height: 0; }
h1 { font-size: 2.125em; }
h2 { font-size: 1.6875em; }
h3, #toctitle, .sidebarblock > .content > .title { font-size: 1.375em; }
h4 { font-size: 1.125em; }
h5 { font-size: 1.125em; }
h6 { font-size: 1em; }
hr { border: solid #dcd2c9; border-width: 1px 0 0; clear: both; margin: 1.25em 0 1.1875em; height: 0; }
/* Helpful Typography Defaults */
em, i { font-style: italic; line-height: inherit; }
strong, b { font-weight: bold; line-height: inherit; }
small { font-size: 60%; line-height: inherit; }
code { font-family: Consolas, "Liberation Mono", Courier, monospace; font-weight: bold; color: #691816; }
/* Lists */
ul, ol, dl { font-size: 1em; line-height: 1.6; margin-bottom: 1.25em; list-style-position: outside; font-family: inherit; }
ul, ol { margin-left: 1.5em; }
ul.no-bullet, ol.no-bullet { margin-left: 1.5em; }
/* Unordered Lists */
ul li ul, ul li ol { margin-left: 1.25em; margin-bottom: 0; font-size: 1em; /* Override nested font-size change */ }
ul.square li ul, ul.circle li ul, ul.disc li ul { list-style: inherit; }
ul.square { list-style-type: square; }
ul.circle { list-style-type: circle; }
ul.disc { list-style-type: disc; }
ul.no-bullet { list-style: none; }
/* Ordered Lists */
ol li ul, ol li ol { margin-left: 1.25em; margin-bottom: 0; }
/* Definition Lists */
dl dt { margin-bottom: 0.3125em; font-weight: bold; }
dl dd { margin-bottom: 1.25em; }
/* Abbreviations */
abbr, acronym { text-transform: uppercase; font-size: 90%; color: #211306; border-bottom: 1px dotted #dddddd; cursor: help; }
abbr { text-transform: none; }
/* Blockquotes */
blockquote { margin: 0 0 1.25em; padding: 0.5625em 1.25em 0 1.1875em; border-left: 1px solid #dddddd; }
blockquote cite { display: block; font-size: 0.8125em; color: #655241; }
blockquote cite:before { content: "\2014 \0020"; }
blockquote cite a, blockquote cite a:visited { color: #655241; }
blockquote, blockquote p { line-height: 1.6; color: #846b55; }
/* Microformats */
.vcard { display: inline-block; margin: 0 0 1.25em 0; border: 1px solid #dddddd; padding: 0.625em 0.75em; }
.vcard li { margin: 0; display: block; }
.vcard .fn { font-weight: bold; font-size: 0.9375em; }
.vevent .summary { font-weight: bold; }
.vevent abbr { cursor: auto; text-decoration: none; font-weight: bold; border: none; padding: 0 0.0625em; }
@media only screen and (min-width: 768px) { h1, h2, h3, #toctitle, .sidebarblock > .content > .title, h4, h5, h6 { line-height: 1.4; }
h1 { font-size: 2.75em; }
h2 { font-size: 2.3125em; }
h3, #toctitle, .sidebarblock > .content > .title { font-size: 1.6875em; }
h4 { font-size: 1.4375em; } }
/* Print styles. Inlined to avoid required HTTP connection: www.phpied.com/delay-loading-your-print-css/ Credit to Paul Irish and HTML5 Boilerplate (html5boilerplate.com)
*/
.print-only { display: none !important; }
@media print { * { background: transparent !important; color: #000 !important; /* Black prints faster: h5bp.com/s */ box-shadow: none !important; text-shadow: none !important; }
a, a:visited { text-decoration: underline; }
a[href]:after { content: " (" attr(href) ")"; }
abbr[title]:after { content: " (" attr(title) ")"; }
.ir a:after, a[href^="javascript:"]:after, a[href^="#"]:after { content: ""; }
pre, blockquote { border: 1px solid #999; page-break-inside: avoid; }
thead { display: table-header-group; /* h5bp.com/t */ }
tr, img { page-break-inside: avoid; }
img { max-width: 100% !important; }
@page { margin: 0.5cm; }
p, h2, h3, #toctitle, .sidebarblock > .content > .title { orphans: 3; widows: 3; }
h2, h3, #toctitle, .sidebarblock > .content > .title { page-break-after: avoid; }
.hide-on-print { display: none !important; }
.print-only { display: block !important; }
.hide-for-print { display: none !important; }
.show-for-print { display: inherit !important; } }
/* Tables */
table { background: white; margin-bottom: 1.25em; border: solid 1px #e4e7ef; }
table thead, table tfoot { background: rgba(105, 60, 22, 0.25); font-weight: bold; }
table thead tr th, table thead tr td, table tfoot tr th, table tfoot tr td { padding: 0.5em 0.625em 0.625em; font-size: inherit; color: #211306; text-align: left; }
table tr th, table tr td { padding: 0.5625em 0.625em; font-size: inherit; color: #211306; }
table tr.even, table tr.alt, table tr:nth-of-type(even) { background: #f4f5f8; }
table thead tr th, table tfoot tr th, table tbody tr td, table tr td, table tfoot tr td { display: table-cell; line-height: 1.6; }
.clearfix:before, .clearfix:after, .float-group:before, .float-group:after { content: " "; display: table; }
.clearfix:after, .float-group:after { clear: both; }
*:not(pre) > code { font-size: inherit; padding: 0; white-space: nowrap; background-color: inherit; border: 0 solid #dddddd; -webkit-border-radius: 6px; border-radius: 6px; text-shadow: none; }
pre, pre > code { line-height: 1.4; color: black; font-family: monospace, serif; font-weight: normal; }
.keyseq { color: #774417; }
kbd:not(.keyseq) { display: inline-block; color: #211306; font-size: 0.75em; line-height: 1.4; background-color: #F7F7F7; border: 1px solid #ccc; -webkit-border-radius: 3px; border-radius: 3px; -webkit-box-shadow: 0 1px 0 rgba(0, 0, 0, 0.2), 0 0 0 2px white inset; box-shadow: 0 1px 0 rgba(0, 0, 0, 0.2), 0 0 0 2px white inset; margin: -0.15em 0.15em 0 0.15em; padding: 0.2em 0.6em 0.2em 0.5em; vertical-align: middle; white-space: nowrap; }
.keyseq kbd:first-child { margin-left: 0; }
.keyseq kbd:last-child { margin-right: 0; }
.menuseq, .menu { color: black; }
b.button:before, b.button:after { position: relative; top: -1px; font-weight: normal; }
b.button:before { content: "["; padding: 0 3px 0 2px; }
b.button:after { content: "]"; padding: 0 2px 0 3px; }
p a > code:hover { color: #541312; }
#header, #content, #footnotes, #footer { width: 100%; margin-left: auto; margin-right: auto; margin-top: 0; margin-bottom: 0; max-width: 62.5em; *zoom: 1; position: relative; padding-left: 0.9375em; padding-right: 0.9375em; }
#header:before, #header:after, #content:before, #content:after, #footnotes:before, #footnotes:after, #footer:before, #footer:after { content: " "; display: table; }
#header:after, #content:after, #footnotes:after, #footer:after { clear: both; }
#header { margin-bottom: 2.5em; }
#header > h1 { color: #693c16; font-weight: normal; border-bottom: 1px solid #dcd2c9;}
#header span { color: #846b55; }
#header #revnumber { text-transform: capitalize; }
#header br { display: none; }
#header br + span { padding-left: 3px; }
#header br + span.author { padding-left: 0; }
#header br + span.author:before { content: ", "; }
#revdate {
display: block;
margin-top: 2.5em;
}
#toc { border-bottom: 1px solid #e6dfd8; padding-bottom: 1.25em; }
#toc > ul { margin-left: 0.25em; }
#toc ul.sectlevel0 > li > a { font-style: italic; }
#toc ul.sectlevel0 ul.sectlevel1 { margin-left: 0; margin-top: 0.5em; margin-bottom: 0.5em; }
#toc ul { list-style-type: none; }
#toctitle { color: #385dbd; }
@media only screen and (min-width: 768px) { body.toc2 { padding-left: 15em; padding-right: 0; }
#toc.toc2 { position: fixed; width: 15em; left: 0; top: 0; border-right: 1px solid #e6dfd8; border-bottom: 0; z-index: 1000; padding: 1em; height: 100%; overflow: auto; }
#toc.toc2 #toctitle { margin-top: 0; font-size: 1.2em; }
#toc.toc2 > ul { font-size: .90em; }
#toc.toc2 ul ul { margin-left: 0; padding-left: 1em; }
#toc.toc2 ul.sectlevel0 ul.sectlevel1 { padding-left: 0; margin-top: 0.5em; margin-bottom: 0.5em; }
body.toc2.toc-right { padding-left: 0; padding-right: 15em; }
body.toc2.toc-right #toc.toc2 { border-right: 0; border-left: 1px solid #e6dfd8; left: auto; right: 0; } }
@media only screen and (min-width: 1280px) { body.toc2 { padding-left: 20em; padding-right: 0; }
#toc.toc2 { width: 20em; }
#toc.toc2 #toctitle { font-size: 1.375em; }
#toc.toc2 > ul { font-size: 0.95em; }
#toc.toc2 ul ul { padding-left: 1.25em; }
body.toc2.toc-right { padding-left: 0; padding-right: 20em; } }
#content #toc { border-style: solid; border-width: 1px; border-color: #d9d9d9; margin-bottom: 1.25em; padding: 1.25em; background: #f2f2f2; border-width: 0; -webkit-border-radius: 6px; border-radius: 6px; }
#content #toc > :first-child { margin-top: 0; }
#content #toc > :last-child { margin-bottom: 0; }
#content #toc a { text-decoration: none; }
#content #toctitle { font-weight: bold; font-family: "Open Sans", Arial, sans-serif; font-size: 1em; padding-left: 0.125em; }
#footer { max-width: 100%; background-color: #23160c; padding: 1.25em; }
#footer-text { color: #deecf9; line-height: 1.44; }
.sect1 { padding-bottom: 1.25em; }
.sect1 + .sect1 { border-top: 1px solid #e6dfd8; }
#content h1 > a.anchor, h2 > a.anchor, h3 > a.anchor, #toctitle > a.anchor, .sidebarblock > .content > .title > a.anchor, h4 > a.anchor, h5 > a.anchor, h6 > a.anchor { position: absolute; width: 1em; margin-left: -1em; display: block; text-decoration: none; visibility: hidden; text-align: center; font-weight: normal; }
#content h1 > a.anchor:before, h2 > a.anchor:before, h3 > a.anchor:before, #toctitle > a.anchor:before, .sidebarblock > .content > .title > a.anchor:before, h4 > a.anchor:before, h5 > a.anchor:before, h6 > a.anchor:before { content: '\00A7'; font-size: .85em; vertical-align: text-top; display: block; margin-top: 0.05em; }
#content h1:hover > a.anchor, #content h1 > a.anchor:hover, h2:hover > a.anchor, h2 > a.anchor:hover, h3:hover > a.anchor, #toctitle:hover > a.anchor, .sidebarblock > .content > .title:hover > a.anchor, h3 > a.anchor:hover, #toctitle > a.anchor:hover, .sidebarblock > .content > .title > a.anchor:hover, h4:hover > a.anchor, h4 > a.anchor:hover, h5:hover > a.anchor, h5 > a.anchor:hover, h6:hover > a.anchor, h6 > a.anchor:hover { visibility: visible; }
#content h1 > a.link, h2 > a.link, h3 > a.link, #toctitle > a.link, .sidebarblock > .content > .title > a.link, h4 > a.link, h5 > a.link, h6 > a.link { color: #152347; text-decoration: none; }
#content h1 > a.link:hover, h2 > a.link:hover, h3 > a.link:hover, #toctitle > a.link:hover, .sidebarblock > .content > .title > a.link:hover, h4 > a.link:hover, h5 > a.link:hover, h6 > a.link:hover { color: #0f1933; }
.imageblock, .literalblock, .listingblock, .mathblock, .verseblock, .videoblock { margin-bottom: 1.25em; }
.admonitionblock td.content > .title, .exampleblock > .title, .imageblock > .title, .listingblock > .title, .literalblock > .title, .mathblock > .title, .openblock > .title, .paragraph > .title, .quoteblock > .title, .sidebarblock > .title, .tableblock > .title, .verseblock > .title, .videoblock > .title, .dlist > .title, .olist > .title, .ulist > .title, .qlist > .title, .hdlist > .title { text-align: left; font-weight: bold; }
.tableblock > caption { text-align: left; font-weight: bold; white-space: nowrap; overflow: visible; max-width: 0; }
table.tableblock #preamble > .sectionbody > .paragraph:first-of-type p { font-size: inherit; }
.admonitionblock > table { border: 0; background: none; width: 100%; }
.admonitionblock > table td.icon { text-align: center; width: 80px; }
.admonitionblock > table td.icon img { max-width: none; }
.admonitionblock > table td.icon .title { font-weight: bold; text-transform: uppercase; }
.admonitionblock > table td.content { padding-left: 1.125em; padding-right: 1.25em; border-left: 1px solid #dcd2c9; color: #846b55; }
.admonitionblock > table td.content > :last-child > :last-child { margin-bottom: 0; }
.exampleblock > .content { border-style: solid; border-width: 1px; border-color: #f3e0ce; margin-bottom: 1.25em; padding: 1.25em; background: #fdfaf7; -webkit-border-radius: 6px; border-radius: 6px; }
.exampleblock > .content > :first-child { margin-top: 0; }
.exampleblock > .content > :last-child { margin-bottom: 0; }
.exampleblock > .content h1, .exampleblock > .content h2, .exampleblock > .content h3, .exampleblock > .content #toctitle, .sidebarblock.exampleblock > .content > .title, .exampleblock > .content h4, .exampleblock > .content h5, .exampleblock > .content h6, .exampleblock > .content p { color: #333333; }
.exampleblock > .content h1, .exampleblock > .content h2, .exampleblock > .content h3, .exampleblock > .content #toctitle, .sidebarblock.exampleblock > .content > .title, .exampleblock > .content h4, .exampleblock > .content h5, .exampleblock > .content h6 { line-height: 1; margin-bottom: 0.625em; }
.exampleblock > .content h1.subheader, .exampleblock > .content h2.subheader, .exampleblock > .content h3.subheader, .exampleblock > .content .subheader#toctitle, .sidebarblock.exampleblock > .content > .subheader.title, .exampleblock > .content h4.subheader, .exampleblock > .content h5.subheader, .exampleblock > .content h6.subheader { line-height: 1.4; }
.exampleblock.result > .content { -webkit-box-shadow: 0 1px 8px #d9d9d9; box-shadow: 0 1px 8px #d9d9d9; }
.sidebarblock { border-style: solid; border-width: 1px; border-color: #d9d9d9; margin-bottom: 1.25em; padding: 1.25em; background: #f2f2f2; -webkit-border-radius: 6px; border-radius: 6px; }
.sidebarblock > :first-child { margin-top: 0; }
.sidebarblock > :last-child { margin-bottom: 0; }
.sidebarblock h1, .sidebarblock h2, .sidebarblock h3, .sidebarblock #toctitle, .sidebarblock > .content > .title, .sidebarblock h4, .sidebarblock h5, .sidebarblock h6, .sidebarblock p { color: #333333; }
.sidebarblock h1, .sidebarblock h2, .sidebarblock h3, .sidebarblock #toctitle, .sidebarblock > .content > .title, .sidebarblock h4, .sidebarblock h5, .sidebarblock h6 { line-height: 1; margin-bottom: 0.625em; }
.sidebarblock h1.subheader, .sidebarblock h2.subheader, .sidebarblock h3.subheader, .sidebarblock .subheader#toctitle, .sidebarblock > .content > .subheader.title, .sidebarblock h4.subheader, .sidebarblock h5.subheader, .sidebarblock h6.subheader { line-height: 1.4; }
.sidebarblock > .content > .title { color: #385dbd; margin-top: 0; line-height: 1.6; }
.exampleblock > .content > :last-child > :last-child, .exampleblock > .content .olist > ol > li:last-child > :last-child, .exampleblock > .content .ulist > ul > li:last-child > :last-child, .exampleblock > .content .qlist > ol > li:last-child > :last-child, .sidebarblock > .content > :last-child > :last-child, .sidebarblock > .content .olist > ol > li:last-child > :last-child, .sidebarblock > .content .ulist > ul > li:last-child > :last-child, .sidebarblock > .content .qlist > ol > li:last-child > :last-child { margin-bottom: 0; }
.literalblock pre:not([class]), .listingblock pre:not([class]) { background: url('../images/golo/pre-bg.png?1370460826'); }
.literalblock pre, .literalblock pre[class], .listingblock pre, .listingblock pre[class] { border-width: 1px; border-style: solid; border-color: rgba(21, 35, 71, 0.1); -webkit-border-radius: 6px; border-radius: 6px; padding: 0.8em; word-wrap: break-word; }
.literalblock pre.nowrap, .literalblock pre[class].nowrap, .listingblock pre.nowrap, .listingblock pre[class].nowrap { overflow-x: auto; white-space: pre; word-wrap: normal; }
.literalblock pre > code, .literalblock pre[class] > code, .listingblock pre > code, .listingblock pre[class] > code { display: block; }
@media only screen { .literalblock pre, .literalblock pre[class], .listingblock pre, .listingblock pre[class] { font-size: 0.72em; } }
@media only screen and (min-width: 768px) { .literalblock pre, .literalblock pre[class], .listingblock pre, .listingblock pre[class] { font-size: 0.81em; } }
@media only screen and (min-width: 1280px) { .literalblock pre, .literalblock pre[class], .listingblock pre, .listingblock pre[class] { font-size: 0.9em; } }
.listingblock pre.highlight { padding: 0; }
.listingblock pre.highlight > code { padding: 0.8em; }
.listingblock > .content { position: relative; }
.listingblock:hover code[class*=" language-"]:before { text-transform: uppercase; font-size: 0.9em; color: #999; position: absolute; top: 0.375em; right: 0.375em; }
.listingblock:hover code.asciidoc:before { content: "asciidoc"; }
.listingblock:hover code.clojure:before { content: "clojure"; }
.listingblock:hover code.css:before { content: "css"; }
.listingblock:hover code.groovy:before { content: "groovy"; }
.listingblock:hover code.html:before { content: "html"; }
.listingblock:hover code.java:before { content: "java"; }
.listingblock:hover code.javascript:before { content: "javascript"; }
.listingblock:hover code.python:before { content: "python"; }
.listingblock:hover code.ruby:before { content: "ruby"; }
.listingblock:hover code.sass:before { content: "sass"; }
.listingblock:hover code.scss:before { content: "scss"; }
.listingblock:hover code.xml:before { content: "xml"; }
.listingblock:hover code.yaml:before { content: "yaml"; }
.listingblock.terminal pre .command:before { content: attr(data-prompt); padding-right: 0.5em; color: #999; }
.listingblock.terminal pre .command:not([data-prompt]):before { content: '$'; }
table.pyhltable { border: 0; margin-bottom: 0; }
table.pyhltable td { vertical-align: top; padding-top: 0; padding-bottom: 0; }
table.pyhltable td.code { padding-left: .75em; padding-right: 0; }
.highlight.pygments .lineno, table.pyhltable td:not(.code) { color: #999; padding-left: 0; padding-right: .5em; border-right: 1px solid #dcd2c9; }
.highlight.pygments .lineno { display: inline-block; margin-right: .25em; }
table.pyhltable .linenodiv { background-color: transparent !important; padding-right: 0 !important; }
.quoteblock { margin: 0 0 1.25em; padding: 0.5625em 1.25em 0 1.1875em; border-left: 1px solid #dddddd; }
.quoteblock blockquote { margin: 0 0 1.25em 0; padding: 0 0 0.5625em 0; border: 0; }
.quoteblock blockquote > .paragraph:last-child p { margin-bottom: 0; }
.quoteblock .attribution { margin-top: -.25em; padding-bottom: 0.5625em; font-size: 0.8125em; color: #655241; }
.quoteblock .attribution br { display: none; }
.quoteblock .attribution cite { display: block; margin-bottom: 0.625em; }
table thead th, table tfoot th { font-weight: bold; }
table.tableblock.grid-all { border-collapse: separate; border-spacing: 1px; -webkit-border-radius: 6px; border-radius: 6px; border-top: 1px solid #e4e7ef; border-bottom: 1px solid #e4e7ef; }
table.tableblock.frame-topbot, table.tableblock.frame-none { border-left: 0; border-right: 0; }
table.tableblock.frame-sides, table.tableblock.frame-none { border-top: 0; border-bottom: 0; }
table.tableblock td .paragraph:last-child p > p:last-child, table.tableblock th > p:last-child, table.tableblock td > p:last-child { margin-bottom: 0; }
th.tableblock.halign-left, td.tableblock.halign-left { text-align: left; }
th.tableblock.halign-right, td.tableblock.halign-right { text-align: right; }
th.tableblock.halign-center, td.tableblock.halign-center { text-align: center; }
th.tableblock.valign-top, td.tableblock.valign-top { vertical-align: top; }
th.tableblock.valign-bottom, td.tableblock.valign-bottom { vertical-align: bottom; }
th.tableblock.valign-middle, td.tableblock.valign-middle { vertical-align: middle; }
tbody tr th { display: table-cell; line-height: 1.6; background: rgba(105, 60, 22, 0.25); }
tbody tr th, tbody tr th p, tfoot tr th, tfoot tr th p { color: #211306; font-weight: bold; }
td > div.verse { white-space: pre; }
ol { margin-left: 1.75em; }
ul li ol { margin-left: 1.5em; }
dl dd { margin-left: 1.125em; }
dl dd:last-child, dl dd:last-child > :last-child { margin-bottom: 0; }
ol > li p, ul > li p, ul dd, ol dd, .olist .olist, .ulist .ulist, .ulist .olist, .olist .ulist { margin-bottom: 0.625em; }
ul.unstyled, ol.unnumbered, ul.checklist, ul.none { list-style-type: none; }
ul.unstyled, ol.unnumbered, ul.checklist { margin-left: 0.625em; }
ul.checklist li > p:first-child > i[class^="icon-check"]:first-child, ul.checklist li > p:first-child > input[type="checkbox"]:first-child { margin-right: 0.25em; }
ul.checklist li > p:first-child > input[type="checkbox"]:first-child { position: relative; top: 1px; }
ul.inline { margin: 0 auto 0.625em auto; margin-left: -1.375em; margin-right: 0; padding: 0; list-style: none; overflow: hidden; }
ul.inline > li { list-style: none; float: left; margin-left: 1.375em; display: block; }
ul.inline > li > * { display: block; }
.unstyled dl dt { font-weight: normal; font-style: normal; }
ol.arabic { list-style-type: decimal; }
ol.decimal { list-style-type: decimal-leading-zero; }
ol.loweralpha { list-style-type: lower-alpha; }
ol.upperalpha { list-style-type: upper-alpha; }
ol.lowerroman { list-style-type: lower-roman; }
ol.upperroman { list-style-type: upper-roman; }
ol.lowergreek { list-style-type: lower-greek; }
.hdlist > table, .colist > table { border: 0; background: none; }
.hdlist > table > tbody > tr, .colist > table > tbody > tr { background: none; }
td.hdlist1 { padding-right: .75em; font-weight: bold; }
td.hdlist1, td.hdlist2 { vertical-align: top; }
.literalblock + .colist, .listingblock + .colist { margin-top: -0.5em; }
.colist > table tr > td:first-of-type { padding: 0 .75em; line-height: 1; }
.colist > table tr > td:last-of-type { padding: 0.25em 0; }
.qanda > ol > li > p > em:only-child { color: #063f40; }
.thumb, .th { line-height: 0; display: inline-block; border: solid 4px white; -webkit-box-shadow: 0 0 0 1px #dddddd; box-shadow: 0 0 0 1px #dddddd; }
.imageblock.left, .imageblock[style*="float: left"] { margin: 0.25em 0.625em 1.25em 0; }
.imageblock.right, .imageblock[style*="float: right"] { margin: 0.25em 0 1.25em 0.625em; }
.imageblock > .title { margin-bottom: 0; }
.imageblock.thumb, .imageblock.th { border-width: 6px; }
.imageblock.thumb > .title, .imageblock.th > .title { padding: 0 0.125em; }
.image.left, .image.right { margin-top: 0.25em; margin-bottom: 0.25em; display: inline-block; line-height: 0; }
.image.left { margin-right: 0.625em; }
.image.right { margin-left: 0.625em; }
a.image { text-decoration: none; }
span.footnote, span.footnoteref { vertical-align: super; font-size: 0.875em; }
span.footnote a, span.footnoteref a { text-decoration: none; }
#footnotes { padding-top: 0.75em; padding-bottom: 0.75em; margin-bottom: 0.625em; }
#footnotes hr { width: 20%; min-width: 6.25em; margin: -.25em 0 .75em 0; border-width: 1px 0 0 0; }
#footnotes .footnote { padding: 0 0.375em; line-height: 1.3; font-size: 0.875em; margin-left: 1.2em; text-indent: -1.2em; margin-bottom: .2em; }
#footnotes .footnote a:first-of-type { font-weight: bold; text-decoration: none; }
#footnotes .footnote:last-of-type { margin-bottom: 0; }
#content #footnotes { margin-top: -0.625em; margin-bottom: 0; padding: 0.75em 0; }
.gist .file-data > table { border: none; background: #fff; width: 100%; margin-bottom: 0; }
.gist .file-data > table td.line-data { width: 99%; }
div.unbreakable { page-break-inside: avoid; }
.big { font-size: larger; }
.small { font-size: smaller; }
.underline { text-decoration: underline; }
.overline { text-decoration: overline; }
.line-through { text-decoration: line-through; }
.aqua { color: #00bfbf; }
.aqua-background { background-color: #00fafa; }
.black { color: black; }
.black-background { background-color: black; }
.blue { color: #0000bf; }
.blue-background { background-color: #0000fa; }
.fuchsia { color: #bf00bf; }
.fuchsia-background { background-color: #fa00fa; }
.gray { color: #606060; }
.gray-background { background-color: #7d7d7d; }
.green { color: #006000; }
.green-background { background-color: #007d00; }
.lime { color: #00bf00; }
.lime-background { background-color: #00fa00; }
.maroon { color: #600000; }
.maroon-background { background-color: #7d0000; }
.navy { color: #000060; }
.navy-background { background-color: #00007d; }
.olive { color: #606000; }
.olive-background { background-color: #7d7d00; }
.purple { color: #600060; }
.purple-background { background-color: #7d007d; }
.red { color: #bf0000; }
.red-background { background-color: #fa0000; }
.silver { color: #909090; }
.silver-background { background-color: #bcbcbc; }
.teal { color: #006060; }
.teal-background { background-color: #007d7d; }
.white { color: #bfbfbf; }
.white-background { background-color: #fafafa; }
.yellow { color: #bfbf00; }
.yellow-background { background-color: #fafa00; }
span.icon > [class^="icon-"], span.icon > [class*=" icon-"] { cursor: default; }
.admonitionblock td.icon [class^="icon-"]:before { font-size: 2.5em; text-shadow: 1px 1px 2px rgba(0, 0, 0, 0.5); cursor: default; }
.admonitionblock td.icon .icon-note:before { content: "\f05a"; color: #095557; color: #064042; }
.admonitionblock td.icon .icon-tip:before { content: "\f0eb"; text-shadow: 1px 1px 2px rgba(155, 155, 0, 0.8); color: #111; }
.admonitionblock td.icon .icon-warning:before { content: "\f071"; color: #bf6900; }
.admonitionblock td.icon .icon-caution:before { content: "\f06d"; color: #bf3400; }
.admonitionblock td.icon .icon-important:before { content: "\f06a"; color: #bf0000; }
.conum { display: inline-block; color: white !important; background-color: #211306; -webkit-border-radius: 100px; border-radius: 100px; text-align: center; width: 20px; height: 20px; font-size: 12px; font-weight: bold; line-height: 20px; font-family: Arial, sans-serif; font-style: normal; position: relative; top: -2px; letter-spacing: -1px; }
.conum * { color: white !important; }
.conum + b { display: none; }
.conum:after { content: attr(data-value); }
.conum:not([data-value]):empty { display: none; }
body { background: url('../images/golo/body-bg.png?1370460826') #fdfaf7 repeat; }
#toc.toc2 ul ul { padding-left: 1.75em; }
#toctitle { color: #152347; }
#header h1 { font-weight: bold; position: relative; left: -0.0625em; }
#header h1 span.lo { color: #dc9424; }
#content h2, #content h3, #content #toctitle, #content .sidebarblock > .content > .title, #content h4, #content h5, #content #toctitle { font-weight: normal; position: relative; left: -0.0625em; }
#content h2 { font-weight: bold; }
.literalblock .content pre.highlight, .listingblock .content pre.highlight { background: url('../images/golo/pre-bg.png?1370460826'); }
.admonitionblock > table td.content { border-color: #e6dfd8; }
table.tableblock.grid-all { -webkit-border-radius: 0; border-radius: 0; }
#footer { background-color: #23160c; }

View File