230 Commits

Author SHA1 Message Date
Gary Russell
3294f6c058 GH-1499: Option to suppress ConsumerRecord logging
Resolves https://github.com/spring-projects/spring-kafka/issues/1499

**I will do the backports because I expect many conflicts.**
2020-06-09 13:08:41 -04:00
Clemens Diebold
423deb0182 Corrects two typos
corrects two small typos in the documentation.
2020-06-07 10:44:39 -04:00
Gary Russell
0cbf56903a Doc Polishing 2020-05-02 09:53:42 -04:00
Gary Russell
650b448a46 Fix KafkaStreamBrancher doc typos
Fixes https://github.com/spring-projects/spring-kafka/issues/1459
2020-04-21 15:18:50 -04:00
Gary Russell
1be02a36a2 Add gradle to test version override appendix
- also capture the current version
2019-11-22 12:20:17 -05:00
Gary Russell
a1d4c96573 Fix typo around NRCEvent properties
- was `pollInterval` instead of `pollTimeout`

**cherry-pick to 2.2.x**

(cherry picked from commit f77dcfbb94)
2019-10-15 15:22:57 -04:00
Gary Russell
b05b04cef8 GH-1259: Handle Failed Record Recovery
Resolves https://github.com/spring-projects/spring-kafka/issues/1259

Previously if the recoverer in a `SeekToCurrentErrorHandler` or
`DefaultAfterRollbackProcessor` failed to recover a record, the
record could be lost; the `FailedRecordTracker` simply logged
the exception.

Change the `SeekUtils` to detect a failure in the recoverer (actually
any failure when determining if the failed record should be recovered)
and include the failed record in the seeks.

In this way the recovery will be attempted once more on each delivery
attempt.

**cherry-pick to 2.2.x**

# Conflicts:
#	spring-kafka/src/main/java/org/springframework/kafka/listener/FailedRecordTracker.java
#	spring-kafka/src/main/java/org/springframework/kafka/support/SeekUtils.java
#	src/reference/asciidoc/kafka.adoc
2019-10-03 12:55:05 -04:00
justinbaur
a52f415978 GH-1226: Update snippet to correct return type and misspelling
Fixes https://github.com/spring-projects/spring-kafka/issues/1226
# Conflicts:
#	src/reference/asciidoc/kafka.adoc
2019-09-26 11:17:55 -04:00
Gary Russell
43ddbf1217 Doc Polishing 2019-09-26 08:42:23 -04:00
Victor Perez Rey
5fae3a927b GH-1125: Add FailedDeserializationInfo
Fixes https://github.com/spring-projects/spring-kafka/issues/1125

Main ideas:

    * It does not enforces one single function to be set,
    it simply makes the new one to take precedence.
    * When setting the function/supplier via configuration,
    one single configuration key/value pair is enough.
    It tries to fit in any of the two possible options, otherwise, raises errors.
    * I'd personally rename `private BiFunction<byte[], Headers, T> failedDeserializationFunction;`
     to `failedDeserializationBiFunction`, respecting the name at setter level.
     However i read about avoiding sugar refactoring as rule of thumb.

* Simplify and unify setFailedDeserializationFunction
& recoverFromSupplier methods from ErrorHandlingDeserializer2
* deprecated since was in the wrong place
* Javadocs for ErrorHandlingDeserializer2 and adocs syntax's amends.
* Correct FailedDeserializationInfo & ErrorHandlingDeserializer2 copyright years
* Correct FailedDeserializationInfo this invocation
& ErrorHandlingDeserializer2 java docs headers
*Polishing code style

**Cherry-pick to 2.2.x**

# Conflicts:
#	spring-kafka/src/main/java/org/springframework/kafka/support/serializer/ErrorHandlingDeserializer2.java
#	src/reference/asciidoc/whats-new.adoc
2019-06-27 16:55:08 -04:00
Gary Russell
b0b7805579 Add ConsumerSeekAware Example 2019-06-13 13:00:51 -04:00
Gary Russell
e487ece915 GH-1118: Add RecordInterceptor
Resolves https://github.com/spring-projects/spring-kafka/issues/1118

**cherry-pick to 2.2.x**

# Conflicts:
#	spring-kafka/src/main/java/org/springframework/kafka/config/AbstractKafkaListenerContainerFactory.java
#	spring-kafka/src/main/java/org/springframework/kafka/listener/AbstractMessageListenerContainer.java
#	spring-kafka/src/main/java/org/springframework/kafka/listener/KafkaMessageListenerContainer.java
#	src/reference/asciidoc/whats-new.adoc
2019-06-11 16:43:04 -04:00
Gary Russell
a587b134de Fix docs for batch error handler 2019-05-03 09:51:31 -04:00
Gary Russell
5601017d7b Doc update for scala 2.12
Fixes https://github.com/spring-projects/spring-kafka/issues/1058
2019-04-04 08:43:51 -04:00
Artem Bilan
cca6fc125c Fix Docs typos 2019-04-02 13:03:32 -04:00
Artem Bilan
1241e9b68d Fix URL to SI Idempotent Receiver 2019-03-26 12:33:39 -04:00
Spring Operator
8e9a648f96 URL Cleanup
This commit updates URLs to prefer the https protocol. Redirects are not followed to avoid accidentally expanding intentionally shortened URLs (i.e. if using a URL shortener).

# Fixed URLs

## Fixed But Review Recommended
These URLs were fixed, but the https status was not OK. However, the https status was the same as the http request or http redirected to an https URL, so they were migrated. Your review is recommended.

* [ ] http://docs.spring.io/spring-integration/reference/html/messaging-endpoints-chapter.html (301) with 1 occurrences migrated to:
  https://docs.spring.io/spring-integration/reference/html/messaging-endpoints-chapter.html ([https](https://docs.spring.io/spring-integration/reference/html/messaging-endpoints-chapter.html) result 404).

## Fixed Success
These URLs were switched to an https URL with a 2xx status. While the status was successful, your review is still recommended.

* [ ] http://docs.spring.io/spring-kafka/docs/current/reference/html/ with 1 occurrences migrated to:
  https://docs.spring.io/spring-kafka/docs/current/reference/html/ ([https](https://docs.spring.io/spring-kafka/docs/current/reference/html/) result 200).
* [ ] http://projects.spring.io/spring-kafka/ with 3 occurrences migrated to:
  https://projects.spring.io/spring-kafka/ ([https](https://projects.spring.io/spring-kafka/) result 200).
* [ ] http://stackoverflow.com/questions/134882/undoing-a-git-rebase with 1 occurrences migrated to:
  https://stackoverflow.com/questions/134882/undoing-a-git-rebase ([https](https://stackoverflow.com/questions/134882/undoing-a-git-rebase) result 200).
* [ ] http://stackoverflow.com/questions/tagged/spring-kafka with 1 occurrences migrated to:
  https://stackoverflow.com/questions/tagged/spring-kafka ([https](https://stackoverflow.com/questions/tagged/spring-kafka) result 200).
* [ ] http://www.enterpriseintegrationpatterns.com/patterns/messaging/IdempotentReceiver.html with 1 occurrences migrated to:
  https://www.enterpriseintegrationpatterns.com/patterns/messaging/IdempotentReceiver.html ([https](https://www.enterpriseintegrationpatterns.com/patterns/messaging/IdempotentReceiver.html) result 200).
* [ ] http://contributor-covenant.org with 1 occurrences migrated to:
  https://contributor-covenant.org ([https](https://contributor-covenant.org) result 301).
* [ ] http://contributor-covenant.org/version/1/3/0/ with 1 occurrences migrated to:
  https://contributor-covenant.org/version/1/3/0/ ([https](https://contributor-covenant.org/version/1/3/0/) result 301).
* [ ] http://static.springsource.org/spring-kafka/reference (301) with 1 occurrences migrated to:
  https://docs.spring.io/spring-kafka/reference ([https](https://static.springsource.org/spring-kafka/reference) result 301).
* [ ] http://docs.spring.io/spring/docs/current/spring-framework-reference/html/beans.html with 1 occurrences migrated to:
  https://docs.spring.io/spring/docs/current/spring-framework-reference/html/beans.html ([https](https://docs.spring.io/spring/docs/current/spring-framework-reference/html/beans.html) result 301).
* [ ] http://help.github.com/fork-a-repo/ with 1 occurrences migrated to:
  https://help.github.com/fork-a-repo/ ([https](https://help.github.com/fork-a-repo/) result 301).
* [ ] http://help.github.com/forking/ with 1 occurrences migrated to:
  https://help.github.com/forking/ ([https](https://help.github.com/forking/) result 301).
* [ ] http://www.springsource.com with 2 occurrences migrated to:
  https://www.springsource.com ([https](https://www.springsource.com) result 301).
* [ ] http://git-scm.com/book/en/Distributed-Git-Contributing-to-a-Project with 1 occurrences migrated to:
  https://git-scm.com/book/en/Distributed-Git-Contributing-to-a-Project ([https](https://git-scm.com/book/en/Distributed-Git-Contributing-to-a-Project) result 302).
* [ ] http://git-scm.com/book/en/Git-Branching-Rebasing with 2 occurrences migrated to:
  https://git-scm.com/book/en/Git-Branching-Rebasing ([https](https://git-scm.com/book/en/Git-Branching-Rebasing) result 302).
* [ ] http://www.springsource.org/node/feed with 1 occurrences migrated to:
  https://www.springsource.org/node/feed ([https](https://www.springsource.org/node/feed) result 302).

# Ignored
These URLs were intentionally ignored.

* http://localhost:8080/send/bar/baz with 1 occurrences
* http://localhost:8080/send/foo/bar with 2 occurrences
* http://localhost:8080/send/foos/a,b,c,d,e with 1 occurrences
* http://localhost:8080/send/unknown/xxx with 1 occurrences
2019-03-26 12:32:55 -04:00
Spring Operator
025ea43129 URL Cleanup
This commit updates URLs to prefer the https protocol. Redirects are not followed to avoid accidentally expanding intentionally shortened URLs (i.e. if using a URL shortener).

# Fixed URLs

## Fixed But Review Recommended
These URLs were fixed, but the https status was not OK. However, the https status was the same as the http request or http redirected to an https URL, so they were migrated. Your review is recommended.

* [ ] http://www.puppycrawl.com/dtds/configuration_1_2.dtd (404) with 1 occurrences migrated to:
  https://www.puppycrawl.com/dtds/configuration_1_2.dtd ([https](https://www.puppycrawl.com/dtds/configuration_1_2.dtd) result 404).
* [ ] http://www.puppycrawl.com/dtds/suppressions_1_1.dtd (404) with 1 occurrences migrated to:
  https://www.puppycrawl.com/dtds/suppressions_1_1.dtd ([https](https://www.puppycrawl.com/dtds/suppressions_1_1.dtd) result 404).

# Ignored
These URLs were intentionally ignored.

* http://maven.apache.org/POM/4.0.0 with 6 occurrences
* http://www.w3.org/2001/XMLSchema-instance with 3 occurrences
2019-03-26 12:09:56 -04:00
Artem Bilan
02de8a2e91 URL Cleanup
This commit updates URLs to prefer the https protocol.
Redirects are not followed to avoid accidentally expanding intentionally shortened URLs (i.e. if using a URL shortener).

# Fixed URLs

## Fixed Success
These URLs were switched to an https URL with a 2xx status.
While the status was successful, your review is still recommended.

* [ ] http://www.apache.org/licenses/ with 1 occurrences migrated to:
  https://www.apache.org/licenses/ ([https](https://www.apache.org/licenses/) result 200).
* [ ] http://www.apache.org/licenses/LICENSE-2.0 with 238 occurrences migrated to:
  https://www.apache.org/licenses/LICENSE-2.0 ([https](https://www.apache.org/licenses/LICENSE-2.0) result 200).
2019-03-21 18:57:31 -04:00
Gary Russell
666fd5c6b0 GH-1012 Allow raw Strings in default header mapper
Resolves https://github.com/spring-projects/spring-kafka/issues/1012

Add configuration to map string-valued headers as raw `byte[]`
instead of adding to the map of json-mapped headers.

* Polishing - PR Comments.
2019-03-21 18:41:03 -04:00
Gary Russell
c56e8e7890 Add gateway links in SIK chapter 2019-03-20 13:29:15 -04:00
Gary Russell
c226c9ee8b GH-990: Run AfterRollbackProcessor in Tx
Resolves https://github.com/spring-projects/spring-kafka/issues/990

Provide a mechanism to start a new transaction within which to invoke
the processor, so if it recovers the failed record, its offset can
be sent to the transaction.

**cherry-pick to 2.2.x**

# Conflicts:
#	spring-kafka/src/main/java/org/springframework/kafka/listener/DeadLetterPublishingRecoverer.java
2019-03-08 13:06:43 -05:00
Gary Russell
45ddac7fea GH-970: Add getGroupId() to containers
Resolves https://github.com/spring-projects/spring-kafka/issues/970

- allow retrieval of the `group.id`, even if not set on the container properties
- also add `getAllListenerContainers` to the `RLERegistry` as a convenience
- also add `getListenerId` to return the id or bean name of the container
2019-02-22 12:15:05 -05:00
Gary Russell
7edf9f4f41 GH-960: Seek-to-Current - commit recovered offset (#961)
* GH-960: Seek-to-Current - commit recovered offset

Resolves https://github.com/spring-projects/spring-kafka/issues/960

When the container is configured with `AckMode.MANUAL_IMMEDIATE`, the
`SeekToCurrentErrorHandler` can be configured to commit the offset of
a recovered record.

* Polishing - PR Comments
2019-02-12 19:04:47 -05:00
Gary Russell
ff174e0a95 @KafkaListener Consumer Property Overrides
Add the `properties` attribute to the `@KafkaListener` annotation to
enhance or override the consumer factory properties.
2019-02-08 13:35:32 -05:00
Gary Russell
c5b2ee47cb GH-946: Detect Unnecessary Replying Template
Resolves https://github.com/spring-projects/spring-kafka/issues/946
2019-02-08 13:32:48 -05:00
Quentin PROUST
ee240006b3 fix outdated references to DEFAULT_STREAMS_CONFIG_BEAN_NAME
Fix existing code that show declaration of bean DEFAULT_STREAMS_CONFIG_BEAN_NAME
as a `Properties` object or a `StreamsConfig` object.
2019-02-08 12:11:56 -05:00
Jay Bryant
f6caf5bb1e Update to the build file to modernize the docs
This commit throws out Docbook and gives us the modern
look and feel for the docs.
It uses Asciidoctor to generate the PDF.

* Polishing for `build.gradle` in regards to asciidoc
* Upgrade to the latest Asciidoc and its plugins
* Combine Asciidoc backends into a single Gradle task
* Fix issues with section levels, duplicated ids and project version in
the docs
* Remove obsolete files: not used by Asciidoc
2019-02-02 11:55:01 -05:00
Jay Bryant
84a7a706ab Editing pass for spring-kafka
I edited for clarity, punctuation, grammar, spelling, usage, and corporate voice.
I also added a few links (mostly to Javadoc).

Further changes to account for changes made while I was working

Someone made changes to a few paragraphs, so I edited the changed material, too.

* Fix "One Sentence per Line" violations
2019-01-31 14:54:33 -05:00
Gary Russell
2907f9cc51 GH-948: Enhance LEFException with group.id
Resolves https://github.com/spring-projects/spring-kafka/issues/948

For example, this can be used by the `DeadLetterPublishingRecoverer`'s
destination resolver to choose a topic based on the group in addition
to the information in the consumer record.

* Polishing - @Nullable on getGroupId().
2019-01-29 14:56:06 -05:00
inponomarev
ab344af41a Introduce KafkaStreamBrancher
review fixes

Test->Tests rename

altered KafkaStreamsBranchTests to use KafkaStreamBrancher

* Polishing code style
* Add `whats-new.adoc`
* Exclude `org.junit.jupiter.api.Assertions` for static imports
2019-01-22 11:03:32 -05:00
Johnny Lim
bda5d99717 Polish doc 2019-01-17 09:49:18 -05:00
SLourenco
4c6d6f5767 GH-917: Specify the port with @EmbeddedKafka
Fixes spring-projects/spring-kafka#917

Added ports attribute to EmbeddedKafka annotation.
It has a default value of 0 to be consistent with the count attribute.

* #917: Added Tests and update adoc files

Created a unit test similarly to the EmbeddedKafkaBroker test for validating the system properties created with the EmbeddedKafka bean creation.

Updated copyright and author/since in relevant files.

Updated .adoc files to reflect the new attribute.

* #917 Corrected since annotations and Unit tests

- Corrected since annotations from last commit
- Moved unit test of EmbeddedKafka annotation from spring-kafka to spring-kafka-test. Refactored unit test to avoid bean initialization and broker creation.

* #917 Corrected white space trailling errors

* #917 Corrected style for spring-kafka-test

* #917 Corrected checkstyle
2019-01-15 11:42:39 -05:00
Gary Russell
6d05a03e23 Doc: Add anchor for annotation properties
https://stackoverflow.com/questions/54038928/how-to-pass-dynamic-topic-name-to-kafkalistenertopics-from-environment-variabl/54040969#54040969

Previously there was no break between discussing batch listeners and general
annotation properties.
2019-01-04 11:22:32 -05:00
Gary Russell
88869e6ad6 JsonDeserializer Header Removal Polishing
Only remove the headers for the object being deserialized.

Previously, it removed all type headers, with explicit reference to the key
headers. This happened to work ok because the key was deserialized first.

If this ever changes, the key headers would be removed when the value is
deserialized and would not be available for key deserialization.

* `JsonDeserializer`: add `spring.json.use.type.headers` configuration property
2019-01-03 11:08:46 -05:00
Gary Russell
e398d3448b SI Outbound result channel docs
Since we added gateways, a dedicated success channel is now provided instead
of using the output channel for result metadata.
2018-12-19 16:04:06 -05:00
Gary Russell
9fef537b38 GH-906: @KafkaListener Javadoc Improvements
Resolves https://github.com/spring-projects/spring-kafka/issues/906
2018-12-17 17:15:58 -05:00
Gary Russell
a2255b8381 Doc polishing 2018-12-14 10:44:02 -05:00
Johnny Lim
e22527f09f Add a missing closing pointy bracket in doc 2018-12-03 21:53:28 -05:00
Gary Russell
b46255162a Polish Replying Template config with Boot
https://stackoverflow.com/questions/53573729/spring-kafka-autoconfiguration-issue-using-replyingkafkatemplate/53575126#53575126
2018-12-03 11:36:36 -05:00
Gary Russell
4390405eff Add appendix for 2.1.x kafka-clients jar 2018-11-27 09:35:09 -05:00
Gary Russell
0a953b0e51 GH-885: Restore infinite retries STCEH
Resolves https://github.com/spring-projects/spring-kafka/issues/885

`SeekToCurrentErrorHandler` and `DefaultAfterRollbackProcessor`,
with negative `maxFailures`, now never recovers and continues to retry
indefinitely.
2018-11-21 10:52:43 -05:00
Gary Russell
810529a978 GH-854: Docs about Filtering Batch adapter
Rsolves https://github.com/spring-projects/spring-kafka/issues/854
2018-11-19 15:53:44 -05:00
Gary Russell
45b5d8b1be GH-878: Add parent container to container events
Fixes: https://github.com/spring-projects/spring-kafka/issues/878
2018-11-19 15:31:19 -05:00
Gary Russell
fd0fbaab9a Add Docs for @SpringBootTest 2018-11-19 13:37:51 -05:00
Gary Russell
c5f485d2dd GH-867: Add ConsumerStoppingEvent
Resolves https://github.com/spring-projects/spring-kafka/issues/867
2018-11-09 11:13:08 -05:00
Gary Russell
074e9613e2 GH-853: Type Safe ErrorHandlingDeserializer
Resolves https://github.com/spring-projects/spring-kafka/issues/853

Since a `null` key is common, we only check for the exception header
if we detect that the error handling deserializer is configured.

* Polishing; add failedDeserializationFunction
2018-11-06 10:00:04 -05:00
Gary Russell
7bf2c647ff GH-862: Fix docs for ErrorHandler
Fixeds https://github.com/spring-projects/spring-kafka/issues/862
2018-11-06 09:55:39 -05:00
Gary Russell
d19189eea2 GH-847: Support placeholders in @SendTo
Resolves https://github.com/spring-projects/spring-kafka/issues/847

**cherry-pick to 2.1.x**
2018-11-01 16:15:27 -04:00
Gary Russell
ac2191c434 Doc fixes 2018-10-29 11:03:17 -04:00