Spring/spring-integration
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,361 Commits 25 Branches 0 Tags
aaaaf86c074ab33307446fe3f3fcbf66f82df58a
Commit Graph

10361 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Gary Russell
aaaaf86c07 GH-296: DSL: Support inline pub/sub subscriptions 
Resolves https://github.com/spring-projects/spring-integration-kafka/issues/296

* * Add generic type to specsto avoid cast to `BroadcastCapableChannel`.

* * Remove unneeded fields in the abstract spec

* Restore log4j config
 2020-06-25 11:29:06 -04:00
Gary Russell
2331eee642 GH-296: Add Kafka-backed MessageChannels 
Resolves https://github.com/spring-projects/spring-integration-kafka/issues/296

* * Add DSL support

* * Polishing and XML support
 2020-06-25 11:29:06 -04:00
Gary Russell
905a5155bc Master to 3.3; support non-Tx if template allows 
* SI to 5.3.0.B-S
 2020-06-25 11:29:06 -04:00
Artem Bilan
f596e49242 Upgrade to latest releases 
* Upgrade to Gradle 6.1.1
* Fix for latest Checkstyle
* Fix for latest SI-Kotlin-DSL
 2020-06-25 11:29:06 -04:00
Artem Bilan
27f0775fb8 Apply SI-Kotlin-DSL in tests (#292) 
* Apply SI-Kotlin-DSL in tests

* Upgrade dependencies

* Upgrade to latest releases
 2020-06-25 11:29:05 -04:00
Gary Russell
386c64c617 Support ProducerRecord outbound payloads 2020-06-25 11:29:05 -04:00
Gary Russell
cf3aa29a23 GH-290: Add ProducerRecordCreator 
Resolves https://github.com/spring-projects/spring-integration-kafka/issues/290
 2020-06-25 11:29:05 -04:00
Gary Russell
6cddb2c169 Revert "Apply SI-Kotlin-DSL in tests" 
This reverts commit b03ddcb953c18469564994921efb84a072a0e358.
 2020-06-25 11:29:05 -04:00
Artem Bilan
9b0b87d37e Apply SI-Kotlin-DSL in tests 2020-06-25 11:29:05 -04:00
Artem Bilan
bf0134cb52 Upgrade to latest GAs 
* Move all the test to JUnit 5; remove redundant JUnit 4 dependencies
 2020-06-25 11:29:04 -04:00
tomvandenberge
0e74cd1cee Add XML attribute for header-mapper 
* Added support for header-mapper to outbound-channel-adapter and
outbound-gateway XML

* Follow up on review

* Corrected mistake

* Corrected indentation
 2020-06-25 11:29:04 -04:00
Gary Russell
8cc2b2cc00 KMessageSource getter for consumer properties 
- allow customization after creation - e.g. SCSt `MessageSourceCustomizer`
- also fix deprecation in `ReplyingKafkaTemplate`
 2020-06-25 11:29:04 -04:00
Gary Russell
adf91edf05 KMSource: Consumer Property Overrides 
- support property overrides in the message source
 2020-06-25 11:29:04 -04:00
Artem Bilan
f6d858c7aa Polishing JavaDocs and possible NPEs 
https://build.spring.io/browse/INTEXT-KAFKA-310/
 2020-06-25 11:29:04 -04:00
Gary Russell
3831919cbc GH-282: KMSource - support async commits etc 
Resolves https://github.com/spring-projects/spring-integration-kafka/issues/282

Use sync-related properties from `ConsumerProperties`.
 2020-06-25 11:29:03 -04:00
Anshul Mehra
c20ef85168 GH-280: Add topicPattern support to KafkaMessageSource 
Fixes https://github.com/spring-projects/spring-integration-kafka/issues/280

* Refactored to topics/topicPattern to be a property to
prevent constructor telescoping

* Add support for manual partition assignment

- Cleanup after rebasing from #283
- Add additional tests

* Fix checkstyle

* Remove redundant factory methods

* Add better test for static assignment

* Use ConsumerProperties 

Fix typo where ContainerProperties was used 
instead of ConsumerProperties

* Simplify construction

* Remove unused import

* Use ObjectUtils

* Use component name for clientId

* Collapse static assignment for loop

* Fix lint warning
 2020-06-25 11:29:03 -04:00
Anshul Mehra
e52c535f7c Use ConsumerProperties in KafkaMessageSource 
* Preserve existing constructors

* Add @deprecated in java docs
 2020-06-25 11:29:03 -04:00
Gary Russell
e6cdb8818f GH-279: Fix tests for latest SK Snapshots 
- add tests for tx.id overrides
 2020-06-25 11:29:03 -04:00
Urs Keller
4cd81fda78 Fix NPE for retryTemp!=null but recoveryCall==null 
Starting with version `3.0.x`, the `KafkaInboundGateway` and
`KafkaMessageDrivenChannelAdapter` rely on the
`RetryContext.getRetryCount()` for the
`IntegrationMessageHeaderAccessor.DELIVERY_ATTEMPT` message header, but
the `RetryContext` is obtained from the `TheadLocal` `attributesHolder`,
which is populated from the `RetryListener.open()` only if
`recoveryCallback != null`

* Fix `KafkaInboundGateway` and `KafkaMessageDrivenChannelAdapter` to
populate `TheadLocal` `attributesHolder` with the `RetryContext`
from the `RetryListener.open()` when only `retryTemplate != null`

**Cherry-pick to `master` & 3.0.x**

# Conflicts:
#	src/test/java/org/springframework/integration/kafka/inbound/MessageDrivenAdapterTests.java
 2020-06-25 11:29:02 -04:00
Artem Bilan
d96a2d930c Add HTTPS entries for XSD into spring.schemas 
Related to spring-projects/spring-integration#2987

**Cherry-pick to 3.1.x, 3.0.x & 2.3.x**
 2020-06-25 11:29:02 -04:00
Gary Russell
41de7495e6 Resolve deprecations and other compiler warnings 2020-06-25 11:29:02 -04:00
Cameron Mayfield
600e88ec56 GH-272: Add KafkaMDrivenChAdapterSpec.payloadType 
Fixes https://github.com/spring-projects/spring-integration-kafka/issues/272

Add `payloadType` option into `KafkaMessageDrivenChannelAdapterSpec`
 2020-06-25 11:29:02 -04:00
Gary Russell
a1b51c9f47 Fix tests for SK 2.3.0.M3 2020-06-25 11:29:02 -04:00
Gary Russell
bc443474cb Pausable package change 2020-06-25 11:29:01 -04:00
Gary Russell
14302088c5 SIGH-2958: Add source record to common SI Header 
Resolves https://github.com/spring-projects/spring-integration/issues/2958

Enables applications to be agnostic regarding the source of
a message. e.g Kafka Vs. AMQP.
 2020-06-25 11:29:01 -04:00
Gary Russell
c25b3b45f4 Allow fetching multiple records 
Resolves https://github.com/spring-projects/spring-integration-kafka/issues/267

- add an option to allow fetching multiple records per poll

* DSL for new constructor arg
 2020-06-25 11:29:01 -04:00
Anshul Mehra
cd6d17e4b0 GH-265: Ensure assignTimeout is used when no partitions are assigned 
* Refactored to rely on assignedPartitions for determining whether to
use pollTimeout or assignTimeout
* Removed redundant assigned flag

fixes #265
 2020-06-25 11:29:01 -04:00
Gary Russell
28fc819fe9 Change MessageSource.payloadType to Class<?> 
- consistency with other endpoints
- avoid the need for SpEL in XML config
 2020-06-25 11:29:01 -04:00
Gary Russell
22b1e70e13 GH-195: Add XML Schema support for new components 
Resolves https://github.com/spring-projects/spring-integration-kafka/issues/195

- gateways
- message source

Polishing

Don't start adapters in parser tests

Fix XML filename for test
 2020-06-25 11:29:00 -04:00
Gary Russell
f59a3687a9 Fix race in KafkaProducerMessageHandlerTests. 2020-06-25 11:29:00 -04:00
Gary Russell
e9833af732 Fix javadoFix javadoc 2020-06-25 11:29:00 -04:00
Gary Russell
e76fd2bea2 Remove Lifecyle since it's the super of Pausable 2020-06-25 11:29:00 -04:00
Gary Russell
ca125a8a6f Polishing - PR comments. 2020-06-25 11:29:00 -04:00
Gary Russell
087247d96f GH-259: Add pause/resume to KafkaMessageSource 
Resolves https://github.com/spring-projects/spring-integration-kafka/issues/259

Enable pause/resume on the polled consumer.

**cherry-pick to 3.1.x**
 2020-06-25 11:28:59 -04:00
Gary Russell
ceb68b40ef GH-260: Fix outbound tx synchronization 
Resolves https://github.com/spring-projects/spring-integration-kafka/issues/260

The outbound endpoint used
`TransactionSynchronizationManager.isActualTransactionActive()`
to detect a global transaction instead of
`TransactionSynchronizationManager.getResource(this.kafkaTemplate.getProducerFactory()) == null`.

The KTM uses `SYNCHRONIZATION_NEVER` by default so the first will
return false unless the synchronization is explicitly set. This
causes a new transaction to be started.

Use the presence of the resource holder instead.

**cherry-pick to 3.1.x**
 2020-06-25 11:28:59 -04:00
Artem Bilan
cc3895f40d Fix XSD for the current version 2020-06-25 11:28:59 -04:00
Artem Bilan
eb7f94ae0d URL Cleanup 
This commit updates URLs to prefer the https protocol.
Redirects are not followed to avoid accidentally expanding intentionally
shortened URLs (i.e. if using a URL shortener).

# Fixed URLs

## Fixed Success
These URLs were switched to an https URL with a 2xx status.
While the status was successful, your review is still recommended.

* [ ] http://www.apache.org/licenses/ with 1 occurrences migrated to:
  https://www.apache.org/licenses/ ([https](https://www.apache.org/licenses/) result 200).
* [ ] http://www.apache.org/licenses/LICENSE-2.0 with 28 occurrences migrated to:
  https://www.apache.org/licenses/LICENSE-2.0 ([https](https://www.apache.org/licenses/LICENSE-2.0) result 200).
 2020-06-25 11:28:59 -04:00
Gary Russell
71cdeee0b3 GH-235: KPMH: Flush the template in stop() 
Resolves https://github.com/spring-projects/spring-integration-kafka/issues/235

**cherry-pick to 3.1.x**
 2020-06-25 11:28:58 -04:00
Spring Operator
4f54c4cab3 URL Cleanup 
This commit updates URLs to prefer the https protocol. Redirects are not followed to avoid accidentally expanding intentionally shortened URLs (i.e. if using a URL shortener).

# Fixed URLs

## Fixed But Review Recommended
These URLs were fixed, but the https status was not OK. However, the https status was the same as the http request or http redirected to an https URL, so they were migrated. Your review is recommended.

* [ ] http://help.github.com/send-pull-requests (404) with 1 occurrences migrated to:
  https://help.github.com/send-pull-requests ([https](https://help.github.com/send-pull-requests) result 404).

## Fixed Success
These URLs were switched to an https URL with a 2xx status. While the status was successful, your review is still recommended.

* [ ] http://kafka.apache.org/ with 1 occurrences migrated to:
  https://kafka.apache.org/ ([https](https://kafka.apache.org/) result 200).
* [ ] http://www.apache.org with 1 occurrences migrated to:
  https://www.apache.org ([https](https://www.apache.org) result 200).
* [ ] http://www.springframework.org/schema/integration/spring-integration.xsd with 1 occurrences migrated to:
  https://www.springframework.org/schema/integration/spring-integration.xsd ([https](https://www.springframework.org/schema/integration/spring-integration.xsd) result 200).
* [ ] http://static.springsource.org/spring-integration/reference (301) with 1 occurrences migrated to:
  https://docs.spring.io/spring-integration/reference ([https](https://static.springsource.org/spring-integration/reference) result 301).
* [ ] http://www.springframework.org with 1 occurrences migrated to:
  https://www.springframework.org ([https](https://www.springframework.org) result 301).
* [ ] http://www.springsource.com with 2 occurrences migrated to:
  https://www.springsource.com ([https](https://www.springsource.com) result 301).

# Ignored
These URLs were intentionally ignored.

* http://www.springframework.org/schema/beans with 1 occurrences
* http://www.springframework.org/schema/integration with 2 occurrences
* http://www.springframework.org/schema/integration/kafka with 2 occurrences
* http://www.springframework.org/schema/tool with 2 occurrences
* http://www.w3.org/2001/XMLSchema with 1 occurrences
 2020-06-25 11:28:58 -04:00
Spring Operator
9b670950a5 URL Cleanup 
This commit updates URLs to prefer the https protocol. Redirects are not followed to avoid accidentally expanding intentionally shortened URLs (i.e. if using a URL shortener).

# Fixed URLs

## Fixed But Review Recommended
These URLs were fixed, but the https status was not OK. However, the https status was the same as the http request or http redirected to an https URL, so they were migrated. Your review is recommended.

* http://www.puppycrawl.com/dtds/configuration_1_2.dtd (404) with 1 occurrences migrated to:
  https://www.puppycrawl.com/dtds/configuration_1_2.dtd ([https](https://www.puppycrawl.com/dtds/configuration_1_2.dtd) result 404).
* http://www.puppycrawl.com/dtds/suppressions_1_1.dtd (404) with 1 occurrences migrated to:
  https://www.puppycrawl.com/dtds/suppressions_1_1.dtd ([https](https://www.puppycrawl.com/dtds/suppressions_1_1.dtd) result 404).

## Fixed Success
These URLs were switched to an https URL with a 2xx status. While the status was successful, your review is still recommended.

* http://www.springframework.org/schema/beans/spring-beans.xsd with 2 occurrences migrated to:
  https://www.springframework.org/schema/beans/spring-beans.xsd ([https](https://www.springframework.org/schema/beans/spring-beans.xsd) result 200).
* http://www.springframework.org/schema/context/spring-context.xsd with 1 occurrences migrated to:
  https://www.springframework.org/schema/context/spring-context.xsd ([https](https://www.springframework.org/schema/context/spring-context.xsd) result 200).
* http://www.springframework.org/schema/integration/kafka/spring-integration-kafka.xsd with 2 occurrences migrated to:
  https://www.springframework.org/schema/integration/kafka/spring-integration-kafka.xsd ([https](https://www.springframework.org/schema/integration/kafka/spring-integration-kafka.xsd) result 200).
* http://www.springframework.org/schema/integration/spring-integration.xsd with 1 occurrences migrated to:
  https://www.springframework.org/schema/integration/spring-integration.xsd ([https](https://www.springframework.org/schema/integration/spring-integration.xsd) result 200).

# Ignored
These URLs were intentionally ignored.

* http://www.springframework.org/schema/beans with 4 occurrences
* http://www.springframework.org/schema/context with 2 occurrences
* http://www.springframework.org/schema/integration with 2 occurrences
* http://www.springframework.org/schema/integration/kafka with 4 occurrences
* http://www.w3.org/2001/XMLSchema-instance with 2 occurrences

* Upgrade Checkstyle and its DTDs for HTTPS
 2020-06-25 11:28:58 -04:00
Gary Russell
7fae571c2d Changes for compatibility with Spring Kafka 2.3 
Also remove test for bad partition in DSL tests - the send now gives an error
about the topic not being present in metadata instead of "not in range".

Use sync for publishing in DSL tests, to fail fast.
 2020-06-25 11:28:58 -04:00
Artem Bilan
daedfb9210 Upgrade dependencies and fix issues 2020-06-25 11:28:58 -04:00
Gary Russell
e5b861dcea Remove unnecessary throws Exception clause 2020-06-25 11:28:57 -04:00
Gary Russell
1873e9ae41 GH-211: MessageSource acks; fix concurrency 
Fixes https://github.com/spring-projects/spring-integration-kafka/issues/211

Use `ConcurrentHashMap` with synchronized set values to avoid concurrency issues
when acks are processed on multiple threads.
 2020-06-25 11:28:57 -04:00
Gary Russell
759e0e236d Nullable fixes 2020-06-25 11:28:57 -04:00
Gary Russell
19c3c4607f Fix TODOs in outbound channel adapter 
- always use `successChannel` for metadata
- unwrap `ExecutionException`
 2020-06-25 11:28:57 -04:00
Artem Bilan
f29bd3e322 GH-212: Add ConsumerSeekAware impl to Inbounds (#213) 
* GH-212: Add ConsumerSeekAware impl to Inbounds

Fixes spring-projects/spring-integration-kafka#212

**Cherry-pick to 3.0.x**

* GH-212: Add ConsumerSeekAware impl to Inbounds

Fixes spring-projects/spring-integration-kafka#212

* Introduce a new `IntegrationKafkaHeaders.CONSUMER_SEEK_CALLBACK`
header to be populated to messages for sending to the channel
* Populate that header from the `KafkaInboundGateway` and
`KafkaMessageDrivenChannelAdapter` into the message from the
`seekCallBack` property if `ListenerContainer` is single-threaded or
from the `ThreadLocal<ConsumerSeekAware.ConsumerSeekCallback>` otherwise;
and only if newly introduced `setAdditionalHeaders` is `true`
* Populate `seekCallBack` property or `ThreadLocal<?>` from the
`registerSeekCallback()` implementation from the internal listeners
* Add `setOnPartitionsAssignedSeekCallback(BiConsumer)` and
`setOnIdleSeekCallback(BiConsumer)` options to react for the appropriate
event from the underlying container and perform appropriate seek management
* Add new options to the DSL classes and cover them with tests, including
check for new `IntegrationKafkaHeaders.CONSUMER_SEEK_CALLBACK` header

**Cherry-pick to 3.0.x**

* Address PR comments: remove unnecessary API

* *Polishing `setOnPartitionsAssignedSeekCallback()` JavaDocs
*Close producers in the `KafkaProducerMessageHandlerTests`
 2020-06-25 11:28:57 -04:00
Gary Russell
bedc0c8fdc GH-209: Upgrade to kafka-clients 2.0.0 
Fixes https://github.com/spring-projects/spring-integration-kafka/issues/209

* README Polishing
 2020-06-25 11:28:56 -04:00
Artem Bilan
0c1cfbc977 Upgrade to SI-5.1 M1, SK-2.2 M1 
* Fixes according the upgrades
* Polishing some code style
 2020-06-25 11:28:56 -04:00
Gary Russell
8decbe7810 GH-204: DSL Register container as bean if needed 
Fixes https://github.com/spring-projects/spring-integration-kafka/issues/204

When an external container is provided to the DSL, register it as a bean
if it is not already a bean.

Polishing id from PR comments; add gateway support too.

* Polishing JavaDocs and omissions in the test-case
 2020-06-25 11:28:56 -04:00