Spring/spring-integration-samples
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

INTSAMPLES-147: Add Splunk example

Browse Source 
JIRA: https://jira.spring.io/browse/INTSAMPLES-147
This commit is contained in:
Filippo Balicchia
2015-11-22 22:09:05 +01:00
committed by Artem Bilan
parent 8f7fb189c0
commit b9697909d0
11 changed files with 523 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

51
basic/splunk/README.md Normal file
View File

@@ -0,0 +1,51 @@
Splunk Sample
==============
This example demonstrates the use of Splunk **Outbound** and **Inbound** Adapter.
As configuration example is used a xml file
## Running the sample
Start Splunk server and change references in application.yml file.
type from the root project
$ gradlew :splunk:run
#### Using an IDE such as eclipse
Type from <spring-integration>/basic/splunk
mvn eclipse:eclipse
import the project and run it **org.springframework.integration.samples.splunk.Application** like any other main
### Output
The application sends 10 "custom" SplunkEvent to a main index and the read all event with yesterday as earliest-time
the received message are printed on sysout with stream adapter
Program output:
Producer:
[2015-11-22 21:16:04:546+0100 _bkt="main~2~783FBE3F-8AA4-4DE3-BC2C-CA406157661C" _cd="2:23" _serial="0" _raw="2015-11-22 18:38:08:923+0100 ean="164" severity="ALERT" email="mail@gmail.com" order_number="21501001010101"" splunk_server="fbalicchia-desktop" index="main" source="tinyMain" _indextime="1448213888" _subsecond=".923" linecount="1" _si="fbalicchia-desktop,main" host="127.0.0.1" _sourcetype="customMessage" sourcetype="customMessage" _time="2015-11-22T18:38:08.923+01:00"]
[2015-11-22 21:16:04:546+0100 _bkt="main~2~783FBE3F-8AA4-4DE3-BC2C-CA406157661C" _cd="2:18" _serial="1" _raw="2015-11-22 18:38:08:822+0100 ean="62" severity="ALERT" email="mail@gmail.com" order_number="21501001010101"" splunk_server="fbalicchia-desktop" index="main" source="tinyMain" _indextime="1448213888" _subsecond=".822" linecount="1" _si="fbalicchia-desktop,main" host="127.0.0.1" _sourcetype="customMessage" sourcetype="customMessage" _time="2015-11-22T18:38:08.822+01:00"]
[2015-11-22 21:16:04:546+0100 _bkt="main~2~783FBE3F-8AA4-4DE3-BC2C-CA406157661C" _cd="2:13" _serial="2" _raw="2015-11-22 16:26:50:770+0100 ean="194" severity="ALERT" email="mail@gmail.com" order_number="21501001010101"" splunk_server="fbalicchia-desktop" index="main" source="tinyMain" _indextime="1448206010" _subsecond=".770" linecount="1" _si="fbalicchia-desktop,main" host="127.0.0.1" _sourcetype="customMessage" sourcetype="customMessage" _time="2015-11-22T16:26:50.770+01:00"]
[2015-11-22 21:16:04:546+0100 _bkt="main~2~783FBE3F-8AA4-4DE3-BC2C-CA406157661C" _cd="2:8" _serial="3" _raw="2015-11-22 16:26:50:670+0100 ean="42" severity="ALERT" email="mail@gmail.com" order_number="21501001010101"" splunk_server="fbalicchia-desktop" index="main" source="tinyMain" _indextime="1448206010" _subsecond=".670" linecount="1" _si="fbalicchia-desktop,main" host="127.0.0.1" _sourcetype="customMessage" sourcetype="customMessage" _time="2015-11-22T16:26:50.670+01:00"]
]
In this snippet we see toString() of message that send from program to Splunk, you can see the custom fields included email, ean code ecc.ecc
Consumer:
[2015-11-22 21:32:33:467+0100 _bkt="main~2~783FBE3F-8AA4-4DE3-BC2C-CA406157661C" _cd="2:158" _serial="0" _raw="2015-11-22 21:25:50:956+0100 ean="216" severity="ALERT" email="mail@gmail.com" order_number="21501001010101"" splunk_server="fbalicchia-desktop" index="main" _kv="1" source="tinyMain" _indextime="1448223950" _subsecond=".956" ean="216" linecount="1" _si="fbalicchia-desktop,main" host="127.0.0.1" _sourcetype="customMessage" sourcetype="customMessage" _time="2015-11-22T21:25:50.956+01:00"]
[2015-11-22 21:32:33:467+0100 _bkt="main~2~783FBE3F-8AA4-4DE3-BC2C-CA406157661C" _cd="2:153" _serial="1" _raw="2015-11-22 21:25:50:936+0100 ean="504" severity="ALERT" email="mail@gmail.com" order_number="21501001010101"" splunk_server="fbalicchia-desktop" index="main" _kv="1" source="tinyMain" _indextime="1448223950" _subsecond=".936" ean="504" linecount="1" _si="fbalicchia-desktop,main" host="127.0.0.1" _sourcetype="customMessage" sourcetype="customMessage" _time="2015-11-22T21:25:50.936+01:00"]
[2015-11-22 21:32:33:467+0100 _bkt="main~2~783FBE3F-8AA4-4DE3-BC2C-CA406157661C" _cd="2:148" _serial="2" _raw="2015-11-22 21:25:50:916+0100 ean="357" severity="ALERT" email="mail@gmail.com" order_number="21501001010101"" splunk_server="fbalicchia-desktop" index="main" _kv="1" source="tinyMain" _indextime="1448223950" _subsecond=".916" ean="357" linecount="1" _si="fbalicchia-desktop,main" host="127.0.0.1" _sourcetype="customMessage" sourcetype="customMessage" _time="2015-11-22T21:25:50.916+01:00"]
The layout of the message is the same but we found entry with ean code greater than '135'. Splunk support out of the box a good query languages where more info can be found at splunk project documentation
> Please note that Before use this example, please, verify that **jdk.tls.disabledAlgorithms=SSLv3** entry in **lib/security/java.security** is commenting out

135
basic/splunk/pom.xml Normal file
View File

@@ -0,0 +1,135 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.3.0.RELEASE</version>
</parent>
<groupId>org.springframework.integration.samples</groupId>
<artifactId>splunk</artifactId>
<version>4.1.0.BUILD-SNAPSHOT</version>
<name>Apache splunk Sample</name>
<description>Apache splunk Sample</description>
<url>http://projects.spring.io/spring-integration</url>
<organization>
<name>SpringIO</name>
<url>https://spring.io</url>
</organization>
<licenses>
<license>
<name>The Apache Software License, Version 2.0</name>
<url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
<distribution>repo</distribution>
</license>
</licenses>
<developers>
<developer>
<id>garyrussell</id>
<name>Gary Russell</name>
<email>grussell@pivotal.io</email>
<roles>
<role>project lead</role>
</roles>
</developer>
<developer>
<id>markfisher</id>
<name>Mark Fisher</name>
<email>mfisher@pivotal.io</email>
<roles>
<role>project founder and lead emeritus</role>
</roles>
</developer>
<developer>
<id>ghillert</id>
<name>Gunnar Hillert</name>
<email>ghillert@pivotal.io</email>
</developer>
<developer>
<id>abilan</id>
<name>Artem Bilan</name>
<email>abilan@pivotal.io</email>
</developer>
</developers>
<scm>
<connection>scm:git:scm:git:git://github.com/spring-projects/spring-integration-samples.git</connection>
<developerConnection>scm:git:scm:git:ssh://git@github.com:spring-projects/spring-integration-samples.git</developerConnection>
<url>https://github.com/spring-projects/spring-integration-samples</url>
</scm>
<dependencies>
<dependency>
<groupId>org.springframework.integration</groupId>
<artifactId>spring-integration-core</artifactId>
<version>4.2.1.RELEASE</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-integration</artifactId>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>com.splunk</groupId>
<artifactId>splunk</artifactId>
<version>1.5.0.0</version>
</dependency>
<dependency>
<groupId>org.springframework.integration</groupId>
<artifactId>spring-integration-splunk</artifactId>
<version>1.1.0.RELEASE</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
</exclusions>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
<version>4.2.0.RELEASE</version>
<scope>test</scope>
</dependency>
</dependencies>
<repositories>
<repository>
<id>repo.spring.io.milestone</id>
<name>Spring Framework Maven Milestone Repository</name>
<url>https://repo.spring.io/libs-milestone</url>
</repository>
<repository>
<id>repo.spring.io.snapshot</id>
<name>Spring Framework Maven Snapshot Repository</name>
<url>https://repo.spring.io/libs-snapshot</url>
</repository>
</repositories>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>

87
basic/splunk/src/main/java/org/springframework/integration/samples/splunk/Application.java Normal file
View File

@@ -0,0 +1,87 @@
/*
* Copyright 2015 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.integration.samples.splunk;
import java.util.concurrent.ThreadLocalRandom;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ImportResource;
import org.springframework.integration.samples.splunk.event.OrderEvent;
import org.springframework.integration.splunk.event.SplunkEvent;
import org.springframework.integration.splunk.support.SplunkServer;
import org.springframework.integration.support.MessageBuilder;
import org.springframework.messaging.MessageChannel;
/**
* @author Filippo Balicchia
* @since 4.2
*/
@SpringBootApplication
@ImportResource("si-splunk-example-context.xml")
public class Application {
@Value("${splunk.host}")
private String host;
@Value("${splunk.port}")
private String port;
@Value("${splunk.username}")
private String username;
@Value("${splunk.password}")
private String password;
@Value("${splunk.owner}")
private String owner;
public static void main(String[] args) throws Exception {
ConfigurableApplicationContext context = SpringApplication.run(Application.class, args);
MessageChannel channelRestOutput = context.getBean("toSplunk", MessageChannel.class);
sendWithRest(channelRestOutput);
System.out.println("Consumer");
}
private static void sendWithRest(MessageChannel channel) {
for (int i = 0; i < 10; i++) {
channel.send(MessageBuilder.withPayload(createEvent()).build());
}
}
private static SplunkEvent createEvent(){
int nextInt = ThreadLocalRandom.current().nextInt(1, 1000);
OrderEvent sd = new OrderEvent();
sd.setEan(String.valueOf (nextInt));
sd.setEmailuser("mail@gmail.com");
sd.setOrderNumber("21501001010101");
return sd;
}
@Bean
public SplunkServer splunkServerRef() {
SplunkServer splunkServer = new SplunkServer();
splunkServer.setPort(Integer.valueOf(port));
splunkServer.setHost(host);
splunkServer.setUsername(username);
splunkServer.setOwner(owner);
splunkServer.setPassword(password);
return splunkServer;
}
}

50
basic/splunk/src/main/java/org/springframework/integration/samples/splunk/event/OrderEvent.java Normal file
View File

@@ -0,0 +1,50 @@
/*
* Copyright 2015 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.integration.samples.splunk.event;
import org.springframework.integration.splunk.event.SplunkEvent;
/**
* @author Filippo Balicchia
* @since 4.2
*/
public class OrderEvent extends SplunkEvent {
private static final long serialVersionUID = 4203808769092252241L;
public static String ORDER_NUMBER = "order_number";
public static String EAN_ITEM = "ean";
public static String EMAIL_USER = "email";
public OrderEvent() {
super();
}
public void setOrderNumber(String orderNumber) {
addPair(ORDER_NUMBER, orderNumber);
}
public void setEan(String eanNumber) {
addPair(EAN_ITEM, eanNumber);
}
public void setEmailuser(String email) {
addPair(EMAIL_USER, email);
}
}

6
basic/splunk/src/main/resources/application.yml Normal file
View File

@@ -0,0 +1,6 @@
splunk:
host: localhost
port: 8089
username: admin
password: admin
owner: admin

15
basic/splunk/src/main/resources/banner.txt Normal file
View File

@@ -0,0 +1,15 @@
${AnsiColor.GREEN} _____ _ _____ _ _ _
${AnsiColor.GREEN} / ____| (_) |_ _| | | | | (_)
${AnsiColor.GREEN} | (___ _ __ _ __ _ _ __ __ _ | | _ __ | |_ ___ __ _ _ __ __ _| |_ _ ___ _ __
${AnsiColor.GREEN} \___ \| '_ \| '__| | '_ \ / _` | | | | '_ \| __/ _ \/ _` | '__/ _` | __| |/ _ \| '_ \
${AnsiColor.GREEN} ____) | |_) | | | | | | | (_| | _| |_| | | | || __/ (_| | | | (_| | |_| | (_) | | | |
${AnsiColor.GREEN} |_____/| .__/|_| |_|_| |_|\__, | |_____|_| |_|\__\___|\__, |_| \__,_|\__|_|\___/|_| |_|
${AnsiColor.GREEN} | | __/ | __/ | ${AnsiColor.BRIGHT_BLUE} ____________
${AnsiColor.GREEN} _____|_| _ |___/ _____ |___/ _ ${AnsiColor.BRIGHT_BLUE} \ \ \ \ \ \ \
${AnsiColor.GREEN} / ____| | | | | / ____| | | ${AnsiColor.BRIGHT_BLUE} \ \ \ \ \ \ \
${AnsiColor.GREEN} | (___ _ __ | |_ _ _ __ | | __ | (___ __ _ _ __ ___ _ __ | | ___ ${AnsiColor.BRIGHT_BLUE} \ \ \ \ \ \ \
${AnsiColor.GREEN} \___ \| '_ \| | | | | '_ \| |/ / \___ \ / _` | '_ ` _ \| '_ \| |/ _ \${AnsiColor.BRIGHT_BLUE} / / / / / / /
${AnsiColor.GREEN} ____) | |_) | | |_| | | | | < ____) | (_| | | | | | | |_) | | __/${AnsiColor.BRIGHT_BLUE} / / / / / / /
${AnsiColor.GREEN} |_____/| .__/|_|\__,_|_| |_|_|\_\ |_____/ \__,_|_| |_| |_| .__/|_|\___|${AnsiColor.BRIGHT_BLUE} / / / / / / /
${AnsiColor.GREEN} ======| |===============================================| |===========
${AnsiColor.GREEN} |_| |_|${AnsiColor.DEFAULT}

14
basic/splunk/src/main/resources/logback.xml Normal file
View File

@@ -0,0 +1,14 @@
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are assigned the type
ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="STDOUT" />
</root>
</configuration>

34
basic/splunk/src/main/resources/si-splunk-example-context.xml Normal file
View File

@@ -0,0 +1,34 @@
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:int="http://www.springframework.org/schema/integration"
xmlns:stream="http://www.springframework.org/schema/integration/stream"
xmlns:int-splunk="http://www.springframework.org/schema/integration/splunk"
xsi:schemaLocation="http://www.springframework.org/schema/integration/stream http://www.springframework.org/schema/integration/stream/spring-integration-stream.xsd
http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/integration/splunk http://www.springframework.org/schema/integration/splunk/spring-integration-splunk.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
<int:channel id="toSplunk" />
<int-splunk:outbound-channel-adapter
id="splunkOutboundChannelAdapter" auto-startup="true" order="1"
channel="toSplunk" splunk-server-ref="splunkServerRef" source-type="customMessage"
source="tinyMain">
<int-splunk:submit-writer index="tinyindex" />
</int-splunk:outbound-channel-adapter>
<!-- Inbound -->
<int:channel id="inputFromSplunk" />
<stream:stdout-channel-adapter id="stdout"
channel="inputFromSplunk" append-newline="true" />
<int-splunk:inbound-channel-adapter
id="splunkInboundChannelAdapter" auto-startup="true" search="search ean > 135"
splunk-server-ref="splunkServerRef" channel="inputFromSplunk" mode="BLOCKING"
earliest-time="-1d" latest-time="now" init-earliest-time="-1d"
field-list="field1, field2">
<int:poller fixed-rate="5000" />
</int-splunk:inbound-channel-adapter>
</beans>

40
basic/splunk/src/test/java/org/springframework/integration/samples/splunk/ApplicationTests.java Normal file
View File

@@ -0,0 +1,40 @@
/*
* Copyright 2015 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.integration.samples.splunk;
import org.junit.ClassRule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.boot.test.SpringApplicationConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
/**
* @author Filippo Balicchia
* @since 4.2
*/
@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = Application.class)
public class ApplicationTests {
@ClassRule
public static SplunkWatcher splunkWatcher = new SplunkWatcher();
@Test
public void contextLoads() {
}
}

68
basic/splunk/src/test/java/org/springframework/integration/samples/splunk/SplunkWatcher.java Normal file
View File

@@ -0,0 +1,68 @@
/*
* Copyright 2015 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.integration.samples.splunk;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.junit.Assume;
import org.junit.rules.TestWatcher;
import org.junit.runner.Description;
import org.junit.runners.model.Statement;
import org.springframework.integration.splunk.support.SplunkServer;
import org.springframework.integration.splunk.support.SplunkServiceFactory;
import com.splunk.Service;
/**
* @author Filippo Balicchia
* @since 4.2
*/
public class SplunkWatcher extends TestWatcher {
private static final Log logger = LogFactory.getLog(SplunkWatcher.class);
private final int port;
public SplunkWatcher() {
this(Service.DEFAULT_PORT);
}
public SplunkWatcher(int port) {
this.port = port;
}
@Override
public Statement apply(Statement base, Description description) {
try {
SplunkServer splunkServer = new SplunkServer();
splunkServer.setPassword("admin");
splunkServer.setUsername("admin");
SplunkServiceFactory splunkServiceFactory = new SplunkServiceFactory(splunkServer);
splunkServiceFactory.getService().open(this.port);
System.setProperty("splunk.port", "" + this.port);
}
catch (Exception e) {
logger.warn(
"Not executing tests because basic connectivity test failed");
Assume.assumeNoException(e);
}
return super.apply(base, description);
}
}

23
build.gradle
View File

@@ -195,6 +195,7 @@ subprojects { subproject ->
springIntegrationVersion = '4.2.2.RELEASE'
springIntegrationDslVersion = '1.1.1.RELEASE'
springIntegrationKafkaVersion = '1.3.0.RELEASE'
springIntegrationSplunkVersion = '1.1.0.RELEASE'
springVersion = '4.2.3.RELEASE'
springSecurityVersion = '4.0.2.RELEASE'
springWebFlowVersion = '2.3.3.RELEASE'
@@ -564,6 +565,28 @@ project('http') {
}
project('splunk') {
description = 'Splunk Sample'
apply plugin: 'spring-boot'
dependencies {
compile 'org.springframework.boot:spring-boot-starter-integration'
compile "org.springframework.integration:spring-integration-core:$springIntegrationVersion"
compile "org.springframework.integration:spring-integration-splunk:$springIntegrationSplunkVersion"
testCompile 'org.springframework.boot:spring-boot-starter-test'
}
springBoot {
mainClass = 'org.springframework.integration.samples.splunk.Application'
}
task run(type: JavaExec) {
main 'org.springframework.integration.samples.splunk.Application'
classpath = sourceSets.main.runtimeClasspath
}
}
project('kafka') {
description = 'Apache Kafka Sample'