SpringCloud/stream-applications
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

semantic segmentation processor

Browse Source
This commit is contained in:
Christian Tzolov
2020-06-25 12:41:39 +02:00
parent 12b83fc940
commit e2932661dd
32 changed files with 783 additions and 81 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
applications/processor/object-detection-processor/README.adoc
View File

@@ -5,17 +5,17 @@
The https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-stream-processor-object-detection[Object Detection] processor provides out-of-the-box support for the https://github.com/tensorflow/models/blob/master/research/object_detection/README.md[TensorFlow Object Detection API]. It allows for real-time localization and identification of multiple objects in a single image or image stream. The Object Detection processor uses one of the pre-trained https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md[object detection] models and corresponding https://github.com/tensorflow/models/tree/865c14c/research/object_detection/data[object labels].
If the pre-trained model is not set explicitly set then following defaults are used:
Sensible defaults when the pre-trained model is not configured:
* `tensorflow.modelFetch` : `detection_scores,detection_classes,detection_boxes,num_detections`
* `tensorflow.model` : `https://storage.googleapis.com/scdf-tensorflow-models/object-detection/faster_rcnn_resnet101_coco_2018_01_28_frozen_inference_graph.pb`
* `tensorflow.object.detection.labels` : `https://storage.googleapis.com/scdf-tensorflow-models/object-detection/mscoco_label_map.pbtxt`
* `object.detection.model` : `https://storage.googleapis.com/scdf-tensorflow-models/object-detection/faster_rcnn_resnet101_coco_2018_01_28_frozen_inference_graph.pb`
* `object.detection.labels` : `https://storage.googleapis.com/scdf-tensorflow-models/object-detection/mscoco_label_map.pbtxt`
* `object.detection.with-masks` : `false`
The following diagram illustrates a Spring Cloud Data Flow streaming pipeline that predicts object types from the images in real-time.
The following diagram illustrates a Spring Cloud Data Flow a streaming pipeline that predicts object types from the images in real-time.
image::{image-root}/scdf-tensorflow-object-detection-arch.png[]
Processor's input is an image byte array and the output is a JSON message in this format:
Processor's input is an image byte array, and the output is an augmented image byte array and a JSON header `detected_objects` in this format:
```json
{
@@ -28,7 +28,7 @@ Processor's input is an image byte array and the output is a JSON message in thi
}
```
The output format is:
The output JSON format is:
* *object-name*:**confidence** - human readable name of the detected object (e.g. label) with its confidence as a float between [0-1]
* *x1*, *y1*, *x2*, *y2* - Response also provides the bounding box of the detected objects represented as (x1, y1, x2, y2). The coordinates are relative to the size of the image size.
@@ -36,7 +36,7 @@ The output format is:
=== Payload
If the incoming type is `byte[]` and the content type is set to `application/octet-stream` , then the application process the input `byte[]` image into and outputs augmented `byte[]` image payload and json header.
The incoming type is `byte[]`, and the content type is `application/octet-stream`. The processor processes the input `byte[]` image and outputs augmented `byte[]` image payload and json header.
== Options

4
applications/processor/object-detection-processor/src/test/java/org/springframework/cloud/stream/app/processor/object/detection/ObjectDetectionProcessorTests.java
View File

@@ -48,8 +48,8 @@ public class ObjectDetectionProcessorTests {
"--object.detection.model=https://download.tensorflow.org/models/object_detection/ssdlite_mobilenet_v2_coco_2018_05_09.tar.gz#frozen_inference_graph.pb",
"--object.detection.labels=https://storage.googleapis.com/scdf-tensorflow-models/object-detection/mscoco_label_map.pbtxt",
"--object.detection.responseSize=10",
"--object.detection..debugOutput=true",
"--object.detection..debugOutputPath=./target/object-detection-1.png")) {
"--object.detection.debugOutput=true",
"--object.detection.debugOutputPath=./target/object-detection-1.png")) {
InputDestination processorInput = context.getBean(InputDestination.class);
OutputDestination processorOutput = context.getBean(OutputDestination.class);