SpringCloud/stream-applications
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

further improvements of the object detection processor README

Browse Source
This commit is contained in:
Christian Tzolov
2020-06-30 12:32:41 +02:00
parent 2109331400
commit f7467c98ce
1 changed files with 8 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
applications/processor/object-detection-processor/README.adoc
View File

@@ -3,19 +3,21 @@
= Object Detection Processor
The Object Detection processor provides out-of-the-box support for the https://github.com/tensorflow/models/blob/master/research/object_detection/README.md[TensorFlow Object Detection API]. It allows for real-time localization and identification of multiple objects in a single image or image stream. The Object Detection processor is built on top of the https://github.com/spring-cloud/stream-applications/tree/master/functions/function/object-detection-function[Object Detection Function]. It one of the pre-trained https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md[object detection] models and corresponding https://github.com/tensorflow/models/tree/865c14c/research/object_detection/data[object labels].
The Object Detection processor provides out-of-the-box support for the https://github.com/tensorflow/models/blob/master/research/object_detection/README.md[TensorFlow Object Detection API]. It allows for real-time localization and identification of multiple objects in a single image or image stream. The Object Detection processor is built on top of the https://github.com/spring-cloud/stream-applications/tree/master/functions/function/object-detection-function[Object Detection Function].
Sensible defaults when the pre-trained model is not configured:
You have to provide the Processor with a pre-trained https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md[object detection model], and the corresponding https://github.com/tensorflow/models/tree/865c14c/research/object_detection/data[object labels].
Here are some sensible configuration defaults:
* `object.detection.model` : `https://storage.googleapis.com/scdf-tensorflow-models/object-detection/faster_rcnn_resnet101_coco_2018_01_28_frozen_inference_graph.pb`
* `object.detection.labels` : `https://storage.googleapis.com/scdf-tensorflow-models/object-detection/mscoco_label_map.pbtxt`
* `object.detection.with-masks` : `false`
The following diagram illustrates a Spring Cloud Data Flow a streaming pipeline that predicts object types from the images in real-time.
The following diagram shows a https://dataflow.spring.io/docs/concepts/streams/[Spring Cloud Data Flow], streaming pipeline, that predicts, in real-time, the object types in input image stream.
image::{image-root}/scdf-tensorflow-object-detection-arch.png[]
Processor's input is an image byte array, and the output is an augmented image byte array, and a JSON header `detected_objects` in this format:
Processor's input is an image byte array, and the output is an augmented image, and a header, called `detected_objects`, that provides textual description of the detected objects:
```json
{
@@ -28,7 +30,7 @@ Processor's input is an image byte array, and the output is an augmented image b
}
```
The output `detected_objects` header contains the following filed:
The `detected_objects` header format is:
* *object-name*:**confidence** - human readable name of the detected object (e.g. label) with its confidence as a float between [0-1]
* *x1*, *y1*, *x2*, *y2* - Response also provides the bounding box of the detected objects represented as `(x1, y1, x2, y2)`. The coordinates are relative to the size of the image size.
@@ -36,7 +38,7 @@ The output `detected_objects` header contains the following filed:
=== Payload
The incoming type is `byte[]`, and the content type is `application/octet-stream`. The processor processes the input `byte[]` image and outputs augmented `byte[]` image payload and json header.
The incoming type is `byte[]`, and the content type is `application/octet-stream`. The processor processes the input `byte[]` image and outputs an augmented `byte[]` image payload and a JSON header (`detected_objects`).
== Options