Difference between revisions of "Sending data to Senfi"
(8 intermediate revisions by 4 users not shown) | |||
Line 1: | Line 1: | ||
+ | <translate> | ||
+ | <!--T:1--> | ||
Senfi monitors and stores time series data from your existing systems. At the moment, Senfi provides a [https://mqtt.org/ MQTT] interface for receiving data from your systems. To start sending data to Senfi, you need to do the following: | Senfi monitors and stores time series data from your existing systems. At the moment, Senfi provides a [https://mqtt.org/ MQTT] interface for receiving data from your systems. To start sending data to Senfi, you need to do the following: | ||
+ | <!--T:2--> | ||
# Create an integration key in the CMS | # Create an integration key in the CMS | ||
# Design and create a measurement in the CMS | # Design and create a measurement in the CMS | ||
# Write a program to read from your system and send it to Senfi's MQTT endpoint | # Write a program to read from your system and send it to Senfi's MQTT endpoint | ||
− | === Create integration key === | + | === Create integration key === <!--T:3--> |
+ | <!--T:4--> | ||
# Login to the [https://ems.senfi.io/cms CMS] | # Login to the [https://ems.senfi.io/cms CMS] | ||
# Go to the Developer > Integration tab | # Go to the Developer > Integration tab | ||
Line 12: | Line 16: | ||
# Note down the values of the '''Key''' (your integration key) and '''Secret''' (your integration secret) | # Note down the values of the '''Key''' (your integration key) and '''Secret''' (your integration secret) | ||
− | === Design/create measurement === | + | === Design/create measurement === <!--T:5--> |
+ | <!--T:6--> | ||
The key to a successful integration is to have a well-designed [[Measurement|measurement]]. So if you are not familiar with the concept of measurement/metric/tag, please read [[Concepts#Measurement.2C_Metric.2C_Tag|this section]] first. | The key to a successful integration is to have a well-designed [[Measurement|measurement]]. So if you are not familiar with the concept of measurement/metric/tag, please read [[Concepts#Measurement.2C_Metric.2C_Tag|this section]] first. | ||
+ | <!--T:7--> | ||
# When you are designing a measurement to represent your system, note down the metrics you want to capture. | # When you are designing a measurement to represent your system, note down the metrics you want to capture. | ||
# Decide on a set of tags to use. Make sure that there is a combination of tags that will uniquely identify the unit/device/equipment that outputs those metrics. | # Decide on a set of tags to use. Make sure that there is a combination of tags that will uniquely identify the unit/device/equipment that outputs those metrics. | ||
Line 27: | Line 33: | ||
# Save when done. | # Save when done. | ||
− | === Publish to MQTT === | + | === Publish to MQTT === <!--T:8--> |
+ | <!--T:9--> | ||
A typical integration follows this pattern: | A typical integration follows this pattern: | ||
− | External system --> Your application --> Senfi MQTT endpoint | + | <!--T:10--> |
+ | External system --> Your application --> Senfi MQTT endpoint | ||
+ | <!--T:11--> | ||
Either the external system you are monitoring pushes to your application, or your application polls the external system for its current values. Either way, once your application has the values of its measurement, you can prepare to send it to Senfi's MQTT endpoint. | Either the external system you are monitoring pushes to your application, or your application polls the external system for its current values. Either way, once your application has the values of its measurement, you can prepare to send it to Senfi's MQTT endpoint. | ||
− | ==== MQTT Endpoint ==== | + | ==== MQTT Endpoint ==== <!--T:12--> |
; Host: mqtt.senfi.io | ; Host: mqtt.senfi.io | ||
; Port: 1883 | ; Port: 1883 | ||
Line 42: | Line 51: | ||
; MQTT topic: ingestor/1/0/live/text/array/ems/''<your integration key>/<your measurement code>'' | ; MQTT topic: ingestor/1/0/live/text/array/ems/''<your integration key>/<your measurement code>'' | ||
− | ==== Message Format ==== | + | ==== Message Format ==== <!--T:13--> |
The message body should be in valid [https://www.json.org/ JSON] format. | The message body should be in valid [https://www.json.org/ JSON] format. | ||
− | Single measurement | + | <!--T:14--> |
+ | Single measurement</translate> | ||
{ | { | ||
"data": [ | "data": [ | ||
Line 59: | Line 69: | ||
} | } | ||
− | Multiple measurements | + | <translate><!--T:15--> Multiple measurements</translate> |
{ | { | ||
"data": [ | "data": [ | ||
Line 82: | Line 92: | ||
] | ] | ||
} | } | ||
− | + | <translate> | |
− | You must indicate which [[Site]] the measurement is meant for. Hence, you need to include the site ID (<tt>site_id</tt>) in your MQTT message. If your measurement has other tags, you must send them in the message as well. | + | <!--T:16--> |
− | + | You must indicate which [[Site]] the measurement is meant for. Hence, you need to include the site ID (<tt><tvar|site_id>site_id</></tt>) in your MQTT message. If your measurement has other tags, you must send them in the message as well. | |
− | <div class="important">Note: '''All''' tags and metrics must be sent in your message body, or they will be treated as invalid data and discarded.</div> | + | </translate> |
− | + | <div class="important"><translate><!--T:17--> Note: '''All''' tags and metrics must be sent in your message body, or they will be treated as invalid data and discarded.</translate></div> | |
− | ==== Data Types ==== | + | <translate> |
+ | ==== Metric Data Types ==== <!--T:18--> | ||
Senfi supports the following data types | Senfi supports the following data types | ||
* Float | * Float | ||
Line 93: | Line 104: | ||
* Boolean | * Boolean | ||
* String | * String | ||
− | + | </translate> | |
− | < | + | <translate> |
− | + | <!--T:20--> | |
;'''Float''' | ;'''Float''' | ||
:Only valid JSON Number is accepted. | :Only valid JSON Number is accepted. | ||
+ | <tvar|float_valid_num> | ||
:* <tt>0</tt> | :* <tt>0</tt> | ||
:* <tt>0.1</tt> | :* <tt>0.1</tt> | ||
:* <tt>1.0e+10</tt> | :* <tt>1.0e+10</tt> | ||
+ | </> | ||
:Invalid examples: | :Invalid examples: | ||
+ | <tvar|float_invalid_num> | ||
:* <tt>1.0.0</tt> | :* <tt>1.0.0</tt> | ||
:* <tt>"1.0"</tt> | :* <tt>"1.0"</tt> | ||
− | + | </> | |
;'''Integer''' | ;'''Integer''' | ||
:Only valid JSON Number is accepted. Decimal values are truncated. | :Only valid JSON Number is accepted. Decimal values are truncated. | ||
+ | <tvar|integer_valid_num> | ||
:* <tt>0</tt> | :* <tt>0</tt> | ||
:* <tt>-15</tt> | :* <tt>-15</tt> | ||
:* <tt>1e10</tt> | :* <tt>1e10</tt> | ||
+ | </> | ||
:Invalid examples: | :Invalid examples: | ||
:* Same as '''Float''' | :* Same as '''Float''' | ||
+ | <!--T:21--> | ||
;'''Boolean''' | ;'''Boolean''' | ||
:Only JSON Boolean is accepted. | :Only JSON Boolean is accepted. | ||
Line 119: | Line 136: | ||
:Invalid examples: | :Invalid examples: | ||
:* "true" | :* "true" | ||
− | :* " | + | :* "FALSE" |
:* 0 | :* 0 | ||
:* "1" | :* "1" | ||
+ | <!--T:22--> | ||
;'''String''' | ;'''String''' | ||
:Any valid JSON String is accepted. | :Any valid JSON String is accepted. | ||
− | + | ==== Sample Application ==== <!--T:23--> | |
You can take a look at [https://github.com/gelement-com/mqtt-ingestor-sample this GitHub repository] for a sample application to send data to Senfi's MQTT endpoint. | You can take a look at [https://github.com/gelement-com/mqtt-ingestor-sample this GitHub repository] for a sample application to send data to Senfi's MQTT endpoint. | ||
− | ==== Publishing Strategy ==== | + | ==== Publishing Strategy ==== <!--T:24--> |
You can either choose to publish data to Senfi regularly (eg. every second), or only do so upon change in the metrics. In practice, it is recommended that you publish regularly. This is so that the system is able to tell when data has stopped coming in. | You can either choose to publish data to Senfi regularly (eg. every second), or only do so upon change in the metrics. In practice, it is recommended that you publish regularly. This is so that the system is able to tell when data has stopped coming in. | ||
− | In addition, you can choose to batch the sending of measurements to Senfi. For example, collect 50 measurements and then publish it in a single message. This may be useful if you have fast changing metrics. However, note that <tt>tm_source</tt> should represent when the measurement is sampled and not when it is sent. | + | <!--T:25--> |
+ | In addition, you can choose to batch the sending of measurements to Senfi. For example, collect 50 measurements and then publish it in a single message. This may be useful if you have fast changing metrics. However, note that <tt><tvar|tm_source>tm_source</></tt> should represent when the measurement is sampled and not when it is sent. | ||
+ | </translate> | ||
+ | |||
+ | |||
+ | <span class="right"><translate><!--T:26--> [[Adding_a_rule|Next: Adding a rule]]</translate></span> |
Latest revision as of 09:49, 19 August 2020
Senfi monitors and stores time series data from your existing systems. At the moment, Senfi provides a MQTT interface for receiving data from your systems. To start sending data to Senfi, you need to do the following:
- Create an integration key in the CMS
- Design and create a measurement in the CMS
- Write a program to read from your system and send it to Senfi's MQTT endpoint
Contents
Create integration key
- Login to the CMS
- Go to the Developer > Integration tab
- Click the + button to add a new integration key/secret pair
- Note down the values of the Key (your integration key) and Secret (your integration secret)
Design/create measurement
The key to a successful integration is to have a well-designed measurement. So if you are not familiar with the concept of measurement/metric/tag, please read this section first.
- When you are designing a measurement to represent your system, note down the metrics you want to capture.
- Decide on a set of tags to use. Make sure that there is a combination of tags that will uniquely identify the unit/device/equipment that outputs those metrics.
- Login to the CMS.
- Go to the Measurement tab.
- Click the + button to add a new measurement.
- Give the measurement a descriptive name. This will be shown to users of the dashboard.
- Specify a measurement code.
- Add tags.
- Add metrics and specify the type of each metric.
- Save when done.
Publish to MQTT
A typical integration follows this pattern:
External system --> Your application --> Senfi MQTT endpoint
Either the external system you are monitoring pushes to your application, or your application polls the external system for its current values. Either way, once your application has the values of its measurement, you can prepare to send it to Senfi's MQTT endpoint.
MQTT Endpoint
- Host
- mqtt.senfi.io
- Port
- 1883
- Username
- <Your integration key>
- Password
- <Your integration secret>
- MQTT topic
- ingestor/1/0/live/text/array/ems/<your integration key>/<your measurement code>
Message Format
The message body should be in valid JSON format.
Single measurement
{ "data": [ { "tm_source": xxxxxxxxxx, "site_id" xxxxxxxx, "tag1": "xxxxxxxx", "tag2": "xxxxxxxx", "metric1": xxxxxxxxx, "metric2": xxxxxxxxx ... } }
Multiple measurements
{ "data": [ { "tm_source": xxxxxxxxxx, "site_id" xxxxxxxx, "tag1": "xxxxxxxx", "tag2": "xxxxxxxx", "metric1": xxxxxxxxx, "metric2": xxxxxxxxx ... }, { "tm_source": xxxxxxxxxx, "site_id" xxxxxxxx, "tag1": "xxxxxxxx", "tag2": "xxxxxxxx", "metric1": xxxxxxxxx, "metric2": xxxxxxxxx, ... } ] }
You must indicate which Site the measurement is meant for. Hence, you need to include the site ID (site_id) in your MQTT message. If your measurement has other tags, you must send them in the message as well.
Metric Data Types
Senfi supports the following data types
- Float
- Integer
- Boolean
- String
- Float
- Only valid JSON Number is accepted.
- 0
- 0.1
- 1.0e+10
- Invalid examples:
- 1.0.0
- "1.0"
- Integer
- Only valid JSON Number is accepted. Decimal values are truncated.
- 0
- -15
- 1e10
- Invalid examples:
- Same as Float
- Boolean
- Only JSON Boolean is accepted.
- true
- false
- Invalid examples:
- "true"
- "FALSE"
- 0
- "1"
- String
- Any valid JSON String is accepted.
Sample Application
You can take a look at this GitHub repository for a sample application to send data to Senfi's MQTT endpoint.
Publishing Strategy
You can either choose to publish data to Senfi regularly (eg. every second), or only do so upon change in the metrics. In practice, it is recommended that you publish regularly. This is so that the system is able to tell when data has stopped coming in.
In addition, you can choose to batch the sending of measurements to Senfi. For example, collect 50 measurements and then publish it in a single message. This may be useful if you have fast changing metrics. However, note that tm_source should represent when the measurement is sampled and not when it is sent.