Difference between revisions of "Computed measurement"
(70 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
+ | <translate> | ||
+ | <!--T:1--> | ||
This page describes computed measurement in detail. | This page describes computed measurement in detail. | ||
− | === Overview === | + | === Overview === <!--T:2--> |
As [[Concepts#Computed_measurement|described here]], computed measurement is a type of data that is generated from other data sources. | As [[Concepts#Computed_measurement|described here]], computed measurement is a type of data that is generated from other data sources. | ||
− | + | <!--T:3--> | |
− | + | At core of a computed measurement is a '''computation script''' that takes in '''input measurements''', performs '''logic and calculations''', and outputs a set of '''derived metrics''' as a '''computed measurement'''. | |
− | |||
− | === Design === | + | <!--T:4--> |
+ | [[File:CM-overview_v1.PNG|Computed Measurement Overview|link=]] | ||
+ | |||
+ | <!--T:5--> | ||
+ | To use a computed measurement, you have to: | ||
+ | * Choose the input measurements | ||
+ | * Choose the output metrics | ||
+ | * Write a script that calculates the output metrics from the input measurements' metrics | ||
+ | |||
+ | <!--T:6--> | ||
+ | The computed measurement script will be initialized immediately after the computed measurement is created and is active until the computed measurement is deleted. Senfi will execute the script whenever measurement data from the selected input measurements arrive. | ||
+ | |||
+ | === Design === <!--T:7--> | ||
The first thing to do when designing a computed measurement is to decide what are the '''inputs''' and what to '''output'''. A computed measurement can specify multiple [[Measurement|measurements]] as input. Metrics and tags in those measurement will then be available to the function that is responsible for generating the output. | The first thing to do when designing a computed measurement is to decide what are the '''inputs''' and what to '''output'''. A computed measurement can specify multiple [[Measurement|measurements]] as input. Metrics and tags in those measurement will then be available to the function that is responsible for generating the output. | ||
− | ==== Input Measurements ==== | + | ==== Input Measurements ==== <!--T:8--> |
− | Any [[Measurement|measurement]] | + | Any [[Measurement|measurement]] that is available to you in the [https://ems.senfi.io/cms CMS] can be used as an input measurement for your computed measurement. |
+ | <!--T:9--> | ||
You should choose only the measurements with metrics that are needed for you to calculate your computed measurement as your input measurements. | You should choose only the measurements with metrics that are needed for you to calculate your computed measurement as your input measurements. | ||
− | ==== Output Measurements ==== | + | ==== Output Measurements ==== <!--T:10--> |
Any data that you want to create with your computed measurement should be packaged in an output measurement. An output measurement has the same composition (metrics, tags, timestamp) as other [[Measurement|measurements]] in Senfi. | Any data that you want to create with your computed measurement should be packaged in an output measurement. An output measurement has the same composition (metrics, tags, timestamp) as other [[Measurement|measurements]] in Senfi. | ||
+ | <!--T:11--> | ||
When designing the output measurement metrics and tags, should consider the following: | When designing the output measurement metrics and tags, should consider the following: | ||
* Metrics: The metrics that you want and how to calculate them from your input measurement(s). | * Metrics: The metrics that you want and how to calculate them from your input measurement(s). | ||
Line 24: | Line 39: | ||
* Timestamp: Whether to use the input measurement's timestamp, or the time the computed measurement was output. | * Timestamp: Whether to use the input measurement's timestamp, or the time the computed measurement was output. | ||
− | === Implementation === | + | === Implementation === <!--T:12--> |
To generate computed measurements, you will write a script that can process your selected input measurements, perform any calculations or logic necessary, and output the computed measurement data. | To generate computed measurements, you will write a script that can process your selected input measurements, perform any calculations or logic necessary, and output the computed measurement data. | ||
+ | <!--T:13--> | ||
+ | The scripting language used to author the computed measurement scripts is [https://developer.mozilla.org/en-US/docs/Web/JavaScript JavaScript]. | ||
+ | |||
+ | <!--T:14--> | ||
The computed measurement script consists of two functions: | The computed measurement script consists of two functions: | ||
* Initialization: Sets up the script | * Initialization: Sets up the script | ||
* Computation: Processes data from input measurements, performs calculations and output your computed measurement | * Computation: Processes data from input measurements, performs calculations and output your computed measurement | ||
− | + | <!--T:15--> | |
− | + | Both the initialization and computation functions must be present in your script. | |
+ | ==== init(): Initialization Function ==== <!--T:16--> | ||
+ | The script initialization function can be used to initialized any data structures you need for logic and state management of your script. It is executed only once when your script is created, modified, or after a system maintenance in which services are restarted. | ||
+ | |||
+ | <!--T:17--> | ||
This is the script initialization function template: | This is the script initialization function template: | ||
+ | </translate> | ||
− | + | <syntaxhighlight lang="Javascript"> | |
− | + | /** | |
− | + | * @name: init | |
− | + | * @description: Perform one-time initialization of your script | |
− | + | * param {string} | |
− | + | **/ | |
− | + | async function init() { | |
− | + | // Perform initialization of script here | |
− | + | // TODO | |
+ | } | ||
+ | </syntaxhighlight> | ||
+ | <translate> | ||
+ | <!--T:18--> | ||
You may choose to leave this function empty if your computed measurement does not need to keep track of data across multiple or consecutive input measurements, such as summation of metric values or checking the difference between a pairs of input measurements. | You may choose to leave this function empty if your computed measurement does not need to keep track of data across multiple or consecutive input measurements, such as summation of metric values or checking the difference between a pairs of input measurements. | ||
+ | </translate> | ||
− | + | <syntaxhighlight lang="Javascript"> | |
− | + | async function init() { | |
− | + | // Initialization not required | |
+ | } | ||
+ | </syntaxhighlight> | ||
+ | <translate> | ||
+ | <!--T:19--> | ||
If your computed measurement needs to keep track of data across multiple or consecutive input measurements, you should create the appropriate data structures here. | If your computed measurement needs to keep track of data across multiple or consecutive input measurements, you should create the appropriate data structures here. | ||
+ | <!--T:20--> | ||
You can also declare global variables. For example: | You can also declare global variables. For example: | ||
− | + | </translate> | |
− | + | ||
− | + | <syntaxhighlight lang="Javascript"> | |
− | + | // Declare global variable for a summation value | |
− | + | let sum; | |
− | + | ||
− | + | async function init() { | |
− | + | // Initialize sum to zero | |
+ | sum = 0; | ||
+ | } | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | <translate> | ||
+ | ==== compute(): Computation Function ==== <!--T:21--> | ||
+ | The script computation function is executed every time a new measurement data, or batch of measurement data, from your specified input measurement arrives. In this function, you will read the data from those input measurements, performs any logic or calculations required, and output your computed measurement data. | ||
− | + | <!--T:22--> | |
This is the script computation function template: | This is the script computation function template: | ||
+ | </translate> | ||
+ | |||
+ | <syntaxhighlight lang="Javascript"> | ||
+ | /** | ||
+ | * @name: compute | ||
+ | * @description: Perform computation on input measurements. This function is called when new measurements arrive | ||
+ | * @param {string} measurement - The input measurement name | ||
+ | * @param {Array.<object>} data - The measurement array data | ||
+ | **/ | ||
+ | function compute(measurement, data) { | ||
+ | // Perform calculation on data here | ||
+ | // TODO | ||
+ | |||
+ | // Example output | ||
+ | const outputData = { | ||
+ | metric1: 1, | ||
+ | metric2: true, | ||
+ | metric3: 1, | ||
+ | }; | ||
+ | |||
+ | // Output computed measurement. See API docs below | ||
+ | output(outputData); | ||
+ | } | ||
+ | |||
+ | // Output API function | ||
+ | /** | ||
+ | * @function output | ||
+ | * @description: User-invoked function to output a computed measurement | ||
+ | * @param {object} outputData - Object containing output metrics (required tags, optional tags, measurement metrics) | ||
+ | * @param {string} outputData.lsid - [Example] lsid tag (for lifts) | ||
+ | * @param {string} outputData.country - [Example] country tag | ||
+ | * @param {number} outputData.metric1 - [Example] metric1 | ||
+ | * @param {boolean} outputData.metric2 - [Example] metric2 | ||
+ | * @param {integer} outputData.metric3 - [Example] metric3 | ||
+ | **/ | ||
+ | </syntaxhighlight> | ||
− | + | <translate> | |
− | + | <!--T:23--> | |
− | + | Do note that data is an array of measurement data, so typically you would loop through the data input to perform calculation tasks on each variable like this: | |
− | + | </translate> | |
− | + | ||
− | + | <syntaxhighlight lang="Javascript"> | |
− | + | function compute(measurement, data) { | |
− | + | // Loop through the array of incoming measurements | |
− | + | for (let i = 0; i < data.length; i++) { | |
− | + | const inputMeasurement = data[i]; | |
− | // | + | |
− | + | // Perform calculation and output | |
− | + | // ... | |
− | + | ||
− | + | // Output | |
− | + | output(...); | |
− | |||
− | // Output | ||
− | output( | ||
} | } | ||
− | + | } | |
− | + | </syntaxhighlight> | |
− | + | ||
− | + | <translate> | |
− | + | <!--T:24--> | |
− | + | At the end of your computations, once you have constructed the output data for the computed measurement, you should call '''output()''', with the output data, to output your computed measurement. This will inform the script engine to ingest your computed measurement into Senfi. | |
− | + | ||
− | + | <!--T:25--> | |
− | + | <div class="important">Note: The computation function does not return or output a computed measurement. You must call '''output()''' in order to output a computed measurement data to Senfi.</div> | |
− | + | ||
− | + | ==== output(): Script Output ==== <!--T:26--> | |
− | + | The script output function is a pre-defined function call that you should invoke whenever you want to output your computed measurement. | |
+ | |||
+ | <!--T:27--> | ||
+ | You must invoke the output function with a valid measurement data (e.g. all declared metrics and tags) for your computed measurement. Note the output measurement will automatically be attributed to the computed measurement name you have chosen in the [https://ems.senfi.io/cms CMS]. It is not possible for the script to output another computed measurement. | ||
+ | |||
+ | === Testing === <!--T:28--> | ||
+ | You can test your computed measurement in the [https://ems.senfi.io/cms CMS] when you are creating or editing the computed measurement. | ||
+ | |||
+ | <!--T:29--> | ||
+ | Your testing data should be identical in format and values to the actual data from your input measurements. You can test with single or multiple sets of data. | ||
+ | |||
+ | <!--T:30--> | ||
+ | The test data format is in [https://www.json.org/ JSON], as an Array of Objects of the form: | ||
+ | </translate> | ||
+ | |||
+ | { | ||
+ | "measurement": | ||
+ | "data": [] | ||
+ | } | ||
+ | |||
+ | <translate> | ||
+ | <!--T:31--> | ||
+ | where <tt>measurement</tt> is the name of the input measurement, and <tt>data</tt> is a list of one or more measurement data from the input measurement following the [[Sending_data_to_Senfi#Message_Format|Senfi data message format]]. | ||
+ | |||
+ | <!--T:32--> | ||
+ | Example of sending 1 set of measurement data from the measurement <tt>temperature_v1</tt>. The script computation function will be executed once during the test: | ||
+ | </translate> | ||
+ | |||
+ | [ | ||
+ | { | ||
+ | "measurement": "temperature_v1", | ||
+ | "data": [{ | ||
+ | "tm_source": xxxxxxxxxx, | ||
+ | "site_id" xxxxxxxx, | ||
+ | "tag1": "xxxxxxxx", | ||
+ | "tag2": "xxxxxxxx", | ||
+ | "temp": xxxxxxxx, | ||
+ | }] | ||
+ | } | ||
+ | ] | ||
+ | |||
+ | <translate> | ||
+ | <!--T:33--> | ||
+ | Abbreviated example of sending a batch of 3 sets of measurement data from the measurement <tt>temperature_v1</tt>. The script computation function will be executed once during the test: | ||
+ | </translate> | ||
+ | |||
+ | [ | ||
+ | { | ||
+ | "measurement": "temperature_v1", | ||
+ | "data": [{ | ||
+ | ... | ||
+ | "temp": xxxxxxxx, | ||
+ | },{ | ||
+ | ... | ||
+ | "temp": xxxxxxxx, | ||
+ | },{ | ||
+ | ... | ||
+ | "temp": xxxxxxxx, | ||
+ | }] | ||
+ | } | ||
+ | ] | ||
+ | |||
+ | <translate> | ||
+ | <!--T:34--> | ||
+ | Abbreviated example of sending 3 sets of measurement data from the measurement <tt>temperature_v1</tt>. The script computation function will be executed 3 times during the test: | ||
+ | </translate> | ||
+ | |||
+ | [ | ||
+ | { | ||
+ | "measurement": "temperature_v1", | ||
+ | "data": [{ | ||
+ | ... | ||
+ | "temp": xxxxxxxx, | ||
+ | }] | ||
+ | }, | ||
+ | { | ||
+ | "measurement": "temperature_v1", | ||
+ | "data": [{ | ||
+ | ... | ||
+ | "temp": xxxxxxxx, | ||
+ | }] | ||
+ | }, | ||
+ | { | ||
+ | "measurement": "temperature_v1", | ||
+ | "data": [{ | ||
+ | ... | ||
+ | "temp": xxxxxxxx, | ||
+ | }] | ||
+ | } | ||
+ | ] | ||
+ | |||
+ | <translate> | ||
+ | <!--T:35--> | ||
+ | If your script performs computation with multiple input measurements, you can specify multiple measurements. In this example, the computation function will be executed twice, once for <tt>temperature_v1</tt> and once for <tt>humidity_v1</tt>: | ||
+ | </translate> | ||
+ | |||
+ | [ | ||
+ | { | ||
+ | "measurement": "temperature_v1", | ||
+ | "data": [{ | ||
+ | ... | ||
+ | "temp": xxxxxxxx, | ||
+ | }] | ||
+ | }, | ||
+ | { | ||
+ | "measurement": "humidity_v1", | ||
+ | "data": [{ | ||
+ | ... | ||
+ | "humidity": xxxxxxxx, | ||
+ | }] | ||
+ | } | ||
+ | ] | ||
+ | |||
+ | <translate> | ||
+ | === Execution === <!--T:36--> | ||
+ | After you create or edit your computed measurement in the [https://ems.senfi.io/cms CMS], your computed measurement script will be compiled and the initialization and computation functions called: | ||
+ | |||
+ | <!--T:37--> | ||
+ | * When you finish creating the script: The initialization function will be run | ||
+ | * When you finish editing the script: The initialization function will be run | ||
+ | * When data from each of your input measurements arrives: The computation function will be run. | ||
+ | |||
+ | <!--T:38--> | ||
+ | <div class="important">Note: If many measurements from the same input measurement arrive within a very short period of time, the measurements from that input measurement may be batched together. For example, if your input measurement '''input_measurement_1''' sends data 10 times per second, you may receive 30 sets of measurements from '''input_measurement_1''' every 3 seconds.</div> | ||
− | + | <!--T:39--> | |
− | + | <div class="important"> | |
+ | Note: If you have multiple input measurements, each invocation of the computation function will be for only one input measurement. For example, if you have the following input measurements '''input_measurement_1''' and '''input_measurement_2''', expect the computation function to be run twice, once for '''input_measurement_1''' and another for '''input_measurement_1''', for each set of data.</div> | ||
− | === | + | ==== Errors and Debugging ==== <!--T:40--> |
− | + | Errors that occur during the initialization or execution of your script will show up in the '''Debugger''' in the '''Developer''' tab in the [https://ems.senfi.io/cms CMS]. | |
− | |||
− | + | <!--T:41--> | |
− | + | Console messages logged by your script also appear in the '''Debugger'''. | |
− | |||
− | === Examples === | + | === Examples === <!--T:42--> |
==== Example 1: Temperature Scale Conversion ==== | ==== Example 1: Temperature Scale Conversion ==== | ||
+ | <!--T:43--> | ||
This example shows a simple usage of computed measurement. | This example shows a simple usage of computed measurement. | ||
− | Imagine you have a temperature sensor that sends raw temperature values in degrees | + | <!--T:44--> |
+ | Imagine you have a temperature sensor that sends raw temperature values in degrees Fahrenheit (℉) and you want to show the temperature in degrees Celsius (°C) instead. This is the temperature measurement from the temperature sensor that you have as input, where <tt>temperatureF</tt> is the temperature data in ℉: | ||
+ | </translate> | ||
{ | { | ||
Line 126: | Line 325: | ||
} | } | ||
− | And this is the computed measurement that you want to output: | + | <translate> |
+ | <!--T:45--> | ||
+ | And this is the computed measurement that you want to output, where <tt>temperatureC</tt> is the temperature metric in °C. You will retain the input measurement's tags in order to correctly attribute the computed measurement to the original temperature sensor: | ||
+ | </translate> | ||
{ | { | ||
Line 136: | Line 338: | ||
} | } | ||
− | + | <translate> | |
+ | <!--T:46--> | ||
+ | To convert the <tt>temperatureF</tt> temperature metric in ℉ into <tt>temperatureC</tt> in °C, you would use the ℉ to °C formula: | ||
+ | </translate> | ||
+ | |||
+ | temperatureC = (temperatureF − 32) * 5/9; | ||
+ | |||
+ | <translate> | ||
+ | <!--T:47--> | ||
+ | As you are just performing conversion when data arrives, there is no need to perform any initialization tasks or declare any global variables. So you can leave the initialization function empty: | ||
+ | </translate> | ||
− | /** | + | <syntaxhighlight lang="Javascript"> |
− | + | async function init() { | |
− | + | // No initialization tasks required for this simple example | |
− | + | } | |
− | + | </syntaxhighlight> | |
− | + | ||
− | + | <translate> | |
− | + | <!--T:48--> | |
+ | Putting everything together, the full script would be similar to: | ||
+ | </translate> | ||
+ | |||
+ | <syntaxhighlight lang="Javascript"> | ||
+ | /** | ||
+ | * @name: init | ||
+ | * @description: Perform one-time initialization of your script | ||
+ | * param {string} | ||
+ | **/ | ||
+ | async function init() { | ||
+ | // No initialization tasks required for this simple example | ||
+ | } | ||
− | + | /** | |
− | + | * @name: compute | |
− | + | * @description: Perform computation on input measurements. This function is called when new measurements arrive | |
− | + | * @param {string} measurement - The input measurement name | |
− | + | * @param {Array.<object>} data - The measurement array data | |
− | + | **/ | |
− | + | function compute(measurement, data) { | |
− | + | // Loop through the array of incoming measurements, | |
− | + | // and convert 'temperatureF' to 'temperatureC' | |
− | + | for (let i = 0; i < data.length; i++) { | |
− | + | const inputMeasurement = data[i]; | |
− | + | const temperatureF = inputMeasurement.temperatureF; | |
+ | |||
+ | // Calculate the temperature in C | ||
+ | const temperatureC = (temperatureF − 32) * 5/9; | ||
− | + | const outputMeasurement = { | |
− | + | tm_source: inputMeasurement.tm_source, | |
− | + | site_id: inputMeasurement.site_id, | |
− | + | tag1: inputMeasurement.tag1, | |
− | + | tag2: inputMeasurement.tag2, | |
− | + | temperatureC: temperatureC, | |
− | + | }; | |
− | + | ||
− | + | // Output computed measurement | |
− | + | output(outputMeasurement); | |
− | + | } | |
− | + | } | |
− | + | ||
+ | /** | ||
+ | * @function output | ||
+ | * @description: User-invoked function to output a computed measurement | ||
+ | * @param {Object} outputData - Object containing output metrics (required tags, optional tags, measurement metrics) | ||
+ | * @param {string} outputData.tm_source - Timestamp (Same as input measurement) | ||
+ | * @param {string} outputData.site_id - Site Id (Same as input measurement) | ||
+ | * @param {string} outputData.tag1 - Tag1 (Same as input measurement) | ||
+ | * @param {number} outputData.tag2 - Tag2 (Same as input measurement) | ||
+ | * @param {boolean} outputData.temperatureC - Calculated temperature in C | ||
+ | **/ | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | <translate> | ||
+ | ==== Example 2: Lift Total Floors Moved ==== <!--T:49--> | ||
+ | |||
+ | <!--T:50--> | ||
+ | This example shows a more complex usage of computed measurement. | ||
+ | |||
+ | <!--T:51--> | ||
+ | Imagine you have a lift controller that sends floor position values (i.e. 1 when the lift car is at the 1st Floor; 2 when it is at the 2nd floor, etc). You want to use the lift car position data to calculate the cumulative total number of floors it has moved. | ||
+ | |||
+ | <!--T:52--> | ||
+ | For example, if the lift was previously reported to be at the 1st Floor and is now reported to be at the 5th Floor, the total number of floors moved is 5 - 1 = 4. If it is next reported to be at the 7th Floor, the new total number of floors moved is (7 - 5) + 4 = 6. Etc | ||
+ | |||
+ | <!--T:53--> | ||
+ | This is the (abbreviated) measurement <tt>lift_controller_v1</tt> from the lift controller that you have as input, where <tt>lsid</tt> is the lift ID tag, and <tt>position</tt> is the lift car position data: | ||
+ | </translate> | ||
+ | |||
+ | { | ||
+ | "tm_source": xxxxxxxxxx, | ||
+ | "site_id" xxxxxxxx, | ||
+ | "lsid": "xxxxxxxx", | ||
+ | "position": xxxxxxxxx | ||
+ | } | ||
+ | |||
+ | <translate> | ||
+ | <!--T:54--> | ||
+ | And this is the computed measurement that you want to output, where <tt>floorsMoved</tt> is to cumulative total number of floors moved. You will retain the input measurement's tags in order to correctly attribute the computed measurement to the original lift: | ||
+ | </translate> | ||
+ | |||
+ | { | ||
+ | "tm_source": xxxxxxxxxx, | ||
+ | "site_id" xxxxxxxx, | ||
+ | "lsid": "xxxxxxxx", | ||
+ | "floorsMoved": xxxxxxxxx | ||
+ | } | ||
+ | |||
+ | <translate> | ||
+ | <!--T:55--> | ||
+ | As you want to accumulate the total number of floors moved over time, and not just performing per-measurement-data calculations, you will declare a global variable <tt>totalFloorsMoved</tt> for this purpose: | ||
+ | </translate> | ||
+ | |||
+ | <syntaxhighlight lang="Javascript"> | ||
+ | let totalFloorsMoved; | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | <translate> | ||
+ | <!--T:56--> | ||
+ | Your computation function will perform calculation across consecutive <tt>lift_controller_v1</tt> input measurements in the computation function. You will declare a global variable <tt>prevFloors</tt> to store the previous value of the <tt>floorsMoved</tt> metric, so the next time the computation function is executed on the next <tt>lift_controller_v1</tt> input measurement, you can perform the calculation: | ||
+ | </translate> | ||
+ | |||
+ | <syntaxhighlight lang="Javascript"> | ||
+ | let prevFloors; | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | <translate> | ||
+ | <!--T:57--> | ||
+ | You can also use the API functions '''getStoredValue''' and '''setStoredValue''' to store accumulated values to persistent storage. This is so you will not lose the accumulated data if the Senfi undergoes system maintenance where computed measurements are terminated and reinitialized. | ||
+ | |||
+ | <!--T:58--> | ||
+ | For example, to read previously calculated total floors moved: | ||
+ | </translate> | ||
+ | |||
+ | <syntaxhighlight lang="Javascript"> | ||
+ | let prevTotalFloorsMoved = await getStoredValue('floorsMoved'); | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | <translate> | ||
+ | <!--T:59--> | ||
+ | Thus, the completed initialization function would be: | ||
+ | </translate> | ||
+ | |||
+ | <syntaxhighlight lang="Javascript"> | ||
+ | let totalFloorsMoved; | ||
+ | let prevFloor; | ||
+ | |||
+ | /** | ||
+ | * @name: init | ||
+ | * @description: Perform one-time initialization of your script | ||
+ | * param {string} | ||
+ | **/ | ||
+ | async function init() { | ||
+ | // Perform initialization of script here | ||
+ | let _totalFloorsMovedVal = await getStoredValue('floorsMoved'); | ||
+ | let _prevFloorVal = await getStoredValue('prevFloor'); | ||
+ | |||
+ | // Return value from getStoredValue is a JSON string. | ||
+ | // We need to convert it back to the actual values (an integer) | ||
+ | // or null if no value was previously stored | ||
+ | let _totalFloorsMoved = JSON.parse(_totalFloorsMovedVal); | ||
+ | let _prevFloor = JSON.parse(_prevFloorVal); | ||
+ | |||
+ | if (_totalFloorsMoved == null) { | ||
+ | totalFloorsMoved = 0; // This is the first time we run the script. Set to 0 | ||
+ | } else { | ||
+ | totalFloorsMoved= _totalFloorsMoved ; // Set to the previously stored value | ||
+ | } | ||
+ | |||
+ | if (_prevFloor == null) { | ||
+ | prevFloor = 0; // This is the first time we run the script. Set to 0 | ||
+ | } else { | ||
+ | prevFloor = _prevFloor ; // Set to the previously stored value | ||
+ | } | ||
+ | } | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | <translate> | ||
+ | <!--T:60--> | ||
+ | Now we need to write the computation logic: | ||
+ | |||
+ | <!--T:61--> | ||
+ | * Upon a new batch input measurements, for each individual measurement, calculate the absolute number of floors moved since the last measurement: <tt>floors moved = abs(current position - previous position)</tt> | ||
+ | * Add the absolute number of floors moved to the cumulative total number of floors moved: <tt>total floors moved += floors moved</tt> | ||
+ | * Construct and output the computed measurement | ||
+ | * Store <tt>current position</tt> as <tt>previous position</tt>, for the next iteration | ||
+ | |||
+ | <!--T:62--> | ||
+ | The computation function would be: | ||
+ | </translate> | ||
+ | |||
+ | <syntaxhighlight lang="Javascript"> | ||
+ | /** | ||
+ | * @name: compute | ||
+ | * @description: Perform computation on watching measurements. This function is called when new measurements arrive | ||
+ | * @param {string} measurement - The source measurement name | ||
+ | * @param {Array.<object>} data - The measurement array data | ||
+ | **/ | ||
+ | function compute(measurement, data) { | ||
+ | // Perform calculation on data here | ||
+ | for (let i = 0; i < data.length; i++) { | ||
+ | const d = data[i]; | ||
+ | |||
+ | // On the very first measurement where _prevFloor is 0, set _prevFloor to current position | ||
+ | if (_prevFloor == 0) { | ||
+ | _prevFloor = d.positon; | ||
} | } | ||
+ | |||
+ | // Calculate the difference in floors moved | ||
+ | const floorsMoved = Math.abs(d.position - _prevFloor) + _totalFloorsMoved; | ||
+ | |||
+ | // Constuct the output measurement | ||
+ | const outputMeasurement = { | ||
+ | site_id: d.site_id, | ||
+ | country: d.country, | ||
+ | lsid: d.lsid, | ||
+ | floorsMoved: floorsMoved | ||
+ | }; | ||
+ | |||
+ | // Output computed measurement | ||
+ | output(outputMeasurement ); | ||
+ | |||
+ | // Store current values | ||
+ | _prevFloor = d.position; | ||
+ | _totalFloorsMoved = floorsMoved; | ||
} | } | ||
− | + | ||
− | /** | + | // Persist global values, to survive script restart |
− | + | await setStoredValue('floorsMoved', _totalFloorsMoved); | |
− | + | await setStoredValue('prevFloor', _prevFloor); | |
− | + | } | |
− | + | </syntaxhighlight> | |
− | + | ||
− | + | <translate> | |
− | + | === Computed Measurement Script API === <!--T:63--> | |
− | + | ||
− | + | ==== Output API function ==== <!--T:64--> | |
− | + | </translate> | |
+ | * '''output''' | ||
+ | <syntaxhighlight lang="Javascript"> | ||
+ | /** | ||
+ | * @function output | ||
+ | * @description: User-invoked function to output a computed measurement | ||
+ | * @param {object} outputData - Object containing output metrics (required tags, optional tags, measurement metrics) | ||
+ | * @param {string} outputData.lsid - [Example] lsid tag (for lifts) | ||
+ | * @param {string} outputData.country - [Example] country tag | ||
+ | * @param {number} outputData.metric1 - [Example] metric1 | ||
+ | * @param {boolean} outputData.metric2 - [Example] metric2 | ||
+ | * @param {integer} outputData.metric3 - [Example] metric3 | ||
+ | **/ | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | <translate> | ||
+ | ==== Persistent Storage API functions ==== <!--T:65--> | ||
+ | </translate> | ||
+ | * '''setStoredValue''' | ||
+ | <syntaxhighlight lang="Javascript"> | ||
+ | /** | ||
+ | * @async - Use await to wait for the function to complete | ||
+ | * @function setStoredValue | ||
+ | * @description Persist a value to senfi database. You may retrieve this value using 'getStoreValue' | ||
+ | * @param {string} key | ||
+ | * @param {any} value | ||
+ | **/ | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | * '''getStoredValue''' | ||
+ | <syntaxhighlight lang="Javascript"> | ||
+ | /** | ||
+ | * @async - Use await to wait for the function to complete | ||
+ | * @function getStoredValue | ||
+ | * @description Retrieve a value previously stored in senfi database. | ||
+ | * @param {string} key | ||
+ | * @return {Promise.<string>} stringified value. Use JSON.parse() to restore the value to original form | ||
+ | **/ | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | <translate> | ||
+ | ==== Timing API functions ==== <!--T:66--> | ||
+ | </translate> | ||
+ | * '''setInterval''' | ||
+ | <syntaxhighlight lang="Javascript"> | ||
+ | /** | ||
+ | * @function setInterval | ||
+ | * @param {function} fn | ||
+ | * @param {integer} interval value, in miliseconds | ||
+ | * @returns {string} token | ||
+ | **/ | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | * '''clearInterval''' | ||
+ | <syntaxhighlight lang="Javascript"> | ||
+ | /** | ||
+ | * @function clearInterval | ||
+ | * @param {string} token - The return value from setInterval() | ||
+ | **/ | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | * '''setTimeout''' | ||
+ | <syntaxhighlight lang="Javascript"> | ||
+ | /** | ||
+ | * @function setTimeout | ||
+ | * @param {function} fn | ||
+ | * @param {integer} timeout value, in miliseconds | ||
+ | * @returns {string} token | ||
+ | **/ | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | * '''clearTimeout''' | ||
+ | <syntaxhighlight lang="Javascript"> | ||
+ | /** | ||
+ | * @function clearTimeout | ||
+ | * @param {string} token - The return value from setTimeout() | ||
+ | **/ | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | <translate> | ||
+ | ==== Logging API functions ==== <!--T:67--> | ||
+ | The following functions allow you to log messages which will show up in the '''Debugger''' in the [https://ems.senfi.io/cms CMS] '''Developer''' Tab. | ||
+ | </translate> | ||
+ | |||
+ | * '''console.log''' | ||
+ | * '''console.error''' | ||
+ | * '''console.warn''' | ||
+ | * '''console.info''' | ||
+ | * '''console.debug''' | ||
− | = | + | <translate> |
− | + | <!--T:68--> | |
+ | <span class="right">[[Sending_data_to_Senfi|Next: Sending data to Senfi]]</span> | ||
+ | </translate> |
Latest revision as of 18:29, 11 November 2019
This page describes computed measurement in detail.
Contents
Overview
As described here, computed measurement is a type of data that is generated from other data sources.
At core of a computed measurement is a computation script that takes in input measurements, performs logic and calculations, and outputs a set of derived metrics as a computed measurement.
To use a computed measurement, you have to:
- Choose the input measurements
- Choose the output metrics
- Write a script that calculates the output metrics from the input measurements' metrics
The computed measurement script will be initialized immediately after the computed measurement is created and is active until the computed measurement is deleted. Senfi will execute the script whenever measurement data from the selected input measurements arrive.
Design
The first thing to do when designing a computed measurement is to decide what are the inputs and what to output. A computed measurement can specify multiple measurements as input. Metrics and tags in those measurement will then be available to the function that is responsible for generating the output.
Input Measurements
Any measurement that is available to you in the CMS can be used as an input measurement for your computed measurement.
You should choose only the measurements with metrics that are needed for you to calculate your computed measurement as your input measurements.
Output Measurements
Any data that you want to create with your computed measurement should be packaged in an output measurement. An output measurement has the same composition (metrics, tags, timestamp) as other measurements in Senfi.
When designing the output measurement metrics and tags, should consider the following:
- Metrics: The metrics that you want and how to calculate them from your input measurement(s).
- Tags: What tags that allow you to differentiate computed measurements derived from different sensors. You can re-use the tags from input measurements or specify different tags depending on the outcome of your computed measurement's computation.
- Timestamp: Whether to use the input measurement's timestamp, or the time the computed measurement was output.
Implementation
To generate computed measurements, you will write a script that can process your selected input measurements, perform any calculations or logic necessary, and output the computed measurement data.
The scripting language used to author the computed measurement scripts is JavaScript.
The computed measurement script consists of two functions:
- Initialization: Sets up the script
- Computation: Processes data from input measurements, performs calculations and output your computed measurement
Both the initialization and computation functions must be present in your script.
init(): Initialization Function
The script initialization function can be used to initialized any data structures you need for logic and state management of your script. It is executed only once when your script is created, modified, or after a system maintenance in which services are restarted.
This is the script initialization function template:
/**
* @name: init
* @description: Perform one-time initialization of your script
* param {string}
**/
async function init() {
// Perform initialization of script here
// TODO
}
You may choose to leave this function empty if your computed measurement does not need to keep track of data across multiple or consecutive input measurements, such as summation of metric values or checking the difference between a pairs of input measurements.
async function init() {
// Initialization not required
}
If your computed measurement needs to keep track of data across multiple or consecutive input measurements, you should create the appropriate data structures here.
You can also declare global variables. For example:
// Declare global variable for a summation value
let sum;
async function init() {
// Initialize sum to zero
sum = 0;
}
compute(): Computation Function
The script computation function is executed every time a new measurement data, or batch of measurement data, from your specified input measurement arrives. In this function, you will read the data from those input measurements, performs any logic or calculations required, and output your computed measurement data.
This is the script computation function template:
/**
* @name: compute
* @description: Perform computation on input measurements. This function is called when new measurements arrive
* @param {string} measurement - The input measurement name
* @param {Array.<object>} data - The measurement array data
**/
function compute(measurement, data) {
// Perform calculation on data here
// TODO
// Example output
const outputData = {
metric1: 1,
metric2: true,
metric3: 1,
};
// Output computed measurement. See API docs below
output(outputData);
}
// Output API function
/**
* @function output
* @description: User-invoked function to output a computed measurement
* @param {object} outputData - Object containing output metrics (required tags, optional tags, measurement metrics)
* @param {string} outputData.lsid - [Example] lsid tag (for lifts)
* @param {string} outputData.country - [Example] country tag
* @param {number} outputData.metric1 - [Example] metric1
* @param {boolean} outputData.metric2 - [Example] metric2
* @param {integer} outputData.metric3 - [Example] metric3
**/
Do note that data is an array of measurement data, so typically you would loop through the data input to perform calculation tasks on each variable like this:
function compute(measurement, data) {
// Loop through the array of incoming measurements
for (let i = 0; i < data.length; i++) {
const inputMeasurement = data[i];
// Perform calculation and output
// ...
// Output
output(...);
}
}
At the end of your computations, once you have constructed the output data for the computed measurement, you should call output(), with the output data, to output your computed measurement. This will inform the script engine to ingest your computed measurement into Senfi.
output(): Script Output
The script output function is a pre-defined function call that you should invoke whenever you want to output your computed measurement.
You must invoke the output function with a valid measurement data (e.g. all declared metrics and tags) for your computed measurement. Note the output measurement will automatically be attributed to the computed measurement name you have chosen in the CMS. It is not possible for the script to output another computed measurement.
Testing
You can test your computed measurement in the CMS when you are creating or editing the computed measurement.
Your testing data should be identical in format and values to the actual data from your input measurements. You can test with single or multiple sets of data.
The test data format is in JSON, as an Array of Objects of the form:
{ "measurement": "data": [] }
where measurement is the name of the input measurement, and data is a list of one or more measurement data from the input measurement following the Senfi data message format.
Example of sending 1 set of measurement data from the measurement temperature_v1. The script computation function will be executed once during the test:
[ { "measurement": "temperature_v1", "data": [{ "tm_source": xxxxxxxxxx, "site_id" xxxxxxxx, "tag1": "xxxxxxxx", "tag2": "xxxxxxxx", "temp": xxxxxxxx, }] } ]
Abbreviated example of sending a batch of 3 sets of measurement data from the measurement temperature_v1. The script computation function will be executed once during the test:
[ { "measurement": "temperature_v1", "data": [{ ... "temp": xxxxxxxx, },{ ... "temp": xxxxxxxx, },{ ... "temp": xxxxxxxx, }] } ]
Abbreviated example of sending 3 sets of measurement data from the measurement temperature_v1. The script computation function will be executed 3 times during the test:
[ { "measurement": "temperature_v1", "data": [{ ... "temp": xxxxxxxx, }] }, { "measurement": "temperature_v1", "data": [{ ... "temp": xxxxxxxx, }] }, { "measurement": "temperature_v1", "data": [{ ... "temp": xxxxxxxx, }] } ]
If your script performs computation with multiple input measurements, you can specify multiple measurements. In this example, the computation function will be executed twice, once for temperature_v1 and once for humidity_v1:
[ { "measurement": "temperature_v1", "data": [{ ... "temp": xxxxxxxx, }] }, { "measurement": "humidity_v1", "data": [{ ... "humidity": xxxxxxxx, }] } ]
Execution
After you create or edit your computed measurement in the CMS, your computed measurement script will be compiled and the initialization and computation functions called:
- When you finish creating the script: The initialization function will be run
- When you finish editing the script: The initialization function will be run
- When data from each of your input measurements arrives: The computation function will be run.
Errors and Debugging
Errors that occur during the initialization or execution of your script will show up in the Debugger in the Developer tab in the CMS.
Console messages logged by your script also appear in the Debugger.
Examples
Example 1: Temperature Scale Conversion
This example shows a simple usage of computed measurement.
Imagine you have a temperature sensor that sends raw temperature values in degrees Fahrenheit (℉) and you want to show the temperature in degrees Celsius (°C) instead. This is the temperature measurement from the temperature sensor that you have as input, where temperatureF is the temperature data in ℉:
{ "tm_source": xxxxxxxxxx, "site_id" xxxxxxxx, "tag1": "xxxxxxxx", "tag2": "xxxxxxxx", "temperatureF": xxxxxxxxx }
And this is the computed measurement that you want to output, where temperatureC is the temperature metric in °C. You will retain the input measurement's tags in order to correctly attribute the computed measurement to the original temperature sensor:
{ "tm_source": xxxxxxxxxx, "site_id" xxxxxxxx, "tag1": "xxxxxxxx", "tag2": "xxxxxxxx", "temperatureC": xxxxxxxxx }
To convert the temperatureF temperature metric in ℉ into temperatureC in °C, you would use the ℉ to °C formula:
temperatureC = (temperatureF − 32) * 5/9;
As you are just performing conversion when data arrives, there is no need to perform any initialization tasks or declare any global variables. So you can leave the initialization function empty:
async function init() {
// No initialization tasks required for this simple example
}
Putting everything together, the full script would be similar to:
/**
* @name: init
* @description: Perform one-time initialization of your script
* param {string}
**/
async function init() {
// No initialization tasks required for this simple example
}
/**
* @name: compute
* @description: Perform computation on input measurements. This function is called when new measurements arrive
* @param {string} measurement - The input measurement name
* @param {Array.<object>} data - The measurement array data
**/
function compute(measurement, data) {
// Loop through the array of incoming measurements,
// and convert 'temperatureF' to 'temperatureC'
for (let i = 0; i < data.length; i++) {
const inputMeasurement = data[i];
const temperatureF = inputMeasurement.temperatureF;
// Calculate the temperature in C
const temperatureC = (temperatureF − 32) * 5/9;
const outputMeasurement = {
tm_source: inputMeasurement.tm_source,
site_id: inputMeasurement.site_id,
tag1: inputMeasurement.tag1,
tag2: inputMeasurement.tag2,
temperatureC: temperatureC,
};
// Output computed measurement
output(outputMeasurement);
}
}
/**
* @function output
* @description: User-invoked function to output a computed measurement
* @param {Object} outputData - Object containing output metrics (required tags, optional tags, measurement metrics)
* @param {string} outputData.tm_source - Timestamp (Same as input measurement)
* @param {string} outputData.site_id - Site Id (Same as input measurement)
* @param {string} outputData.tag1 - Tag1 (Same as input measurement)
* @param {number} outputData.tag2 - Tag2 (Same as input measurement)
* @param {boolean} outputData.temperatureC - Calculated temperature in C
**/
Example 2: Lift Total Floors Moved
This example shows a more complex usage of computed measurement.
Imagine you have a lift controller that sends floor position values (i.e. 1 when the lift car is at the 1st Floor; 2 when it is at the 2nd floor, etc). You want to use the lift car position data to calculate the cumulative total number of floors it has moved.
For example, if the lift was previously reported to be at the 1st Floor and is now reported to be at the 5th Floor, the total number of floors moved is 5 - 1 = 4. If it is next reported to be at the 7th Floor, the new total number of floors moved is (7 - 5) + 4 = 6. Etc
This is the (abbreviated) measurement lift_controller_v1 from the lift controller that you have as input, where lsid is the lift ID tag, and position is the lift car position data:
{ "tm_source": xxxxxxxxxx, "site_id" xxxxxxxx, "lsid": "xxxxxxxx", "position": xxxxxxxxx }
And this is the computed measurement that you want to output, where floorsMoved is to cumulative total number of floors moved. You will retain the input measurement's tags in order to correctly attribute the computed measurement to the original lift:
{ "tm_source": xxxxxxxxxx, "site_id" xxxxxxxx, "lsid": "xxxxxxxx", "floorsMoved": xxxxxxxxx }
As you want to accumulate the total number of floors moved over time, and not just performing per-measurement-data calculations, you will declare a global variable totalFloorsMoved for this purpose:
let totalFloorsMoved;
Your computation function will perform calculation across consecutive lift_controller_v1 input measurements in the computation function. You will declare a global variable prevFloors to store the previous value of the floorsMoved metric, so the next time the computation function is executed on the next lift_controller_v1 input measurement, you can perform the calculation:
let prevFloors;
You can also use the API functions getStoredValue and setStoredValue to store accumulated values to persistent storage. This is so you will not lose the accumulated data if the Senfi undergoes system maintenance where computed measurements are terminated and reinitialized.
For example, to read previously calculated total floors moved:
let prevTotalFloorsMoved = await getStoredValue('floorsMoved');
Thus, the completed initialization function would be:
let totalFloorsMoved;
let prevFloor;
/**
* @name: init
* @description: Perform one-time initialization of your script
* param {string}
**/
async function init() {
// Perform initialization of script here
let _totalFloorsMovedVal = await getStoredValue('floorsMoved');
let _prevFloorVal = await getStoredValue('prevFloor');
// Return value from getStoredValue is a JSON string.
// We need to convert it back to the actual values (an integer)
// or null if no value was previously stored
let _totalFloorsMoved = JSON.parse(_totalFloorsMovedVal);
let _prevFloor = JSON.parse(_prevFloorVal);
if (_totalFloorsMoved == null) {
totalFloorsMoved = 0; // This is the first time we run the script. Set to 0
} else {
totalFloorsMoved= _totalFloorsMoved ; // Set to the previously stored value
}
if (_prevFloor == null) {
prevFloor = 0; // This is the first time we run the script. Set to 0
} else {
prevFloor = _prevFloor ; // Set to the previously stored value
}
}
Now we need to write the computation logic:
- Upon a new batch input measurements, for each individual measurement, calculate the absolute number of floors moved since the last measurement: floors moved = abs(current position - previous position)
- Add the absolute number of floors moved to the cumulative total number of floors moved: total floors moved += floors moved
- Construct and output the computed measurement
- Store current position as previous position, for the next iteration
The computation function would be:
/**
* @name: compute
* @description: Perform computation on watching measurements. This function is called when new measurements arrive
* @param {string} measurement - The source measurement name
* @param {Array.<object>} data - The measurement array data
**/
function compute(measurement, data) {
// Perform calculation on data here
for (let i = 0; i < data.length; i++) {
const d = data[i];
// On the very first measurement where _prevFloor is 0, set _prevFloor to current position
if (_prevFloor == 0) {
_prevFloor = d.positon;
}
// Calculate the difference in floors moved
const floorsMoved = Math.abs(d.position - _prevFloor) + _totalFloorsMoved;
// Constuct the output measurement
const outputMeasurement = {
site_id: d.site_id,
country: d.country,
lsid: d.lsid,
floorsMoved: floorsMoved
};
// Output computed measurement
output(outputMeasurement );
// Store current values
_prevFloor = d.position;
_totalFloorsMoved = floorsMoved;
}
// Persist global values, to survive script restart
await setStoredValue('floorsMoved', _totalFloorsMoved);
await setStoredValue('prevFloor', _prevFloor);
}
Computed Measurement Script API
Output API function
- output
/**
* @function output
* @description: User-invoked function to output a computed measurement
* @param {object} outputData - Object containing output metrics (required tags, optional tags, measurement metrics)
* @param {string} outputData.lsid - [Example] lsid tag (for lifts)
* @param {string} outputData.country - [Example] country tag
* @param {number} outputData.metric1 - [Example] metric1
* @param {boolean} outputData.metric2 - [Example] metric2
* @param {integer} outputData.metric3 - [Example] metric3
**/
Persistent Storage API functions
- setStoredValue
/**
* @async - Use await to wait for the function to complete
* @function setStoredValue
* @description Persist a value to senfi database. You may retrieve this value using 'getStoreValue'
* @param {string} key
* @param {any} value
**/
- getStoredValue
/**
* @async - Use await to wait for the function to complete
* @function getStoredValue
* @description Retrieve a value previously stored in senfi database.
* @param {string} key
* @return {Promise.<string>} stringified value. Use JSON.parse() to restore the value to original form
**/
Timing API functions
- setInterval
/**
* @function setInterval
* @param {function} fn
* @param {integer} interval value, in miliseconds
* @returns {string} token
**/
- clearInterval
/**
* @function clearInterval
* @param {string} token - The return value from setInterval()
**/
- setTimeout
/**
* @function setTimeout
* @param {function} fn
* @param {integer} timeout value, in miliseconds
* @returns {string} token
**/
- clearTimeout
/**
* @function clearTimeout
* @param {string} token - The return value from setTimeout()
**/
Logging API functions
The following functions allow you to log messages which will show up in the Debugger in the CMS Developer Tab.
- console.log
- console.error
- console.warn
- console.info
- console.debug