Difference between revisions of "Computed measurement"

Revision as of 13:31, 17 September 2019

This page describes computed measurement in detail.

The first thing to do when designing a computed measurement is to decide what are the inputs and what to output. A computed measurement can specify multiple measurements as input. Metrics and tags in those measurement will then be available to the function that is responsible for generating the output.

Input Measurements

Any measurement measurements that that are available to you in the CMS can be used an input measurements for your computed measurement.

You should choose only the measurements with metrics that are needed for you to calculate your computed measurement as your input measurements.

Output Measurements

Any data that you want to create with your computed measurement should be packaged in an output measurement. An output measurement has the same composition (metrics, tags, timestamp) as other measurements in Senfi.

When designing the output measurement metrics and tags, should consider the following:

Metrics: The metrics that you want and how to calculate them from your input measurement(s).
Tags: What tags that allow you to differentiate computed measurements derived from different sensors. You can re-use the tags from input measurements or specify different tags depending on the outcome of your computed measurement's computation.
Timestamp: Whether to use the input measurement's timestamp, or the time the computed measurement was output.

Implementation

To generate computed measurements, you will write a script that can process your selected input measurements, perform any calculations or logic necessary, and output the computed measurement data.

The scripting language used to author the computed measurement scripts is JavaScript.

The computed measurement script consists of two functions:

Initialization: Sets up the script
Computation: Processes data from input measurements, performs calculations and output your computed measurement

Both the initialization and computation functions must be present in your script.

init(): Initialization Function

The script initialization function can be used to initialized any data structures you need for logic and state management of your script. It is executed only once when your script is created, modified, or after a system maintenance in which services are restarted.

This is the script initialization function template:

/**
 * @name: init
 * @description: Perform one-time initialization of your script
 * param {string}
 **/
async function init() {
  // Perform initialization of script here
  // TODO
}

You may choose to leave this function empty if your computed measurement does not need to keep track of data across multiple or consecutive input measurements, such as summation of metric values or checking the difference between a pairs of input measurements.

async function init() {
   // Initialization not required
}

If your computed measurement needs to keep track of data across multiple or consecutive input measurements, you should create the appropriate data structures here.

You can also declare global variables. For example:

// Declare global variable for a summation value
let sum;

async function init() {
  // Initialize sum to zero
  sum = 0;
}

compute(): Computation Function

The script computation function is executed every time a new measurement data, or batch of measurement data, from your specified input measurement arrives. In this function, you will read the data from those input measurements, performs any logic or calculations required, and output your computed measurement data.

This is the script computation function template:

/**
 * @name: compute
 * @description: Perform computation on input measurements. This function is called when new measurements arrive
 * @param {string} measurement - The input measurement name
 * @param {Array.<object>} data - The measurement array data
 **/
function compute(measurement, data) {
  // Perform calculation on data here
  // TODO

  // Example output
  const outputData = {
    metric1: 1,
    metric2: true,
    metric3: 1,
  };

  // Output computed measurement. See API docs below
  output(outputData);
}
 
// Output API function
/**
 * @function output
 * @description: User-invoked function to output a computed measurement
 * @param {object} outputData - Object containing output metrics (required tags, optional tags, measurement metrics)
 * @param {string} outputData.lsid - [Example] lsid tag (for lifts)
 * @param {string} outputData.country - [Example] country tag
 * @param {number} outputData.metric1 - [Example] metric1
 * @param {boolean} outputData.metric2 - [Example] metric2
 * @param {integer} outputData.metric3 - [Example] metric3
 **/

Do note that data is an array of measurement data, so typically you would loop through the data input to perform calculation tasks on each variable like this:

function compute(measurement, data) {    
  // Loop through the array of incoming measurements
  for (let i = 0; i < data.length; i++) {
    const inputMeasurement = data[i];

    // Perform calculation and output
    // ...
    
    // Output
    output(...); 
  }
}

At the end of your computations, once you have constructed your output measurement data, you should call output(), with the computed measurement data, to output your computed measurement. This will inform the script engine to ingest your computed measurement into Senfi.

Note: The computation function does not return or output a computed measurement. You must call output() in order to output a computed measurement data to Senfi.

output(): Script Output

The script output function is a pre-defined function call that you should invoke whenever you want to output your computed measurement.

You must invoke the output function with a valid measurement data (e.g. all declared metrics and tags) for your computed measurement. Note the output measurement will automatically be attributed to the computed measurement name you have chosen in the CMS. It is not possible for the script to output another computed measurement.

Testing

You can test your computed measurement in the CMS when you are creating or editing the computed measurement.

Your testing data should be identical in format and values to the actual data from your input measurements. You can test with single or multiple sets of data.

The test data format is in JSON, as an Array of Objects of the form:

{
  "measurement":
  "data": []
}

where measurement is the name of the input measurement, and data is a list of one or more measurement data from the input measurement following the Senfi data message format.

Example of sending 1 set of measurement data from the measurement temperature_v1. The script computation function will be executed once during the test:

[
  {
    "measurement": "temperature_v1",
    "data": [{
      "tm_source": xxxxxxxxxx,
      "site_id" xxxxxxxx,
      "tag1": "xxxxxxxx",
      "tag2": "xxxxxxxx",
      "temp": xxxxxxxx,
    }]
  }
]

Abbreviated example of sending a batch of 3 sets of measurement data from the measurement temperature_v1. The script computation function will be executed once during the test:

[
  {
    "measurement": "temperature_v1",
    "data": [{
      ...
      "temp": xxxxxxxx,
    },{
      ...
      "temp": xxxxxxxx,
    },{
      ...
      "temp": xxxxxxxx,
    }]
  }
]

Abbreviated example of sending 3 sets of measurement data from the measurement temperature_v1. The script computation function will be executed 3 times during the test:

[
  {
    "measurement": "temperature_v1",
    "data": [{
      ...
      "temp": xxxxxxxx,
    }]
  },
  {
    "measurement": "temperature_v1",
    "data": [{
      ...
      "temp": xxxxxxxx,
    }]
  },
  {
    "measurement": "temperature_v1",
    "data": [{
      ...
      "temp": xxxxxxxx,
    }]
  }
]

If your script performs computation with multiple input measurements, you can specify multiple measurements. In this example, the computation function will be executed twice, once for temperature_v1 and once for humidity_v1:

[
  {
    "measurement": "temperature_v1",
    "data": [{
      ...
      "temp": xxxxxxxx,
    }]
  },
  {
    "measurement": "humidity_v1",
    "data": [{
      ...
      "humidity": xxxxxxxx,
    }]
  }
]

Execution

After you create or edit your computed measurement in the CMS, your computed measurement script will be compiled and the initialization and computation functions called:

When you finish creating the script: The initialization function will be run
When you finish editing the script: The initialization function will be run
When data from each of your input measurements arrives: The computation function will be run.

Note: If many measurements from the same input measurement arrive within a very short period of time, the measurements from that input measurement may be batched together. For example, if your input measurement input_measurement_1 sends data 10 times per second, you may receive 30 sets of measurements from input_measurement_1 every 3 seconds.

Note: If you have multiple input measurements, each invocation of the computation function will be for only one input measurement. For example, if you have the following input measurements input_measurement_1 and input_measurement_2, expect the computation function to be run twice, once for input_measurement_1 and another for input_measurement_1, for each set of data.

Errors and Debugging

Errors that occur during the initialization or execution of your script will show up in the Debugger in the Developer tab in the CMS.

Console messages logged by your script also appear in the Debugger.

Examples

Example 1: Temperature Scale Conversion

This example shows a simple usage of computed measurement.

Imagine you have a temperature sensor that sends raw temperature values in degrees Farenheit (℉) and you want to show the temperature in degrees Celsius (°C) instead. This is the temperature measurement from the temperature sensor that you have as input:

 {
   "tm_source": xxxxxxxxxx,
   "site_id" xxxxxxxx,
   "tag1": "xxxxxxxx",
   "tag2": "xxxxxxxx",
   "temperatureF": xxxxxxxxx
 }

And this is the computed measurement that you want to output:

 {
   "tm_source": xxxxxxxxxx,
   "site_id" xxxxxxxx,
   "tag1": "xxxxxxxx",
   "tag2": "xxxxxxxx",
   "temperatureC": xxxxxxxxx
 }

To convert the temperatureF temperature metric in ℉ into temperatureC in °C, you would use the ℉ to °C formula:

temperatureC = (temperatureF − 32) * 5/9;

As you are just performing conversion when data arrives, there is no need to perform any initialization tasks or declare any global variables. So you can leave the initialization function empty:

async function init() {
  // No initialization tasks required for this simple example
}

Putting everything together, the full script would be similar to:

 /**
  * @name: init
  * @description: Perform one-time initialization of your script
  * param {string}
  **/
 async function init() {
   // No initialization tasks required for this simple example
 } 
 
 /**
  * @name: compute
  * @description: Perform computation on input measurements. This function is called when new measurements arrive
  * @param {string} measurement - The input measurement name
  * @param {Array.<object>} data - The measurement array data
  **/
 function compute(measurement, data) {    
   // Loop through the array of incoming measurements,
   // and convert 'temperatureF' to 'temperatureC'
   for (let i = 0; i < data.length; i++) {
     const inputMeasurement = data[i];
     const temperatureF = inputMeasurement.temperatureF;
  
     // Calculate the temperature in C
     const temperatureC = (temperatureF − 32) * 5/9; 
  
     const outputMeasurement = {
       tm_source: inputMeasurement.tm_source,
       site_id: inputMeasurement.site_id,
       tag1: inputMeasurement.tag1,
       tag2: inputMeasurement.tag2,
       temperatureC: temperatureC,
     };  
   
     // Output computed measurement
     output(outputMeasurement);
   }
 }
 
 /**
  * @function output 
  * @description: User-invoked function to output a computed measurement
  * @param {Object} outputData - Object containing output metrics (required tags, optional tags, measurement metrics)
  * @param {string} outputData.tm_source - Timestamp (Same as input measurement)
  * @param {string} outputData.site_id - Site Id (Same as input measurement)
  * @param {string} outputData.tag1  - Tag1 (Same as input measurement)
  * @param {number} outputData.tag2 - Tag2 (Same as input measurement)
  * @param {boolean} outputData.temperatureC - Calculated temperature in C
 **/

Example 2

 ... TODO: give a more complex example ...