Prerequisites

This tutorial assumes you have completed the entirety of the Introductory Tutorial and the App Component Tutorial. Please complete both tutorials if you have not done so already, then return to this page to learn how to use Stateful Services to improve them.

Overview

The App constructed in the Introductory Tutorial does not store any of the sensor information.
Events are received and processed by the Service and then discarded. There may be situations where storing the most recent sensor value for an engine may be useful. However, storing each event into the database as it arrives is a slow and inefficient solution that will not scale. In this Tutorial, you will learn to use the state properties of the Service to store sensor events in memory. Then, if necessary, the values stored in memory can be written to the database at an infrequent interval.

Part 1: Enhance the Component

1. Review Global State Variable Example

Recall that in the Introductory Tutorial, you introduced an AccumulateState task called SaveSpeed in the Inbound Service Event Handler SpeedEvent. The SaveSpeed task saves speed sensor readings in the speed Global State variable.

The SaveSpeed task also causes three Procedures to be generated: com.vantiq.engines.EngineMonitor.speedGet(), com.vantiq.engines.EngineMonitor.speedReset(), and com.vantiq.engines.EngineMonitor.speedUpdate(). These may be used to retrieve, reset and update the speed Global State variable.

2. Enhance the App Component

In the App Component Tutorial, some common tasks were used to create the FilterAndDiagnose App Component. Let’s add an additional task to the FilterAndDiagnose Component to save engine state whenever an unusual engine alert message is generated by the App Component.

Open the FilterAndDiagnose App Component by clicking on it from within the Project Contents tree:

From the Modifiers section of the palette, drag and drop an AccumulateState task over the link between the AddDiagnosis task and the Diagnosis connector. Select event as the Downstream Event and click OK.

Click the new AccumulateState task and rename it SaveAlertMessage.

Click the Click to Edit link to edit the configuration for the new SaveAlertMessage.

Enter lastAlert as the stateProperty property.

Click the “vailScript” link next to the procedure property. This allows you to write a VAIL Block code snippet for updating the Service’s Global State based on incoming events.

The following state-updating code tests to determine if the engine status alert message isn’t empty and, if so, creates a JSON object that contains four properties: systemId, lastAlert, temperature, and speed.

// Update the value of state using event.
var msg = event.alertMsg
if (msg != "") {
    var newState = {systemId: event.systemId, lastAlert: msg, temperature: event.temperature, speed: event.speed}
    state = newState
}

Click OK to complete editing the SaveAlertMessage task, then Save the component, which contains the new SaveAlertMessage task.

When an App Component is updated, the Service Event Handlers that use it must be resaved to include the Component changes.

Open the com.vantiq.engines.EngineMonitor Service, click on the Implement tab, then open the TemperatureEvent Event Handler in the Inbound section. The TemperatureEvent Inbound Event Type uses the FilterAndDiagnose App Component, so it needs to be updated. The easiest way to cause an update of the TemperatureEvent handler is to click the SensorReading task, make a small change to its name then change the name back to SensorReading, then save the event handler.

3. Test the New Global State

If you’ve imported the App Components completed tutorial, you will need to enable the two Sources, com.vantiq.engines.SpeedSensor and com.vantiq.engines.TemperatureSensor in order to begin the flow of simulated sensor events.

Click the Active Resource Control Center (lightning bolt) icon in the IDE Navigation Bar to display the Active Resource Control Center pane:

Activate the SpeedSensor and TemperatureSensor Sources by clicking their Active Slider. Once the Sources have been activated, the TemperatureEvent handler in the com.vantiq.engines.EngineMonitor Service will display badges as each of the tasks process sensor events:

To check that the lastAlert Global State is being correctly saved:

• click the State section in the Implement tab,
• open the Global State Type section,
• click the View (play icon) Action button next to the lastAlert variable. Select lastAlertGet as the Procedure.

Let the lastAlertGet Procedure run for a minute or so and observe the lastAlert JSON object that the SaveAlertMessage Component task is producing:

Part 2: Write State to the Database

1. Write System Status to the Database

As indicated in the Introductory Tutorial, the traditional approach of storing event data in a database is too slow and unnecessary for most purposes. In the engine status system modified in Part 1 above, processing sensor events and displaying them in the EngineMonitor Client is all accomplished by using in-memory state. However, there are circumstances where certain data need to be persistently stored on an infrequent basis.

As a persistent state example, we will create a mechanism to commit the last alert (lastAlert) Service State object to the database at a regular interval.

In the com.vantiq.engines.EngineMonitor pane, navigate to the Implement tab and click on the + button next to the Procedures header and select New Procedure.

Copy and paste the following code which retrieves the State object and updates the com.vantiq.engines.EngineLastAlert instance in the database for the matching systemId.

package com.vantiq.engines
PROCEDURE EngineMonitor.writeLastAlert()

var alertState = EngineMonitor.lastAlertGet()
if (alertState) {
    UPSERT com.vantiq.engines.EngineLastAlert(alertState)
}

Click Save to save the Service. You will be prompted to repair the Service Interface because you have created a public Procedure that is not part of the Service Interface. Click Repair Interface.

Notice that when you saved the Service, the Procedure shows an error. That is because the com.vantiq.engines.EngineLastAlert Type does not exist yet. To create a new Type by that name, use the Add menu to select Type, then click the New Type button. Enter the following values as shown below:

• Name: EngineLastAlert
• Package: com.vantiq.engines
• Description: Last engine alert written periodically by the EngineMonitor Service
• Role: standard

Click OK to display the com.vantiq.engines.EngineLastAlert Type pane. In the Properties tab, add the following four properties using the Add Property button.

• Name: systemId, Type: String
• Name: lastAlert, Type: String
• Name: temperature, Type: Integer
• Name: speed, Type: Integer

Navigate to the Indexes tab and select Add Index. Select systemId as the key and check the Is Index Unique? checkbox.
Click OK to close the popup.

Navigate to the Natural Keys tab and click Edit Keys. Click the + button to add a key and select systemId and click OK.

Click Save to save the Type. Notice that saving the Type removes the error from the writeLastAlert Procedure.

2. Schedule the Procedure

In the Service: com.vantiq.engines.EngineMonitor pane, click on the State section at the top of the Implement tab.

Click on the Scheduled Procedures: to expand the section. This will show a list of the Procedures defined in the Service that are available to be scheduled. Only procedures with no parameters may be scheduled.

Next to the writeLastAlert entry in the list, click the Add Schedule button. This allows you to set the frequency at which this Procedure is automatically executed. For the purposes of testing, leave the interval at 1 minute. However, in a production system a longer interval such as 10 minutes, 1 hour, or 1 day is more appropriate.

Notice that the scheduled event is not yet active. Once any Procedure in the Service is executed, the scheduled Procedures will become active. To ensure the writeLastAlert Scheduled Procedure starts, navigate back to the writeLastAlert Procedure and click the Execute (play) button at the upper-left.

3. Monitoring the Database

Using the Project Contents tree on the left-hand side, open the com.vantiq.engines.EngineLastAlert Type pane then click Show All Records in the pane.
In the resulting pane select Auto Refresh, set the interval to 5 seconds, and click OK.

Wait about a minute for the Scheduled Procedures to execute and write the most recent Service state to the database.

Notice that the EngineLastAlert instance for systemId 0123456789 contains the most recent speed value of 40, a temperature value of 215, and a lastAlert indicating an engine overheat condition.

Part 3: Using Partitioned State

The Service you have defined uses Global State. However, to truly scale your application to support a large number of distinct engines, you will have to use Partitioned State instead. By partitioning a Service’s state, we can divide it up between the members of a Vantiq cluster and allow each partition to be accessed independently. This allows for concurrent processing and optimizes for performance at scale. For more information about Partitioned State, see Stateful Services.

1. Split by Group

The first step in converting the Service to use Partitioned State is to define how to partition the data.

In the com.vantiq.engines.EngineMonitor Service, navigate to the Implement tab and open the SpeedEvent Inbound Event Handler. From the Flow Control section of the Palette, drag and drop a SplitByGroup task over the link between the SensorReading task and the SaveSpeed task. Click on the SplitByGroup task and rename it SplitBySystemId.

Click on Click to Edit to configure the SplitBySystemId task. Set the groupBy property to event.systemId. This will partition the inbound events by system/engine ID.

Next, open the TemperatureEvent Inbound Event Handler. Drag and drop a SplitByGroup task over the link between the SensorReading task and the OverheatCheck task. Click on the SplitByGroup task and rename it SplitBySystemId. Right-click the SplitBySystemId task and select Link Existing Task. Select NoAlert as the Task Name and click OK. Delete the link between the SensorReading and NoAlert tasks.

Finally, click on Click to Edit to configure the SplitBySystemId task. Set the groupBy property to event.systemId. This will partition the inbound events by system/engine ID.

Save the Service.

2. Update Event Handler Tasks

With the TemperatureEvent Inbound Event Handler of the com.vantiq.engines.EngineMonitor Service still open, click on the RetrieveSpeed task and click Click to Edit. Click on the link to the right of the transformation property.

Since the RetrieveSpeed Transformation task is now below a SplitByGroup task, the transformation will be operating on a state object that only has a single piece of the Partitioned State; that is, the entry for a single groupBy key. So, the configured Visual Transformation needs to be changed to

com.vantiq.engines.EngineMonitor.speedGet(event.systemId)

as the transformation.

Click OK to save the transformation, click OK again to save the configuration. Save the Service.

3. View Partition Type

Click on the State section at the top of the Implement tab. Click on the Global State Type: to expand the section. Notice that your Service still contains one Global State Type property, lastAlert. Delete that Global State Type property since it is no longer needed.

Next click on the Partitioned State Type: to expand that section.

Notice that two new Partitioned State properties were automatically added to your Service’s definition.
The speed property is a Concurrent.Map() object keyed by the engine’s systemId and contains the most recent speed state. The lastAlert property is a Concurrent.Map() object keyed by the engine’s systemId and contains the most recent alert state. These Maps will be partitioned by engine systemId across the members of the Vantiq cluster.

These are generated State properties and should not be updated.

4. Update writeLastAlert Procedure

Update the Procedure used to store the state in the database so that it works correctly with the Partitioned State.
The Procedure must be declared as multi partition rather than global since it is accessing a Partitioned State rather than Global State. Multi-partitioned Procedures have access to the Service’s Partitioned State and when scheduled, will run on all partitions.

Navigate to the Implement tab and select the writeEngineStatus Procedure. Update the Procedure text with the following VAIL:

package com.vantiq.engines
multi partition PROCEDURE EngineMonitor.writeLastAlert()

var alertStates = lastAlert
if (alertStates) {
    for (alertState in alertStates) {
        UPSERT com.vantiq.engines.EngineLastAlert(alertState.value)
    }
}

The only differences when converting the Procedure from global to multi-partitioned are

• Updating the Procedure header to use the multi partition state access modifier
• Directly accessing the state object rather than using the getter Procedure.

The Procedure header now includes multi partition and directly accesses the state rather than using the getter Procedure. Multi-partitioned Procedures
have access to the Service’s Partitioned State and when scheduled, will run on all partitions.

Save the Service.

5. Confirm Outputs

In Part 1, the two Sources, com.vantiq.engines.SpeedSensor and com.vantiq.engines.TemperatureSensor were activated to begin the flow of simulated sensor events. The events produced by these two Sources should now be driving the Partitioned State version of the engine monitoring system.

Using the Project Contents tree on the left-hand side, open the com.vantiq.engines.EngineLastAlert Type pane and click Show All Records.
In the resulting pane select Auto Refresh, set the interval to 5 seconds, and click OK. Click Delete All to remove all previous records.

Wait about a minute for the Scheduled Procedure to execute and write the most recent Service states to the database.

Navigate to the Implement tab of the com.vantiq.engines.EngineMonitor Service and click lastAlertGet Procedure. This Procedure will fetch the state of your Service by engine id. Notice that now the state getter requires a partitionKey.

Click the blue Play button in the upper left-hand corner to execute the Procedure.
Set 0123456789 as the partition key and click Execute.

Expect the result to look similar to:

{
   "systemId": "0123456789",
   "lastAlert": "Your engine is overheating: check for a malfunctioning fan or a coolant leak."
   "temperature": 215,
   "speed": 40
}

Conclusion

Congratulations! You have completed the Stateful Services Tutorial.

In this Tutorial you have learned:

• How to update an App Component and the Event Handlers that use that Component
• How to store your application state in memory using a Service’s State properties.
• How to schedule a Procedure to execute at a regular interval
• How to use partitioned state to support a very large number of keys

As a parting message: always think twice about how you are using the database. Whenever possible, store your application’s state in memory using Stateful Services. The less reliant your system is on the database, the better it will perform and the easier it will be to scale into production.