Objectives

Upon completion of this tutorial, a new developer will be able to:

• Easily navigate around the Vantiq IDE
• Create a simple real-world Vantiq application

Tutorial Overview

This tutorial uses a practical example from the Internet of Things (IoT): processing data from engine temperature and speed sensors to detect and diagnose an overheating engine. The basic steps to creating this application are:

• Create a new Project
• Use the Design Modeler to create the initial Project resources
• Create Sources to receive simulated sensor data
• Use the Service Builder’s visual event handler to specify how the application reacts to asynchronous, real-time engine conditions
• Create a visual front-end with a Client
• Run the application in simulation

Note: if needed, you can import a finished version of this project using the Projects button and select Import. Select Tutorials for Import Type, then select Introductory from the second drop-down, then click Import.

Namespace Privileges

In addition, when following the Tutorials, please make sure you are logged in as the administrator of the Vantiq namespace to which you are assigned. You can check your privilege level by clicking on your username in the title bar. If the bottom of the popup says, “Developer” or “Namespace Admin” at the bottom, you have the needed privileges. If it says, “User (Developer),” then you need to change to a Developer namespace. You may need to first create your own Developer namespace, as described here: Creating a Developer Namespace.

1: Creating an Engine Monitoring Project

The first task in building the engine monitoring system is to create a Project in the Vantiq IDE.

Note: You will be alerted if you do not have permission to create a project in the current namespace. You need to change to, or create, a namespace where you have Developer or Namespace Admin privilege (see above).

Use the Project menu to select New Project, which displays the New Project Wizard. Either create a new Namespace (recommended) or add a new project to the current Namespace, select Design Model as the Project type, and click Continue.

Set the project name to EngineMonitorProject and click Go to New Design Model Dialog.

Define the Design Model as:

• Model Name: EngineMonitor
• Package: com.vantiq.engines
• Description: Determine if there is an engine malfunction
• Design Template: IoT System
• Service Name: com.vantiq.engines.EngineMonitor
• Number of Streams: 2

The Design Template selected is IoT System since it most closely matches the requirements of the engine simulation: receiving data from IoT devices (the sensors), processing that data, and updating a Client that represents an engine dashboard.

The Number of Streams value is set to 2 since our system will be receiving data from two engine sensors: speed and temperature.

Click OK to confirm. After the Design Model generation is complete, the com.vantiq.engines.EngineMonitor Design Model pane is displayed:

2: Understanding the Design Model

The Design Modeler is a visual IDE tool for building Vantiq systems. It allows the user to easily connect Services and Clients, which are the building blocks of a Vantiq system. The Design Model generated for the engine monitor Project contains six nodes:

• Sensors: this is an External System Design Modeler node, which graphically represents the engine’s temperature and speed sensors. External System nodes are used for documentation purposes only. They are meant to represent how the Vantiq system being built receives or transmits data external to the Vantiq system.
• EngineMonitorSensors: this is a Service Design Modeler node, which represents an actual Vantiq Service. This Service has a limited purpose: to receive messages from the temperature and speeds sensors and to forward those events to the EngineMonitor Service for processing.
• EngineMonitor: this is another Service Design Modeler node which represents the Service that will use data from the sensors, determine if that data indicates possible problems with the engine, then forwards the sensor data and engine status to the EngineMonitor Client for display. In this tutorial, the Service Builder is used to modify both Services in this Design Model.
• EngineMonitor: this is a Client Design Modeler node which represents the Client that will display engine status: speed, temperature and any error status. In this tutorial, the Client Builder is used to modify the Client in this Design Model.
• DataWarehouse: this is another External System Design Modeler node. It could represent a database, either Vantiq’s native database or an external database system, to store data received or calculated by the engine system. Since this is a real-time engine status system, this node won’t be used and is deleted as this tutorial progresses.
• OutsideWorld: like the DataWarehouse node and for the same reasons, this node is deleted later on.

This Design Model represents a flow of events in an IoT system. Here is how the data and events for the engine monitoring system will inform us to modify the Services and Clients to develop a working system:

1. Sensor data is captured from two MQTT Sources. The speed and temperature data is simulated by a public MQTT server hosted by Vantiq.
2. The receipt of the Sensor data is handled by the EngineMonitorSensors Service described above. This Service doesn’t need to transform data for our engine monitor system but instead simply publishes it as Outbound Event Types to the EngineMonitor Service for processing. The IoT System Design Template uses two Services to separate the receipt of raw sensor data from the processing of the data as Event Types. This architecture allows the EngineMonitor Service to be reused in other Vantiq systems that process temperature and speed similarly.
3. The EngineMonitor Service receives the temperature and speed Event Types in two separate Event Handlers.a. The speed Event Handler (SpeedEvent) simply saves the sensor data in a Service’s speed Global State variable and then republishes that event to an Outbound Event Type for use by the EngineMonitor Client. The speed Global State variable is necessary so the temperature Event Handler may retrieve and use the speed data without having to store it in the Vantiq database.b. The temperature Event Handler (TemperatureEvent) reads the temperature data to determine if there is an overheating condition (if the temperature reading is 210F or greater). If the engine is overheating, the handler reads the speed Global State variable and constructs an alert message based on the combination of the temperature and speed sensor values. The handler then publishes an event that contains temperature and overheat status data to an Outbound Event Type for use by the EngineMonitor Client.
4. The EngineMonitor Client receives speed, temperature, and overheat status from the EngineMonitor Service’s two Outbound Event Types. These events are processed by two Client Data Streams, each of which is used to run Client widgets (gauges, line charts, etc.) to display the data in a dashboard-like display.

Please note this is a simple example of a working Vantiq system. It demonstrates basic event handling and filtering and works well for a system with a low rate of data receipt (the simulated speed and temperature MQTT messages are received at a rate of one per second). The EngineMonitor Client can easily process the events produced by the EngineMonitor Service, which are produced at no more than two per second.

The system demonstrates the basic architectural principles required to build scalable systems that process thousands of events each second. The core principle is to process the events in memory as they arrive, maintaining any state required for real-time processing in memory as Service Global State. The traditional approach of storing the events in a database when they arrive and then querying the database to process the events is too slow when processing thousands of events each second.

This simple example is not scalable to thousands or more events per second because it does not exploit the flow control and filtering features that the Event Handler system provides to filter or combine data and reduce the overall number of events processed by the system and delivered to Clients.

Finally, this system also demonstrates a Client that does not have to retrieve data from a database and process it in order to display useful system status. It should be a goal of any scalable system to limit data transformation and modification operations to Service Event Handlers rather than delegating that work to Clients.

3: Create the Engine Sensor Sources

Two MQTT Sources must be created to simulate the speed and temperature data from a public MQTT server hosted by Vantiq.

Use the Add menu to select Source…:

Use the New Source button to display the New Source pane. Enter the following values to configure the source:

1. Source Name: SpeedSensor
2. Package: com.vantiq.engines
3. Source Type: MQTT

Add a Server URI from the Server URI tab: tcp://public.vantiq.com:1883

Add a Topic from the Topic tab: com.vantiq.mqtt.enginespeed

Save the Source then click the Test Data Receipt button from the pane’s toolbar. The Source pane will switch to the Subscription pane to display data received from the MQTT server:

Note the simulated data is sent approximately once per second and contains a JSON object with two properties: systemId and speed.

Follow similar steps to create a second Source to receive simulated temperature events. Enter the following values to configure the source:

1. Source Name: TemperatureSensor
2. Package: com.vantiq.engines
3. Source Type: MQTT

Add a Server URI from the Server URI tab: tcp://public.vantiq.com:1883

Add a Topic from the Topic tab: com.vantiq.mqtt.enginetemp

Save the Source.

4: Renaming the Design Model’s Events

In Lesson 2 above, a four-step description described the flow of data and events for the engine monitoring system to be built. This lesson shows how to rename the Service Event Types and Client Data Streams to more accurately reflect that flow of data and events.

1. Right-click the EngineMonitorSensors Service in the Design Model, then select Edit in Service Builder menu item. The pane for that Service is displayed. Expand the Outbound section under the Interface tab, then select Sensor0, as shown below:Change the Name of the Outbound Event to SpeedEvent. Save that change using the Save Changes button in the pane’s title bar.
2. Select Sensor1 from the same Outbound section. Change the Name of the Outbound Event to TemperatureEvent and again save that change.
3. Right-click the EngineMonitor Service (Service nodes have a green title bar) in the Design Model, then select Edit in Service Builder menu item. The pane for that Service is displayed. Expand the Inbound section under the Interface tab, then select SensorReading0. Change the Name of the Inbound Event to SpeedEvent and again save that change.
4. Select SensorReading1 from the same Inbound section. Change the Name of the Inbound Event to TemperatureEvent and again save the change.
5. Expand the Outbound section under the Interface tab, then select AnomalyDetected0. Change the Name of the Outbound Event to EngineStatus and again save that change.
6. Select AnomalyDetected1 from the same Outbound section. Change the Name of the Outbound Event to EngineSpeed and again save that change.
7. Right-click the EngineMonitor Client (Client nodes have a purple title bar) in the Design Model, then select Edit in Client Builder menu item. The pane for that Client is displayed. Click the Edit tab, expand the Data Streams tree item, as shown below:Click the AnomalyDetected0 Data Stream to display the Edit Data Stream dialog. Make the following changes in that dialog:a. Data Stream Name: EngineStatusb. Service: com.vantiq.engines.EngineMonitor

   c. Event: EngineStatus

   Click Save to save the Data Stream.

8. Click the AnomalyDetected1 Data Stream to display the Edit Data Stream dialog. Make the following changes in that dialog:a. Data Stream Name: EngineSpeedb. Service: com.vantiq.engines.EngineMonitorc. Event: EngineSpeedClick Save to save the Data Stream, then save the Client using the Save Changes icon in the pane’s title bar.

Note after these changes, the Service and Client node in the Design Model pane changes to reflect the new Event Type and Data Stream names. Use the Save Changes button in the Design Model pane’s title bar to save those changes.

5: Remove Unneeded External Systems

As explained in Lesson 2 above, this system doesn’t require storing any event data in an External System. This means we can delete the DataWarehouse and OutsideWorld Design Model nodes. Click each of those nodes then use the Delete toolbar button to delete them from the Design Model, then save the Design Model. Even though those two nodes are used for documentation purposes only, deleting them more accurately reflects the system being developed. The Design Model pane should now look like this:

6: Address the To Do List

Notice that in the screenshot from the previous lesson, there is a To Do List in the lower, left corner of the Design Model pane. The To Do List contains errors found in the two Services contained in the Design Model. At this point in the system development process, these errors are normal and are all associated with Inbound Event tasks in those Services that need to be configured for the engine system.

Clicking on a To Do List item in a Design Model causes the Service pane that contains the error to be displayed. Click on the first error in the list which will cause the com.vantiq.engines.EngineMonitor Service to display and automatically navigate to the Implement tab which contains the SpeedEvent Inbound Event Handler that is associated with the error. The Service pane should look like this:

This is the Event Handler which processes speed sensor data from the EngineMonitorSensors Service. Recall from Lesson 2 that the SpeedEvent Inbound Event Handler saves the sensor data in a Service’s speed Global State variable then republishes that event to an Outbound Event Type for use by the EngineMonitor Client. This involves the following changes to the Visual Event Handler shown in the screenshot above (it contains the SensorReading task (the orange rectangle contained in the Ingest task context):

1. From the Modifiers section of the Event Handler palette, drag and drop an AccumulateState task on the SensorReading task. Then click on the AccumulateState task and rename it SaveSpeed.Click the Click to Edit link to edit the configuration for this task. Click on the link next to the procedure property. This allows you to write the code snippet for updating your Service’s state based on the incoming events. Select VAIL Block in the Edit union parameter dialog then replace the placeholder VAIL code with the following:var curSpeed = event.speed
state = curSpeedClick OK to return to the task configuration parameters. Enter speed as the stateProperty property. Click OK to complete editing the SaveSpeed task. The SaveSpeed task has now been configured to update the speed Global State variable every time a sensor reading is received by the SensorReading task. When the EngineMonitor Service gets saved, the SaveSpeed task will cause three Procedures to be generated: com.vantiq.engines.EngineMonitor.speedGet(), com.vantiq.engines.EngineMonitor.speedReset(), and com.vantiq.engines.EngineMonitor.speedUpdate(). These may be used to retrieve, reset and update the speed Global State variable.
2. Right-click the SaveSpeed task, select Link Existing Task to display a dialog. In the dialog, select React as the Task Name and event as the Downstream Event. Click OK. This will link the SaveSpeed task to the React task, which was the goal outlined in Lesson 2: save the speed sensor value then republish that data via the React task.
3. Click the React task and rename it to PublishSpeed. Click the Click to Edit link to edit the configuration for this task. The service property is correctly set (com.vantiq.engines.EngineMonitor). However, the eventTypeName property needs to reflect the renamed Outbound Event: replace AnomalyDetected0 with EngineSpeed. Click OK. The Event Handler graph should now look like this: