Purpose

To familiarize developers with the powerful features of the Vantiq Catalog.

Objectives

By the end of this tutorial, the developer should be able to:

• Access and configure a Catalog
• Link subscribers and publishers to the Catalog
• Set up access tokens to the Catalog for maximum security
• Subscribe to events from the Vantiq IDE
• Create and publish simulated events with VAIL Rules and Procedures
• Be able to verify event receipt to subscribers
• Create and publish a Service to the Catalog
• Subscribe to a Catalog Service and execute Service Procedures in an Application

Tutorial Overview

In this tutorial, we will create a simple application that publishes “overheat” events when a simulated machine sensor reports a temperature that is too high. Our Catalog set-up will allow such events and related “reaction” events to communicate across different Namespaces.

The basic steps involved to accomplish this are:

• Create separate Namespaces for the Catalog, publisher, and subscriber
• Register the publisher and subscriber to the Catalog
• Define Event Types
• Define a Catalog Service
• Receive events
• Generate and publish events
• Execute Procedures from a Catalog Service

Part 1: Setup Catalog Infrastructure

Step 1: Create Catalog Namespaces

This tutorial requires three Namespaces, which means you should first familiarize yourself with creating and managing Namespaces in the User and Namespace Administration Tutorial.

The first step is to sign into a Namespace in which you have developer or higher privileges. From there, you will create Namespaces to act as each of the three Catalog roles: Catalog, Subscriber and Publisher.

Use the Administer button to select Namespaces, then use the New button in the Namespaces list to create three Namespaces with the following names: catalog, publisher and subscriber. You should be the administrator of each of these Namespaces.

Once you’ve created the three Namespaces, switch into the catalog Namespace using the Current Namespace button at the top right of the IDE Navigation Bar and selecting catalog from the list of available Namespaces.

Step 2: Configure Catalog

Now that all of the Namespaces have been created and you’re in the catalog Namespace, it’s time to configure each Namespace for its respective role. The catalog Namespace is currently just a standard developer Namespace. To allow this Namespace to host a Catalog of events and Services, use the Administer button to select Advanced>Catalog which displays the Manage Catalog pane. Click the Create Catalog button and confirm that you want to create a Catalog.

This Namespace is now ready to host events and Services in the Catalog. Other Namespaces that connect to the catalog Namespace can read the Catalog of events and Services and register as publishers and subscribers of events and Services.

Of course, we’ll be publishing important information through this catalog. We don’t want just anyone to be able to send out messages. To avoid this, we can change the default permissions for connected namespaces.

In the Manage Catalog pane, find the catalog namespace (it should be the only one present right now) and click on Edit Permissions. Toggle off the box for publish under Event Types then click OK. Now no members of the catalog will be allowed to publish to Event Types unless we explicitly let them.

Step 3: Connect Publisher and Subscriber to Catalog

The next step is to connect the publisher and subscriber Namespaces to the catalog Namespace.

To connect the publisher and subscriber Namespaces to the catalog Namespace, we need to get credentials that can be used by those Namespaces to communicate with the catalog Namespace. To that end, we create an access token by clicking the Create Token button in the Manage Catalog pane. Name the access token “CatalogToken” then click Save. This creates a new token and copies it to the browser clipboard. Save this token string somewhere readily accessible as we’ll need it to configure the subscriber and publisher Namespace Catalog connections.

Switch into the publisher Namespace using the Current Namespace button at the top right of the IDE Navigation Bar and selecting publisher from the list of available Namespaces. You will see the page reload and you will now be in the publisher Namespace. Verify this by checking that the Namespace name in the Navigation Bar has changed from catalog to publisher.

Once in the publisher Namespace, we must connect to the catalog Namespace using the access token we just created. Use the Show button to select Catalogs to display the Catalogs pane. Use the New button to display the Connect to Catalog dialog. Paste the token string into the New Catalog Access Token field, verify the URL in the URL to Connect to Catalog Namespace field, and click the Connect button.

Switch to the subscriber Namespace and repeat the process of connecting to the catalog. Afterwards, both the publisher and subscriber Namespaces are now able to view the Catalog of Event Types and Services hosted by the Catalog Namespace. To view the Catalog of Event Types and Services from any connected Namespace, use the Show button to select Catalogs to display the Catalogs pane. Click on any listed Catalog Namespace to display its list of known Event Types and Services.

Both the publisher and the subscriber namespace need to perform publishes as part of this tutorial. Because we changed the default catalog permissions to disallow publishing, we must now override the default permissions for both namespaces. To do this, return to the catalog namespace and open the Manage Catalog pane. For the publisher and subscriber namespaces in the list, click Edit Permissions. Select “Event Types” and “publish” in the dropdowns, then click Add Override. Make sure the box is checked, then click Save.

Part 2: Creating Events

Step 1: Create Event Schema

To get started on creating the machine temperature and overheat Event Types, switch back to the catalog Namespace.

Before we can create the Event Types, we first need to define a schema type which defines the schema for the machine temperature and overheat events. Use the Add button to select Type and create a new type. In the resulting dialog, name the type MachineData, change the role to schema then click OK. From the Type: MachineData pane, assign the type the following properties, making sure to check them all as required:

• machineId (String) – A unique identifier that associates a reading with a specific machine.
• temperature (Integer) – The sensor reading in degrees fahrenheit.
• timestamp (DateTime) – The time at which the sensor reading was recorded.

Step 2: Define Event Types

Now we finally get to add events to the Catalog. While still in the catalog Namespace, open the Catalogs pane by using the Show button to select Catalogs. Select the catalog Catalog Namespace from the list, then use the New button to select Create New Event Type. The Event Type pane is displayed in which we create our first Event Type. Configure this Event Type with the following values:

• Event Name: /machine/temp
• Description: A new machine temperature sensor reading.
• Schema: Select MachineData from the droplist

Note that the Event Type name is a Topic. Topics must contain a leading slash but often contain additional slashes, which is a good practice for organizing events in a hierarchical manner. The prefix /machine indicates this is an event on a machine and the suffix /temp indicates this is a temperature event. In the future, we will add another event with the same /machine event prefix. Giving events common prefixes makes it easier to find them in the Catalog and helps users looking at the Catalog get a sense for how different events are related.

Once you’ve completed the new Event Type pane, click the Save Changes icon and you’ll see a list of suggested keywords found by parsing the specified description. Leave machine and temperature, delete the rest, then click OK. This will add some keyword tags to the newly created Event Type, which offers users a different way to query for interesting Event Types.

Follow the same process to create an Event Type for the overheating event. Use the following configuration:

• Event Name: /machine/overheat
• Description: The temperature of a machine has risen above the normal operating range.
• Schema: Select MachineData from the droplist

Save the Event Type, then uncheck any unhelpful suggested keywords. (In this case, everything except machine and temperature.) Click OK. You should now see two Event Types listed in the Catalog pane.

Step 3: Register a Publisher

Now that we have the necessary events in the Catalog, it’s time to switch to the publisher Namespace and start producing temperature events. Create a project by entering a name “MachineData” to the right of the Vantiq logo in the Navigation Bar then clicking the green Save button. After each step, save the project by clicking the green Save button again.

Next, open the Catalog by using the Show button to select Catalogs. Select the catalog Catalog Namespace from the list, then click on the /machine/temp event in the list. In the resulting Event Type pane, click on Click to View next to Publishers. In the Publishers of /machine/temp dialog, specify a local Topic, /my/machine/temp/event, for the /machine/temp event. Click + Become Publisher.

The local name of an Event Type is the Topic on which the events will occur in this Namespace. For this tutorial, we set the local name to /my/machine/temp/event. This means that when an event occurs on /my/machine/temp/event, it will be forwarded to all of the subscribers of /machine/temp.

Step 4: Publish Events

We now need to generate events in this publisher Namespace that will be published to /my/machine/temp/event so the subscriber Namespace will have some data to consume. To generate random data, we create a rule that runs every five seconds and publishes the data to /my/machine/temp/event. To create a rule, use the Add button to select Rule then click the New Rule button.

Create a rule with the following text:

RULE tempEventTrigger
WHEN PUBLISH OCCURS ON "/random"
// Publish one random temperature event each for 3 machineIds
PUBLISH {temperature: random(0, 100), machineId: "Machine1", timestamp: now()} TO TOPIC "/my/machine/temp/event"
PUBLISH {temperature: random(0, 100), machineId: "Machine2", timestamp: now()} TO TOPIC "/my/machine/temp/event"
PUBLISH {temperature: random(0, 100), machineId: "Machine3", timestamp: now()} TO TOPIC "/my/machine/temp/event"

This rule will run every time an event publishes to the /random Topic. Next we need to create a Scheduled Event that will publish to /random once every five seconds. To create the Scheduled Event, use the Add button to select Advanced>Scheduled Event then click the New Scheduled Event button.

Create a new Scheduled Event with the following properties:

   Name: "RandomTempGenerator"
   Topic: /random
   Periodic?: True
   Active?: True
   Scheduling Type: Periodically, starting now
   Interval: 5 seconds

If you want to verify that the events are being published as you expect, use the Show button to select Resource Graph then find and click on the Topic /my/machine/temp/event. Click the Test Data Receipt button in the top left of the resulting pane. This displays a subscription pane that streams all of the events in real time that occur on that Topic, all of which are forwarded to all subscribers of the Event Type /machine/temp. Wait five seconds and you should see an event like this:

Step 5: Register a Subscriber

Now that the publisher is generating /machine/temp events, it’s time to switch to the subscriber Namespace and register a subscriber. In the subscriber Namespace, give the project a name such as “OverheatDetector” and click the Save button in the Navigation Bar.

Once you’ve created the project, open the Catalog by using the Show button to select Catalogs. Select the catalog Catalog Namespace from the list, then right-click on the /machine/temp entry in the list. Select Subscribe to confirm that you want to subscribe to /machine/temp events.

At this point, all events published to /my/machine/temp/event in the publisher Namespace should be forwarded to the subscriber Namespace on the Topic /machine/temp. You may verify this by clicking the Test Data Receipt button from the /machine/temp Topic pane.

Part 3: Building Apps

Step 1: Trigger App on /machine/temp Events

While still in the subscriber Namespace, create a new Vantiq App by using the Add button to select App then clicking the New App button. This App will process the /machine/temp events to identify when a machine overheats. When the App identifies a machine that is overheating, it will produce a /machine/overheat event and publish that event to the Catalog. We define overheating to mean the temperature has risen above 100 degrees. Below 100 degrees, the machine is deemed to be operating within the normal operating temperature range.

With the new App pane open, name the App OverheatDetector by using the Properties button. Every App starts with an empty yellow box representing an initial stream of events. Click on the yellow box and name the task TempStream. Next, click Click to Edit next to Configuration label to edit the TempStream configuration. The inboundResource for the event stream is topics, and the inboundResourceId (/machine/temp) can be selected from the dropdown next to the input field:

Note that once you select /machine/temp from the droplist, the schema property at the bottom of the configuration dialog is automatically updated to show MachineData. This is because the /machine/temp event in the Catalog has an associated schema type so subscribing to the event through the Catalog added a copy of the schema type to this Namespace.

It’s also possible to subscribe to events in the Catalog directly from the App Builder configuration dialog. Click the plus (+) button next to the dropdown for the inboundResourceId property and you will see a dialog where you could subscribe to any event from the Catalog. This would have the same effect as subscribing from the Catalogs pane displayed by using the Show button and selecting Catalogs.

To verify the event stream is working, save the App, right click on the TempStream task and click View Task Events. This opens a subscription pane which shows events as they arrive.

Step 2: Split by MachineId

The random temperature readings published to /machine/temp from the publisher Namespace each contain a machineId. We want all downstream processing to be separated for each machine to ensure that temperature readings from one machine are not matched against readings from a different machine. To do this, we must add a SplitByGroup task below the TempStream task. Drag and drop SplitByGroup from the Activity Pattern palette on the left side of the App Builder (found in the Flow Control palette section) on top of the TempStream task. Rename the task SplitByMachineId by clicking on the newly added task. Then click Click to Edit next to Configuration label to edit the SplitByMachineId configuration.

The property we want to split by is machineId. Click on the downward arrow next to the groupBy parameter and select event.machineId from the property menu. Click OK to close the configuration dialog and then save the App again.

Step 3: Detect Machine Overheating

The random temperature readings published to /machine/temp from the publisher Namespace range from 0 to 100, so we’ll say the machine is overheating whenever the temperature rises above 100 degrees. To detect when the temperature crosses from under 100 to over 100, we must add a threshold task below the SplitByMachineId task. Drag and drop Threshold from the Activity Pattern palette on the left side of the App Builder (found in the Filters palette section) on top of the SplitByMachineId task. Rename the task DetectOverheat by clicking on the newly added task. Then click Click to Edit next to Configuration label to edit the DetectOverheat configuration.

The condition we want to check is event.temperature>100, so enter that in the condition input field. Notice that the droplist to the left includes all of the properties from the MachineData schema type because the TempStream task above is known to have that schema type.

The direction property should be set to true because we only care when the condition changes from false to true, meaning the temperature was under 100 degrees and is now over 100 degrees. Click OK to close the configuration dialog and then save the App again.

Step 4: Test Threshold Crossing

You’ll notice while the OverheatDetector App is running that the threshold task is never triggered. That’s because the data generated in the publisher Namespace is always below the threshold. To check that the threshold task is properly configured, we can simulate an overheating event in this Namespace by publishing an event to the /machine/temp Topic with a temperature outside the normal range. This published event won’t go through the Catalog, but it will behave as if it came from the Catalog within the subscriber Namespace. To publish this event, click on the /machine/temp Topic in the Project Contents list (found at the left of the IDE) to open the Topic detail pane. Edit the Publish Message field to use Machine1 as the machineId and set the temperature to any number greater than 100:

Click the Publish button to publish the event. After you publish the event, you should see the counter next to the DetectOverheat task display the number 1. Change the machineId value in the Topic detail pane to Machine2 and publish the event again and you’ll see the counter increment to 2.

One important note is that after a threshold is crossed, no other event will be produced by the threshold task until the boundary is crossed again. If the generator in the publisher Namespace is producing data below the threshold and the threshold is crossed by a manual publish in this Namespace, publishing a second event with a temperature over 100 will not produce more output from the threshold until the generator generates another temperature below 100.

Step 5: Publish Overheat Events

Now that we can detect when a machine is overheating, the last step is to publish the overheat events back to the Catalog.To do this, we must add a PublishToTopic task below the DetectOverheat task. Drag and drop PublishToTopic from the Activity Pattern palette on the left side of the App Builder (found in the Actions palette section) on top of the DetectOverheat task.

Rename the task PublishToCatalog by clicking on the newly added task. Then click Click to Edit next to Configuration label to edit the PublishToCatalog configuration.

In the resulting dialog, type /local/machine/overheat as the topic. Select the name of the catalog from the catalog droplist. Finally, select /machine/overheat as the event and click OK. Now, all events that flow through this task will be published to the topic /local/machine/overheat which will be registered as a publisher for the /machine/overheat in the Catalog. All /machine/overheat subscribers will receive the event.

Now, save the App and you will see an extra node appear in the graph below the PublishToCatalog task marking the task as a catalog publisher for the /machine/overheat event type.

Step 6: Create Overheat Type

In addition to using the Catalog for events, the Catalog may also be used to host Vantiq Services. Services are containers for Procedures which can be used to organize a collection of related Procedures into a common package. In the next few steps, we will create and host a Service, machineTemperatureService. However, before we can create machineTemperatureService, we first need to define a data type which represents an instance of a machine overheat event.

Use the Add button to select Type and create a new type. In the resulting dialog, name the type OverheatedMachine and assign it the following properties:

• machineId (String) – A unique identifier that associates a reading with a specific machine.
• temperature (Integer) – The sensor reading in degrees fahrenheit.
• timestamp (DateTime) – The time at which the sensor reading was recorded.
• hasBeenMaintenanced (Boolean) – If the machine has been subject to maintenance since it last malfunctioned

Step 7: Save to Overheat Type

When a machine overheats, we want to document the occurrence for historical purposes by inserting an instance of the OverheatedMachine type. Drag and drop SaveToType from the Activity Pattern palette on the left side of the App Builder (found in the Actions palette section) on top of the DetectOverheat task.

Click on the SaveToType Task and name the task RecordOverheat. Next, click Click to Edit next to Configuration label to edit the RecordOverheat configuration. Select OverheatedMachine as the type. Click OK and then save the App.

Step 8: Create the machineTemperatureService Service

Next, you will create a Service which will be published to the Catalog. This Service will be used by subscribers of the /machine/overheat event to determine whether or not the overheating machine requires maintenance.

Use the Add button to select Services and click + New to create a new Service. Create a new Service named machineTemperatureService and click OK.

In the Service pane add the following description: Service to keep track of machine status.

Navigate to the Interface tab in the Service pane and click the Add button next to Procedures to add a new Procedure interface to the service.

Define a new Procedure Signature with the following properties:

• Name: doesMachineNeedMaint
• Description: Returns true if a machine has overheated three times this week
• Return Type: Boolean

Click + New Parameter then on the pencil icon next to it, and add a parameter with the following properties:

• Name: machineDef
• Description: machine descriptor
• Type: Type MachineData

To generate the Procedure stub, go to the Implement tab and click on the procedure. Save the Service.

Next, we need to fill in the machineTemperatureService.doesMachineNeedMaint Procedure. Add the following Procedure text that returns true if the machine has overheated three times in the past week.

//Returns true if a machine has overheated three times this week
PROCEDURE machineTemperatureService.doesMachineNeedMaint(machineDef MachineData):Boolean
var oneWeekAgo = now().minusMillis(1 week)
var overheatedMachines = SELECT * FROM OverheatedMachine WHERE timestamp > oneWeekAgo  and machineId == machineDef.machineId and hasBeenMaintenanced == null
return overheatedMachines.size() > 3

Save the Procedure. Return to the Interface tab, click General, then click the Publish button. After publishing the Service to the Catalog, the Service is now available to all Namespaces that have access to the Catalog.

Step 9: Subscribe to Overheat Events

Now we will subscribe to /machine/overheat events in the publisher Namespace. First, switch into the publisher Namespace.

We will create an App which reacts to /machine/overheat events and detects whether or not the overheating machine requires maintenance. If maintenance is required, the App will trigger another App with collaboration tasks.

Use the Add button to select App then click the New App button. Name the App OverheatApp.

Click on the yellow EventStream box. Name the EventStream overheatEventStream. Click Click to Edit to configure the EventStream. Select topics as the inboundResource. Click on the “+” button to the left of the inboundResourceId box. This will allow you to subscribe directly to the /machine/overheat/ event from the Catalog.

Select /machine/overheat/, then subscribe, and then /machine/overheat again.

Notice that the MachineData type was automatically set as the schema for this EventStream.

Click OK to close the dialog.

Drag and drop Procedure from the Activity Pattern palette on the left side of the App Builder (found in the Actions palette section) on top of the overheatEventStream task. Rename the task doesMachineNeedMaintenance by clicking on the newly added task then clicking Click to Edit next to Configuration label to edit the doesMachineNeedMaintenance configuration.

Click on the Search icon next to the Procedure droplist to subscribe to a Service in the Catalog.

Select machineTemperatureService as the Service. Click subscribe.

Click OK to subscribe to the Service. When the dialog closes, select machineTemperatureService.doesMachineNeedMaint from the Procedure droplist and click OK.

This App is now using the Procedure defined in the Service that was published to the Catalog by the subscriber Namespace. By subscribing to the Service through the Catalog, this Namespace invokes the Procedure as if it were defined locally.

Next, add a Filter task below the doesMachineNeedMaintenance task, by dragging and dropping Filter on top of the doesMachineNeedMaintenance task.

Click on the Filter task, name this task machineNeedsMaintenance. Then click Click to Edit to configure the task. Set the condition to event == true. The Procedure we called above will return a Boolean so we only want to continue downstream when the Procedure returns true.

Click OK.

Finally, drag a PublishToTopic task onto the machineNeedsMaintenance filter Task.

Click on the new PublishToTopic Task and name the task maintenanceCollab. Then click Click to Edit to configure the task. Set the topic to /start/maintenanceCollab and click OK to close the pop-up.

In a fully complete system, this PublishToTopic would trigger another App which would likely find and request a repair person to fix the overheating machine.
For brevity, this step has been removed from this tutorial. For more information on Collaborations, see the Collaboration Tutorial.

Click the Save Changes icon to save the App.

Step 10: Catalog Round Trip

The last step in this tutorial is to test the complete round trip by:

• Publishing a temperature event from the publisher Namespace with a temperature over 100
• Identifying the overheating machine in the subscriber Namespace
• Publishing the overheat event from the subscriber Namespace
• Verifying the overheat event shows back up in the publisher Namespace
• Calling a Service Procedure in the publisher Namespace which was defined in the subscriber Namespace
• Detecting whether or not the machine requires maintenance

We already have a rule running in the publisher Namespace that produces “good” temperature readings, so all we need to do to trigger an overheat event is publish a single “bad” temperature reading.

We will do this in a Procedure so it’s easily repeatable. Use the Add button to select Procedure then click New Procedure. Define the Procedure with the following VAIL code:

PROCEDURE publishBadTempEvent()
var machineId = "Machine1"
var badTemp = random(101,200)
PUBLISH {machineId: machineId, temperature: badTemp, timestamp: now()} TO TOPIC "/my/machine/temp/event"

Save the Procedure and execute it. After verifying that there are no parameters provided, the execution will return the “true” condition:

You should also see an event show up in the OverheatDetector App. The event will stop at the doesMachineNeedMaintenance task because the machine has not overheated more than three times this week.

Execute the Procedure three more times. On the fourth time, the event will travel all the way down the App and trigger the next App with its collaboration.

Conclusion

At the completion of this tutorial, a developer should be familiar with:

• The relationship between Catalogs, Event Subscribers and Event Producers
• The concept that Subscribers can also be producers and producers can also be subscribers
• How to register Namespaces with Catalogs
• Proper Topic nomenclature
• Populating Catalogs with events
• Subscribing to events from other Namespaces
• Ways to publish simulated events
• Creating a Service
• Publishing a Service to the Catalog
• Subscribing to a Service and using Service Procedures in a subscriber Namespace

For more details on the Vantiq Catalog, please refer to the Catalog documentation.