Event Driven Thinking

1.1.                     Introduction 

Event-based thinking is unfamiliar to many business and technology leaders, even as their organizations must evolve into 21^st century digital enterprises. Without making this critical transition to event-driven systems, enterprises cannot meet the real-time needs of modern business operations.

This guide introduces the basics of event-driven thinking and how the Vantiq platform supports event-driven applications.

1.2.                     The Digital Enterprise and Events

1.2.1.     Enterprise Trends

Many business leaders and analysts have declared that the future is the real-time, digital enterprise. Customer engagements, partner and supplier interactions, and
internal operations are all required to be more responsive and more efficient: 

·        Customers expect online, real-time interactions when engaging with the enterprise:

o  Phone 

o  Chat 

o  Web  

·        Customers expect the enterprise to engage on their terms and on their devices. Let’s set up an appointment for next Wednesday doesn’t fly; let’s complete this transaction on my mobile phone while I have a couple of minutes is the new norm. 

·        Products must report status in real-time, triggering immediate responses to failures and proactive notifications to the customers so that they can optimize their use of the product. 

·        Partners deliver real-time business events tomanage their relationship with the enterprise. 

·        Data brokers provide additional context in real-time with which the enterprise can interpret real-time business events. Examples:

o   Continuous analysis of the company’s reputation on social media

o   Buying & market trends

·        Resources are dynamically scheduled to minimize response time and maximize productivity. 

·        Supply-chains are monitored in real-time so that anomalies can be dealt with immediately before they have an impact on downstream operations. 

·        Finance operates in real-time with visibility on the financial position of the company, updated continuously. 

There are a number of well-known examples of real-time operations in the e-commerceand logistics worlds:  

·        Amazon provides customers with on-demand buying experiences tied to real-time inventory management, real-time checkpoint tracking of purchases, and real-time delivery.  Returns are also convenient, with real-time refunds when the product is taken to local return centers.

·        Banking operations have evolved into near real-time systems as you can now get account status up to the last transaction for credit and debit cards, and make real-time transfers between accounts.

·        In the couture industry, customers can be scanned for measurements, then track the progress of their fashion choices in real-time all the way to their doorsteps.

·        The healthcare industry is undergoing especially rapid advancement to meet customer demand in real-time.  Just a few examples:

o   On-demand drone delivery of medicines

o   Real-time organ availability and transport tracking

o   Drugs designed and manufactured for individual patients

o   Real-time vital sign monitoring

o   Real-time blood glucose management

·        Automakers that used to have a handful of models, can now offer dozens as they fill more specialized niches and manufacturing technology makes it more cost effective to produce smaller, more personalized batches of products.

Key to all these customer-focused examples is speed, maintaining relevant context and monitoring detailed, real-time status of the enterprise. To accomplish these, business systems must adopt a real-time architecture, taking in status updates in real-time, and processing them in real-time.

1.2.2.  Enterprise Technology

In addition to business drivers, technology is also driving the evolution of the digital enterprise: 

·        IoT 

·        AI-powered user experiences such as ChatGPT, Siri, Alexa and Cortana 

·        Augmented reality and virtual reality 

IoT revolutionizes the digital enterprise by providing detailed information on the environment, products, and people. Physical products can be deeply instrumented to provide full visibility into their performance and behavior. Monitoring can be set up to understand the behavior of the artifacts in a space, including people, to drive deeper customer engagement. 

AI-powered user experiences can change the way people interact with the enterprise; improving the experience of customers, employees and partners. Interactions are real-time
and deeply engaging, also making the data more accurate, with less effort on the part of the user. 

AR and VR take this to the next level providing the user with augmented views of reality, optimized for their interactions with the enterprise. 

Another key to the digital enterprise is to realize that technology should be applied to access data automatically, avoiding error-prone and often incomplete manual data entry.  For instance,

·        Assembly lines automatically monitor items and report their status as they move through the manufacturing process, whether the product is a car or a sweater.

·        Basic customer information is supplied directly by the customer, who will be less likely to mistype the data.

·        Sensors and smart devices augment customer data.

·        Systems such as on-site sensors with facial recognition, or apps on mobile devices, automatically detect the presence of the customer, greet them, and begin providing them with immediate real-time information based on their interested behavior.

·        Product usage patterns, even after purchase, can be monitored with an eye toward making personalized suggestions to the consumer for how to maximize the utility of the product, maintain the product proactively, and for making complementary purchases to go with the product, whether that product is a physical one or an information service.

All these data acquisition technologies produce events and are best supported by event-drivensystems. 

The business drivers and the technology drivers for the digital enterprise all depend on re-imagining business systems as event-driven systems: 

·        Information is delivered to the system inreal-time.

·        Information must be processed in real-time.

·        Full integration of sources and systems across the extended enterprise is required. 

The key observation with respect to these characteristics is that information is DELIVERED TO the system, the information is not ACQUIRED BY the system. In these applications, the system cannot read stored data at intervals of its choosing; the system must ingest EVENTS produced by the outside world as they happen, and likewise respond to those events in real-time. This is the heart of why the digital enterprise must be event-driven. 

1.3.                     Event-Driven Migration

An enterprise does not need to wait until the entire organization commits to becoming a real-time, event-driven digital enterprise. Components that make up an event-driven system are loosely coupled, which makes integration with other systems easier.

A well-integrated event-driven system can orchestrate connections between event-driven applications and legacy systems, allowing businesses to move down thedigitalization path at their own pace.   

1.3.1.     Real-time 

Being able to respond in real-time is critical to building modern applications that support the digital enterprise. Users want an immediate response before they are distracted by some other demand on their time. Even in fully automated business operations, the system cannot afford to hang on to context, which might change, when a response is delayed.

1.3.2.     Loosely-coupled 

Event-driven applications can be considered functional capabilities tied together by events. Asynchronous business functions subscribe to the events they are responsible for processing, ingest them as they come in, perform their functions, then return a response, all in real-time.  Multiple business functions might be listening for the same events, processing them in different ways, all simultaneously, and independently of each other.

Maintaining an event-driven system is therefore much easier than with other types.  If a new function needs to process the same events, existing applications do not have to be upgraded; the new business function simply subscribes to the relevant events and ALL EXISTING APPLICATIONS are automatically extended to include the new business function. There is no way to overstate how powerful this model is and how much more responsive IT can be as the enterprise reaches critical mass in migrating to event-driven architectures. 

1.4.                     Event Focused Thinking

A key skill required for designing and developing event-driven systems is thinking of applications in terms of their events, in addition to thinking about business logic and data
requirements. 

Traditional systems are architected to store data in a database.

·        A system that knows something happened stores that fact in a database.  It will likely queue the write and do it when convenient. This means there is a delay before the relevant data is even available.

·        Another system that needs to know if something happened must poll the database looking for the event. If polling occurs often enough to find the events quickly, the application will be massively inefficient. Polling less frequently means responses are less timely and there is a longer delay before the event is processed. 

Also, traditionally, system design focuses on the business processes and the data that models the business. This approach is embodied in methods such as Business Process
Management (BPM), Object-Oriented Design and Domain Driven Design. 

Each of these tend to reinforce what one might call “database thinking”. In database-centric systems, the bulk of the systems that have been historically constructed, actions are
recorded in the database. Thus, in BPM each step records results in a database and the next step reads the results it needs and proceeds. In Object-Oriented design, the operations on the objects tend to read and write the database to retain object state. This results in developers thinking not so much about the flow of the system, but manipulating state for each activity. 

Event-based systems work differently. They are focused on the events that occur throughout the system and the responses the business must make to those events. In order to effectively develop event-driven systems, developers need to consider the event-driven nature of the system during the entire software development life cycle, from requirements to retirement. 

But first, what is an event?

An event is something of interest that has happened in the business. The key element of the definition is that the event is something that occurred in the past.

Most importantly, it is not something that needs to be done. For example, submitOrder is a command. It is an order to do something, and it tightly-couples the whole system to accomplish this one task, then do the next task.  It is NOT an event.

In contrast, orderSubmitted is an event that represents the fact an order was submitted. The orderSubmitted event is something that can elicit reaction(s) from a digital enterprise.

The design is no longer sensitive to how the order was placed since any mechanism that can properly produce an orderSubmitted event drives the correct reaction. 

The event focused design also extends to additional or multi-domain extensions by adding additional reactions. For example, orderSubmitted certainly drives the order process but, in a digital enterprise, may also trigger customer experience reactions, marketing reactions and finance reactions without the order management app knowing anything about these other domains or the reactions they implement. 

The various business functions can all act on the event independently, making the system more robust and more flexible. This makes parallel activities easier to handle – something
we would expect in streaming event-based systems – all reactions to an event can be executed in parallel; reactions that are independent can be executed in any order. 

1.5.                     Context

1.5.1.  Events and Services 

There is a major focus on services and microservices in contemporary software architecture. 

Services are a powerful mechanism for encapsulating functionality in a network consumable fashion. Microservices decrease service granularity into even smaller deployable units, allowing each to be upgraded independently, (at least if sufficient testing is available to guarantee there are no backward compatibility issues introduced in the new deployment). 

Events and services are complementary, as events identify critical things that happen in the enterprise and services specify the business logic required to react to those events. An
event-driven application binds the events and the services that implement the reactions. At runtime, as events are received, perhaps in a streaming manner, the application accepts the event, determines the relevant reactions, and invokes the proper services and microservices to react to the event. 

1.5.2.  Mediation for Flexibility and Scale

A common best practice in the use of services is to exploit API management. API management can be thought of as a mediation scheme for services. Those wishing to use a service
obtain access to the service from the mediation layer. If the service is a commercial service, part of the provisioning process is agreeing on commercial terms for the use of the service. At runtime, requests are routed to the service by the mediator. The mediation approach decouples the service requester from the service provider, allowing the mediation layer to provide security, monitoring, chargeback, and API independence by transforming requests into the current formats accepted by the service. 

A similar best practice will evolve for events by including an event management capability. Those wishing to receive an event contact the mediation layer which will provision the subscriber for access to the event. At runtime, events are routed by the mediator to the subscriber, decoupling the event publishers from the subscribers allowing value added services to be incorporated into the mediation layer and allowing events to evolve over time without impacting subscribers. 

1.5.3.  Events versus Processes

Some processes don’t seem to lend themselves well to event driven operations, such as those which must follow a particular order:

1.    Find the part 

2.    Obtain the part from inventory 

3.    Find suitable packing 

4.    Pack the part 

5.    Address the label 

6.    Attach label to package 

7.    Deliver package to shipper 

8.    Record the completion of the shipping business process

However, if the process has many variations or unknowns, an event-based approach might be better. For example, suppose the picker and transporter machine fails during step 2 (obtain
the part from inventory). Here is a great opportunity to apply event thinking:

·        Start with an automationFailed event that is picked up by Maintenance to start the repair process.

·        The event is also simultaneously picked up by Shipping so they can find an alternate way to obtain the part for packing and shipping

·        Perhaps a process starts that sends a human out to obtain the part or another machine is assigned the task

All these exceptional activities are asynchronous and, in some cases collaborative with other systems or human beings, which makes an event-based system a more natural approach than a process system. 

Bottom line: Only use processes where the process is well understood and mostly invariant. Use events where the order of activities may vary, or a number of unusual conditions and exceptions must be accommodated. 

1.6.                     Tactical and Strategic

Another thorny issue in deploying event-based systems, or any systems for that matter, is the tension
between:

·        tactical initiatives that can be implemented quickly and move the enterprise incrementally toward digitization

·        strategic initiatives that are longer-term, but have a greater impact on reaching the ultimate digitization goals

Vantiq offers a unique platform supporting real-time, event-driven applications that can seamlessly integrate tactical application initiatives with strategic initiatives giving the enterprise the best of both worlds. The platform features that make this integration possible include: 

·        Design Model Tool

·        Intelligent, Advanced Catalog Event Broker 

·        Services 

·        Visual Event Handlers, including Collaboration activity patterns

The following discussion outlines the tactical and strategic issues and the use of Vantiq to assemble strategic capabilities from tactical initiatives. 

1.6.1.  Strategic Digitization

To remain competitive in the 21^st century, enterprises must become highly integrated, real-time operations. This means that relevant data on all aspects, across all entities of the operation must be collected in real-time, analyzed in real-time to determine the current situation and, if the situation is not optimal, actions must be taken immediately to adjust the business to achieve the optimal state. 

Obtaining this level of integration is currently extremely difficult, if not impossible, because it requires a top-down design of the entire system. Much like enterprise architecture efforts, it is expensive, time-consuming and almost never completes, as the enterprise changes faster than the architecture can be captured. This discourages many enterprises from even considering strategic digitization initiatives. 

1.6.2.  Tactical Digitization

Given the difficulties in executing strategic digitization initiatives, many organizations opt exclusively for simpler tactical initiatives. These tactical initiatives are successful both because they satisfy a clear, immediate business requirement and can be implemented in a relatively short period of time and, usually, within a single department. 

The problem with tactical initiatives is that, even when aggregated over time, the fact that each tactical application is its own silo means the notion of a real-time, digitized enterprise can never be fully achieved. 

Conclusion

Enterprises that want to continue business operations into the 21^st Century must fully embrace real-time business operations, which means their systems need to be event-driven and
widely-integrated.  Vantiq is a platform architected to help solutions builders rapidly assemble and integrate fully event-driven applications and deploy them across a whole spectrum of platforms.