Introduction
Event-based thinking is unfamiliar to many business and technology leaders, even as their organizations must evolve into 21st-century digital enterprises. Without making this critical transition to event-driven systems, enterprises cannot meet the real-time needs of modern business operations.
This guide introduces the basics of event-driven thinking and how the Vantiq platform supports event-driven applications.
The Digital Enterprise and Events
Enterprise Trends
Many business leaders and analysts have declared that the future is the real-time, digital enterprise. Customer engagements, partner and supplier interactions, and internal operations are all required to be more responsive and more efficient:
- Customers expect online, real-time interactions when engaging with the enterprise — by phone, chat and web.
- Customers expect the enterprise to engage on their terms and on their devices. “Let’s set up an appointment for next Wednesday” doesn’t fly; “let’s complete this transaction on my mobile phone while I have a couple of minutes” is the new norm.
- Products must report status in real time, triggering immediate responses to failures and proactive notifications to customers so they can optimize their use of the product.
- Partners deliver real-time business events to manage their relationship with the enterprise.
- Data brokers provide additional context in real time with which the enterprise can interpret real-time business events — for example, continuous analysis of the company’s reputation on social media, or buying and market trends.
- Resources are dynamically scheduled to minimize response time and maximize productivity.
- Supply chains are monitored in real time so anomalies can be dealt with immediately, before they impact downstream operations.
- Finance operates in real time, with continuously updated visibility into the company’s financial position.
There are a number of well-known examples of real-time operations in the e-commerce and logistics worlds:
- Amazon provides on-demand buying tied to real-time inventory management, real-time checkpoint tracking of purchases, and real-time delivery. Returns are also convenient, with real-time refunds when the product is taken to local return centers.
- Banking operations have evolved into near real-time systems: you can now see account status up to the last credit or debit card transaction and make real-time transfers between accounts.
- Couture customers can be scanned for measurements, then track the progress of their fashion choices in real time all the way to their doorsteps.
- Healthcare is advancing especially rapidly to meet customer demand in real time:
- On-demand drone delivery of medicines
- Real-time organ availability and transport tracking
- Drugs designed and manufactured for individual patients
- Real-time vital sign monitoring
- Real-time blood glucose management
- Automakers that used to have a handful of models can now offer dozens, as they fill more specialized niches and manufacturing technology makes smaller, more personalized batches cost effective.
Key to all these customer-focused examples is speed, maintaining relevant context, and monitoring detailed, real-time status of the enterprise. To accomplish this, business systems must adopt a real-time architecture — taking in status updates in real time and processing them in real time.
Enterprise Technology
In addition to business drivers, technology is also driving the evolution of the digital enterprise:
- IoT
- AI-powered user experiences such as ChatGPT, Siri, Alexa and Cortana
- Augmented reality (AR) and virtual reality (VR)
IoT revolutionizes the digital enterprise by providing detailed information on the environment, products and people. Physical products can be deeply instrumented to provide full visibility into their performance and behavior. Monitoring can be set up to understand the behavior of the artifacts in a space, including people, to drive deeper customer engagement.
AI-powered user experiences can change the way people interact with the enterprise, improving the experience of customers, employees and partners. Interactions are real-time and deeply engaging, which also makes the data more accurate with less effort from the user.
AR and VR take this to the next level, providing users with augmented views of reality optimized for their interactions with the enterprise.
Another key to the digital enterprise is applying technology to access data automatically, avoiding error-prone and often incomplete manual data entry. For instance:
- Assembly lines automatically monitor items and report their status as they move through manufacturing, whether the product is a car or a sweater.
- Basic customer information is supplied directly by the customer, who is less likely to mistype it.
- Sensors and smart devices augment customer data.
- On-site sensors with facial recognition, or apps on mobile devices, automatically detect the customer’s presence, greet them, and begin providing immediate, real-time information based on their interests.
- Product usage patterns, even after purchase, can be monitored to make personalized suggestions — how to get the most from the product, maintain it proactively, or make complementary purchases — whether the product is physical or an information service.
All these data acquisition technologies produce events and are best supported by event-driven systems.
The business and technology drivers for the digital enterprise all depend on re-imagining business systems as event-driven systems:
- Information is delivered to the system in real time.
- Information must be processed in real time.
- Full integration of sources and systems across the extended enterprise is required.
Key observation: information is delivered to the system, not acquired by it. The system cannot read stored data at intervals of its choosing; it must ingest events produced by the outside world as they happen, and respond to them in real time. This is the heart of why the digital enterprise must be event-driven.
Event-Driven Migration
An enterprise does not need to wait until the entire organization commits to becoming a real-time, event-driven digital enterprise. Components of an event-driven system are loosely coupled, which makes integration with other systems easier.
A well-integrated event-driven system can orchestrate connections between event-driven applications and legacy systems, allowing businesses to move down the digitalization path at their own pace.
Real-time
Being able to respond in real time is critical to building modern applications that support the digital enterprise. Users want an immediate response before they are distracted by some other demand on their time. Even in fully automated business operations, the system cannot afford to hold on to context — which might change — when a response is delayed.
Loosely-coupled
Event-driven applications can be considered functional capabilities tied together by events. Asynchronous business functions subscribe to the events they are responsible for processing, ingest them as they come in, perform their functions, then return a response — all in real time. Multiple business functions might listen for the same events and process them in different ways, simultaneously and independently of each other.
Maintaining an event-driven system is therefore much easier than with other types. If a new function needs to process the same events, existing applications do not have to be upgraded; the new business function simply subscribes to the relevant events, and all existing applications are automatically extended to include it. It is hard to overstate how powerful this model is, and how much more responsive IT can be as the enterprise reaches critical mass in migrating to event-driven architectures.
Event-Focused Thinking
A key skill for designing and developing event-driven systems is thinking of applications in terms of their events, in addition to their business logic and data requirements.
Traditional systems are architected to store data in a database:
- A system that knows something happened stores that fact in a database. It will likely queue the write and do it when convenient, so there is a delay before the data is even available.
- Another system that needs to know if something happened must poll the database looking for it. Polling often enough to find events quickly is massively inefficient; polling less often means slower responses and longer delays.
Traditionally, system design also focuses on the business processes and the data that models the business. This approach is embodied in methods such as Business Process Management (BPM), Object-Oriented Design and Domain-Driven Design.
Each of these tends to reinforce what one might call “database thinking.” In database-centric systems — the bulk of systems historically built — actions are recorded in the database. In BPM, each step records results in a database, and the next step reads the results it needs and proceeds. In Object-Oriented Design, operations on objects tend to read and write the database to retain object state. As a result, developers think less about the flow of the system and more about manipulating state for each activity.
Event-based systems work differently. They focus on the events that occur throughout the system and the responses the business must make to them. To develop event-driven systems effectively, developers need to consider the event-driven nature of the system throughout the entire software development life cycle, from requirements to retirement.
What is an event?
An event is something of interest that has happened in the business. The key element of the definition is that the event occurred in the past.
Most importantly, an event is not something that needs to be done:
submitOrderis a command. It is an order to do something, and it tightly couples the whole system to accomplish this one task, then the next. It is not an event.orderSubmittedis an event — it represents the fact that an order was submitted, and it can elicit reactions from a digital enterprise.
The design is no longer sensitive to how the order was placed, since any mechanism that can properly produce an orderSubmitted event drives the correct reaction.
Event-focused design also extends to multi-domain scenarios simply by adding reactions. For example, orderSubmitted certainly drives the order process, but in a digital enterprise it may also trigger customer experience, marketing and finance reactions — without the order management app knowing anything about those domains or the reactions they implement.
The various business functions can all act on the event independently, making the system more robust and more flexible. This also makes parallel activities easier to handle — something we would expect in streaming event-based systems. All reactions to an event can run in parallel, and independent reactions can run in any order.
Context
Events and Services
There is a major focus on services and microservices in contemporary software architecture.
Services are a powerful mechanism for encapsulating functionality in a network-consumable way. Microservices break services into even smaller deployable units, so each can be upgraded independently (at least if there is enough testing to guarantee the new deployment introduces no backward-compatibility issues).
Events and services are complementary: events identify the critical things that happen in the enterprise, and services specify the business logic required to react to them. An event-driven application binds the events to the services that implement the reactions. At runtime, as events are received — perhaps as a stream — the application accepts each event, determines the relevant reactions, and invokes the proper services and microservices.
Mediation for Flexibility and Scale
A common best practice with services is API management, which can be thought of as a mediation scheme for services. Anyone wishing to use a service obtains access to it from the mediation layer; for a commercial service, provisioning includes agreeing on commercial terms. At runtime, the mediator routes requests to the service. This decouples the service requester from the service provider, letting the mediation layer provide security, monitoring, chargeback and API independence by transforming requests into the formats the service currently accepts.
A similar best practice will evolve for events through an event management capability. Anyone wishing to receive an event contacts the mediation layer, which provisions the subscriber for access. At runtime, the mediator routes events to subscribers — decoupling publishers from subscribers, allowing value-added services in the mediation layer, and letting events evolve over time without impacting subscribers.
Events versus Processes
Some processes don’t seem to lend themselves well to event-driven operations, such as those that must follow a particular order:
- Find the part
- Obtain the part from inventory
- Find suitable packing
- Pack the part
- Address the label
- Attach the label to the package
- Deliver the package to the shipper
- Record the completion of the shipping business process
However, if the process has many variations or unknowns, an event-based approach might be better. For example, suppose the picker-and-transporter machine fails during step 2 (obtain the part from inventory). This is a great opportunity to apply event thinking:
- Start with an
automationFailedevent that Maintenance picks up to start the repair process. - Shipping picks up the same event simultaneously, so they can find an alternate way to obtain the part for packing and shipping.
- Perhaps a process starts that sends a person out to obtain the part, or assigns the task to another machine.
All these exceptional activities are asynchronous and, in some cases, collaborative with other systems or people — which makes an event-based system a more natural approach than a process system.
Bottom line: Use processes only where the process is well understood and mostly invariant. Use events where the order of activities may vary, or where many unusual conditions and exceptions must be accommodated.
Tactical and Strategic
Another thorny issue in deploying event-based systems — or any systems — is the tension between:
- Tactical initiatives that can be implemented quickly and move the enterprise incrementally toward digitization
- Strategic initiatives that are longer-term but have a greater impact on reaching the ultimate digitization goals
Vantiq offers a unique platform for real-time, event-driven applications that seamlessly integrates tactical initiatives with strategic ones, giving the enterprise the best of both worlds. The platform features that make this possible include:
- Design Model Tool
- Intelligent, Advanced Catalog Event Broker
- Services
- Visual Event Handlers, including Collaboration activity patterns
The following discussion outlines the tactical and strategic issues, and how Vantiq assembles strategic capabilities from tactical initiatives.
Strategic Digitization
To remain competitive in the 21st century, enterprises must become highly integrated, real-time operations. Relevant data on all aspects of the operation, across all entities, must be collected in real time and analyzed in real time to determine the current situation — and if the situation is not optimal, action must be taken immediately to adjust the business toward the optimal state.
Achieving this level of integration is currently extremely difficult, if not impossible, because it requires a top-down design of the entire system. Much like enterprise architecture efforts, it is expensive, time-consuming and almost never completes, as the enterprise changes faster than the architecture can be captured. This discourages many enterprises from even considering strategic digitization initiatives.
Tactical Digitization
Given the difficulty of strategic digitization, many organizations opt exclusively for simpler tactical initiatives. These succeed because they satisfy a clear, immediate business requirement and can be implemented in a relatively short time, usually within a single department.
The problem is that, even when aggregated over time, each tactical application is its own silo — so a real-time, digitized enterprise can never be fully achieved this way.
Conclusion
Enterprises that want to continue operating in the 21st century must fully embrace real-time business operations, which means their systems need to be event-driven and widely integrated. Vantiq is a platform architected to help solution builders rapidly assemble and integrate fully event-driven applications and deploy them across a whole spectrum of platforms.