Chapter 1 Overview : What’s Different About Complex Event Processing

What’s Different About Complex Event Processing
Complex Event Processing (CEP) is a set of technologies that allows "events" to be processed on a continuous basis.
Most conventional event processing software is used either for Business Process Management (BPM), TIBCO iProcess for example, or for Service Oriented Architecture (SOA), for example TIBCO ActiveMatrix BusinessWorks software.
CEP is unlike conventional event processing technologies, however, in that it treats all events as potentially significant and records them asynchronously. Applications that are appropriate for CEP are event-driven, which implies some aspect of real-time behavior. To be more specific, the typical CEP application area can be identified as having some aspect of "situation awareness," "sense and respond," or "track and trace," aspects which overlap in actual business situations.
Situation awareness is about "knowing" the state of the product, person, document, or entity of interest at any point in time. Achieving this knowledge requires continuous monitoring of events to do with the entity, events that indicate what situation or state the entity is in, or about to be in. As an example, a dashboard indicates all performance indicators for a runtime production process. All the production plant events are monitored and the situation, or health, of the production process is determined via some performance indicators that are shown in real-time to one or more operators.
Sense and respond is about detecting some significant fact about the product, person, document or entity of interest, and responding accordingly. To achieve this result the software does the following:
As an example, you may monitor cell phone or credit card usage, detect that a cell phone or credit card is being used consecutively at locations that are too far apart for real-time person-to-business transactions. Detection of such transactions indicates that an act of fraud is in progress. The system responds accordingly, denying the transactions, and invoking the necessary workflow to handle the situation as defined in standard procedures.
Track and trace is about tracking the product, person, document or entity of interest over time and tracing pertinent facts like location, owner, or general status. An example would be tracking events from an RFID-enabled inventory control system where at any point in time you need to know how many widgets are in what location.
"Situation awareness," "sense and respond," and "track and trace" can all be classified as types of activity monitoring, for which the continuous evaluation of incoming events is suitable. For this reason, CEP is often described as a generalization of Business Activity Monitoring (BAM), although the event processing task may be only indirectly be related to business, as in the case of an engine monitoring application or process routing task.
Technical Requirements of a CEP System
CEP systems must be able to receive and record events and identify patterns of these events and any related data. CEP systems must also handle temporal or time-based constraints, especially for handling the non-occurrence of events. The following TIBCO BusinessEvents features satisfy these requirements:
A rich event model, incorporating event channels (for different event mechanisms) and destinations (for different types of events).
A state model mechanism that allows entities to be described in terms of state, and in particular allows modelling of time-out events to handle the non-occurrence of events. (State modeling is available in the Enterprise Suite version only.)
A Model-Driven Approach
TIBCO BusinessEvents can be described not only as a CEP engine but also as an event-driven rule engine or real-time rule engine. TIBCO BusinessEvents enables CEP problems to be solved through a model-driven approach, in which the developer defines the event, rule, concept (class) and state models which are then compiled so that at run-time incoming events are processed as efficiently as possible. The various models are as follows:
Event model  The event model describes the inputs into BusinessEvents. Events provide information through their properties and (optionally) through an XML payload. The event model provides the primary interface between BusinessEvents and the outside world, for input as well as output. Typical event sources (or channels) are messages from TIBCO Rendezvous and TIBCO Enterprise Message Service middleware, events generated explicitly by BusinessEvents, and custom mechanisms for non-standard event sources. Events can be used to trigger rules. See Event Model, for a UML diagram of the event model.
Concept model  The concept model describes the data concepts used in BusinessEvents, which may be mapped from events or their payloads, or loaded by some other mechanism into BusinessEvents. The concept model is based on standard UML Class and Class Diagram principles. See Concept Model, for a UML diagram of the concept model.
Rule and ruleset model  Rules, arranged into rule sets, provide one of the main behavioral mechanisms in BusinessEvents. Rules are defined in terms of declarations (events and concepts of interest), conditions (filters and joins on and between the attributes of the events and concepts), and actions. The underlying rule engine is based on an algorithm called the Rete algorithm, which mixes all rules together into a type of look-up tree, so that any additional concept instance or event can near-instantly cause the appropriate rule or rules to fire and invoke the appropriate actions. Rules are almost always defined in general terms (concepts or classes and events), so they apply to however many combinations of those events and classes exist in memory at any one time. The combination of rule declaration and condition defines the event pattern required for CEP operation. Rule actions that update other concepts may cause other rules to become available for firing, a process called inferencing or forward chaining. These types of rules are generally called Production Rules. The appropriate UML Production Rule Representation is still under development. See Rule and Ruleset Model, for a UML diagram of the rule and ruleset model.
Rule functions: Algorithms, procedures or functions may be usefully defined as parameterized rule functions and re-used as required in rule actions and other areas where a behavior can be specified.
State model: an important item of metadata for a concept or object is its state according to some state machine or state model representation. Typically a state model describes the states that an entity can hold, and the transitions between states that are allowed, and the conditions for such transitions. Internally the state model is just additional metadata, but it is more useful to describe the state model as a visual model of states and transitions. The state transition rules can be viewed as special customizations of standard rules. The state model is based on the standard UML State Model principles. See State Model—Overview and State Model—State Detail, for UML diagrams of the state model.
Stateful Rule Engine
At run-time, the rule engine executes rules based on new events and data sources on a continuous basis. The rule memory is never "reset" (unless by design), so that future events can always be compared to past events. For this reason, the rule engine is described as a stateful rule engine. If required, the working memory can be cleared and a new set of data asserted for each "transaction," in which case the engine is operating as a stateless rule engine.
Object Management Options
To ensure resilience, BusinessEvents provides two persistence mechanisms for the events and data loaded into the system. One is called Persistence. It is a checkpoint mechanism that causes the data in the working memory to be saved to a lightweight database engine, to be restored when and as required. The other is a high performance distributed cache that allows data to be persisted and removed from memory or returned to memory, as required to handle extremely large problem domains (that would not typically fit into a runtime memory model).
These characteristics provide BusinessEvents with its enterprise and extreme transaction processing capabilities. Note that no rule operations are stored in the databases: this is because it is more efficient to simply rerun the rules and recreate the appropriate actions, than it is to persist the internal workings of the rule engine.