Overview : What’s Different About Complex Event Processing

What’s Different About Complex Event Processing
Complex Event Processing (CEP) is a set of technologies that allows "events" to be processed on a continuous basis.
Most conventional event processing software is used either for Business Process Management (BPM), TIBCO iProcess for example, or for Service Oriented Architecture (SOA), for example TIBCO ActiveMatrix BusinessWorks software.
CEP is unlike conventional event processing technologies, however, in that it treats all events as potentially significant and records them asynchronously. Applications that are appropriate for CEP are event-driven, which implies some aspect of real-time behavior. To be more specific, the typical CEP application area can be identified as having some aspect of "situation awareness," "sense and respond," or "track and trace," aspects which overlap in actual business situations.
Situation awareness is about "knowing" the state of the product, person, document, or entity of interest at any point in time. Achieving this knowledge requires continuous monitoring of events to do with the entity, events that indicate what situation or state the entity is in, or about to be in. As an example, a dashboard indicates all performance indicators for a runtime production process. All the production plant events are monitored and the situation, or health, of the production process is determined via some performance indicators that are shown in real-time to one or more operators.
Sense and respond is about detecting some significant fact about the product, person, document or entity of interest, and responding accordingly. To achieve this result the software does the following:
As an example, you may monitor cell phone or credit card usage, detect that a cell phone or credit card is being used consecutively at locations that are too far apart for real-time person-to-business transactions. Detection of such transactions indicates that an act of fraud is in progress. The system responds accordingly, denying the transactions, and invoking the necessary workflow to handle the situation as defined in standard procedures.
Track and trace is about tracking the product, person, document or entity of interest over time and tracing pertinent facts like location, owner, or general status. An example would be tracking events from an RFID-enabled inventory control system where at any point in time you need to know how many widgets are in what location.
"Situation awareness," "sense and respond," and "track and trace" can all be classified as types of activity monitoring, for which the continuous evaluation of incoming events is suitable. For this reason, CEP is often described as a generalization of Business Activity Monitoring (BAM), although the event processing task may be only indirectly be related to business, as in the case of an engine monitoring application or process routing task.
Technical Requirements of a CEP System
CEP systems must be able to receive and record events and identify patterns of these events and any related data. CEP systems must also handle temporal or time-based constraints, especially for handling the non-occurrence of events. The following TIBCO BusinessEvents features satisfy these requirements:
A rich event model, incorporating event channels (for different event mechanisms) and destinations (for different types of events).
A state model mechanism that allows entities to be described in terms of state, and in particular allows modelling of time-out events to handle the non-occurrence of events. (State modeling requires TIBCO BusinessEvents Data Modeling, purchased separately.)
A Model-Driven Approach
TIBCO BusinessEvents can be described not only as a CEP engine but also as an event-driven rule engine or real-time rule engine. TIBCO BusinessEvents enables CEP problems to be solved through a model-driven approach, in which the developer defines the event, rule, concept (class) and state models which are then compiled so that at run-time incoming events are processed as efficiently as possible. The various models are as follows:
Event model  The event model describes the inputs into BusinessEvents. Events provide information through their properties and (optionally) through an XML payload. The event model provides the primary interface between BusinessEvents and the outside world, for input as well as output. Typical event sources (or channels) are messages from TIBCO Rendezvous and TIBCO Enterprise Message Service middleware, events generated explicitly by BusinessEvents, and custom mechanisms for non-standard event sources. Events can be used to trigger rules.
Concept model  The concept model describes the data concepts used in BusinessEvents, which may be mapped from events or their payloads, or loaded by some other mechanism into BusinessEvents. The concept model is based on standard UML Class and Class Diagram principles.
Rule model  Rules provide one of the main behavioral mechanisms in BusinessEvents. Rules are defined in terms of declarations (events and concepts of interest), conditions (filters and joins on and between the attributes of the events and concepts), and actions. The underlying rule engine is based on an algorithm called the Rete algorithm, which mixes all rules together into a type of look-up tree, so that any additional concept instance or event can near-instantly cause the appropriate rule or rules to fire and invoke the appropriate actions. Rules are almost always defined in general terms (concepts or classes and events), so they apply to however many combinations of those events and classes exist in memory at any one time. The combination of rule declaration and condition defines the event pattern required for CEP operation. Rule actions that update other concepts may cause other rules to become available for firing, a process called inferencing or forward chaining. These types of rules are generally called Production Rules. The appropriate UML Production Rule Representation is still under development.
Rule functions  Algorithms, procedures or functions may be usefully defined as parameterized rule functions and re-used as required in rule actions and other areas where a behavior can be specified.
State model  An important item of metadata for a concept or object is its state. Typically a state model describes the states that an entity can hold, and the transitions between states that are allowed, and the conditions for such transitions. Internally the state model is just additional metadata, but it is more useful to describe the state model as a visual model of states and transitions. The state transition rules can be viewed as special customizations of standard rules. The state model is based on the standard UML State Model principles. Requires TIBCO BusinessEvents Data Modeling add-on software.
Query model  Queries can provide both snapshot and continuous views of the data in a BusinessEvents cache. Queries can also provide continuous views of data arriving through channels. They are constructed and executed from rule functions in a specialized agent (called a query agent). Queries provide event stream processing or set operations to derive information that can then be used in rule functions, or shared (via events or the cache). Requires TIBCO BusinessEvents Event Stream Processing add-on software.
Stateful Rule Engine
At run-time, the rule engine executes rules based on new events and data sources on a continuous basis. The rule memory is never "reset" (unless by design), so that future events can always be compared to past events. For this reason, the rule engine is described as a stateful rule engine. If required, the working memory can be cleared and a new set of data asserted for each "transaction," in which case the engine is operating as a stateless rule engine.
Object Management Options
To ensure resilience, BusinessEvents provides two persistence mechanisms for the events and data loaded into the system. One is called Persistence. It is a checkpoint mechanism that causes the data in the working memory to be saved to a lightweight database engine, to be restored when and as required. The other is a high performance distributed cache that allows data to be persisted and removed from the Rete network or returned to the Rete network, as required to handle extremely large problem domains (that would not typically fit into a runtime memory model). A backing store can be added to provide additional reliability and object management options. Just as data can be moved between the Rete network and the cache, so can less used data be moved between the backing store and the cache, to balance storage, memory, and performance requirements.
These characteristics provide BusinessEvents with its enterprise and extreme transaction processing capabilities. Note that no rule operations are stored in the databases: this is because it is more efficient to simply rerun the rules and recreate the appropriate actions, than it is to persist the internal workings of the rule engine.