Replication

Overview

Data stored in the TIBCO EBX® repository can be mirrored to dedicated relational tables to enable direct access to the data by SQL requests and views.

Like history, this data replication is transparent to end-users and client applications. Certain actions trigger automatic changes to the replica in the database:

• Activating replication at the model-level updates the database schema by automatically executing the necessary DDL statements.

• Data model evolutions that impact replicated tables, such as creating a new column, also automatically update the database schema using DDL statements.

• When using the 'onCommit' refresh mode: updating data in the EBX® repository triggers the associated inserts, updates, and deletions on the replica database tables.

Configuring replication

Enabling replication

To define a replication unit on a data model, use the element osd:replication under the elements annotation/appinfo. Each replication unit specifies tables in a single dataset in a specific dataspace.

The nested elements are as follows:

For example:

<xs:schema>
	<xs:annotation>
		<xs:appinfo>
			<osd:replication>
				<name>ProductRef</name>
				<dataSpace>ProductReference</dataSpace>
				<dataSet>productCatalog</dataSet>
				<refresh>onCommit</refresh>
				<table>
					<path>/root/domain1/tableA</path>
					<nameInDatabase>PRODUCT_REF_A</nameInDatabase>
				</table>
				<table>
					<path>/root/domain1/tableB</path>
					<nameInDatabase>PRODUCT_REF_B</nameInDatabase>
					<element>
						<path>/retailers</path>
						<nameInDatabase>PRODUCT_REF_B_RETAILERS</nameInDatabase>
					</element>
				</table>
			</osd:replication>
		</xs:appinfo>
	</xs:annotation>
	...
</xs:schema>

Notes:

• See Data model restrictions for replicated tables

• If, at data model compilation, the specified dataset and/or dataspace does not exist in the current repository, a warning is reported, but the replica table is created in the database. Once the specified dataspace and dataset are created, the replication becomes active.

• At data model compilation, if a table replication is removed, or if some of the above properties has changed, the replica table is dropped from the database, and then recreated with the new definition if needed.

Disabling replication on a specific field or group

For a replicated table, the default behavior is to replicate all its supported elements (see Data model restrictions for replicated tables).

It is possible to disable replication for a specific field or group, either through the data model assistant, or by editing the underlying data model.

To disable the replication of a field or group by editing the data model, use the element osd:replication with the attribute disable="true".

<xs:element name="longDescription" type="xs:string">
    <xs:annotation>
        <xs:appinfo>
            <osd:replication disable="true" />
        </xs:appinfo>
    </xs:annotation>
</xs:element>

To disable the replication of a field or group through the data model assistant, use the Replication property in the Advanced properties of the element.

When this property is defined on a group, replication is disabled recursively for all its descendents. Once a group disables replication, it is not possible to specifically re-enable replication on a descendant.

Accessing a replica table using SQL

This section describes how to directly access a replica table using SQL.

Finding the replica table in the database

For every replicated EBX® table, a corresponding table is generated in the RDBMS. Using the EBX® user interface, you can find the name of this database table by clicking on the documentation pane of the table.

Access restrictions

The replica database tables must only be directly accessed in read-only mode. It is the responsibility of the database administrator to block write-access to all database users except the one that EBX® uses.

SQL reads

Direct SQL reads are possible in well-managed, preferably short-lived transactions. However, for such accesses, EBX® permissions are not taken into account. As a result, applications given the privilege to perform reads must be trusted through other authentication processes and permissions.

Requesting an 'onDemand' replication refresh

The 'onDemand' refresh policy requires an explicit request to refresh the replicated table data.

There are several ways to request a replication refresh:

• User interface: In the dataset actions menu, use the action 'Refresh replicas' under the group 'Replication' to launch the replication refresh wizard.

• Data services: Use the replication refresh data services operation. See Replication refresh for data services for more information.

• Java API: Call the ReplicationUnit.performRefresh methods in the ReplicationUnit API to launch a refresh of the replication unit.

Impact and limitations of replication

The replication feature has some known limitations and side-effects, which are listed below. If using replication, it is strongly recommended to read this section carefully and to contact Cloud Software Group, Inc. support in case of questions.

See Supported databases for the databases for which replication is supported.

Validation

Some EBX® data model constraints become blocking constraints when replication is enabled. For more information, see Structural constraints.

Data model restrictions for replicated tables

Some restrictions apply to data models containing tables that are replicated:

• Data model restrictions due to mapped mode

• Dataset inheritance is not supported for the 'onCommit' refresh policy if the specified dataset is not a root dataset or has not yet been created. See dataset inheritance for more information.

• Field inheritance is also only supported for the 'onDemand' refresh policy. This means that, at data model compilation, an error is reported if the refresh mode is 'onCommit' and the table to be replicated has an inherited field. See inherited fields for more information.

• Computed values are ignored.

• Linked fields are ignored.

• Limitations exist for two types of aggregated lists: aggregated lists under another aggregated list, and aggregated lists under a terminal group. Data models that contain such aggregated lists can be used, however these lists will be ignored (not replicated).

• User-defined attributes are not supported. A compilation error is raised if they are included in a replication unit.

Data model evolutions may also be constrained by the underlying RDBMS, depending on the data already contained in the concerned tables.

Database configuration

The refresh operation is optimized to transmit only the rows of the source table that have been modified (with respect to creation and deletion) since the last refresh. However, depending on the volume of data exchanged, this can be an intensive operation, requiring large transactions. In particular, the first refresh operation can concern a large number of rows. It is necessary for the database to be configured properly to allow such transactions to run under optimal conditions.

For instance, with Oracle:

• It is mandatory for the bulk of all replica tables in a replication unit to fit into the 'UNDO' tablespace.

• It is recommended to provide enough space in the buffer cache to allow those transactions to run with minimal disk access.

• It is recommended to provision 'REDO' log groups big enough to avoid those transactions to wait for the 'db_writer' process.

Distributed data delivery (D3)

Replication is available on both D3 primary and replica delivery dataspaces. On the primary dataspace, the replication behavior is the same as in a standard semantic dataspace, but on replica dataspaces, the replicated content is that of the last broadcast snapshot.

In a replica delivery dataspace, some restrictions occur:

• The refresh policy defined in the data model has no influence on the behavior described above: replication always happens on snapshot.

• The action item Refresh replicas is not available.

• It is not allowed to invoke the ReplicationUnit.performRefresh method.

Other limitations of replication

• Limitations of supported databases

• For inheritance, a replica record field cannot hold the "inherit value" flag (AdaptationValue.INHERIT_VALUE). It only holds the inherited value in such cases. More generally, it is not possible to distinguish inheriting state from overwriting state.