Patterns Data Transfer Tool - Overview

This section describes the process of using Patterns Data transfer tool.

TIBCO® Patterns stores multiple types of data while performing its role as an inexact matching engine. Transferring this data between Patterns installations can be difficult, particularly if the installation architectures are incompatible.
For example, transferring data from a single-server installation to a clustered installation.

The Patterns Data Transfer Tool (PDTT) is a Java™-based command-line utility, which transfers data between installations with a single command, regardless of installation architectures.

Limitations

PDTT has limited support for TIBCO Patterns versions prior to 6.1.2:
- The source server and target server must be at 5.6 version or higher.
- When the source server is 5.6 or 5.7 version, only tables can be transferred.
Transferring from a higher version server to a lower version server is not supported.

PDTT does not block such attempts, but it is not a supported use case and is likely to fail.
PDTT should be run in isolation. No other applications must be modifying data on the source or target server while PDTT is running. Applications can read data but not write it.
PDTT only performs full transfers and attempts to transfer all objects.
Transfer of an object fails if the object exists on the target server.

For character-maps, thesauri, and RLink models behavior can be changed with the -i command-line option. The -i option enables over-writing existing thesauri and RLink-models. Tables and character-maps cannot be overwritten, even with the -i option.
One transaction is used per object.

If PDTT fails, the following actions occur:
- Objects that were completely transferred remain on the target server.
- Objects that were not completely transferred are removed from the target server.
Parallel transfer

Tables can be large. To minimize the transfer time and memory usage, PDTT transfers tables in parallel. All other objects are transferred one at a time.
Resource usage
- Memory: PDTT typically uses less than 2 GB of memory. Transferring large RLink Learn models requires up to an additional 6 GB of memory.
- CPU: PDTT uses one thread per table. Transfer of other objects is single-threaded and takes place before transferring tables.

Logging

PDTT is compatible with Apache log4j2. This provides for setting the location of the log output, log levels, and so on. The default log level is info.

Usage

As Patterns Data Transfer Tool is a command-line based tool, the following command is used for data transfer:

java [jvm_options] -jar  TIB_tps_datatransfer.jar  source_server  target_server  [-i] [-o]

Command	Description
`jvm_options`	Any applicable options to your Java Virtual Machine
`source_server`	Patterns installation that data is transferred from
`target_server`	Patterns installation that data is transferred to
`-i`	Objects that exist on the target server do not cause an error
`-o`	Override server-version validations.

Both, source_server and target_server use the following format:

version:hostname:port[:SSL]

Option	Description
version	Version of the Patterns server. Example: 6.1.2
hostname	DNS-name or IP-address where the Patterns server is running
port	The TCP port that the Patterns server is listens on
SSL	Use SSL-encrypted communications

The versions specified on the command line are checked against the versions returned by the servers. If either does not match, an error is logged and the transfer is not attempted. You can override this behavior with the -o option. If -o is specified and the versions do not match, a warning is logged and the transfer is attempted.

Example:

java –jar TIB_tps_datatransfer.jar 5.6:localhost:8051:SSL  6.1:otherhost:5055

This command is used to commence transfer from a TIBCO® Patterns 5.6 server to an TIBCO® Patterns 6.1.0 version. The 5.6 Patterns server uses SSL-encrypted communications on port 8051 on the local host. The 6.1.2 Patterns server uses plain-text communications on port 5055 on machine otherhost. Since -i is not specified, any object that already exists on otherhost causes an error.

A Sample Run of Patterns Data Transfer Tool

Note: The z440 machine is associated with the 5051 port and localhost is associated with the 5055 port. It is important to maintain consistency with these associations throughout this section when executing commands.

This example uses PDTT to transfer data from a Patterns 5.6.0 instance on port 5051, on the z440 machine to a Patterns 6.1.0 instance on the local machine on port 5055.

Using the NSC command-line tool, we can see the tables on the source instance:

java -jar NSC.jar -host z440 -port 5051 -cmd tbllist

>>> nsc start.
List: GENERIC
DBDESCRIPTOR: join--names
CHECKPOINT_STATUS: Last Checkpoint: 2022/05/11-10:31:33
TRAN_ID: 0
TRAN_OBJ_STATE: 1
DBINFO:
DBNUMFIELDS: 6
FIELDNAMES: ["last","first","ssn","street","city","zip"]
FIELDTYPES: [5, 5, 5, 5, 5, 5]
FIELDBYTELCOUNTS: [337, 281, 486, 930, 506, 270]
FIELDCHARLCOUNTS: [337, 281, 486, 930, 506, 270]
CHARMAPS: ["=STD=","=STD=","=STD=","=STD=","=STD=","=STD="]
DBGIPFILTER: true
DBGIPGPU: false
GIPGRAMLEN: 3
DBSORTFILTER: false
DBPSIFILTER: false
DBNUMRECORDS: 54
DBKEYTREEKBYTES: 12
DBRECFIELDKBYTES: 3
DBHEADERKBYTES: 130
DBIDXTOTALKBYTES: 644
DBTOTALKBYTES: 1814
PROCTOTALKBYTES: 334076
TABLE_TYPE: Child
PARENT_TBL: join--states

List: GENERIC
DBDESCRIPTOR: join--states
CHECKPOINT_STATUS: Last Checkpoint: 2022/05/11-10:31:33
TRAN_ID: 0
TRAN_OBJ_STATE: 1
DBINFO:
DBNUMFIELDS: 4
FIELDNAMES: ["codeIdx","name","flower","latin-flower"]
FIELDTYPES: [5, 5, 5, 5]
FIELDBYTELCOUNTS: [100, 422, 647, 800]
FIELDCHARLCOUNTS: [100, 422, 647, 800]
CHARMAPS: ["=STD=","=STD=","=STD=","=STD="]
DBGIPFILTER: true
DBGIPGPU: false
GIPGRAMLEN: 3
DBSORTFILTER: false
DBPSIFILTER: false
DBNUMRECORDS: 50
DBKEYTREEKBYTES: 12
DBRECFIELDKBYTES: 2
DBHEADERKBYTES: 130
DBIDXTOTALKBYTES: 3833
DBTOTALKBYTES: 5001
PROCTOTALKBYTES: 334076
TABLE_TYPE: Parent
CHILD_TBLS: ["join--zips","join--names"]

List: GENERIC
DBDESCRIPTOR: join--zips
CHECKPOINT_STATUS: Last Checkpoint: 2022/05/11-10:31:33
TRAN_ID: 0
TRAN_OBJ_STATE: 1
DBINFO:
DBNUMFIELDS: 4
FIELDNAMES: ["ZipCodeType","City","StateIdx","Location"]
FIELDTYPES: [5, 5, 5, 5]
FIELDBYTELCOUNTS: [438, 531, 106, 1020]
FIELDCHARLCOUNTS: [438, 531, 106, 1020]
CHARMAPS: ["=STD=","=STD=","=STD=","=STD="]
DBGIPFILTER: true
DBGIPGPU: false
GIPGRAMLEN: 3
DBSORTFILTER: false
DBPSIFILTER: false
DBNUMRECORDS: 56
DBKEYTREEKBYTES: 12
DBRECFIELDKBYTES: 2
DBHEADERKBYTES: 130
DBIDXTOTALKBYTES: 1201
DBTOTALKBYTES: 2370
PROCTOTALKBYTES: 334076
TABLE_TYPE: Child
PARENT_TBL: join—states
>>> nsc end.

Enter the following command to run the transfer:

java –jar TIB_tps_datatransfer.jar 5.6:z440:5051 6.1:localhost:5055

After the transfer, the following command is used to view the tables on the target instance:

java -jar NSC.jar -host localhost -port 5055 -cmd tbllist

//Tables on target server//
>>> nsc start.
List: GENERIC
DBDESCRIPTOR: join--names
OBJ_ID: 56485e82-8513-4a14-9b8f-9c79184bb62c
CHECKPOINT_STATUS: Never Checkpointed
TRAN_ID: 0
TRAN_OBJ_STATE: 1
DBINFO:
DBNUMFIELDS: 6
FIELDNAMES: ["last","first","ssn","street","city","zip"]
FIELDTYPES: [5, 5, 5, 5, 5, 5]
FIELDBYTELCOUNTS: [337, 281, 486, 930, 506, 270]
FIELDCHARLCOUNTS: [337, 281, 486, 930, 506, 270]
CHARMAPS: ["=STD=","=STD=","=STD=","=STD=","=STD=","=STD="]
DBGIPFILTER: true
DBGIPGPU: false
GIPGRAMLEN: 3
DBSORTFILTER: false
DBPSIFILTER: false
DBNUMRECORDS: 54
DBKEYTREEKBYTES: 12
DBRECFIELDKBYTES: 3
DBHEADERKBYTES: 130
DBIDXTOTALKBYTES: 644
DBTOTALKBYTES: 1816
TABLE_TYPE: Child
PARENT_TBL: join--states

List: GENERIC
DBDESCRIPTOR: join--states
OBJ_ID: 1b95de52-4dd1-4863-ba6f-fe2e02a7b80f
CHECKPOINT_STATUS: Never Checkpointed
TRAN_ID: 0
TRAN_OBJ_STATE: 1
DBINFO:
DBNUMFIELDS: 4
FIELDNAMES: ["codeIdx","name","flower","latin-flower"]
FIELDTYPES: [5, 5, 5, 5]
FIELDBYTELCOUNTS: [100, 422, 647, 800]
FIELDCHARLCOUNTS: [100, 422, 647, 800]
CHARMAPS: ["=STD=","=STD=","=STD=","=STD="]
DBGIPFILTER: true
DBGIPGPU: false
GIPGRAMLEN: 3
DBSORTFILTER: false
DBPSIFILTER: false
DBNUMRECORDS: 50
DBKEYTREEKBYTES: 12
DBRECFIELDKBYTES: 2
DBHEADERKBYTES: 130
DBIDXTOTALKBYTES: 3833
DBTOTALKBYTES: 5003
TABLE_TYPE: Parent
CHILD_TBLS: ["join--names","join--zips"]

List: GENERIC
DBDESCRIPTOR: join--zips
OBJ_ID: 88d2b00b-faf6-4cbd-9506-b3d2e8abc697
CHECKPOINT_STATUS: Never Checkpointed
TRAN_ID: 0
TRAN_OBJ_STATE: 1
DBINFO:
DBNUMFIELDS: 4
FIELDNAMES: ["ZipCodeType","City","StateIdx","Location"]
FIELDTYPES: [5, 5, 5, 5]
FIELDBYTELCOUNTS: [414, 515, 106, 996]
FIELDCHARLCOUNTS: [414, 515, 106, 996]
CHARMAPS: ["=STD=","=STD=","=STD=","=STD="]
DBGIPFILTER: true
DBGIPGPU: false
GIPGRAMLEN: 3
DBSORTFILTER: false
DBPSIFILTER: false
DBNUMRECORDS: 54
DBKEYTREEKBYTES: 12
DBRECFIELDKBYTES: 2
DBHEADERKBYTES: 130
DBIDXTOTALKBYTES: 1201
DBTOTALKBYTES: 2372
TABLE_TYPE: Child
PARENT_TBL: join--states
>>> nsc end.

Did you find this helpful?

Yes No