Open topic with navigation

TIBCO® Patterns 6.1.2

# Patterns Data Transfer Tool - Overview

This section describes the process of using Patterns Data transfer tool.

TIBCO® Patterns stores multiple types of data while performing its role as an inexact matching engine. Transferring this data between Patterns installations can be difficult, particularly if the installation architectures are incompatible.
For example, transferring data from a single-server installation to a clustered installation.

The Patterns Data Transfer Tool (PDTT) is a Java™-based command-line utility, which transfers data between installations with a single command, regardless of installation architectures.

### Limitations

1. PDTT has limited support for TIBCO Patterns versions prior to 6.1.2:

   - The source server and target server must be at 5.6 version or higher.

   - When the source server is 5.6 or 5.7 version, only tables can be transferred.

2. Transferring from a higher version server to a lower version server is not supported.

   PDTT does not block such attempts, but it is not a supported use case and is likely to fail.

3. PDTT should be run in isolation. No other applications must be modifying data on the source or target server while PDTT is running. Applications can read data but not write it.

4. PDTT only performs full transfers and attempts to transfer all objects.

5. Transfer of an object fails if the object exists on the target server.

   For character-maps, thesauri, and RLink models behavior can be changed with the `-i` command-line option. The `-i` option enables over-writing existing thesauri and RLink-models. Tables and character-maps cannot be overwritten, even with the `-i` option.

6. One transaction is used per object.

   If PDTT fails, the following actions occur:

   - Objects that were completely transferred remain on the target server.

   - Objects that were not completely transferred are removed from the target server.

7. Parallel transfer

   Tables can be large. To minimize the transfer time and memory usage, PDTT transfers tables in parallel. All other objects are transferred one at a time.

8. Resource usage

   - Memory: PDTT typically uses less than 2 GB of memory. Transferring large RLink Learn models requires up to an additional 6 GB of memory.

   - CPU: PDTT uses one thread per table. Transfer of other objects is single-threaded and takes place before transferring tables.

9. Logging

   PDTT is compatible with Apache log4j2. This provides for setting the location of the log output, log levels, and so on. The default log level is info.

   ### Usage

   As Patterns Data Transfer Tool is a command-line based tool, the following command is used for data transfer:

       java [jvm_options] -jar  TIB_tps_datatransfer.jar  source_server  target_server  [-i] [-o]

   Command | Description
   --- | ---
   `jvm_options` | Any applicable options to your Java Virtual Machine
   `source_server` | Patterns installation that data is transferred from
   `target_server` | Patterns installation that data is transferred to
   `-i` | Objects that exist on the target server do not cause an error
   `-o` | Override server-version validations.

   Both, source_server and target_server use the following format:

   `version:hostname:port[:SSL]`

   Option | Description
   --- | ---
   version | Version of the Patterns server. Example: 6.1.2
   hostname | DNS-name or IP-address where the Patterns server is running
   port | The TCP port that the Patterns server is listens on
   SSL | Use SSL-encrypted communications

   The versions specified on the command line are checked against the versions returned by the servers. If either does not match, an error is logged and the transfer is not attempted. You can override this behavior with the -o option. If -o is specified and the versions do not match, a warning is logged and the transfer is attempted.

   Example:

       java –jar TIB_tps_datatransfer.jar 5.6:localhost:8051:SSL  6.1:otherhost:5055

   This command is used to commence transfer from a TIBCO® Patterns 5.6 server to an TIBCO® Patterns 6.1.0 version. The 5.6 Patterns server uses SSL-encrypted communications on port 8051 on the local host. The 6.1.2 Patterns server uses plain-text communications on port 5055 on machine `otherhost`. Since -i is not specified, any object that already exists on otherhost causes an error.

   ## A Sample Run of Patterns Data Transfer Tool

   Note: The z440 machine is associated with the 5051 port and localhost is associated with the 5055 port. It is important to maintain consistency with these associations throughout this section when executing commands.

   This example uses PDTT to transfer data from a Patterns 5.6.0 instance on port 5051, on the `z440` machine to a Patterns 6.1.0 instance on the local machine on port 5055.

   Using the NSC command-line tool, we can see the tables on the source instance:

       java -jar NSC.jar -host z440 -port 5051 -cmd tbllist

       >>> nsc start.
       List: GENERIC
       DBDESCRIPTOR: join--names
       CHECKPOINT_STATUS: Last Checkpoint: 2022/05/11-10:31:33
       TRAN_ID: 0
       TRAN_OBJ_STATE: 1
       DBINFO:
       DBNUMFIELDS: 6
       FIELDNAMES: ["last","first","ssn","street","city","zip"]
       FIELDTYPES: [5, 5, 5, 5, 5, 5]
       FIELDBYTELCOUNTS: [337, 281, 486, 930, 506, 270]
       FIELDCHARLCOUNTS: [337, 281, 486, 930, 506, 270]
       CHARMAPS: ["=STD=","=STD=","=STD=","=STD=","=STD=","=STD="]
       DBGIPFILTER: true
       DBGIPGPU: false
       GIPGRAMLEN: 3
       DBSORTFILTER: false
       DBPSIFILTER: false
       DBNUMRECORDS: 54
       DBKEYTREEKBYTES: 12
       DBRECFIELDKBYTES: 3
       DBHEADERKBYTES: 130
       DBIDXTOTALKBYTES: 644
       DBTOTALKBYTES: 1814
       PROCTOTALKBYTES: 334076
       TABLE_TYPE: Child
       PARENT_TBL: join--states

       List: GENERIC
       DBDESCRIPTOR: join--states
       CHECKPOINT_STATUS: Last Checkpoint: 2022/05/11-10:31:33
       TRAN_ID: 0
       TRAN_OBJ_STATE: 1
       DBINFO:
       DBNUMFIELDS: 4
       FIELDNAMES: ["codeIdx","name","flower","latin-flower"]
       FIELDTYPES: [5, 5, 5, 5]
       FIELDBYTELCOUNTS: [100, 422, 647, 800]
       FIELDCHARLCOUNTS: [100, 422, 647, 800]
       CHARMAPS: ["=STD=","=STD=","=STD=","=STD="]
       DBGIPFILTER: true
       DBGIPGPU: false
       GIPGRAMLEN: 3
       DBSORTFILTER: false
       DBPSIFILTER: false
       DBNUMRECORDS: 50
       DBKEYTREEKBYTES: 12
       DBRECFIELDKBYTES: 2
       DBHEADERKBYTES: 130
       DBIDXTOTALKBYTES: 3833
       DBTOTALKBYTES: 5001
       PROCTOTALKBYTES: 334076
       TABLE_TYPE: Parent
       CHILD_TBLS: ["join--zips","join--names"]

       List: GENERIC
       DBDESCRIPTOR: join--zips
       CHECKPOINT_STATUS: Last Checkpoint: 2022/05/11-10:31:33
       TRAN_ID: 0
       TRAN_OBJ_STATE: 1
       DBINFO:
       DBNUMFIELDS: 4
       FIELDNAMES: ["ZipCodeType","City","StateIdx","Location"]
       FIELDTYPES: [5, 5, 5, 5]
       FIELDBYTELCOUNTS: [438, 531, 106, 1020]
       FIELDCHARLCOUNTS: [438, 531, 106, 1020]
       CHARMAPS: ["=STD=","=STD=","=STD=","=STD="]
       DBGIPFILTER: true
       DBGIPGPU: false
       GIPGRAMLEN: 3
       DBSORTFILTER: false
       DBPSIFILTER: false
       DBNUMRECORDS: 56
       DBKEYTREEKBYTES: 12
       DBRECFIELDKBYTES: 2
       DBHEADERKBYTES: 130
       DBIDXTOTALKBYTES: 1201
       DBTOTALKBYTES: 2370
       PROCTOTALKBYTES: 334076
       TABLE_TYPE: Child
       PARENT_TBL: join—states
       >>> nsc end.

   Enter the following command to run the transfer:

       java –jar TIB_tps_datatransfer.jar 5.6:z440:5051 6.1:localhost:5055

   After the transfer, the following command is used to view the tables on the target instance:

       java -jar NSC.jar -host localhost -port 5055 -cmd tbllist

       //Tables on target server//
       >>> nsc start.
       List: GENERIC
       DBDESCRIPTOR: join--names
       OBJ_ID: 56485e82-8513-4a14-9b8f-9c79184bb62c
       CHECKPOINT_STATUS: Never Checkpointed
       TRAN_ID: 0
       TRAN_OBJ_STATE: 1
       DBINFO:
       DBNUMFIELDS: 6
       FIELDNAMES: ["last","first","ssn","street","city","zip"]
       FIELDTYPES: [5, 5, 5, 5, 5, 5]
       FIELDBYTELCOUNTS: [337, 281, 486, 930, 506, 270]
       FIELDCHARLCOUNTS: [337, 281, 486, 930, 506, 270]
       CHARMAPS: ["=STD=","=STD=","=STD=","=STD=","=STD=","=STD="]
       DBGIPFILTER: true
       DBGIPGPU: false
       GIPGRAMLEN: 3
       DBSORTFILTER: false
       DBPSIFILTER: false
       DBNUMRECORDS: 54
       DBKEYTREEKBYTES: 12
       DBRECFIELDKBYTES: 3
       DBHEADERKBYTES: 130
       DBIDXTOTALKBYTES: 644
       DBTOTALKBYTES: 1816
       TABLE_TYPE: Child
       PARENT_TBL: join--states

       List: GENERIC
       DBDESCRIPTOR: join--states
       OBJ_ID: 1b95de52-4dd1-4863-ba6f-fe2e02a7b80f
       CHECKPOINT_STATUS: Never Checkpointed
       TRAN_ID: 0
       TRAN_OBJ_STATE: 1
       DBINFO:
       DBNUMFIELDS: 4
       FIELDNAMES: ["codeIdx","name","flower","latin-flower"]
       FIELDTYPES: [5, 5, 5, 5]
       FIELDBYTELCOUNTS: [100, 422, 647, 800]
       FIELDCHARLCOUNTS: [100, 422, 647, 800]
       CHARMAPS: ["=STD=","=STD=","=STD=","=STD="]
       DBGIPFILTER: true
       DBGIPGPU: false
       GIPGRAMLEN: 3
       DBSORTFILTER: false
       DBPSIFILTER: false
       DBNUMRECORDS: 50
       DBKEYTREEKBYTES: 12
       DBRECFIELDKBYTES: 2
       DBHEADERKBYTES: 130
       DBIDXTOTALKBYTES: 3833
       DBTOTALKBYTES: 5003
       TABLE_TYPE: Parent
       CHILD_TBLS: ["join--names","join--zips"]

       List: GENERIC
       DBDESCRIPTOR: join--zips
       OBJ_ID: 88d2b00b-faf6-4cbd-9506-b3d2e8abc697
       CHECKPOINT_STATUS: Never Checkpointed
       TRAN_ID: 0
       TRAN_OBJ_STATE: 1
       DBINFO:
       DBNUMFIELDS: 4
       FIELDNAMES: ["ZipCodeType","City","StateIdx","Location"]
       FIELDTYPES: [5, 5, 5, 5]
       FIELDBYTELCOUNTS: [414, 515, 106, 996]
       FIELDCHARLCOUNTS: [414, 515, 106, 996]
       CHARMAPS: ["=STD=","=STD=","=STD=","=STD="]
       DBGIPFILTER: true
       DBGIPGPU: false
       GIPGRAMLEN: 3
       DBSORTFILTER: false
       DBPSIFILTER: false
       DBNUMRECORDS: 54
       DBKEYTREEKBYTES: 12
       DBRECFIELDKBYTES: 2
       DBHEADERKBYTES: 130
       DBIDXTOTALKBYTES: 1201
       DBTOTALKBYTES: 2372
       TABLE_TYPE: Child
       PARENT_TBL: join--states
       >>> nsc end.

Did you find this helpful?

Yes No
Great!
Thanks for taking the time to give us some feedback.
Sorry about that

Why wasn't this helpful? (check all that apply)

Could not find what I was looking for
Instructions confusing or unclear
Instructions did not work

Want to tell us more?

Please select at least one option or provide details about the issue!!

Error occurred while submitting feedback.

Error2 occurred while submitting feedback.

TIBCO® Patterns

Copyright © 2010-2025. Cloud Software Group, Inc. All Rights Reserved.