Reading a Database From a File (lkt_dbread)

The dbread command is used to facilitate database loading. It's designed to read records from a file and produce a record list and database parameter list suitable for input to lkt_dbload.

dvkerr_t lkt_dbread( lpar_t file, lpar_t parameters, 
                     lpar_t *reclist, lpar_t *dbpars );

The following is a sample CSV file and the code that would be used to parse it.

first,middle,last,order
5,5,5,6
key1,William,Jefferson,Clinton,42
key2,George,W,Bush,43
lpar_t filepar,configpars,reclist,dbpars,dbname;
filepar=lpar_create_str(LPAR_STR_CSVFILE,"filename.csv");
configpars=lpar_create_lst(LPAR_LST_GENERIC);
lpar_append_list(configpars,lpar_create_bool(LPAR_BOOL_FNAMESFIRST,TRUE));
lpar_append_list(configpars,lpar_create_bool(LPAR_BOOL_FTYPESFIRST,TRUE));
lpar_append_list(configpars,lpar_create_bool(LPAR_BOOL_LEADINGKEY,TRUE));
lpar_append_list(configpars,lpar_create_int(LPAR_INT_DBNUMFIELDS,4));
lkt_dbread(filepar,configpars,&reclist,&dbpars);
dbname = lpar_create_str(LPAR_STR_DBDESCRIPTOR, "dbname");
lkt_dbload(LPAR_NULL,dbname,dbpars,reclist,LPAR_NULL);

Two file formats are supported: Fixed Width text files and CSV files. For Fixed Width files a width is defined for each field of the record, no separators exist between fields. Every record is exactly one line, each line has exactly the same number of bytes. CSV files are standard format comma separated values files. Microsoft Excel format is used: fields containing commas, double quotes, or new-lines must be enclosed in double quotes, a double quote within a quoted field is given as two double quotes.

In addition to the data records, files might contain one or both of the special header lines: field names—the names of the fields, field types—the types of the fields given as integer values or field type names. If both are specified, the field names always appear first.

The field types are specified using any of the names or numbers below (the letters in the names are not case sensitive):

Field Types

Field Type	Recognized Names or Numbers
key field in CSV	2, key
parent key field	16, parent-key, parent_key, parentkey
non-searchable text	4, text
searchable text	5, searchtext, srchtext
integer	6, integer, int
floating point	8, float, double
date	10, date
Searchable Date	11, srchdate, searchdate
date and time	12, datetime, datet
variable attributes	15, attributes, attrs

The record key can be specified in one of 4 ways:

•

Leading Key Each record starts with an unnamed, untyped field that holds the record key. Thus, each data record has one more field than any field names or field types header record. This is the format that was used up to release 4.4.3.

•

Field Index One of the record fields is identified via field index position as the key field. This field is included in any field name or field type header. Thus each data record has the same number of fields as the header records. However, this field is NOT loaded as a field in the table.

Thus, the record as loaded into the table has one fewer field than the file.

•

Field Name This is similar to Field Index except the key field is specified by field name instead of index position. To use this, "the field names first" option must be used.

•

No Key The record key is not given, instead keys are generated automatically when the records are read.

If the file contains child records, the parent key can be specified by field index or by field name.

An example file for each method of specifying the key is given below. Each example has both field names and field types header records. In all cases, the records have 4 fields: first,middle,last,order.

Leading Key
First,middle,last,order 5,5,5,6 key1,William,Jefferson,Clinton,42 key2,George,W,Bush,43

Key Index (with key index being 4)
first,middle,last,order,key searchtext,searchtext,searchtext,int,key William,Jefferson,Clinton,42,key1 George,W,Bush,43,key2

Key Name (with key name being "record key")
First,middle,last,record key,order 5,5,5,2,6 William,Jefferson,Clinton,key1,42 George,W,Bush,key2,43

No Key
First,middle,last,order 5,5,5,6 William,Jefferson,Clinton,42 George,W,Bush,43

Input

host
(required)

is the name of a file from which the database must be read. This must be either an LPAR_STR_CSVFILE or an LPAR_STR_FWFFILE for comma separated value or fixed width field files respectively. For more information, see Communicating with TIBCO Patterns Servers.

parameters (optional)

is a list of parameters which describe the file contents. It might contain any of the following lpars:

•

LPAR_STR_ENCODING defines the character encoding used in the file to be read. It should be one of "latin-1" or "UTF-8".

Default value: "latin-1"

•

LPAR_INTARR_RECSRCHFLDS contains field widths measured in bytes. This lpar is required for fixed width field files and it is ignored if used with comma separated value files. If LPAR_BOOL_LEADINGKEY is set a width must be given for the key field in addition to each record field.

•

LPAR_INT_DBNUMFIELDS is the number of fields in the file. If this lpar is not provided, lkt_dbread attempt to guess this value. This count includes only actual data fields, it does not include the key field as part of the field count.

•

LPAR_BOOL_LEADINGKEY is true if the first field in the file should be used as the record key.

Default value: false

•

LPAR_INT_KEYFIELD gives the index (zero based) of the field that is to be used as the key field.

Default value: no field is used as the key field

•

LPAR_STR_KEYFIELD gives the name of the field that is to be used as the key field.

Default value: no field is used as the key field.

•

LPAR_INT_INITIALKEY is the number used to generate the key for the first record in the file when no key field was specified. Each successive record is numbered one higher than the previous.

Default value: 0

•

LPAR_BOOL_FNAMESFIRST is true if the first line of the file contains field names rather than a record.

Default value: false

•

LPAR_BOOL_FTYPESFIRST is true if the first line of the file (after field names, if they exist) are field type codes. See The TIBCO Patterns Table for a description of field types.

Default value: false

•

LPAR_INT_BATCHSIZE indicates that the records read should be broken into multiple lists of at most this size. This is useful if you wish to implement an interactive database load with a periodic status update. If this lpar is not given, all of the records are placed in a single list.

•

LPAR_BOOL_DOMAXWORK indicates that errors should be ignored whenever possible. For instance, if there is no record content for a given line (i.e. all the fields are empty), a DVK_ERR_FILEFORMAT is usually returned. If LPAR_BOOL_DOMAXWORK is set to true, this line is skipped and parsing begins on the next line.

Default value: false

•

LPAR_INT_PARENT_KEYFIELD gives the index (zero based) of the field that is used as the parent key field.

Default value: no field is used as the parent key field

•

LPAR_STR_PARENT_KEYFIELD gives the name of the field that is used as the parent key field.

Default value: no field is used as the parent key field.

Only one of LPAR_BOOL_LEADINGKEY, LPAR_INT_KEYFIELD, LPAR_STR_KEYFIELD can be specified. If none of these are specified then this is considered the "No Key" case and keys are generated automatically for each record.

Output

reclist (required)

is either a list of records, or a list of lists of records, depending on whether a batch size is specified. Each list of records can be passed to the lkt_dbload, lkt_dbrecadd, lkt_dbrecreplace or lkt_dbrecupdate commands to add or update them in a database.

dbpars (optional)

is a list of database parameters determined from reading the file. These parameters are intended to be passed to the lkt_dbload command as the dbpars parameter. Extra parameters can be added to the list as necessary.

 

Error codes and items returned by lkt_dbread

ARRAYLEN	item that has the wrong length
EXPECTLIST	item that should have been a list type lpar
FILEFORMAT	a line from the file which could not be parsed
IOERROR	LPAR_STR_SYSERROR describing error
PARAMCONFLICT	lpar in conflict with an earlier lpar
	FILEFORMATERROR describing what is missing
PARAMVAL	lpar that contains an illegal value