Variable Attributes - Usage Details
The Variable Attributes feature provides the ability to associate with a record a variable number of labeled pieces of data and then search for records based on these labeled values. These labeled pieces of information are called Attributes of the record. The feature is called Variable Attributes because any particular record is allowed to have anywhere from zero to 1,000 Attributes associated with it. Each record is allowed to have a completely different set of Attributes. Labels are added dynamically as they are received, there is no need to predefine them, thus it is not necessary to know the entire set of possible Attributes when a table is created.
While searching, an Attribute value can be specified and used anywhere a standard field value is used. This includes all the query types and predicate expressions. However, note that an Attribute value is always considered to be of type LKT_FLD_TYPE_SRCHTEXT and can be used only where a text field is used. For instance, it cannot be the target of a custom date search.
An example used for the Variable Attributes feature might be a products table. There can be a very large number of different characteristics for a product (e.g. height, weight, color, power requirements, operating temperature) the particular set of characteristics being dependent on the type of product. To provide a separate column in a table for each possible characteristic is impractical from several points of view: the number of fields needed could easily be in the thousands or even tens of thousands, too large to be practical, the set of characteristics is likely to change frequently as new products are introduced, adding and removing columns in a large table on a frequent basis is not practical. With the Variable Attributes feature handling this kind of situation becomes straightforward.
All Variable Attribute values are kept in a special field of the table that is of type LKT_FLD_TYPE_ATTRIBUTES. A table might have only one field of type LKT_FLD_TYPE_ATTRIBUTES and that field must be the last field of the table. The presence of such a field in a table enables Variable Attributes to be attached to the records of the table. This field is referred to as the Variable Attributes field. If a table does not have a Variable Attributes field, variable attributes cannot be loaded into records of that table.
A record might never have two attribute values with the same name.
The ordering of attributes is maintained. Attributes are always returned in the same order they were added.
Attributes are referenced by using a "field name" with the format:
"Variable-Attribute-Field:attribute-name"
where Variable-Attribute-Field is the name assigned to the Variable Attribute Field for the table, a colon is a literal colon character and attribute-name is the name or label for the attribute being referenced. It is an error if Variable-Attribute-Field does not exist in the table or is not a field of type LKT_FLD_TYPE_ATTRIBUTES.
The above convention implies that the name of the Variable Attributes Field might not contain the colon character. Other sizes and limits to keep in mind:
| • | There is a limit of 1,000 Attributes per record. |
| • | Variable Attribute data must be UTF-8 encoded characters. |
| • | The total Attribute data for a particular record is limited to 50,000 bytes. |
| • | There might be at most 60,000 different Attribute names in one table. |
| • | The variable attribute name is case sensitive and allows letters, digits, the underscore, and the hyphen character, other characters are not supported. |
| • | An Attribute name is a null-terminated string of from 1 to 2048 characters. |
When referencing a Variable Attribute in a query or predicate, if the named attribute does not exist in the record it is treated exactly as if the record had a zero length (empty) value for that Attribute. No distinction is made between an Attribute with a zero length value and an Attribute that doesn't exist in the record.
If you reference a Variable Attribute field without an Attribute name qualifier (i.e. reference it as a regular searchable text field) the value of the field is the values of each of the Attributes in the field concatenated together with a single space character between each value. The order of the values is maintained as originally given. Thus if you had a Variable Attribute set:
street="123 Albany St"
city="Boston"
state="MA"
zip="02201"
the value of the Variable Attribute field as a whole (with no Attribute name qualifier) would be "123 Albany St Boston MA 02201". Thus it is possible to do a simple query against all attribute values by querying the field as if it were a normal searchable text field.
In addition to the previously described means of referencing Variable Attributes in a table, a new query structure also provides a convenient means of performing a query across a set of attributes. This query structure is described in Query Construction along with the other query constructs.
Adding and Accessing Variable Attributes in a Record
As we saw in Building the Input: Data Records all of the field data for a record is loaded as a single block of text. Thus all fields, regardless of field type, must have a text representation. This is true of the Variable Attributes Field just as it is true for integer, float, date and date-time fields. A set of Attributes is encoded as a text string using the following syntax:
Attribute-Name:Attribute-Value[;Attribute-Name:Attribute-Value]*
I.e. A list of Attribute-Name - Attribute-Value pairs, where the name and value are separated by a colon character and pairs in the list are separated by semicolon characters. If an Attribute-Value contains a semicolon character it must be escaped by giving it twice. An example:
For address: "123; Suite 107 Main St" Split into street-name and street-number.
"street-number:123;; Suite 107;street-name:Main St"
A zero length string is the string encoded form of an empty Attributes set.
There is no need to worry too much about the details of this format because a set of functions is provided to encode and decode a set of attribute values to and from this format. These functions work with an opaque object that represents a set of Attributes. These functions provide a means of creating one of these objects from either a list of attribute names and values, or an encoded text form of an attribute set. You can then retrieve attribute values in a number of ways and generate the encoded text string for the attribute set. The opaque object representing an Attribute set is declared as type:
dvk_va_t
The special constant dvk_va_null represents a non-existent attribute set (as opposed to an empty attribute set). This value might be assigned to a variable of type dvk_va_t to indicate that it is not currently set. For example,
dvk_va_t our_va_set = dvk_va_null ;
The functions available are:
dvk_varattr_create
dvk_va_t
dvk_varattr_create(int num_attrs, const char **attr_names,
const char **attr_values, int *val_lens)
This function creates a Variable Attributes object from a list of Attribute names and values. It returns a new Variable Attributes object. It returns dvk_va_null if invalid arguments are given. The returned item must be cleaned up by the caller using the dvk_varattr_destroy function.
The arguments are:
| • | num_attrs The count of the number of attributes. All of the following arrays must be of this length. |
| • | attr_names The list of attribute names. Each name is a null-terminated string. Names are subject to the restrictions for attribute names given above. All names in the list must be unique. |
| • | attr_values The list of attribute values. Each value is a text string whose length in bytes, is given by the corresponding entry in val_lens. Values must be UTF-8 encoded character strings, but need not be null-terminated strings. |
| • | val_lens This is a list of integer values that give the length of the Attribute values in bytes. |
dvk_varattr_decode
dvk_va_t
dvk_varattr_decode(const char *enc_str, int enc_len)
This function creates a new Variable Attributes set object from the encoded text string format for an Attribute set. It returns a new Attributes set object if successful. It returns dvk_va_null if the given string is not a properly encoded Attribute set. The returned item must be cleaned up by the caller using the dvk_varattr_destroy function.
The arguments are:
| • | enc_str This is the encoded text string version of the attribute set. This might not be NULL. To represent an empty set, use a zero length string. |
| • | enc_len This is the length in bytes of the encoded text string. |
dvk_varattr_destroy
void
dvk_varattr_destroy(dvk_va_t va)
This function destroys a Variable Attributes set object. If you do not call this function for each dvk_va_t object you create, a major memory leak occurs. Note that this also destroys all data that might have been returned from this Attribute set, so all such data is invalid after this call is made.
The argument is:
va The Attribute set to be destroyed.
dvk_varattr_num_attrs
int
dvk_varattr_num_attrs(dvk_va_t va)
This function returns the number of attributes in an Attribute set. It returns a value less than zero if the given set is not valid.
The argument is:
va The Attribute set.
dvk_varattr_get_names
const char **
dvk_varattr_get_names(dvk_va_t va, int *num_attrs)
This function returns a list of the names of all of the Attributes in the given Variable Attributes set. The names are null-terminated strings. The names appear in the order they were originally given when the set was created. If va is invalid a NULL pointer is returned.
Note that the list returned and the names in the list become invalid when the given Variable Attribute set is destroyed. If they are to be used after the set is destroyed the caller must make a copy of the list and all values in the list.
The arguments are:
| • | va The Attribute set. |
| • | num_attrs If this is a non-null pointer the number of items in the returned list is placed in the integer pointed to by this value. |
dvk_varattr_get_values
const char **
dvk_varattr_get_values(dvk_va_t va, int *num_attrs)
This function returns a list of the values of all of the Attributes in the given Variable Attributes set. The values are not null-terminated strings. To get the lengths of the values dvk_varattr_get_lengths must be used. The values appear in the order they were originally given when the set was created. This is the same order names are returned in by the dvk_varattr_get_names function, so names and values can be matched up by index position. If va is invalid a NULL pointer is returned.
Note that the list returned and the values in the list become invalid when the given Variable Attribute set is destroyed. If they are to be used after the set is destroyed the caller must make a copy of the list and all values in the list.
The arguments are:
| • | va The Attribute set. |
| • | num_attrs If this is a non-null pointer the number of items in the returned list is placed in the integer pointed to by this value. |
dvk_varattr_get_lengths
const int *
dvk_varattr_get_lengths(dvk_va_t va, int *num_attrs)
This function returns a list of the lengths of the values of all of the Attributes in the given Variable Attributes set. The values appear in the order they were originally given when the set was created. This is the same order names are returned in by the dvk_varattr_get_names function, so names, values and value lengths can be matched up by index position. If va is invalid a NULL pointer is returned.
Note that the list returned becomes invalid when the given Variable Attribute set is destroyed. If the list is to be used after the set is destroyed the caller must make a copy of the list. The caller must not attempt to update any value in the list returned.
The arguments are:
| • | va The Attribute set. |
| • | num_attrs If this is a non-null pointer the number of items in the returned list is placed in the integer pointed to by this value. |
dvk_varattr_get_attr
const char *
dvk_varattr_get_attr(dvk_va_t va, const char *attr_name,
int *val_len)
This function returns the value of an Attribute by Attribute name. Given an Attribute set and an Attribute name a pointer to the value of that attribute is returned. If the set has no Attribute with the given value a pointer to a zero length string is returned. If the Attribute set is invalid or null, a null pointer is returned. The value returned is a static value that must not be updated. It is valid only until the Attribute set is destroyed.
Note that with this function you can't distinguish between the case of an Attribute that is in the set and has a zero length value and an attribute that is not in the set at all. If you wish to distinguish these cases, you should retrieve the list of Attribute names, values and lengths. An Attribute that is not in the set of course is not returned in the list, but an Attribute with a zero length value is in the list, with the corresponding value being of length zero.
The arguments are:
| • | va: The Attribute set. |
| • | attr_name: The null-terminated Attribute name. It must not be NULL. |
| • | val_len: If this is non-null the length of the attribute value is placed in the integer pointed to by this value. |
dvk_varattr_encode
const char *
dvk_varattr_encode(dvk_va_t va, int *enc_len)
This function is used to translate an Attribute set into the text string encoded form for the set. This can be used to encode an attribute set into a record to be sent to the TIBCO Patterns server. If the Attribute set is invalid a NULL pointer is returned. The value returned is a static value that must not be updated. It is valid only until the Attribute set is destroyed.
The arguments are:
| • | va The Attribute set. |
| • | enc_len If this is non-null the length of the encoded string in bytes is placed in the integer pointed to by this value. |
Let's look back at the example of building records in Building the Input: Data Records, only this time let's suppose the input record has four fields: "street", "city", "state", and "zip" instead of a single "addr" field, and the table has an attributes field instead of a text address field that we wish to encode these values into. This could be done as shown below.
typedef unsigned char UCHAR;
/* ... */
{
int srchtxtlen;
int fldlens[2];
int srchtxt[1024];
UCHAR *keystr;
static const char *addr_names[4] = { "street","city","state","zip"};
const char *addr_values[4];
int addr_lens[4];
const char *enc_str ;
dvk_va_t va ;
lpar_t reclst;
lpar_t record;
reclst = lpar_create_lst(LPAR_LST_GENERIC);
for ( /*all records*/ ) {
/* ... */
/* set standard field */*/
fldlens[0] = strlen(cur_record.name);
strcpy(srchtxt, cur_record.name);
/* create attribute set for record attributes */
addr_values[0] = cur_record.street;
addr_lens[0] = strlen(addr_values[0]);
addr_values[1] = cur_record.city;
addr_lens[1] = strlen(addr_values[1]);
addr_values[2] = cur_record.state;
addr_lens[2] = strlen(addr_values[2]);
addr_values[3] = cur_record.zip;
addr_lens[3] = strlen(addr_values[3]);
va = dvk_varattr_create(4, addr_names, addr_values, addr_lens);
if (va == dvk_va_null) {
/* handle error here */
}
/* get the encoded text string for it. */
enc_str = dvk_varattr_encode(va, &(fldlens[1]));
if (enc_str == NULL) {
/* handle error here */
}
/* append it to search text, note it is not null terminated string */
memcpy(srchtext+fldlens[0], enc_str, fldlens[1]);
/* we've copied what we need - clean up the attribute set */
dvk_varattr_destroy(va);
va = dvk_va_null;
/* now create record as before */
srchtxtlen = fldlens[0] + fldlens[1];
record = lpar_create_record();
lpar_set_record_key(record, keystr);
lpar_set_record_srchtxt(record, srchtxt, srchtxtlen);
lpar_set_record_srchflds(record, fldlens, 2);
lpar_append_lst(reclst, record);
}
/* ... Pass list of records to a command function ... */
lpar_destroy(reclst);
}
For fetching attribute values from a returned record let's look at the example from Processing the Output of DevKit Commands again. This time assuming the record has the above structure and we wish to print out each attribute value in the returned record.
/* ... */
{
int i, item, numitems, recnum, numrecs, arrlen;
int namelen;
lpar_t query, lpar;
lpar_t names, srchpars, stats, matches, minfo;
dvkerr_t dvkerr;
const int *pvals, *dvals;
const double *vvals;
const UCHAR *name;
const lpar_t *records;
const lpar_t *recinfo;
int numattrs;
const char **attrnames = {"street","city","state","zip",NULL};
const char *attrval;
int attrlen;
int attridx;
const UCHAR *enc_str;
int enc_len;
dvk_va_t va;
/* ... Build names list, query, and search parameter list ... */
/* .... as before .... */
records = lpar_get_lst_item_array(matches, &numrecs);
for (recnum=0 ; recnum < numrecs ; recnum++) {
printf("[%2d] ", recnum+1);
name = lpar_get_record_field(records[recnum], 0, &namelen);
printf("Name field `%.*s'\n", namelen, (char *)name);
/* pull the Attributes encoded string and decode it */
enc_str = lpar_get_record_field(records[recnum],1,&enc_len);
va = dvk_varattr_decode(enc_str, enc_len);
if (va == (dvk_va_t *)0) {
/* handle error here */
printf("Invalid Attributes\n");
} else {
/* successfully decoded, print our name-values */
for (attridx = 0; attrnames[attridx] != NULL; attridx++) {
attrval = dvk_varattr_get_attr(va,
attrnames[attridx],
&attrlen);
if ((attrval == NULL)||(attrlen < 0)) {
/* handle error here */
printf("Error retrieving value for: %s.",
attrnames[attridx]);
} else {
/* Output the value. */
printf("%s `%.*s'\n", attrnames[attridx],
attrlen,
attrval) ;
}
}
/* clean up, done with values now */
dvk_varattr_destroy(va);
va = dvk_va_null;
}
/* the rest is as before.... */
}
lpar_destroy(names);
lpar_destroy(query);
lpar_destroy(srchpars);
lpar_destroy(stats);
lpar_destroy(matches);
lpar_destroy(minfo);
}
In the example above we have assumed we know the names of all of the Attributes of interest and pulled them out by name. If we wanted to retrieve all attributes in the record the above could be modified as follows:
/* ... */
{
int i, item, numitems, recnum, numrecs, arrlen;
int namelen;
lpar_t query, lpar;
lpar_t names, srchpars, stats, matches, minfo;
dvkerr_t dvkerr;
const int *pvals, *dvals;
const double *vvals;
const UCHAR *name;
const lpar_t *records;
const lpar_t *recinfo;
int numattrs;
const char **attrnames;
const char **attrvals;
const int *attrlens;
int attridx;
const UCHAR *enc_str;
int enc_len;
dvk_va_t va;
/* ... Build names list, query, and search parameter list ... */
/* .... as before .... */
records = lpar_get_lst_item_array(matches, &numrecs);
for (recnum=0 ; recnum < numrecs ; recnum++) {
printf("[%2d] ", recnum+1);
name = lpar_get_record_field(records[recnum], 0, &namelen);
printf("Name field `%.*s'\n", namelen, (char *)name);
/* pull the Attributes encoded string and decode it */
enc_str = lpar_get_record_field(records[recnum],1,&enc_len);
va = dvk_varattr_decode(enc_str, enc_len);
if (va == (dvk_va_t *)0) {
/* handle error here */
printf("Invalid Attributes\n");
} else {
/* successfully decoded, print all name-values */
attrnames = dvk_varattr_get_names(va, &numattrs);
attrvals = dvk_varattr_get_values(va, &numattrs);
attrlens = dvk_varattr_get_lengths(va, &numattrs);
if ((attrnames == NULL)
|| (attrvals == NULL)
|| (attrlens == NULL)) {
/* handle error here */
printf("Could not retrieve attribute values.\n");
} else {
for (attridx = 0; attridx < numattrs; attridx++) {
printf("%s `%.*s'\n", attrnames[attridx],
attrlens[attridx],
attrvals[attridx]);
}
}
/* clean up, done with values now */
dvk_varattr_destroy(va);
va = dvk_va_null;
}
/* the rest is as before.... */
}
lpar_destroy(names);
lpar_destroy(query);
lpar_destroy(srchpars);
lpar_destroy(stats);
lpar_destroy(matches);
lpar_destroy(minfo);
}