Managing Apache Solr Metadata

When the server accesses a data source, it needs to know how to interpret the data stored there. For each object the server accesses, you create a synonym that describes its structure and the server mapping of the data types.

Creating Synonyms

Synonyms define unique names or aliases for each Apache Solr Search file that is accessible from the server. Synonyms are useful because they hide the underlying data source location and identity from client applications. They also provide support for the extended metadata features of the server, such as virtual fields and additional security mechanisms.

Using synonyms allows an object to be moved or renamed while allowing client applications to continue functioning without modification. The only modification required is a redefinition of the synonym on the server. The result of creating a synonym is a Master File and an Access File, which represent the server metadata.

Create a Synonym

    Procedure
  1. From the WebFOCUS Reporting Server browser interface Application page, click Get Data.
  2. On the Configured Adapters section of the page, in Simple Mode, right-click an adapter and click Show Connections. Right-click a connection.

    Depending on the type of adapter you choose, one of the following options appears on the context menu.

    • Show DBMS objects. This option opens the page for selecting synonym objects and properties.
    • Create metadata objects. This option opens the page for selecting synonym objects and properties.
    • Show files. This option opens a file picker. After you choose a file of the correct type, the page for selecting synonym objects and properties opens.
    • Show local files. This option opens a file picker. After you choose a file of the correct type, the page for selecting synonym objects and properties opens.
    • Show topics. This option opens the page for selecting synonym objects and properties for topics within the environment.
  3. Enter values for the parameters required by the adapter as described in the chapter for your adapter.
  4. After entering the parameter values, click Add.

    This button may be labeled Next, Create Synonym, Create Base Synonyms, Create Cluster Synonym, or Update Base Synonyms.

    The synonym creation process for most adapters has been consolidated so that you can enter all necessary parameters on one page. However, for some adapters such as LDAP, continue clicking Next until you get to a page that has a Create Synonym button.

Synonym Creation Parameters for Apache Solr

The following list describes the synonym creation parameters for which you can supply values.

Customize data type mappings

To change the data type mappings from their default settings, expand the Customize data type mappings section. The customizable mappings are displayed.

The Customize data type mappings are given below.

  • Decompose Date fields into components. When set to ON, decomposes Date fields and date portion of Date-Time fields into Year, Quarter, Month, and Day components. The default value is OFF.
  • Date Order. Controls mapping of the date format into Master File as YYMD, MDYY, and DMYY. The default value is adapter specific.
  • Activate GEOGRAPHIC_ROLE assignment. When set to ON, assigns GEOGRAPHIC_ROLE based on column name analysis according to the vocabulary information stored and roles in the configuration file. Create Synonym maps the GEOGRAPHIC_ROLE Master File attribute to the certain geometry (GEOMETRY_AREA or GEOMETRY_POINT) or geography (e.g. CITY, STATE, and ZIP5) role defined in the server configuration file (edahome/catalog/geo_services.xml). The complete list of values can be obtained by running edahome/catalog/geo_srv_roles.fex. The default value is ON.
  • Set data type DOUBLE for numeric decimal columns Sets data type DOUBLE for numeric decimal columns. This setting applies to XML and JSON adapters. The default value is ON.
  • Set data type AnV (on) / STRING for character columns Sets data type AnV (VARCHAR) for character columns. The setting applies to XML and JSON adapters. The default value is ON.
Miscellaneous settings

The following Miscellaneous settings option is available:

Enable (on) / Disable (off) scan limit

Enables the DFIX or Direct Excel adapter to scan the number of rows that are processed during upload. The default value is ON.

Application

Select an application directory. The default value is baseapp.

Prefix/Suffix

If you have tables with identical table names, assign a prefix or a suffix to distinguish them. For example, if you have identically named human resources and payroll tables, assign the prefix HR to distinguish the synonyms for the human resources tables. Note that the resulting synonym name cannot exceed 64 characters.

If all tables and views have unique names, leave the prefix and suffix fields blank.

Synonym Field Names Processing Options

The Synonym Field Names Processing Options are given below:

  • Validate. Select the Validate checkbox if you wish to convert all special characters to underscores and perform a name check to prevent the use of reserved names. (This is accomplished by adding numbers to the names.) This parameter ensures that names adhere to specifications. For more information, see Validation for Special Characters and Reserved Words.

    When the Validate option is unchecked, only the following characters are converted to underscores: '-'; ' '; ' \'; '/'; ','; '$'. No checking is performed for names.

  • Make unique. Select the Make unique checkbox if you wish to set the scope for field and group names to the entire synonym. This ensures that no duplicate names are used, even in different segments of the synonym. When this option is unchecked, the scope is the segment.
Filter

Filters the selection from the synonym candidates list. To filter your selection from the synonym candidates list, type the string to filter and click the search icon. For example, type ABC to search all objects whose names have the letters ABC in them.

Default Synonym Name

This column displays the name that is assigned to each synonym. To assign a different name, replace the displayed value.

Solr Core

This column displays the name of the Solr core or collection.

Sample Generated Synonym

An Adapter for Apache Solr synonym consists of a Master File and an Access File. This is a synonym for the retail_product.

Master File retail_product.mas 

FILENAME=RETAIL_PRODUCT, SUFFIX=SOLR    , 
REMARKS='Synonym for SOLR collection ''retail_product'' at URL na1devfocsol01.dev.tibco.com:9983', $  
SEGMENT=RETAIL_PRODUCT, SEGTYPE=S0, $    
FIELDNAME=__HITS, ALIAS=__HITS, USAGE=I16, ACTUAL=A31,      
MISSING=ON, ACCESS_PROPERTY=(INTERNAL),      
TITLE='__HITS', $    
FIELDNAME=BRAND, ALIAS=BRAND, USAGE=STRING, ACTUAL=STRING, FIELDTYPE=I,      
MISSING=ON, ACCESS_PROPERTY=(VALUE),      
TITLE='BRAND', $    
FIELDNAME=BRANDTYPE, ALIAS=BRANDTYPE, USAGE=STRING, ACTUAL=STRING, FIELDTYPE=I,      
MISSING=ON,      
TITLE='BRANDTYPE', $
FIELDNAME=ID_PRODUCT, ALIAS=ID_PRODUCT, USAGE=STRING, ACTUAL=STRING, FIELDTYPE=I,      
MISSING=ON,      
TITLE='ID_PRODUCT', $    
FIELDNAME=MODEL, ALIAS=MODEL, USAGE=STRING, ACTUAL=STRING, FIELDTYPE=I,      
MISSING=ON, ACCESS_PROPERTY=(VALUE),      
TITLE='MODEL', $    
FIELDNAME=PRICE_DOLLARS, ALIAS=PRICE_DOLLARS, USAGE=D20.2, ACTUAL=A64, FIELDTYPE=I,      
MISSING=ON,      
TITLE='PRICE_DOLLARS', $    
FIELDNAME=PRODUCT_CATEGORY, ALIAS=PRODUCT_CATEGORY, USAGE=STRING, ACTUAL=STRING, FIELDTYPE=I,      
MISSING=ON, ACCESS_PROPERTY=(VALUE),      
TITLE='PRODUCT_CATEGORY', $    
FIELDNAME=PRODUCT_COST, ALIAS=PRODUCT_COST, USAGE=D20.2, ACTUAL=A64, FIELDTYPE=I,    

      
MISSING=ON,      
TITLE='PRODUCT_COST', $    
FIELDNAME=PRODUCT_DESCRIPTION, ALIAS=PRODUCT_DESCRIPTION, USAGE=STRING, ACTUAL=STRING, FIELDTYPE=I,      
MISSING=ON, ACCESS_PROPERTY=(VALUE),      
TITLE='PRODUCT_DESCRIPTION', $    
FIELDNAME=PRODUCT_NAME, ALIAS=PRODUCT_NAME, USAGE=STRING, ACTUAL=STRING, FIELDTYPE=I,      
MISSING=ON, ACCESS_PROPERTY=(VALUE),      
TITLE='PRODUCT_NAME', $    
FIELDNAME=PRODUCT_SUBCATEG, ALIAS=PRODUCT_SUBCATEG, USAGE=STRING, ACTUAL=STRING, FIELDTYPE=I,     
MISSING=ON, ACCESS_PROPERTY=(VALUE),      
TITLE='PRODUCT_SUBCATEG', $    
FIELDNAME=PRODUCT_WEIGHT, ALIAS=PRODUCT_WEIGHT, USAGE=D20.2, ACTUAL=A64, FIELDTYPE=I,      
MISSING=ON,      
TITLE='PRODUCT_WEIGHT', $    
FIELDNAME=PRODUCT_WEIGHT_UNITS, ALIAS=PRODUCT_WEIGHT_UNITS, USAGE=STRING, ACTUAL=STRING, FIELDTYPE=I,      
MISSING=ON,      
TITLE='PRODUCT_WEIGHT_UNITS', $    
FIELDNAME=_NEST_PATH_, ALIAS=_nest_path_, USAGE=STRING, ACTUAL=STRING, FIELDTYPE=I,      
MISSING=ON, ACCESS_PROPERTY=(INTERNAL),      
TITLE='_nest_path_', $    
FIELDNAME=_ROOT_, ALIAS=_root_, USAGE=STRING, ACTUAL=STRING, FIELDTYPE=I,     
MISSING=ON, ACCESS_PROPERTY=(INTERNAL),      
TITLE='_root_', $    
FIELDNAME=_TEXT_, ALIAS=_text_, USAGE=STRING, ACTUAL=STRING, FIELDTYPE=I,      
MISSING=ON, ACCESS_PROPERTY=(INTERNAL, VALUE),      
TITLE='_text_', $    
FIELDNAME=_VERSION_, ALIAS=_version_, USAGE=P31, ACTUAL=A31,      
MISSING=ON, ACCESS_PROPERTY=(INTERNAL),      
TITLE='_version_', $    
FIELDNAME=ID, ALIAS=id, USAGE=STRING, ACTUAL=STRING, FIELDTYPE=I,      
MISSING=ON, ACCESS_PROPERTY=(INTERNAL),      
TITLE='id', $    
FIELDNAME=_SEARCH_STRING_, ALIAS=_Search_string_, USAGE=STRING, ACTUAL=STRING, ACCESS_PROPERTY=(INTERNAL),      
TITLE='_Search_string_', $

Access File retail_product.acx 

SEGNAME=RETAIL_PRODUCT,   
CONNECTION=CON01,   
COLLECTION=retail_product,   
UNIQUE_KEY=ID_PRODUCT, $

Access File Keywords

Attribute

Description 

SEGNAME

This value must be identical to the SEGNAME value in the Master File. 

CONNECTION

Indicates a previously declared connection. The syntax is:

CONNECTION=connection
 
COLLECTION

Indicates the name of the selected Solr collection.

 
UNIQUE_KEY

Indicates the name of the column that contains the unique key.

Managing Synonyms

Once you have created a synonym, you can right-click the synonym name in the navigation pane of the WebFOCUS Reporting Server browser interface or the ibi Data Migrator desktop interface to access the following options.

Option

Description

Open

Opens the Master File for viewing and editing by using a graphical interface. If an Access file is used, it is also available.

Sample Data

Retrieves up to 50 rows from the associated data source.

Data Profiling

Provides the data characteristics for synonym columns.

It is displayed with the following options, if applicable.

Statistics. Alphanumeric columns provide the count of distinct values, total count, maximum, minimum, average length, and the number of nulls.

Count. The numeric column provides the data count value.

Row Count. Numeric columns provide the count of distinct values, the total count, duplicate values, unique, non-unique values, and empty values.

Key Analysis. Allows you to select data set columns for analysis. You can also analyze entire data in a single operation.

Analysis Report

Provides information about synonym location and its properties. Additionally, it enables you to see the applications and synonyms that contain or have references to this synonym.

It is displayed with the following options, if applicable.

Impact Analysis. Generates a report showing where this synonym is stored and used, with links to the synonym instances. Impact Analysis reports enable you to evaluate changes before they are made by showing which components are affected. See the ibi™ WebFOCUS® Reporting Server Administration manual for details about Impact Analysis reports.

Dependencies Analysis. Generates a report showing information about the synonym and other synonyms and objects that are referenced within it.

Metadata Management

Describes the characteristics of data sources and provides the information needed to access those data sources.

It is displayed with the following options, if applicable.

Create Cluster Synonym. Creates a cluster synonym in WebFOCUS Reporting Server Browser Interface.

Classify Data. Lists options that enable you to classify columns of the data source.

Refresh. Regenerates the synonym. Use this option if the underlying object has been altered.

Edit as Text. Enables you to view and manually edit the Master File synonym.

Note: To update the synonym, it is strongly recommended that you use the graphical interface provided by the Open option, rather than manually editing the Master File.

Edit Access File as Text. Enables you to view and manually edit the Access File synonym.

Note: This option is available only when an Access File is created as part of the synonym.

Download Master File. Allows you to download the Master File to your computer.

Download Access File. Allows you to download the Access File to your computer.

Prepare Translation File. Allows you to prepare metadata translation files for the selected data set.

Note: You must have a Language File before using this option.

Data Management

Enables you to recreate a DBMS table, drop a table, insert sample data, delete all data, and show/modify data.

It is displayed with the following options, if applicable.

Create Indexes. Creates indexes on the data source table. You are asked to confirm this selection before the indexes are created.

Delete All Data. Deletes all existing data. You are asked to confirm this selection before the data is deleted.

Drop Table. Drops the table so that it is removed from the DBMS.

Insert Sample Data. Inserts specified number of sample records, populating all fields with counter values.

Show/Modify Data. Opens a window that shows the data in the data source with buttons you can click to insert values, filter values, reload the data source, and customize the view.

Quick Copy. Enables you to copy the data to a table in a target adapter

Flow

Opens the data flow page for creating flows.

Copy

Copies the synonym to the clipboard.

Delete

Deletes the synonym. You are asked to confirm this selection before the synonym is deleted.

Cut

Deletes the synonym and places it on the clipboard.

Rename

Allows you to rename the synonym.

Properties

Displays the properties of the synonym, including physical location, last modified date, description, and privileges.