Managing Data Caches

In this section:

How to:

In Release 82 Version 05, WebFOCUS introduced the use of the Data Values Cache to enhance system performance and shorten response time for variable values retrieved from master file sources identified with a two-part name in the FIND parameter syntax, which is typically seen in procedures that create autoprompt reports, Insight charts, or with output developed using WebFOCUS Designer, as highlighted in the following example. 

.
.
.
-DEFAULTH &WF_SUMMARY='Summary';
-DEFAULTH &WF_TITLE='WebFOCUS Report';
TABLE FILE ibisamp/car
SUM CAR.BODY.DEALER_COST
CAR.BODY.RETAIL_COST
BY CAR.ORIGIN.COUNTRY
BY CAR.COMP.CAR
WHERE CAR.ORIGIN.COUNTRY EQ &COUNTRY.(FIND CAR.ORIGIN.COUNTRY IN ibisamp/CAR |FORMAT=A10,SORT=ASCENDING).COUNTRY:.QUOTEDSTRING;
ON TABLE PCHOLD FORMAT HTML
ON TABLE NOTOTAL
ON TABLE SET CACHELINES 100
ON TABLE SET GRWIDTH 1
ON TABLE SET STYLE *
INCLUDE=IBFS:/FILE/IBI_HTML_DIR/ibi_themes/Warm.sty,$
TYPE=REPORT, TITLETEXT=&WF_TITLE.QUOTEDSTRING, SUMMARY=&WF_SUMMARY.QUOTEDSTRING, ORIENTATION=LANDSCAPE, $
ENDSTYLE
END

-RUN

This cache allocates a portion of Application Server memory to parameter values retrieved in response to these procedures. Once data values are cached, they can be retrieved more quickly when the procedure is run again. The time it takes to rerun a report, chart, or other output is significantly shorter. This enhanced response time improves system performance and the user experience by reducing the time it takes to retrieve data for additional iterations of reports, charts, and other output.

The WebFOCUS User Interface support the Data Values Cache in three different locations.

Reference: Application Caches Settings

Settings on the Application Caches page configure the size and contents of the Data Values Cache on the Application Server, as shown in the following image. These settings establish the default configuration for Data Values Cache operations in your environment. Administrators and users who have the privilege to work with the Session Viewer can temporarily change these default settings using options in the Caching list. For more information, see Managing Session Cache Exceptions.

The Application Caches Settings page
Data Values Max Cache Memory (MB) (IBI_DATAVALUES_CACHE_MAXMEG)

Defines the maximum amount of memory allocated to the Data Values Cache. This cache holds the data source values retrieved from queries to the Server issued by procedures that identify their master file sources with a two-part name in the FIND parameter syntax, along with the IBFS path to the master file and the ID of the user that ran the procedure. The Data Values Cache uses memory on the machine that hosts the WebFOCUS Application Server.

This setting is assigned a value of zero (0), by default, meaning that no memory is allocated to the data values cache, and no data values are cached.

To activate the use of the data values cache, an administrator must assign a number from one (1) to five hundred (500) to this setting. A value of ten (10) megabytes can accommodate the caching requirements for most organizations.

Data Values User Cache Paths (IBI_DATAVALUES_CACHE_INCLUDEPATHS)

Identifies the IBFS paths to those resources whose data source values should only be made available to the individual user who retrieved the data.

The paths defined in this setting should only identify resources that contain data that is subject to DBA or row level security restrictions. For calls to cache resources that are not subject to these restrictions, WebFOCUS uses the paths defined in the Data Values Global Cache Paths (IBI_DATAVALUES_CACHE_GLOBALPATHS) setting.

If no IBFS Paths are defined in this setting, no data source values are cached. This is the default value.

If one or more IBFS Paths are defined in this setting, data values retrieved from all of the resources identified in those paths can be included in the Data Values User Cache for an individual user. If this setting must include multiple paths, they must be separated by a semicolon with no space following, as shown in the following example, /EDA/EDASERVE/retail_samples;/EDA/EDASERVE/ibisamp.

Paths defined in this setting use the IBFS Path format. At a minimum they must include the /EDA IBFS subsystem component followed by the name of the Server node. Shorter, higher-level paths define a broad range of folders and the master files they contain. Longer paths that extend to a lower level define a narrower range of application folders and master files. Paths can extend down to the level of an individual application folder or a single master file.

To include multiple folders or master files in this setting, you can use a short higher level path that includes a range of folders or a series of longer and more detailed paths that identify a very specific group of folders and master files. If this setting must include multiple paths, each one must be separated by a semicolon with no space following, as shown in the following example, /EDA/EDASERVE/retail_samples;/EDA/EDASERVE/ibisamp.

IBFS Paths in this setting use the following structure. Only the first two components of the path are required. You can use as many of the remaining path components as you wish to narrow the focus of the path. 

/EDA/Node/ApplicationFolder/SubFolder1 ... SubFolderN/Resource

where:

EDA

Is the EDA IBFS subsystem. This component is required for all paths. The initial slash before this component is required.

Node

Is the name of the Server node. This component is required for all paths.

ApplicationFolder

Is the name of the application folder that contains the resources from which data will be added to the data values cache.

SubFolder1 ... SubFolderN

Are the names of any folders in the path underneath the application folder that connect to the end point of the path. Include as many folders as are necessary.

Resource

Is the end point of the path. If this is the name of a folder that contains one or more master file resources, then data from all master files in the folder is included in the cache. If this is the name of a master file without the .mas extension, then only data from this specific master file is included in the cache.

Note:

  • Paths to any folders or master files identified in the Data Values User Cache Paths setting take precedence over those identified in the Data Values Global Cache Paths setting. This means that if the same path appears in the Global Cache Path setting and the User Session Class Path setting, then data retrieved from the master files in that path moves into the User Session Cache, instead of the Global Cache.
Data Values Global Cache Paths (IBI_DATAVALUES_CACHE_GLOBALPATHS)

Identifies the IBFS paths to those resources whose data source values can be made available to all users without restriction in the Global Data Values Cache.

The paths defined in this setting should only identify resources that contain data that is not subject to DBA or row-level security restrictions. For calls to cache resources that contain data that is subject to these restrictions, WebFOCUS uses the paths defined in the Data Values User Cache Paths (IBI_DATAVALUES_CACHE_INCLUDEPATHS) setting.

If no IBFS Paths are defined in this setting, then no data source values are included in the Data Values Global Cache. This is the default value.

If one or more IBFS Paths are defined in this setting, then data values retrieved from the resources identified in those paths are included in the Data Values Global Cache.

Paths defined in this setting use the IBFS Path format. At a minimum they must include the /EDA IBFS subsystem component followed by the name of the Server node. Shorter, higher-level paths define a broad range of folders and the master files they contain. Longer paths that extend to a lower level define a narrower range of application folders and master files. Paths can extend down to the level of an individual application folder or a single master file.

To include multiple folders or master files in this setting, you can use a short higher level path that includes a range of folders or a series of longer and more detailed paths that identify a very specific group of folders and master files. If this setting must include multiple paths, each one must be separated by a semicolon with no space following, as shown in the following example, /EDA/EDASERVE/retail_samples;/EDA/EDASERVE/ibisamp.

IBFS Paths in this setting use the following structure. Only the first two components of the path are required. You can use as many of the remaining path components as you wish to narrow the focus of the path. 

/EDA/Node/ApplicationFolder/SubFolder1 ... SubFolderN/Resource

where:

EDA

Is the EDA IBFS subsystem. This component is required for all paths. The initial slash before this component is required.

Node

Is the name of the Server node. This component is required for all paths.

ApplicationFolder

Is the name of the application folder that contains the resources from which data will be added to the data values cache.

SubFolder1 ... SubFolderN

Are the names of any folders in the path underneath the application folder that connect to the end point of the path. Include as many folders as are necessary.

Resource

Is the end point of the path. If this is the name of a folder that contains one or more master file resources, then data from all master files in the folder is included in the cache. If this is the name of a master file without the .mas extension, then only data from this specific master file is included in the cache.

Data Values Cache Paths to Exclude (IBI_DATAVALUES_CACHE_EXCLUDEPATHS)

Identifies the IBFS paths to those folders or master files within the paths defined in the Data Values Global Cache Paths or the Data Values User Cache Paths settings whose values should be not be included in these caches.

If no IBFS Paths are defined in this setting, no data values are excluded from the individual resources or folders defined within the paths in the Data Values Global Cache Paths (IBI_DATAVALUES_CACHE_GLOBALPATHS) or the Data Values User Cache Paths (IBI_DATAVALUES_CACHE_INCLUDEPATHS) settings. This is the default value.

If one or more IBFS Paths are defined in this setting, data values from the individual resources or folders defined within these paths are excluded from the Global Data Values Cache and from individual User Session Caches even though the folder or directory in which they are contained is subject to caching.

Paths defined in this setting use the IBFS Path format. At a minimum they must include the /EDA IBFS subsystem component followed by the name of the Server node. Shorter, higher-level paths define a broad range of folders and the master files they contain. Longer paths that extend to a lower level define a narrower range of application folders and master files. Paths can extend down to the level of an individual application folder or a single master file. Typically this setting requires longer and more specific paths in order to identify the individual folders or master files that must be excluded from the paths defined in the other data cache settings.

When this setting includes multiple paths, each one must be separated by a semicolon with no space following, as shown in the following example, /EDA/EDASERVE/retail_samples;/EDA/EDASERVE/ibisamp.

IBFS Paths in this setting use the following structure. Only the first two components of the path are required. You can use the remaining components to narrow the focus of the path.

/EDA/Node/ApplicationFolder/SubFolder1 ... SubFolderN/ResourceFolder/Resource

where:

EDA

Is the EDA IBFS subsystem. This component is required for all paths. The initial slash before this component is required.

Node

Is the name of the Server node. This component is required for all paths.

ApplicationFolder

Is the name of the application folder that contains the resources whose data will be excluded from the data values cache.

SubFolder1 ... SubFolderN

Are the names of any folders underneath the application folder that connect to the end point of the path. Include as many folders as are necessary.

Resource

Is the end point of the path. If this is the name of a folder that contains one or more master file resources, then data from all master files in the folder is excluded from the cache. If this is the name of a master file without the .mas extension, then only data from this specific master file is excluded from the cache.

Procedure: How to Configure a Data Values Cache

  1. Sign in to WebFOCUS as an Administrator, and then open the Administration Console.
  2. On the Configuration tab, under the Application Settings folder, click Application Caches to display the Application Caches page.
  3. Type the number of megabytes of Application Server memory that should be allocated to the Data Values Cache in the Data Values Cache Max Memory (MB) (IBI_DATAVALUES_CACHE_MAXMEG) setting.
  4. Type the paths to all resources containing data that can be included in the global cache and made available to anyone in the Data Values Global Cache Paths (IBI_DATAVALUES_CACHE_GLOBALPATHS) setting.

    Note: If you must type multiple paths in this setting or in the following settings, end each path with a semi-colon (;) and leave no space between paths.

  5. Type the paths to all resources containing data that can only be included in the individual cache assigned to the user who requested it and unavailable to others in the Data Values User Cache Paths (IBI_DATAVALUES_CACHE_INCLUDEPATHS) setting.
  6. If you must exclude any folders or master files within the paths you identified in either the Data Values Global Cache Paths setting or the Data Values User Cache Paths setting from the data values cache, type the paths to them in the Data Values Cache Paths to Exclude (IBI_DATAVALUES_CACHE_EXCLUDEPATHS) setting.

    This step is optional, and is not typically included in a data values cache configuration.

  7. Click Save.

    Your changes become effective immediately. Do not click Clear Cache.

Working with LRU Cache Statistics

The Lru (Least Recently Used) Cache Statistics page displays current cache usage statistics, as shown in the following image.

The LRU Cache Statistics Page

This page gives you an overview of the current amount of resources allocated to each cache, the number of entries they contain, the memory allocated to them, and the number of objects or events that added data to the cache, retrieved data from it, cleared data from it, or removed old and unused data from it to make way for new data.

Based on these statistics, you can infer the current status of activity in a cache, determine if metadata or data source values are successfully loading into a cache, identify when a cache is operating at peak capacity, and learn if data or metadata was removed or cleared from the cache.

The name LRU Cache is derived from the fact that all caches follow a Least Recently Used rule. Under this rule, resources that have been unused the longest are the first to be cleared from the cache when the addition of new resources would cause the cache to exceed the amount of memory allocated to it.

To open the LRU Cache Statistics page, open the Administration Console, click the Diagnostics tab, and then click LRU Cache Statistics.

Values assigned to the statistics on this page result from events that occurred during cache operations, and the layout of entries on this page follows the organization of the individual caches that comprise the total cache allocation within the Application Server memory, and the structure of records within individual caches.

Understanding the Cache Statistics Page Layout

In this section:

Entries on the LRU Cache Statistics page are organized by rows, one row per cache, as shown in the following image.

Lru Cache Statics showing the rows of statistics for the MetaData cache, the DataValues cache, and the ServerConfig cache.

Understanding Cache Entries

Entries for the following caches appear on the Lru Cache Statistics page, by default.

MetaData

This cache captures metadata values produced from calls to the Server initiated automatically by the client whenever a user selects the Insight mode to run a chart. These calls include requests to the server for information, fonts, and metadata that are shared throughout all Insight Charts. Paths to the resources that place metadata in this cache and the amount of memory to be allocated to this cache are pre-defined, and the cache is therefore activated automatically as soon as a user chooses to work with a chart in Insight mode. The settings that configure this cache and its operations are pre-configured, and are not available to administrators. If you need to reconfigure or adapt this cache, please contact the Customer Support team.

DataValues

This cache captures data values from queries in user-initiated procedures that create reports, charts, portals, dashboards, and other content. Paths to the resources that place data in this cache are defined in the Data Values Global Cache Paths (IBI_DATAVALUES_CACHE_ GLOBALPATHS) setting and in the Data Values User Cache Paths (IBI_DATAVALUES_CACHE_INCLUDEPATHS) setting, located on the Application Caches page of the Administration Console configuration tab.

ServerConfig

This cache captures metadata values used by server operations that are not explicitly related to queries from user-initiated procedures. Paths to the resources that place metadata in this cache and the amount of memory to be allocated to this cache are pre-configured, and are not available to administrators. If you need to reconfigure or adapt this cache, please contact the Customer Support team.

Understanding Cache Statistics

The columns at the top of the page identify the statistics captured for each cache. With the exceptions of Current Entries and Current Memory, these statistics reflect cumulative activity through multiple sessions. They are cleared only when the caches close. The Current Entries and Current Memory statistics are cleared whenever a Clear Cache operation takes place.

For each cache, the following statistics are available:

Max Memory (K)

The maximum amount of memory on the machine hosting the Application Server that is allocated to this cache. For the Data Values Cache, the value in this column is determined by the value assigned to the Data Values Max Cache Memory (MB) (IBI_DATAVALUES_CACHE_MAXMEG) setting for the DataValues row. The maximum amount of memory is pre-configured for the MetaData cache and the ServerConfig cache and is not available to administrators for reconfiguration. Note that the amount of memory allocated to these settings is expressed in megabytes in the Application Caches page setting, but the same amount is expressed in kilobytes in this statistic. Therefore, a value of 50 in the Data Values Max Cache Memory (MB) (IBI_DATAVALUES_CACHE_MAXMEG) setting appears as 50,000 in this entry. This value remains the same throughout a session unless an administrator changes the value in the Data Values Max Cache Memory (MB) (IBI_DATAVALUES_CACHE_MAXMEG) setting.

Current Entries

The number of data entries currently held in the cache. Each entry contains the IBFS path to the resource that contains the data retrieved by the query, followed by each individual data source value, the name of the user that ran the procedure that placed data in the cache, and time stamps for the time it was added to the cache and the most recent time it was retrieved for re-use. This value changes as entries are added to, removed from, or cleared from the cache.

Current Memory (K)

The current amount of memory that is being used by the data values in the cache. The value in this statistic less the value in the Max Memory (K) statistic identifies the amount of memory available for any additional data. This value changes as data entries are added to, removed from, or cleared from the cache.

Gets No-Hit

The number of Get objects that did not successfully retrieve metadata or data values from the cache. This number represents the number of procedures that ran but were unable to retrieve data from the cache because data from the query had not previously been added to it. This value increases whenever a Get object is unable to retrieve data or metadata from the cache. This value does not decrease, and is cumulative across multiple sessions. It returns to zero only when the cache itself is closed.

Gets Hit

The number of Get objects that successfully retrieved values from the cache. This number represents the number of procedures that were run and were able to retrieve data that had previously been added to the cache. This value increases whenever a Get object retrieves data or metadata from the cache. This value does not decrease, and is cumulative across multiple sessions. It returns to zero only when the cache itself is closed.

Put Obj Count

The current number of Put Objects that loaded data values retrieved from a procedure into the cache. This value increases whenever a Put object adds data or metadata to the cache. This value does not decrease, and is cumulative across multiple sessions. It returns to zero only when the cache itself is closed.

Remove Obj Count

The current number of Remove Objects that removed data values from the cache in response to the LRU Prune process. This value increases whenever a Remove object removes data or metadata from the cache. This value does not decrease, and is cumulative across multiple sessions. It returns to zero only when the cache itself is closed.

LruPrune Entries

The number of Least Recently Used entries removed from the cache to make room for metadata or data that must be added to the cache in response to a query issued from a new procedure. Least Recently Used entries are the oldest entries in the cache that have not been requested by a Get object for reuse by a procedure. This value increases whenever data entries must be removed from the cache to clear memory space for new data or metadata from a Put object. This value does not decrease, and is cumulative across multiple sessions. It returns to zero only when the cache itself is closed.

LruPrune Memory

The amount of memory freed when Least Recently Used entries were removed from the cache to make room for metadata or data source values that must be added in response to a query issued from a new procedure. Least Recently Used entries are the oldest entries in the cache that have not been requested by a Get object for reuse by a procedure. Note that this value is cumulative, that is, it represents the amount of memory freed by each LRU remove operation. It does not represent the current amount of free memory.

Clear Count

The number of Clear Cache events initiated by users during the current session. Every time a user clicks the Clear Cache command on the Administration Console menu bar, this value increases. Procedures that include a command to clear the cache also cause this value to increase. This value does not decrease, and is cumulative across multiple sessions. It returns to zero only when the cache itself is closed.

Understanding Cache Group Entries

Cache entries can be grouped by User ID or the path to the master file from which the data in the entry was retrieved. Both groupings give you an idea of the origin of data transferred to the cache, and help you assess how recently data in the cache was assigned to it or retrieved from it.

The Group by subsection can appear directly underneath the row for each active cache, as shown in the following image. This subsection opens when user activity causes values to be placed in it during the current session, and remains visible until a User clears the cache or an Application Server shutdown closes the cache.

The Group by statistics entry under the DataValues row.

This subsection displays additional summary cache statistics grouped by Scope, Path, or User. When grouped by Scope, statistics in this section identify summary statistics for all users and paths within the cache. When grouped by User, statistics in this section identify summary statistics for each individual User who added data or metadata to the cache. The section lists one row of statistics for each User. When grouped by Path, statistics in this section identify summary statistics for each individual resource from which data or metadata was added to the cache. The section lists one row of statistics for each Path.

Using these grouping options, you can identify the IBFS paths that were retrieved by queries in this session, and assess the statistics in this subsection by session, user, or query as represented by an IBFS path.

Each option identifies the following group statistics.

Count

The number of metadata or data source entries currently held in the cache for each user or path, depending upon the selected Group By option. If the Group By option is set to Global, the value will match that of the whole cache.

MemoryK

The amount of memory occupied by the metadata or data source entries currently held in the cache for each session, user, or path, depending upon the selected Group By option. If the Group By option is set to Global, this value will match that of the whole cache.

Oldest Age

The number of minutes that the oldest metadata or data source entry has been held in the cache for each user or path.

Last Ref

The number of minutes that have elapsed after the most recent call for data from the cache.

Caches begin to display statistics when they are initiated by system operations, and close their display of statistics when they are closed by an Application Server shutdown.

Statistics on the page are not logged or stored. With the exception of statistics in the Group By subsection and the Current Entries and Current Memory statistics, they are cumulative and persist from session to session. Current statistics serve as a snapshot of the current state of the caches. Cumulative statistics serve as an ongoing record of the activity that took place over the life of each cache.

Using the Session Viewer Caching List

The main page of the Session Viewer contains the Caching list. Options in this list override the default configuration for the caching operations that are defined on the Application Caches page of the Administration Console Configuration tab, as shown in the following image.

The Session Viewer main page with the Caching list displayed.

The options in this list affect caching operations for the current session only. They enable administrators and developers to temporarily suspend caching for application development sessions.

When the next session starts, the Default Scope option is automatically reassigned to the list. This option resets the cache to a persistent status for the new session, making it ready for use in a production session.

The list contains the following options.

Default Scope

When this option, also known as the User Scope, is selected, data source values remain in the cache and Lru Cache statistics are not cleared when a session ends. This is the default option for the Caching list. It allows cached data and cache statistics to persist from session to session.

This option displays the Default Scope label when it is selected and the Set Default Scope label when it is not selected.

Session Scope

Displays the Session Scope label when it is selected and the Set Session Scope label when cleared.

No Caching

Suspends caching in the current session. After you select this option caching is suspended within the session until you select Set Session Scope or Set Default Scope. This option displays the No Caching label when it is selected and the Turn Off Caching label when cleared.

Refresh Cache

Refreshes the cache and immediately restores the option that was previously selected.