Feed Simulation Custom Reader Sample

Contents

Sample Overview
Essential Prerequisite Setup
This Sample's Files
Running This Sample in StreamBase Studio
Running This Sample in Terminal Windows
Importing This Sample into StreamBase Studio
Sample Location

This sample demonstrates how to implement a custom file reader to read non-standard, proprietary, or binary files as the source of a stream of input tuples for feed simulations. Your Java file reading code must extend one of the classes in the com.streambase.sb.feedsim package in the StreamBase Client Library. StreamBase provides a way to use your custom class instead of its internal CSV-reading code in conjunction with the Feed Simulation Editor's Data File option.

For more information about configuring Feed Simulations to use custom readers, see Feed Simulation with Custom File Reader.

Sample Overview

This sample demonstrates and provides Java source code for two implementations of custom file readers for feed simulations:

  • A custom CSV file reader

  • A tuple reader

The custom CSV file reader implementation forces all characters to lowercase if the first character in the file is a hash sign (#), forcing all characters to uppercase otherwise. This sample's implementation class, MyFeedSimCSVPlugin, extends the required FeedSimCSVInputStream interface. Because only the read() method needs to be overridden, this implementation works for any schema CSV file.

The custom Tuple reader implementation incorporates the schema of the sample CSV file and delivers to the StreamBase application the CSV stock price in each incoming tuple divided by 10. This sample's implementation class, MyFeedSimTuplePlugin, extends the required FeedSimTupleInputStream interface.

See Feed Simulation with Custom File Reader for a discussion of the classes in the com.streambase.sb.feedsim package in the StreamBase Client Library.

Essential Prerequisite Setup

This sample cannot load and run until you have completed the following essential setup: you must place the class files containing your feed simulation custom reader on the classpath of StreamBase Studio's JVM. The classpath must be configured before you start Studio. Follow these steps:

  1. Start Studio and load this sample as described above. Studio automatically and silently builds the Java source files in the java-src directory and places the resulting class files in the java-bin directory. (Studio does not display the java-bin directory in the Package Explorer view.)

  2. Exit Studio and set the environment variable STREAMBASE_FEEDSIM_PLUGIN_CLASSPATH to the path to the java-bin subdirectory of this sample's directory in your Studio workspace. For example, for Windows: 

    set STREAMBASE_FEEDSIM_PLUGIN_CLASSPATH=
  C:\Users\sbuser\Documents\StreamBase Studio n.m Workspace\sample_feedsim-plugin\java-bin

    For Linux and Bash: 

    export STREAMBASE_FEEDSIM_PLUGIN_CLASSPATH= \
  /home/sbuser/StreamBase Studio n.m Workspace/sample_feedsim-plugin/java-bin

  3. Restart Studio. If you set the environment variable in a terminal window or StreamBase Command Prompt, then start Studio from the same window with the sbstudio command.

The Custom Reader button in the Data File Options dialog shows an error message and cannot locate the sample's custom classes until you configure the classpath as above, or with the alternative Java system property streambase.feedsim.plugin-classpath.

This Sample's Files

This sample includes the following files:

File Description
MyFeedSimPlugin.sbapp A simple, pass-through EventFlow application.
timestamp-simple.csv A simple three-field CSV file of stock symbol, price, and timestamp data.
simple-csv.sbfs A feed simulation file set up to read timestamp-simple.csv as its input data source, using the custom file reader MyFeedSimCSVPlugin.
simple-tuple.sbfs A feed simulation file set up to read timestamp-simple.csv as its input data source, using the custom file reader MyFeedSimTuplePlugin.
MyFeedSimCSVPlugin.java In the java-src directory, Java source file for the MyFeedSimCSVPlugin class.
MyFeedSimTuplePlugin.java In the java-src directory, Java source file for the MyFeedSimTuplePlugin class.

Running This Sample in StreamBase Studio

The steps to run this sample in Studio are as follows:

  1. Make sure the classpath is set as described in Essential Prerequisite Setup.

  2. In the Package Explorer, double-click to open the MyFeedSimPlugin.sbapp application.

  3. Make sure the application is the currently active tab in the EventFlow Editor, then click the Run button. This opens the SB Test/Debug perspective and starts the application.

  4. In the Feed Simulations tab, select simple-csv.sbfs and click Run.

  5. In the Application Output view, look for data emitted by the StreamBase application similar to the following: 

    symbol=IBM, price=89.57, date=2005-04-01 16:00:02.000-0500
symbol=IBM, price=89.0, date=2005-04-01 16:00:03.000-0500
symbol=IBM, price=88.44, date=2005-04-01 16:00:04.000-0500
symbol=IBM, price=36.0, date=2005-04-01 16:00:05.000-0500
symbol=NYT, price=35.8, date=2005-04-01 16:00:15.000-0500
symbol=NYT, price=35.77, date=2005-04-01 16:00:20.000-0500
symbol=NYT, price=35.78, date=2005-04-01 16:00:21.000-0500
symbol=DELL, price=38.03, date=2005-04-01 16:00:22.000-0500

    Notice that the stock symbols are all uppercase as sent to the StreamBase application. In the CSV file, the symbols are mixed case.

    This feed simulation uses timestamp rate emission, so the first four tuples are emitted with one second between them, the fifth tuple is emitted ten seconds after the fourth, and so on.

  6. In the Feed Simulations tab, click Stop, then select simple-tuple.sbfs and click Run.

  7. In the Application Output view, look for data emitted similar to the following: 

    symbol=Ibm, price=8.956999999999999, date=2005-04-01 16:00:02.000-0500
symbol=Ibm, price=8.9, date=2005-04-01 16:00:03.000-0500
symbol=Ibm, price=8.844, date=2005-04-01 16:00:04.000-0500
symbol=Ibm, price=3.6, date=2005-04-01 16:00:05.000-0500
symbol=Nyt, price=3.5799999999999996, date=2005-04-01 16:00:15.000-0500
symbol=Nyt, price=3.5770000000000004, date=2005-04-01 16:00:20.000-0500
symbol=Nyt, price=3.5780000000000003, date=2005-04-01 16:00:21.000-0500
symbol=Dell, price=3.803, date=2005-04-01 16:00:22.000-0500

    Notice that the price field is a tenth of the value in the CSV file. Notice also that the symbols are in mixed case, just as they are in the CSV file.

    This feed simulation also uses timestamp rate emission, so the tuples emission rate should be paced the same as the same as seen in step 4 above.

  8. When done, press F9 or click the Stop Running Application button.

Running This Sample in Terminal Windows

Follow these steps to run this sample in terminal windows.

This section describes how to run this sample in UNIX terminal windows or Windows command prompt windows. On Windows, be sure to use the StreamBase Command Prompt from the Start menu as described in the Test/Debug Guide, not the default command prompt.

  1. Make sure the classpath is set as described in Essential Prerequisite Setup.

  2. Open three terminal windows on UNIX, or three StreamBase Command Prompts on Windows. In each window, navigate to your workspace copy of the sample.

  3. In window 1, start StreamBase Server running the sample application.

    sbd MyFeedSimPlugin.sbapp

  4. In window 2, type:

    sbc dequeue

    This window is to display tuples dequeued from the application's output ports.

  5. In window 3, type:

    sbfeedsim simple-csv.sbfs

    Look for tuples in window 2 like those described in step 5 of the previous section.

  6. In window 3, type:

    sbfeedsim simple-tuple.sbfs

    Look for tuples in window 2 like those described in step 7 of the previous section.

  7. In window 2, type Ctrl+C to terminate the dequeuer. Then type the following command to shut down the server:

    sbadmin shutdown

Back to top^

Importing This Sample into StreamBase Studio

In StreamBase Studio, import this sample with the following steps:

  • From the top menu, select File → Load StreamBase Sample.

  • Select feedsim-plugin from the Extending StreamBase category.

  • Click OK.

StreamBase Studio creates a project for the sample in your current Studio workspace.

Sample Location

When you load the sample into StreamBase Studio, Studio copies the sample's files to a project folder in your Studio workspace directory. Because the Studio workspace location is normally part of your home directory, you have full access rights there by default.

Important

Load this sample in StreamBase Studio, and thereafter use the Studio workspace copy of the sample to run and test it, even when running from the command prompt.

Using the workspace copy of the sample avoids the permission problems that can occur when trying to work with the initially installed location of the sample. The default workspace location for this sample is: 

studio-workspace/sample_feedsim-plugin

See Default Installation Directories for the location of studio-workspace on your system.

In the default TIBCO StreamBase installation, this sample's files are initially installed in: 

streambase-install-dir/sample/feedsim-plugin

See Default Installation Directories for the location of streambase-install-dir on your system. This location may require administrator privileges for write access, depending on your platform.