Contents
This sample demonstrates how to implement a custom file reader to read non-standard, proprietary, or binary files as the source of a stream of input tuples for feed simulations. Your Java file-reading code must extend one of the classes in the com.streambase.sb.feedsim package in the StreamBase Client Library. StreamBase provides a way to use your custom class instead of its internal CSV-reading code in conjunction with the Feed Simulation Editor's Data File option.
For more information about configuring Feed Simulations to use custom readers, see Feed Simulation with Custom File Reader.
This sample demonstrates and provides Java source code for two implementations of custom file readers for feed simulations:
-
A custom CSV file reader
-
A custom tuple reader
The custom CSV file reader implementation forces all characters to lowercase if the first character in the file is a pound
sign (#), forcing all characters to uppercase otherwise. This sample's implementation class, MyFeedSimCSVPlugin
, extends the required FeedSimCSVInputStream
interface. Because only the read()
method needs to be overridden, this implementation works for any schema CSV file.
The custom Tuple reader implementation parses lines in a sample file that contains web server log data, stored in the NCSA
Common Log Format. The getSchema()
method returns a schema that assigns appropriate field names and types to the log data, and each readTuple()
call returns the Tuple for each successive line of log data.
See Feed Simulation with Custom File Reader for a discussion of the classes in the com.streambase.sb.feedsim package in the StreamBase Client Library.
In StreamBase Studio, import this sample with the following steps:
-
From the top-level menu, select
> . -
Enter
custom reader
in the Search field to narrow the list of options. -
Select the Feed Simulation Custom Reader sample from the Extending StreamBase category.
-
Click
.
StreamBase Studio creates a project for the sample in your current Studio workspace.
The steps to run this sample in Studio are as follows:
-
In the Project Explorer view, open this sample's folder.
Keep an eye on the bottom right status bar of the Studio window. Make sure any
Updating
,Downloading
,Building
, orRebuild project
messages finish before you proceed. -
Open the
src/main/eventflow/
folder.packageName
-
Double-click to open the
MyFeedSimCSVPlugin.sbapp
module. Make sure the module is the currently active tab in the EventFlow Editor. -
Click the
Run button. This opens the SB Test/Debug perspective and starts the module.
-
Wait for the Waiting for fragment to initialize message to clear.
-
The Feed Simulation adapter in this module is configured to be self-running. Thus, soon after the module starts, look in the Output Streams view for data emitted on the
CsvPluginOutputStream
stream.Notice that the stock symbols are all uppercase as sent to the StreamBase application. In the originating CSV file, the symbols are mixed case.
-
To reread the same data file, in the Manual Input view, select the
FeedSimControl
stream, enter the stringstart
in the command field, and click . -
When done, press F9 or click the
Terminate EventFlow Fragment button.
-
Next, open the second module in this sample,
MyFeedSimTuplePlugin.sbapp
. -
Make sure the new module is the currently active tab in the EventFlow Editor, then click the
Run button to start the module.
-
The second sample is also self-running. In the Output Streams view, look for data emitted on the
TuplePluginOutputStream
. -
To re-read the same data file, in the Manual Input view, select the
FeedSimControl
stream, enter the stringstart
in the command field, and click . -
When done, press F9 or click the
Terminate EventFlow Fragment button.
This sample includes the following files that demonstrate a custom CSV file reader plug-in.
File | Description |
---|---|
MyFeedSimCSVPlugin.sbapp |
A simple, pass-through EventFlow application that demonstrates the custom CSV file reader plug-in. |
feedSimForCsvPlugin.sbfs |
A feed simulation file set up to read timestamp-simple.csv as its input data source, using the custom file reader MyFeedSimCSVPlugin .
|
timestamp-simple.csv |
A simple three-field CSV file of stock symbol, price, and timestamp data. |
MyFeedSimCSVPlugin.java |
In the src/main/java/ directory, the Java source file for the MyFeedSimCSVPlugin class.
|
This sample also includes the following files that demonstrate a custom tuple reader plug-in.
File | Description |
---|---|
MyFeedSimTuplePlugin.sbapp |
A simple, pass-through EventFlow application that demonstrates a custom tuple reader plug-in. |
feedSimForTuplePlugin.sbfs |
A feed simulation file set up to read log-data.txt as its input data source, using the custom file reader MyFeedSimTuplePlugin .
|
log-data.txt |
A text file containing a few lines of sample web server log data. |
MyFeedSimTuplePlugin.java |
In the src/main/java/ directory, the Java source file for the MyFeedSimTuplePlugin class.
|
When you load the sample into StreamBase Studio, Studio copies the sample project's files to your Studio workspace, which is normally part of your home directory, with full access rights.
Important
Load this sample in StreamBase Studio, and thereafter use the Studio workspace copy of the sample to run and test it, even when running from the command prompt.
Using the workspace copy of the sample avoids permission problems. The default workspace location for this sample is:
studio-workspace
/sample_feedsim-plugin
See Default Installation Directories for the default location of studio-workspace
on your system.