Creating and Running Test Suites

Creating Test Suites

StreamBase supports batch execution of tests by means of StreamBase test suites. StreamBase test suites are folders that contain tests and resources for the tests. The folders are just like any other folder in the file system except for the way they can be interpreted by StreamBase.

Applications associated with the tests in a test suite can be in the test suite folder, in the parent folder of the test suite folder, or specified on the sbtest command line. All resources required for a test must be in the same test suite folder as the test that requires them.

You can configure and run test suites in StreamBase Studio, or by means of the sbtest command.

Configuring and Running a Test Suite in StreamBase Studio

When you have existing tests that you want to run as a group, you can create a folder anywhere in the file system, copy the tests into the folder, and then import the folder into the StreamBase project containing the application to be tested. You can also create a new folder in a StreamBase project and import tests and resources to make it a test suite.

To specify batch execution of tests, specify their folder when you create the test in the Run dialog, as shown:

In this illustration the test suite is /sample_firstapp/test_folder1.

When you run the test suite you can select an option to use a new StreamBase Server instance for each test, as shown:

You can improve performance by running all the tests in a single server instance, but that option can cause issues resulting from side effects of the tests. Also,if you select this option in a case where tests use different applications each application runs in its own server instance and you will not see a performance improvement by selecting this option.

When you run a test suite containing one or more tests that specify Default as the target application, StreamBase Studio prompts you to choose an application.

If a test explicitly specifies an application, StreamBase Studio looks for the application in the test folder and in its parent folder. Then, if the application is not found, Studio prompts you to choose among several (if more than one with the same name is found).

StreamBase automatically starts servers as required to run the applications being tested.

Example of Using Test Suites in StreamBase Studio

This example shows a test suite for firstapp, which is the StreamBase sample project is used in the Getting Started Tutorial. The application is as follows:

The following directory structure shows sample_firstapp, which contains firstapp.sbapp and test_folder1, a test suite that has been added to the original project. This test suite holds three test files and their associated resources for firstapp.sbapp.

The tests in this test suite have been created with the target application set to Default. See test1.sbtest for an example:

<?xml version="1.0" encoding="UTF-8"?>
<sbtest:Suite xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" 
       xmlns:sbtest="http://www.streambase.com/schemas/sbtest">
 <Tests UseDefaultTarget="true">
   <Outputs URI="test1-AllTheRest.csv" DequeueTimeoutSeconds="20">
     <Stream StreamName="AllTheRest">
       <Fields Name="symbol" DataTypeName="string"/>
       <Fields Name="quantity" DataTypeName="int"/>
     </Stream>
   </Outputs>
   <Outputs URI="test1-BigTrades.csv" DequeueTimeoutSeconds="20">
     <Stream StreamName="BigTrades">
       <Fields Name="symbol" DataTypeName="string"/>
       <Fields Name="quantity" DataTypeName="int"/>
     </Stream>
   </Outputs>
   <Inputs URI="test1-feedsim.sbfs"/>
 </Tests>
</sbtest:Suite>

To run the tests in test_folder_1:

Select the test suite (test_folder_1) and open the Run Dialog.
Select StreamBase Test as the kind of configuration you want to specify.
Type a name for the configuration and select the option to run all the tests in test_folder_1, as shown in the previous illustration.
Click Run.
When the dialog prompts you to select the application to run, click firstapp.sbapp.
Watch the Console and the JUnit view to see test execution progress and final results of the tests.
- The JUnit view provides a quick view of success or failure:
- The Console shows progress while each test is running:
- The Console shows additional information for tests that fail:

Running a Test Suite from the Command Line

Using the sbtest command to run a test suite is similar to running just one test from the command line. To run the test suite shown in the preceding test_folder1 example, issue the following command from the parent folder for the test suite:

sbtest test_folder1

Options to the sbtest command can be used with a test suite just as with a StreamBase test. For example, if you want to run a test suite against an application that is not in the test suite folder nor in the parent folder of the test suite, you can specify the application by using the -a option with sbtest.

sbtest -a../second.sbapp test_folder1

Notes About StreamBase Test Suites

When StreamBase runs a test suite, it executes all files with an .sbtest extension found recursively in the test suite directory.
StreamBase runs tests in the order they appear on the command line (when executing with sbtest) and in alphabetical order within a test suite folder.
If StreamBase Studio cannot find the application to be tested in the test suite or in the parent folder of the test suite, it does the following:
- Searches the current workspace for any and all application files with the name specified
- Displays a list of any applications that it finds
- Prompts you to select the file against which to run the test suite
You can tell StreamBase to share server instances across tests. This option is available as a check box in the Main tab of the Run dialog for test configuration, as shown in the preceding StreamBase Studio example. The option is available when running sbtest on the command line with the –s option. When the check box is not selected (or when –s appears on the command line), tests that run against the same application share a server instance.
All tests in StreamBase test suites must share the same server configuration (.sbconf) file.
To reduce the time required to run a test suite save applications to be tested as StreamBase archives (.sbar) file.