Web Reader Input Adapter Samples

Contents

About The Samples
Importing This Sample into StreamBase Studio
Running The Web Reader Samples in StreamBase Studio
Sample Location

About The Samples

In this sample, the Spotfire Streaming Web Reader input adapter retrieves the contents of a web page by sending an HTTP GET or HTTP POST request to a specified URL. The calls made by the adapter demonstrate how to get data from REST, JSONP, and SOAP web endpoints.

Thus, this sample retrieves an XML file served as a web page on a particular URL, then uses the XML to Tuple operator to parse the XML and convert it to tuple format. In the context of a larger application, this sample's ParsedWebPage output stream would be ready to be processed further by downstream components.

Importing This Sample into StreamBase Studio

In StreamBase® Studio, import this sample with the following steps:

  • From the top-level menu, select File>Import Samples and Community Content.

  • Enter webr to narrow the list of options.

  • Select Web Reader input adapter from the StreamBase Standard Adapters category.

  • Click Import Now.

StreamBase® Studio creates a single project containing the sample files.

Running The Web Reader Samples in StreamBase Studio

HTTP GET

In this sample, the Spotfire Streaming Web Reader input adapter retrieves the contents of a web page by sending an HTTP GET request to the specified URL in the adapter HTTP URL property.

  1. In the Project Explorer view, open the sample you just loaded.

    If you see red marks on a project folder, wait a moment for the project to load its features.

    If the red marks do not resolve themselves after a minute, select the project, right-click, and select Maven>Update Project from the context menu.

  2. Open the src/main/eventflow/packageName folder.

  3. Open the webreader.sbapp file and click the Run button. This opens the SB Test/Debug perspective and starts the module.

  4. In the Manual Input view, click Send Data to send the default null tuple.

  5. In the Output Streams view, observe tuples emitted on the RawWebPage and ParsedWebPage output streams.

  6. Select the tuple emitted from the ParsedWebPage stream, then use the triangles next to the data fields displayed to analyze and understand their structure.

  7. Press F9 or click the Terminate EventFlow Fragment button.

  8. Back on the EventFlow Editor canvas, double-click the WebReader component to edit its properties. Clear the Enable Control Port control, and set the Poll Frequency control to 10,000. Now save the module and run it again.

  9. The application now retrieves the contents of the specified URL every 10 seconds.

  10. When done, press F9 or click the Terminate EventFlow Fragment button.

JSONP

In this sample, the Spotfire Streaming Web Reader input adapter sends a JSONP request to the server using the URLParams and HTTPHeaders input variables. See the defaults map operator on the canvas of this sample to show how to set up the different types of fields required for this operation. The TupleToJSON operator is used to convert the input data into a JSON string to send to the server.

  1. Continuing in this sample project, open the webreaderJSONP.sbapp file.

  2. Click the Run button. This opens the SB Test/Debug perspective and starts the module.

  3. In the Manual Input view, enter some values into the fields and click Send Data to send the tuple.

  4. In the Output Streams view, observe tuples emitted on the RawResponse and ParsedResponse output streams.

  5. Press F9 or click the Terminate EventFlow Fragment button.

POST JSON

In this sample, the Spotfire Streaming Web Reader input adapter sends a JSON request to the server via a POST request using the PostData and HTTPHeaders input variables. See the PostDefaults map operator on the canvas of this sample to show how to set up the different types of fields required for this operation. The TupleToJSON operator is used to convert the input data into a JSON string to send to the server.

  1. Continuing in this sample project, open the webreaderPostJSON.sbapp file.

  2. Click the Run button. This opens the SB Test/Debug perspective and starts the module.

  3. In the Manual Input view, enter some values into the fields and click Send Data to send the tuple.

  4. In the Output Streams view, observe tuples emitted on the RawResponse and ParsedResponse output streams.

  5. Press F9 or click the Terminate EventFlow Fragment button.

POST XML

In this sample, the Spotfire Streaming Web Reader input adapter sends a XML request to the server via a POST request using the URLParams input variable. See the PostDefaults map operator on the canvas of this sample to show how to set up the different types of fields required for this operation. The XMLToTuple operator is used to convert the output string data into a Tuple.

  1. Continuing in this sample project, open the webreaderPostXML.sbapp file.

  2. Click the Run button. This opens the SB Test/Debug perspective and starts the module.

  3. In the Manual Input view, enter some values into the fields and click Send Data to send the tuple.

  4. In the Output Streams view, observe tuples emitted on the RawResponse and ParsedResponse output streams.

  5. Press F9 or click the Terminate EventFlow Fragment button.

SOAP

In this sample, the Web Reader input adapter sends a SOAP request to the server via a POST request using the PostData and HTTPHeaders input variables. See the SOAPDefaults map operator on the canvas of this sample to show how to set up the different types of fields required for this operation. The XMLToTuple operator is used to convert the output string data into a Tuple.

  1. Continuing in this sample project, open the webreaderSOAP.sbapp file.

  2. Click the Run button. This opens the SB Test/Debug perspective and starts the module.

  3. In the Manual Input view, enter some values into the fields and click Send Data to send the tuple.

  4. In the Output Streams view, observe tuples emitted on the RawResponse and ParsedResponse output streams.

  5. Press F9 or click the Terminate EventFlow Fragment button.

Sample Location

When you load the sample into StreamBase® Studio, Studio copies the sample project's files to your Studio workspace, which is normally part of your home directory, with full access rights.

Important

Load this sample in StreamBase® Studio, and thereafter use the Studio workspace copy of the sample to run and test it, even when running from the command prompt.

Using the workspace copy of the sample avoids permission problems. The default workspace location for this sample is: 

studio-workspace/sample_adapter_embedded_webreader

See Default Installation Directories for the default location of studio-workspace on your system.