This topic explains how to use the FTP operator and describes the configuration settings you can make in the operator's Properties view.
The Spotfire Streaming FTP operator is a global Java operator that provides a way to run FTP commands through a Java-based FTP client. This is especially useful for the execution of programmatic file transfers.
Configure the operator to send either a single StreamBase FTP client command, or a single protocol-level FTP command, to a single target FTP server. The operator's command is sent on receipt of a tuple with any schema on the operator's input port. To send more than one FTP command, use more than one instance of the operator in your application, or use a tuple expression as the Command property.
Specify the command to run using the Command field of the Operator Properties tab. The command you select may require parameters or arguments. If so, enter those on other tabs of the Properties view.
The FTP operator creates a single FTP connection for each command it sends; therefore
a command such as CWD does not do anything, since the
user is logged out before any other commands can be executed. When the next command
is executed, the user is logged back in and his CWD will be his home directory.
Select the FTP from the Insert an Operator or Adapter dialog, which you invoke with one of the following methods:
-
Drag the Adapters, Java Operators token from the Operators and Adapters drawer of the Palette view to the canvas.
-
Click in the canvas where you want to place the operator, and invoke the keyboard shortcut
O V -
From the top-level menu, invoke >>.
From the Insert an Operator or Adapter dialog that opens, select FTP and double-click or press OK.
This section describes the properties you can set for an FTP operator, using the various tabs of the Properties view in StreamBase Studio.
This section describes the properties on the General tab in the Properties view for the FTP operator.
Name: Use this required field to specify or change the name of this instance of this component. The name must be unique within the current EventFlow module. The name can contain alphanumeric characters, underscores, and escaped special characters. Special characters can be escaped as described in Identifier Naming Rules. The first character must be alphabetic or an underscore.
Operator: A read-only field that shows the formal name of the operator.
Class name: Shows the fully qualified class name that implements the functionality of this operator. If you need to reference this class name elsewhere in your application, you can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.
Start options: This field provides a link to the Cluster Aware tab, where you configure the conditions under which this operator starts.
Enable Error Output Port: Select this checkbox to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports to learn about Error Ports.
Description: Optionally, enter text to briefly describe the purpose and function of the component. In the EventFlow Editor canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.
This section describes the properties on the Operator Properties tab in the Properties view for the FTP operator.
- Command
-
This field is central to the operation of the operator. Enter a StreamBase expression in the Command field to determine which FTP client command to send with each tuple. The available commands are:
APPEND
PUT
GET
FREE_FORMAll commands except
FREE_FORMcorrespond to a traditional FTP client command. TheFREE_FORMcommand lets you type a protocol-level FTP command, such asDELEorMKD, with any necessary arguments.Once you select an FTP client command with the Command field, you need to provide further information on one of the other tabs of the Properties view.
- Asynchronous Operation
-
This determines if the file transfer operations should be performed asynchronously. NOTE: If operations are performed asynchronously the output tuple of the operation will not be sent until the operation is complete but the system will continue processing the next command tuple.
- Async worker threads
-
This is the number of worker threads to allocate for command execution. The default value of 1 ensures that this operator's commands are executed serially. A positive integer will allocate a fixed number of threads. Setting this property to 0 will allocate an unlimited number of worker threads.
- Enable Progress Updates
-
If enabled the operator will emit progress update tuples on the progress port.
- Server Hostname
-
Enter a hostname or IP address in this field in the form of an expression. This should resolve to the IP address of the FTP server to which you wish to connect.
- Server Port
-
The default value for this property is 21, the default FTP control port. Enter the number of the FTP server's control port in the form of an expression.
- Username
-
Enter a user name in this field in the form of an expression. If the server you are connecting to supports anonymous FTP, be sure to enter the appropriate anonymous username for that server.
- Password
-
Enter a password in this field in the form of an expression. If the server you are connecting to supports anonymous FTP, be sure to enter an appropriate anonymous password for that server.
- Log Level
-
Controls the level of verbosity the adapter uses to issue informational traces to the console. This setting is independent of the containing application's overall log level. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE.
This section describes the properties on the Connection Options tab in the Properties view for the FTP operator. This tab allows you to configure certain parameters regarding the FTP connection.
- Local Passive Mode
-
Check this box to send a PASV command to the server before every data transfer. If left unchecked data transfers will occur in Local Active Mode. Local Passive Mode causes the server to open a port to which the client will connect for data transfer; in Active mode the opposite is ture. Passive mode is generally more likely to be compatible with most firewall configurations.
- Transfer Timeout
-
The default value of 0 indicates an unlimited timeout. Any positive integer will cause all
GETcommands to timeout after being idle for that number of seconds.PUTcommands will not timeout, regardless of this value. - Retry Count
-
The number of times to retry should there be a connection issue or an error with the file transfer. 0 is considered unlimited.
- Poll Period
-
The time, in seconds, to wait between retry attempts.
- Buffer Size
-
The internal data size of the buffer used to for buffered data streams.
This section describes the properties on the Transfer Options tab in the Properties view for the FTP operator. If the expression you entered in the Command field of the Operator Properties tab evaluates to a StreamBase FTP client command, then you need to fill in the fields in this tab.
- Local File
-
Enter an expression that evaluates to the path name of the local file you wish to use in the transfer (for a GET this need not be an existing file; however all directories listed in the filename must exist). If you enter a relative path name, it will be relative to the project directory of the StreamBase application in which the operator is running.
- Local File Is Resource
-
Enter an expression that evaluates to true or false which determines if the Local File should be loaded as a resource file (true) or as a fully qualified path (false).
- Remote File
-
Enter an expression that evaluates to the name of the remote file you wish to use in the transfer. This can either be an absolute path name or relative to the home directory of the user you specified in the Username field in the Operator Properties tab.
- File Type
-
Choose an FTP File Type to be used in the file transfer from the dropdown menu; be sure that the file type you choose is supported by the FTP server you are connecting to (both ASCII and BINARY should be supported by all FTP servers). The supported file types are:
ASCII
BINARY
EBCDIC
LOCAL
This section describes the properties on the Protocol
Command Options tab in the Properties view for the FTP operator. The fields
in this tab will be used when the Command field on
the Operator Properties tab evaluates to either the empty string or the value
FREE_FORM. For any given tuple, if the Command field does not evaluate to either of these values, then
the fields on this tab will not be evaluated.
- FTP Command
-
Enter the name of the FTP command you wish to execute, as if you were typing a protocol-level FTP command (such as
DELEorMKD).Note
Any FTP commands that do not return replies on the control channel are not guaranteed to be supported by this function. If you wish to transfer data, use a StreamBase FTP client command, such as
PUTorGET. - FTP Params
-
As a single String, enter the argument to the FTP command. If no argument is required, this may be left empty. If the command requires or supports multiple arguments, separate them with whitespace. For example:
"TYPE A T"
Use the settings in this tab to enable this operator or adapter for runtime start and stop conditions in a multi-node cluster. During initial development of the fragment that contains this operator or adapter, and for maximum compatibility with releases before 10.5.0, leave the Cluster start policy control in its default setting, Start with module.
Cluster awareness is an advanced topic that requires an understanding of StreamBase Runtime architecture features, including clusters, quorums, availability zones, and partitions. See Cluster Awareness Tab Settings on the Using Cluster Awareness page for instructions on configuring this tab.
Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.
Caution
Concurrency settings are not suitable for every application, and using these settings requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.
The FTP operator has one input port that accepts a tuple with any schema. The operator runs the appropriate command on the receipt of a tuple, based on its contents.
You can also add an optional Error Output port, which outputs a StreamBase error tuple for any error thrown by the operator, as described in General Tab.
The FTP operator has one mandatory output port, which serves as its status port. For each tuple that comes in on the input port, exactly 1 tuple will exit this port. The schema for this port is pre-set by the operator and is described in the following table:
| Field Name | Field Type | Field Description |
|---|---|---|
type
|
string |
Describes the type of status being emitted. Currently the only value
supported is COMMAND.
|
action
|
string |
The action that triggered this status tuple. One of:
|
object
|
string |
The Server response to the FTP command. This will include the three digit
status code as the first three non-whitespace characters. If this is
null, that means either the server did not
respond or a connection could not be established. For a protocol command
such as SIZE, which causes the server to issue
a response on the control connection, this will contain the response with
the relevant information.
|
message
|
string | A message either confirming success or with some information as to why a command failed. This has no well-defined format and is for manual debugging use only. |
inputTuple
|
tuple | The input tuple which caused this command to be issued. |
The FTP operator has an optional progress port. If the progress port is enabled and a file transfer is performed, one or more tuples will exit from the progress port with transfer information. The schema for this port is pre-set by the operator and is described in the following table:
| Field Name | Field Type | Field Description |
|---|---|---|
totalBytesTransferred
|
long | The total bytes that have been transferred so far for this operation. |
bytesTransferred
|
int | The number of bytes that have been transferred since the last progress update (or since the start if this is the first progress tuple). |
inputTuple
|
tuple | The input tuple which caused this command to be issued. |