This topic describes the features and syntax of the StreamBase expression language. You can use StreamBase expressions in the Properties view of most EventFlow operators, and can use them as part of statements in StreamSQL applications.
Reference Pages
StreamBase Environment Variables |
StreamBase Java Properties |
StreamBase Expression Language Functions |
StreamBase provides two types of built-in functions in the expression language:
- Simple Functions
-
Simple functions operate on a single tuple field at a time. Most functions in the StreamBase expression language are simple functions. You can use simple functions in expressions for any StreamBase operator (except the Heartbeat, Metronome, and Union operators, which do not accept expressions). All infix operators (
+
,-
,*
,/
,%
,&&
,||
) are simple functions. - Aggregate Functions
-
Aggregate functions are used on sets of data to return a single result. Aggregate functions evaluate columns of data from windows or tables. In EventFlow applications, aggregate functions can only be used:
-
In aggregate expressions in Aggregate operators
-
In output expressions in Query Operators that perform Read or Delete operations
In StreamSQL applications, aggregate functions can be used in SELECT statements related to aggregate or query read or delete operations.
-
For a complete, categorized list of StreamBase functions and their syntaxes, see StreamBase Expression Language Functions.
StreamBase provides sample applications that feature custom functions for you to load into StreamBase Studio to see how they are built and configured. The custom sample applications are listed in Extending StreamBase Samples.
StreamBase Studio provides the following user assistance features to aid writing expressions:
- Expression Auto-completion
-
When the cursor is in a field in the Properties view that accepts expressions, press Ctrl+Space to open a command-completion dialog. Type a few letters to show the expression language functions that begin with those letters. Select a function to see its syntax and description in a separate pane. Press Enter to finish typing from the selected function. See Expression Auto-Completion and Content Assistance for more on auto-completion.
- Functions Tab in Properties View
-
The Functions tab shows a categorized list of all expression language functions. Type a regular expression in the filter box to narrow the selection to matching functions. Double-click a function name to see its syntax and description in the Detail sub-tab.
- Expression QuickRef Tab in Properties View
-
The Expression QuickRef tab shows a summary of the features of the expression language. Click a Show in Help link to open the corresponding section of this page in the Help window.
See Properties View for more information on these features.
Use the sbd --eval command to evaluate and test your expressions at the command prompt before using them in your StreamBase applications. sbd --eval works with simple functions, but not with aggregate functions. You must evaluate aggregate functions using other methods, such as running a test application and checking the results.
The syntax is either:
sbd --eval "expr to evaluate
" sbd -e "expr to evaluate
"
You can use either double or single quotes around the expression to evaluate. You must escape any quotes contained in the expression itself, as discussed below.
Note
Starting with release 7.6.0 on Windows, sbd --eval can interpret an initial hyphen in the expression to evaluate as an sbd command option when the command is surrounded by single quotes.
On Windows, be sure to run sbd --eval from the StreamBase Command Prompt, not a standard Windows command prompt.
Note
Some functions in the StreamBase expression language do not work with sbd --eval because the --eval
option causes sbd to run with a limited environment and
immediately exit after the function evaluation. This includes any function that
requests the status of the server or its environment, or that requests a further
function evaluation.
The following expression language functions are not expected to work with sbd --eval:
get_conf_param() | getNodeName() |
get_boolean_conf_param() | getParallelRoot() |
get_inf_conf_param() | getPath() |
get_long_conf_param() | getServerURI() |
get_double_conf_param() | securitytag() |
getContainer() | eval() |
getLeadeshipStatus() |
The following examples show how to use sbd --eval. These examples work as shown at the StreamBase Command Prompt on Windows and at the shell prompt on OS X or Linux. See Using Quotes in Expressions for further details on the quoting rules for StreamBase expressions.
-
You can evaluate any expression, whether or not it contains an expression language function. For example:
sbd --eval "1e1 * (15 % -4)"
returns:
(double) 30.0
-
For Windows, you must surround with double quotes the entire expression you are testing. At Linux or OS X shell prompts, you can use either double or single quotes, but using double quotes is recommended. The next example tests the cube root function:
sbd --eval "cbrt(27)"
which returns:
(double) 3.0
-
If you need to quote strings in your expression, you can either escape the extra double quotes with a backslash or use single quotes inside the surrounding double quotes. For example, the following commands are interpreted the same:
sbd --eval "list('alpha', 'beta', 'gamma')" sbd --eval "list(\"alpha\", \"beta\", \"gamma\")"
Both commands return:
(list(string)) [alpha, beta, gamma]
See Using Quotes in Expressions for more on using quotes in expressions.
-
For another example of quote escaping, consider the following format_time() function. Both of these versions work on Windows, OS X, and Linux:
sbd --eval "format_time(now(), 'EEEE, MMM d, yyyy HH:mm zzzz')" sbd --eval "format_time(now(), \"EEEE, MMM d, yyyy HH:mm zzzz\")"
Both versions return the current time, formatted like this example:
(string) Thursday, Nov 27, 2014 18:16 Eastern Standard Time
-
You can use sbd --eval to generate the correct CSV tuple format to use as input for sbc enqueue. For example, when defining a complex tuple that contains nested tuples, use syntax like the following for sbd --eval. (This command is a single unbroken line, shown on two lines for publication clarity):
sbd --eval "tuple(3.14159 as Math, tuple('IBM' as Stock, tuple(1 as X, 2 as Y, 3 as Z) as Inner) as Outer)"
This returns:
((Math double, Outer (Stock string, Inner (X int, Y int, Z int)))) 3.14159,"IBM,""1,2,3"""
Now use the returned value to enter this tuple for sbc enqueue:
3.14159,"IBM,""1,2,3"""
-
With typing patience and some trial and error, you can test any expression with sbd --eval, however complex. The following example tests the example shown in this Guide for the zip() function. First, build the example tuple with the tuple() function. (This command is a single unbroken line, shown on two lines for publication clarity):
sbd --eval "tuple( list(102.51, 96.82, 36.33) as prices, list('AAPL', 'IBM', 'HPQ') as symbols )"
This returns:
((prices list(double), symbols list(string))) "[102.51,96.82,36.33]","[AAPL,IBM,HPQ]"
Once you have the tuple() function returning a valid value, surround it with the zip() function:
sbd --eval "zip(tuple( list(102.51, 96.82, 36.33) as prices, list('AAPL', 'IBM', 'HPQ') as symbols ) )"
This returns:
(list((prices double, symbols string))) [102.51,AAPL, 96.82,IBM, 36.33,HPQ]
If you do not find a built-in function that you need, you can define a function in
the StreamBase expression language. You can use functions you define with any
operator that takes a StreamBase expression. function
is
a StreamBase data type, and like other data types, it can be passed in streams,
specified within output expressions, and defined for modules in a fashion similar to
constants. In some cases, EventFlow functions can avoid the need to create custom
Java functions.
This example describes creating a new EventFlow application that uses a function in two ways to convert temperatures in Celsius to temperatures in Fahrenheit. In the first section, the function is an expression in an input stream. In the second section, the function is a module definition.
-
Invoke
→ → -
Select the check boxes for
. -
Name the project
temperature_conversion
and click . A blank EventFlow canvas opens in StreamBase Studio for a module automatically namedtemperature_conversion.sbapp
. -
Drag an Input Stream, an Output Stream, and a Map operator onto the EventFlow canvas. Name and connect them as shown here:
The following sections show how to provide the function to the convert operator in two ways:
-
Passing the function as an expression on an input stream
-
Creating a module definition for it
Normally, you would use only one of these approaches, depending on your application's purpose and design. Passing functions as fields in streams provides greater flexibility in defining them, but that flexibility may not be needed and can reduce an application's efficiency.
The StreamBase Studio sample EventFlow Function Sample provides an application similar to this example that you can run and modify.
Note
Certain built-in functions take user-defined functions as arguments, such as the
filterlist
list manipulation function.
These are sometimes called higher-order functions.
Configure the input schema by describing the input data, including a function that will perform the data conversion.
-
Select the input stream, celsius and specify its schema. Use the
button to add two fields:-
temp_cel of type double
-
to_fahr of type function
-
-
Select argument schema under
to_fahr
and click the button to create a field for the argument to the function. -
In the new child field, replace
UnnamedField1
withcels
and specify its type as double. -
Set the
to_fahr
return type to double.The input stream schema now looks like this:
-
Edit the output settings of the convert Map operator:
-
Set Input Fields to
to prevent the celsius temperature and the function from passing to the output stream. (You can specify instead if you want to see the input values when you run the application.) -
Add an output field named
fahrenheit
. In its Expression column, typeto_fahr(temp_cel)
The convert Map operator settings now look like this:
-
-
Save the application. Then click the Run button to start it in the SB Test/Debug view.
-
In the Manual Input tab, enter the following values for input stream celsius:
-
Set temp_cel to
20
. -
Set to_fahr to
function (cels double) { (9/5 * cels) + 32 }
The Manual Input tab should look like this:
-
-
Click the
button. StreamBase returns the following tuple:07:23:03 fahrenheit fahrenheit=68.0
-
Keeping the to_fahr function the same, try setting temp_cel to other values to see that the function works correctly, You can change the constants in and syntax of to_fahr as well, as long as it continues to take one double argument and return a double or a value coercible to double.
You can declare a function as a module definition instead of passing it through the input stream. To specify a function in this way:
-
In the
temperature_conversion.sbapp
tab of the EventFlow editor, click the Definitions tab. -
In the Constants, Named Schemas, and Table Schemas pane, click the button. The Add a constant or function dialog opens.
-
Enter a name for the function. To avoid confusing it with the to_fahr function defined in the previous section, set the name of this one to
to_fahrenheit
. -
Define the function. The general syntax for a function is:
function
function_name
(arg1 type
,
arg2 type
,...
) ->return_type
{expr
}Enter the following in the Expression field:
function to_fahrenheit(cels double) -> double { (9/5 * cels) + 32 }
If a function calls itself recursively, you must specify both a name and a return type for it, as shown above. As this function is not recursive, they are not required and you can shorten its definition to:
function (cels double) { (9/5 * cels) + 32 }
-
Edit the Output Settings tab of the convert Map operator to add a field after fahrenheit. Call it
fahrenheit2
. -
For that field's expression, enter
to_fahrenheit(temp_cel)
to use the function you just defined. -
Save the application.
-
Run the application and manually input a value of
30
for temp_cel, keeping the previous definition ofto_fahr
. Because the two functions are identical, the output tuple now has two fields with the value 86.0.08:28:25 fahrenheit fahrenheit=86.0, fahrenheit2=86.0
StreamBase may not successfully compile a user EventFlow function from a stream if
that function is not pure. For example, a function data
type field containing a function such as function randomInt (n) -> int {int(random()*n)}
can cause a compilation exception because random()
is
not a pure function, meaning that its output is not constant or is not entirely
determined by its input arguments. The exception you see is:
exception: Compilation error preparing function "function randomInt (n int) -> int {int(random()*n)}" : current security options prevent calling any operation that is not purely functional from enqueued or compiled functions"
Other impure StreamBase functions that give rise to this condition include
securerandom()
, eval()
,
calljava()
, and callcpp()
.
You can force StreamBase to accept impure functions by setting the system property
streambase.appgen.allow-all-function-input to
true
. By default it is set to false
. See StreamBase
Java Properties.
Starting with release 7.4.0, function
is a StreamBase
data type and a reserved keyword.
The following simple functions in the expression language, sometimes called higher order functions, allow or require passing in a function as one of their arguments to manipulate single or multiple lists.
-
filterlist()
Removes elements of a list that fail a test provided by the input function. -
foldleft()
Applies an accumulator function forward over a set of lists of the same length and compatible types. -
foldright()
Applies an accumulator function backwards over a set of lists of the same length and compatible types. -
maplist()
Applies a function to corresponding elements of multiple lists and collates the results as a single list. -
mergelist()
Merges a list of sorted lists to produce a sorted output list. The input comparison function is used to compare elements of the lists.
In addition, sort()
has a syntax that accepts a custom
comparison function to sort a single list.
Input functions to these higher order functions must obey certain rules:
-
Their arguments must be compatible with the data type of their input lists.
-
The number of calling arguments must equal the number of input lists (+1 for
foldleft
andfoldright
) -
The
filterlist
input function must return true or false for each list element. -
Comparison functions must return integer values less than zero if the first argument is smaller than the second; greater than zero if the first argument is greater than the second; one or zero if the arguments are equal.
-
Accumulator functions can be created to return tuples or lists of a specified data type.
You can pass custom functions to these functions using function aliases that you declare in a server configuration file.
For details, see individual function descriptions in StreamBase Expression Language Functions.
Starting with release 7.6.0:
-
You can have multiple expressions inside a function body, optionally separated by one (or more) semicolons. Semicolons after the last statement are allowed and ignored. The return value of the function is that of the last expression on the block.
-
You can define local variables with
@define(
. The scope of these variables is from the @define to the closing brace of the containing function. Duplicate variable definitions or variables with names identical to arguments are not allowed.name
asexpr
)function foo( a int, b int ) { @define(a + b AS sum, a * b AS prod); // The following returns a value equal to a*a + b*b // Note that sum and prod defined above are visible // inside the following local function as well. // The final parenthesis pair executes the function // immediately after defining it. function sum_of_sqr() { sum * sum - 2 * prod } (); a + b + sum + prod; // Unneeded final semicolon is allowed. }
-
The @define construction can be used to override variables. However, notice that free variables are bound lexically. This means that if you create a function referring to a free variable
a
, the value ofa
is bound when the function is created, not when it is called. For example:sbd -e "function foo(a int) { @define(function() \ {do_print(\"inner_a:\",a)} as bar); @define(a+1 as a); \ do_print(\"outer_a:\", a); bar()} (0)"
This returns the following because the value of
a
inside functionbar
is captured before we modifya
.outer_a:1 inner_a:0 (int) 0
-
You can annotate a function as @cacheable to cache the result of a function so that it is not evaluated again. For example:
function @cacheable foo(a int, b int) { a + b }
If you later call
foo(1, 2) + foo(1, 2)
, the value of foo is evaluated once and cached, instead of being evaluated twice.
You can use the StreamBase custom function APIs to implement your own function and configure your application to use it. Your custom functions can be written by extending the StreamBase APIs for Java or C++. For details on writing your own functions, see these topics in the API Guide:
You can call your custom functions in expressions by calling a function alias you have declared in the server configuration file, or by using calljava() or callcpp().
You can define aliases for your custom functions using the <custom-function>
element in the StreamBase Server
configuration file. This allows you to reference your custom functions directly,
without using a calljava()
or callcpp
() function. For example:
myCustomFunction(x, y)
See <custom-functions> for instructions on defining aliases for your custom functions.
Expressions often need quotes around string values to designate those values as strings. The StreamBase expression language is agnostic about whether to use double or single quotes in expressions. For example, StreamBase interprets the following expressions identically when used in an expression in an operator or adapter in StreamBase Studio or in a StreamSQL statement:
strftime("Traded on %b %d, %Y at %H:%M", now()) strftime('Traded on %b %d, %Y at %H:%M', now())
The only rule is that you must escape any instance of the same quote mark if you need to use it again inside a pair of quote marks. You can escape a quote mark with a preceding backslash, or you can surround it with the opposite quote mark. For example, StreamBase interprets the following lines identically in Studio or in StreamSQL:
strftime('Traded at Miller\'s Crossing on %b %d, %Y at %H:%M', now()) strftime("Traded at Miller's Crossing on %b %d, %Y at %H:%M", now())
The simple rules for using quote marks in StreamBase expressions in Studio are complicated by the shell's quoting rules when testing expressions at the command prompt with sbd --eval.
On Windows, when using the StreamBase Command Prompt, you must surround the entire expression with
double quotes when using sbd --eval.
This is a requirement of the cmd.exe
shell, not
StreamBase. You can still use either backslash escaping or single quotes inside the
surrounding double quotes. For example, StreamBase resolves the following example
commands identically at the StreamBase Command Prompt:
Works on Windows:
sbd --eval "strftime(\"Traded on %b %d, %Y at %H:%M\", now())"
sbd --eval "strftime('Traded on %b %d, %Y at %H:%M', now())"
The Bash shell is more forgiving, both under Linux and under Cygwin on Windows. Bash accepts either double or single quotes surrounding the argument to sbd --eval. For example, StreamBase resolves the following commands identically at the Bash prompt:
Works on Bash under Linux or Cygwin:
sbd --eval "strftime(\"Traded on %b %d, %Y at %H:%M\", now())"
sbd --eval "strftime('Traded on %b %d, %Y at %H:%M', now())"
sbd --eval 'strftime("Traded on %b %d, %Y at %H:%M", now())'
However, Bash does not allow the following version:
Does NOT work on Bash:
sbd --eval 'strftime(\'Traded on %b %d, %Y at %H:%M\', now())'
You may have to experiment with different quoting styles and quote escaping styles to get a complex command to run with sbd --eval.
Expressions use the StreamBase static data types:
blob | list |
bool | long |
capture | string |
double | timestamp |
function | tuple |
int |
These data types are described in detail in StreamBase Data Types.
To recast data types, use one of the StreamBase type conversion
functions. For example, to cast a value to a double in an expression, use the
double()
function. To cast a value to a string, use the
string()
function, and so on. Casting may not be
necessary if type coercion is supported for the data types, as described in the next
section.
Data type coercion is the implicit conversion of a value
from one data type to another. For example, the +
(concatenation) string operator performs coercion from numeric and boolean values to
strings, as in:
"This is " + (2 > 1) => (string) “This is true” "1" + 2 + 3 => (string) "123"
Evaluation proceeds from left to right, so that order matters:
1 + 2 + "3" => (string) "33"
Arithmetic operators coerce values when evaluating expressions with mixed numeric data types, as in:
1 + 2L + 3.0 => (double) 6.0
StreamBase math functions that take arguments and return doubles accept int, long or
double input arguments. For example, the sqrt()
function accepts one int, long, or double input and
returns a double result (or NaN).
The following are the supported data type coercions and conversions:
Input data type | Converts to |
---|---|
int | long (concatenates to string) |
int | double (concatenates to string) |
long | double (concatenates to string) |
list(int) | list(long) |
list(int) | list(double) |
list(long) | list(double) |
list(tuple) merging with different list(tuple) | list(tuple), if a valid supertype of the two tuples can be found |
sub-tuple field with unmatched schema | sub-tuple field with unioned schema |
string (numerals only) | int, long, double (via int(), long(), double()) |
The following rules apply to data type coercions:
-
For long-to-double coercion where the precision of the long cannot be preserved in the double, Java rounding rules apply.
-
When entering data to a stream whose schema contains a field of type tuple, StreamBase attempts to coerce the data into the sub-tuple fields using the same rules as for a loose union in the Union operator.
-
Coercion rules do not apply to output streams with declared, explicit schemas.
Expressions are case sensitive. Function names are usually all lowercase, with a few exceptions. If you add custom functions, follow the lowercase convention.
StreamBase identifier naming rules apply to the name you assign to any StreamBase component, including schemas, fields, operators, tables, modules, containers, and data source names. Specifically, identifiers you create should:
-
Begin with an alphabetic character or an underscore.
-
Contain only alphabetic characters, numbers, and underscores.
-
Not contain hyphens, colons or other non-alphanumeric characters.
Note
StreamBase may create internal identifiers that include colons, but you cannot use colons in your identifiers unless you escape them as described in the next section.
For clarity in your application modules and to make debugging easier, follow the rules below when choosing names for your StreamBase components. Wherever possible, these rules are enforced by typechecking in StreamBase Studio.
-
Do not use the name of a StreamBase data type, EventFlow reserved word, StreamSQL reserved word, or Live Datamart reserved words to name your stream fields or other components.
-
Do not name a stream field the same as any dynamic variable or constant defined in the same module. StreamBase resolves unqualified names first against the names of dynamic variables and constants, and then against the names of fields in currently available streams. See Qualify Field Names to Avoid Name Conflicts for more information.
-
For code clarity, avoid naming a stream field the same as any StreamBase expression language function, or the same as any custom function you have configured for use in the same application. You can refer to the alphabetic listing of expression language function names.
Identifier names can include Unicode characters as long as Unicode support is enabled for both Studio and Server, as described in Unicode Support.
StreamBase supports an escaped identifier syntax to enable the use of arbitrary naming conventions for fields and identifiers. When you provide a name for a component of a StreamBase expression that violates the rules described in the previous section, StreamBase Studio automatically stores the name as an escaped identifier, which wraps the invalid identifier in a syntax that enables its use. However, Studio does not automatically rename fields that violate the naming rules; you must explicitly enter escaped identifier syntax for the name of any field that would otherwise cause Studio to report an error.
The escaped identifier syntax is #"
— a pound or hash symbol,
followed by the otherwise-violating identifier you want to use within double quotes.
Escaped identifiers work throughout StreamBase for the names of all components and
fields. You can use this syntax to name a field the same as a StreamBase reserved
word, such as name
"#"AS"
.
When you name fields and expression components, do not use the following words, which are reserved for use in the expression language:
and | as | between |
else | false | function |
if | or | not |
null | then | true |
In EventFlow applications, StreamBase Studio does not prevent you from naming a field with a reserved word in an input stream's Properties view. However, a schema that contains a reserved word causes one or more typecheck errors in downstream components. The error message in the Typecheck view identifies the offending field's name. To correct the typecheck error, rename the field in the input stream's Properties view.
In StreamSQL applications, Studio prevents you from using a reserved word as a field name when defining schemas, or as a field in an index specification, order-by specification, or gather key.
Although the names of StreamBase data types are not reserved words, TIBCO recommends that you do not use data type names as field names, for clarity when developing and debugging your applications. The data type names are listed in Data Types.
The StreamSQL language has an additional list of reserved words, as shown in StreamSQL Reserved Words.
If a name conflict occurs when referencing a field name used in an expression in an operator, you can qualify the name to clarify the inbound stream you mean.
For example, in the Properties view for a Gather operator, you can qualify the fields
as input[port-number].field-name
.
Example: input1.is_alarm, input2.is_alarm,
input3.is_alarm
.
In the Properties view for a Join operator, you can qualify the fields in the two
input streams as input1.field-name
and
input2.field-name
. The input1.field-name
refers to a field in the stream
arriving at the top port (#1). The input2.field-name
refers to a field in the stream
arriving at the bottom port (#2).
Examples:
input1.SKU input2.SKU
In the Properties view for a Query operator, you can qualify the fields in the two
input streams as input.field-name
and
old.field-name
. The input.field-name
refers to the current input tuple,
while old.field-name
refers to the field's
prior value.
Examples:
input.Symbol old.Symbol
Important
In any expression anywhere in StreamBase, unqualified names are resolved first
against the names of any dynamic variables and constants defined in the current
module, and then against the names of fields in currently available streams. This
means that a dynamic variable or constant named foo
can mask a field also named foo
, depending on the
context.
One consequence of this rule: If you use the same name for a dynamic variable and
for its updating expression field, StreamBase Studio issues a typecheck warning. To
distinguish a dynamic variable from the same-named field in the updating stream,
qualify the field name with input.
, or just rename the field.
fieldname
You can use +, –, *, /,
and %
. The ^
expression is not supported.
The following example shows the arithmetic operators in use. (This expression is shown on two lines for publication clarity.):
((((sector_in_fund_value_t + (shares_traded * price_at_trade)) * 100) / fund_value_t) > 25)
This expression is designed to execute the 25% sector test, where one sector is not
allowed to be more than 25% of the total fund value. is_alarm
is set to true
if the test
fails.
You can also use the +
operator to concatenate strings;
see Concatenation in Expressions for details.
You can also use the *
operator to repeat a string
multiple times; see String Repetition for details.
The modulus operator (%) finds the remainder of a division operation. Division and
modulus are always related such that a/b*b+(a%b) ==
a
, where a/b
is an integer
division. For example, 15%4
can be resolved as follows:
as:
15 / 4 * 4 + MOD == 15 3 * 4 + MOD = 15 12 + MOD = 15 MOD = 3
This relationship is preserved with negative divisors. As a result, whenever the quotient is negative, the modulus is negative. For example:
15 % 4 = 3 15 % -4 = 3 -15 % 4 = -3 -15% -4 = -3
Also note that if you divide a smaller number by a larger number, the modulus is always the smaller number. For example:
4 % 15 = 4
This behavior may be different than modulus operators in some programming languages.
Tip
The easiest way to understand StreamBase modulus operations is to try out different expressions using the sbd --eval command as described in Evaluating Expressions with sbd --eval.
Use the between-and operator in expressions using the following syntax:
test-expr
betweenexpr1
andexpr2
where expr1
and expr2
resolve to instances of the same data type to
which test-expr
resolves. That is, the data
type of test-
expr, expr1
, and expr2
must be the same, or must be coercibly the
same, using the rules described in Data Type Coercion.
This construction returns Boolean true or false, and is thus ideally suited for use
in an if-then-else
construction:
iftest-value
betweenboundary1
andboundary2
thenexpr-if-true
elseexpr-if-false
For example, using the current value of fieldname
in the incoming tuple:
if fieldname
between 100 and 200 then true else false
The between-and operator can be negated with the ! or NOT operator:
if fieldname
not between 100 and 200 then true else false
The between-and operator is inclusive, which means if test-value
exactly matches either boundary1
or boundary2
, the operator returns true. Use this
operator as a more readable version of the following equivalent expression. Using
between-and has the advantage of only specifying test-value
once.
if (test-value
>=boundary1
andtest-value
<=boundary2
) thenexpr-if-true
elseexpr-if-false
The between-and operator works with any StreamBase data type, including list and tuple, using the relational comparison algorithms described for each operator on StreamBase Data Types.
Use the infix operator IN to test for list or string containment, returning Boolean
true or false. For example, the following expressions each return true
:
2 in [1, 2, 3] 2 in list(1, 2, 3)
For strings, IN returns true
if the string on the left
is a substring of the string on the right. For example,
'foo' in 'there is food here'
returns true
. However, if the right side argument is a
list of strings, a list element must exactly match the left side argument. For
example:
'foo' in ['there','is','food','here']
returns false
.
The IN operator is syntactic sugar for the contains() function.
Notice that IN is not among the reserved words for the expression language, and the string IN can still be used as an identifier in other contexts. The IN operator only becomes an operator in the context of a list containment expression as shown above, with the test value on its left and the queried list on its right. The queried list can be expressed in simple bracket format, or can be any function or expression that resolves to a list, including the list() and range() functions.
Unary operators act on only one operand in an expression. –a
is valid for integer, double, and long types.
!a
or NOT a
is valid for the bool type. +a
is not supported.
You can use the following relational operators: <
,
<=
, =
, ==
, >
, >=
, !=
The relational operator <>
is not supported.
StreamBase data types are comparable with relational operators in different ways, as listed in the entry for each data type on StreamBase Data Types.
You can use the following logical operators: &&
,
AND
, ||
, OR
, !
, NOT
.
Logical Operator | Meaning |
---|---|
&& or AND
|
Logical AND |
|| or OR
|
Logical OR |
! or NOT
|
NOT: negates its operand |
Note
The evaluation of expressions using && and || will short-circuit. This means that after StreamBase encounters the first term that evaluates as false, no other terms in a statement using && are evaluated. Similarly, evaluation of a statement using || stops after the first true term.
For related information on Boolean logic and nulls, see Using Nulls.
You can apply operators and functions to all fields in a tuple, using wildcards
(*
) in EventFlow expressions in much the same way as in
StreamSQL. The resulting values can replace the current field values, be assigned as
new fields, or evaluated as a collection as arguments to EventFlow or custom
functions, according on the syntax you use to expand values.
You can use wildcards to produce lists of expressions when passing arguments to
functions. A simple wildcard expression such as input.*
replaces input field values with evaluation results. To assign the results to fields
at a lower level, use the AS
keyword. The following
table illustrates several syntaxes that illustrate the effect of using AS
in wildcard expansions.
Wildcard Expression | Expansion | Remarks |
---|---|---|
f(g(input.*))
|
f(g(input.x)), f(g(input.y)), f(g(input.z))
|
Replaces top-level fields |
f(g(input.* AS *))
|
f(g(input.x, input.y, input.z)))
|
Replaces top-level fields |
f(g(input.*) AS *)
|
f(g(input.x) as x, g(input.y) as y, g(input.z) as
z)
|
Creates lower-level fields with same names |
Expressions such as (input.x AS *)
are not
legal. You must specify field names with wildcards when using AS *
.
For example, to determine whether all fields in a tuple are null, you could use the following expression:
andall(isnull(input.*) AS *)
The isnull
function is applied to all input fields, the
results are collected into boolean fields with the same names, and the resulting list
is evaluated by andall
, which only returns true if
every field is null. In this example, the fields do not have to be of the same type.
For additional information, see Wildcard Rules in StreamSQL SELECT Statements.
The precedence of mathematical operators in expressions, from highest to lowest, is as follows:
-
Unary operators:
!
,NOT
, and–
The logical negation operator,
!
orNOT
, reverses the meaning of its operand.The unary negation operator,
–
, produces the negative of its operand. -
Multiplicative operators:
*
and/
and%
Multiplication:
*
Division:
/
Modulus, the remainder from division:
%
-
Additive operators:
+
and–
Addition:
+
Subtraction:
–
-
Relational operators:
<
and<=
and>
and>=
Less than:
<
Less than or equal to:
<=
Greater than:
>
Greater than or equal to:
>=
-
Equality operator:
==
or=
-
Not equal:
!=
orNOT =
-
Logical AND:
&&
orAND
-
Logical OR:
||
orOR
You can use parentheses in expressions to override the default precedence.
You can add comments to your expressions using either C-style comments or StreamSQL-style comments:
-
StreamSQL comments begin with two hyphens and extend to the end of the same line.
-
C-style comments begin with slash-asterisk and end with asterisk-slash. You can insert C-style comments anywhere in the expression, including the middle of a line, and they can extend over multiple lines.
The following sample expression illustrates several uses of comments:
/* handle special APPL case */ if (input1.symbol=="APPL.N") then input2.symbol == "APPL" /* handle special IBM case */ else if (input1.symbol=="IBM.L") then input2.symbol == "IBM" else input2.symbol == input1.symbol -- accept the rest as given
This section describes how to specify literals in expressions, for each data type. (For information about the data types themselves, see StreamBase Data Types.)
- blob
-
Use the StreamBase blob() function, which converts a string to a blob.
Example:
blob("abcde")
creates a blob containing the bytes representing the string"abcde"
.You can use tobson() to convert strings of text, other literals, or any StreamBase value to a BSON blob. To convert a BSON blob containing a string back to a string literal, use the stringbson() function.
- bool
-
Use the literals
true
andfalse
. - int
-
Enter a number that is in the range of the int data type. Decimal, binary, octal and hex encodings are supported. Examples:
0
and31337
are both intsPrefix non-decimal integers with 0 (zero):
-
Binary:
0b11011
and0B11011
are interpreted as binary (decimal27
) -
Hexadecimal:
0x7F8
and0X7F8
are interpreted as hexadecimal (decimal2040
)
To represent a numeric literal as a long data type, terminate it with letter
l
(orL)
. -
- double
-
Enter a number using a decimal point, or using scientific notation. Use a
D
ord
suffix to avoid ambiguity.Examples:
10.0
and1e1
are both doubles, as are13.85d
and128D
- long
-
Enter a number that is in the range of the long data type, appending the letter
l
orL
to the number. The number can be formatted as decimal, binary, or hexadecimal.Examples:
100L
and3147483650L
are both longs, as are0B10110L
,0456L
, and0XA5B6L
.Note
If the L is omitted, StreamBase interprets your number as an int no matter how large the value is.
- list
-
Enter comma-separated values, surrounded by square brackets.
Examples:
[23, 46, 889]
is a list of three ints.
is a list of three strings.["IBM", "AAPL", "ORCL"]
- string
-
You can use single quotes (
'string'
) or double quotes ("string"
) around strings. Escape characters are supported, as follows:Character Results in \" Double quotation mark \' Single quotation mark \n Newline \t Tab \b Backspace \r Carriage return \f Form feed \\ Backslash \u XXXX
Unicode character with hex code point XXXX
Examples:
String Literal Results in "He said, \"Hello.\"" He said, "Hello." 'She said, \'Hello.\' ' She said, 'Hello.' "A\tB" A<tab>B "A\nB" A<newline>B "C:\\WINDOWS" C:\WINDOWS "Copyright \u00A9 2050" Copyright © 2050 - timestamp
-
You can use timestamp literals only in comparison expressions that use the operators
==
,!=
,<=
,>=
,<
, or>
, when one side of the comparison is a timestamp and the other side is a string literal that can be interpreted as a valid timestamp. Otherwise, you cannot enter timestamp literals directly in expressions. See timestamp in the StreamBase Data Types documentation.In cases other than the one just described, to use timestamps in expressions, convert a string in a particular format to a timestamp using a function such as the StreamBase timestamp() function. The string argument must be in the form of a time format pattern as defined in the
java.text.SimpleDateFormat
class described in the Sun Java Platform SE reference documentation. See timestamp() for examples.Also see the format_time(), parse_time() and format() functions. The maximum precision for timestamps is milliseconds.
- tuple
-
In expressions, define a single tuple using this syntax:
tuple (
value
ASfieldname
[,...]) [AStuplename
]The tuple schema, delimited by parentheses, consists of one or more field specifications separated by commas. Each field name resolves to an instance of a StreamBase data type (including another tuple). The last field name can be followed by a comma, which StreamBase ignores.
Use the AS keyword to specify field names in the result. Omit the outermost
AS
in EventFlow expressions, where the tuple field's name is specified in StreamBase Studio. The outermost AS is required in StreamSQL.tuplename
To cast a set of values as a tuple that conforms to an existing named schema, use the tuple() function.
You can use nulls in the StreamBase expression language, one for each of the simple data types:
blob(null) bool(null) double(null) int(null) long(null) string(null) timestamp(null)
The data type of a null is never implicit. You must specify the data type of any null you are using in the expression.
In general, when you apply any arithmetic operator or function with data-type
(null) as one of the arguments, the result
is null. Three exceptions are the isnull()
and
notnull()
functions, which test whether an expression
evaluates to null, and the coalesce()
and coalesce_tuples()
functions, which select a non-null value from a
set of potentially null arguments.
Expression Example | Result |
---|---|
3 + int(null) | A null int |
int(null) + int(null) | A null int |
int(null) + bool(null) | A typecheck error. You cannot add an int and a bool. |
if bool(null) then 3 else 4 | A null int |
int(null) == int(null) | A null bool, because null is not equal to itself |
int(null) != int(null) | A null bool, because null is not equal to itself |
isnull(int(null)) | A bool that evaluates to true |
notnull(int(null)) | A bool that evaluates to false |
To specify a null list, use the nulllist()
function. See Null Lists for a discussion
of null lists compared to empty lists.
To specify a null tuple that uses a named schema, specify the name of the schema with
empty parentheses. For example, for the schema named nyse_data
, the expression nyse_data()
creates a null tuple. See Null Tuples for a discussion of null tuples compared to empty tuples.
For more detailed information, see Using Nulls.
In an expression such as: if p then e1 else
e2
, p
must be a valid boolean expression,
and both e1
and e2
must be
expressions that have the same type. If p is true, then e1 is evaluated and the
result returned. If p
is false, e2
is evaluated and the result returned. In either case, the other
sub-expression is not evaluated.
Note
In the StreamBase expression language, each if
clause
must have a pair of explicit
then
and else
clauses.
Should you create compound if
clauses, to avoid
ambiguity, remember to specify the then
and
else
clauses for each if
clause. For an example, see the next section.
You can use combinations of if-then
and else-if-then
statements to form compound conditional expressions.
For example:
if i==0 then "Buy" else if i==1 then "Sell" else if
i==2 then "Hold" else "None of the above"
Here is a second example, indented for clarity, where we nest an if then else
in a then
clause:
if p1 then if p2 then A else B else if p3 then C else D
Notice how each if
clause contains a then
clause and an else
clause.
You can enter an expression that concatenates a string with any StreamBase data type. For any such concatenation, a string must be the first value. For two string variables, the concatenation expression is:
stringvar1
+stringvar2
For a numeric type with a static string, the concatenation expression is:
"staticstring
" +number
For example:
"square root of 16 = " + sqrt(16)
->"square root of 16 = 4
For a string and a numeric variable, the concatenation expression is
stringvar
+numvar
For example, where month = "September" and day = 16:
month + day -> "September 16"
Use the *
operator to repeat a string n
times, using the syntax
"staticstring" * n
where n
is a positive integer or an
expression that evaluates to one. For example:
"abc." * 3 -> abc.abc.abc.
The *
operator simplifies creating fixed-length fields.
For example, if a string field needs to be 32 characters long, use an expression like
the following to pad its contents with blanks:
stringvar
+ ' ' * (32-length(stringvar
))
This use of the *
operator is also known as string multiplication.
When used with comparison operators ( == != > < <= >= ) you must compare time fields interval-to-interval, or timestamp-to-timestamp. You cannot use the comparison operators to forma an interval-to-timestamp comparison.
To express constant intervals, you can use the seconds()
and minutes()
functions.
For example, if you want to add 60 seconds to a timestamp, enter expressions such as:
t + seconds(60) t + minutes(1)
For more on comparing timestamp expressions, see timestamp Data Type.
You can declare global parameters in the StreamBase Server configuration file,
usually named sbd.sbconf
, and can define module
parameters in the Parameters tab of the EventFlow Editor. You can reference either
parameter type in StreamBase expressions. See Constants, Variables,
and Parameters for more on the difference between global and module parameters.
To reference a parameter in an expression, wrap the parameter name in braces and
prefix the open brace with a dollar sign: ${parameter}
. If you are referencing a parameter
that is defined as a string, use quotes around the entire reference: "${parameter}"
StreamBase Server Configuration File XML Reference explains how to declare global parameters. For example, consider a server configuration file that defines the following two global parameters:
<operator-parameters> <operator-parameter name="MyInt" value="2"/> <operator-parameter name="MyString" value="somestring"/> </operator-parameters>
You could reference the first parameter (for example, in an output field) using an
expression like 35 * ${MyInt}
. The
expression would evaluate at run time to 70
. The
following StreamSQL statement references the second parameter:
SELECT * FROM InputStream1 "${MyString}" AS
source
Notice the quotation marks, which are needed so that the expression is resolved as a string.
Note
Expression parameters can be referenced in both EventFlow and StreamSQL applications. However, if you convert an EventFlow application file to a StreamSQL file, any expression parameters in the EventFlow module are resolved during the conversion, and the value of the parameter is copied to the StreamSQL file. The parameter itself is not preserved.