Custom Operators
Use the Custom Operator Framework to create an analytic packages in Scala and Java, and then run them in the Team Studio workflow environment.
Data scientists and programmers can use the framework to create a wide variety of custom operators to use within a Team Studio analytics workflow. They can develop the custom operators in Scala and Java using the framework and examples that are provided with the SDK. The framework supports both Spark and Database operators, but this section covers developing only for Spark-enabled operators.
After custom operators are deployed to a Team Studio instance, the user does not need to read, write, or understand any code to use them. Specifically, users can easily integrate with their own specific Spark jobs, or they can run their own custom code against their data within the Team Studio framework. The custom operators can consume one or more tabular data sets and can produce a tabular dataset as output to following operators.
An entire customer team can share pre-existing libraries of analytics and have those incorporated into the Team Studio visual workflows as simply another operator that can be quickly chosen and configured through the Team Studio drag and drop interface.
The Custom Operator SDK is available on Github.
- Required Tools
Before you begin, make sure you have installed and configured the required tools: Java, Maven, and GIT. - Compiling the Samples
After you download the samples locally, you build them for the Team Studio web application. - Uploading an Operator to Team Studio
If mvn package is successful, BUILD SUCCESS appears in the console. The target directory has been created. - Viewing and Running Samples From IntelliJ IDEA
You can build and run the sample operators from an IDE such as IntelliJ. This approach is useful when you are developing your own custom operators. - Building Your First Custom Operator in Scala
Use the Custom Operator framework to create new functionality for your Team Studio workflows. We have provided a set of samples in Scala and Java to showcase some of the things you can do and describe the format an operator takes. - Building a Source Operator
Not every operator falls under the "data set in, data set out" paradigm. This tutorial demonstrates how to build a source operator - that is, one that creates a data source and can be connected to other operators for further transformation or modeling. - Creating a Custom Visualization with Scala and Javascript
Using our Custom Operator SDK, you can create visualizations using the common JavaScript libraries D3, Plotly, and Dojo.