Creating a Custom Visualization with Scala and Javascript

Using our Custom Operator SDK, you can create visualizations using the common JavaScript libraries D3, Plotly, and Dojo.

1. Rename JavascriptVisualization with the preferred name of your operator.

   For example, to create an operator called "MyCoolOperator," use the filenames MyCoolOperator.js and MyCoolOperator.scala.

2. Review the following Scala sample code.

   class JavascriptVisualizationRuntime extends SparkRuntime[HdfsTabularDataset, IONone] {
    
   [...]
    override def createVisualResults(context: SparkExecutionContext,
                                      input: HdfsTabularDataset,
                                      output: IONone,
                                      params: OperatorParameters,
                                      listener: OperatorListener): VisualModel = {
       val dataset = context.visualModelHelper.createTabularDatasetVisualization(input)
       val tabularModel = dataset.asInstanceOf[TabularVisualModel]
    
       val columnName = params.getTabularDatasetSelectedColumn(JavascriptVisualizationUtil.COLUMN_TO_ANALYZE)._2
       val i = tabularModel.columnDefs.indexWhere(columnDef => columnDef.columnName == columnName)
    
       val compositeVisualModel = new CompositeVisualModel()
       if (i != -1) {
         val valuesSet = tabularModel.content.map(row => row(i))
         compositeVisualModel.addVisualModel("Data Cloud",
           new JavascriptVisualModel("execute", Option.apply(valuesSet)))
       } else {
         compositeVisualModel.addVisualModel("Data Cloud",
           new JavascriptVisualModel("execute", Option.apply("No results")))
       }
       compositeVisualModel.addVisualModel("Hello World",
         new JavascriptVisualModel("simpleFunction", Option.apply("")))
       compositeVisualModel
     }

   • JavascriptVisualizationRuntime extends SparkRuntime.
   • The example overrides createVisualResults to insert custom code. For your customization, keep the parameters as defined.
   • CompositeVisualModel() is the core of the example, creating and defining the visualization. CompositeVisualModel means that we can have several "tabs" in the output. This example creates a visualization with two tabs: one with our "word cloud" and one with simple text that says "Hello World."
   • Within CompositeVisualModel, the example creates a JavascriptVisualModel using the following structure.

     case class JavascriptVisualModel(
               functionName:String,
               data: Option[Any]) extends VisualModel

   • The JavaScript code executes the function simpleFunction.
   • The example returns CompositeVisualModel.

   Before editing the example JavaScript code, check the signature class of the operator to ensure that the metadata includes useJavascript=True. JavaScript code does not run without this setting.

3. In your JavaScript file, define the functions called from the Scala code.
   Here is an example of how to use JavaScript for the word cloud example.

   define([
       "dojo/dom-construct",
       "dojo/dom-geometry"
   ], function(domConstruct, domGeo) {
       return {
           /**
            * @param outpane The enclosing html <div> in which your Javascript code should place elements into
            * @param visualData Data specified by your custom operator in createVisualResults() when creating the JavascriptVisualModel
            */
           execute: function(outpane, visualData) {
               var geo = domGeo.position(outpane);
               var canvas = domConstruct.create('canvas', {width: geo.w, height: geo.h}, outpane);
               var context = canvas.getContext('2d');
                 // Trivial visualization example - all elements in visualData (an array) are randomly positioned in the canvas with random colors
               for (var i = 0; i < visualData.length; i++) {
                   context.fillStyle = 'rgb('+Math.floor(Math.random() * 255)+','+Math.floor(Math.random() *
                       255)+','+ Math.floor(Math.random() * 255)+')';
                   context.fillText(visualData[i], 100 + (Math.random() * (geo.w - 200)), 100 + (Math.random() * (geo.h - 200)));
               }
           },
           simpleFunction: function(outpane, visualData) {
               var span = domConstruct.create('span', {}, outpane);
               span.innerHTML = "Hello World"
           }
       };
   });

   Note:

   • The execute function creates the word cloud and uses the JavaScript built-in canvas technology.
   • The function simpleFunction outputs "Hello World" as text.