Creating a Custom Visualization with Scala and Java script

You can create visualizations using the Custom Operator SDK, the common JavaScript libraries D3, Plotly, and Dojo.

Procedure
1. Rename JavascriptVisualization with the preferred name of your operator.

   For example, to create an operator called "MyCoolOperator," use the filenames MyCoolOperator.js and MyCoolOperator.scala.

2. Review the following Scala sample code.

   class JavascriptVisualizationRuntime extends SparkRuntime[HdfsTabularDataset, IONone] {
    
   [...]
    override def createVisualResults(context: SparkExecutionContext,
                                      input: HdfsTabularDataset,
                                      output: IONone,
                                      params: OperatorParameters,
                                      listener: OperatorListener): VisualModel = {
       val dataset = context.visualModelHelper.createTabularDatasetVisualization(input)
       val tabularModel = dataset.asInstanceOf[TabularVisualModel]
    
       val columnName = params.getTabularDatasetSelectedColumn(JavascriptVisualizationUtil.COLUMN_TO_ANALYZE)._2
       val i = tabularModel.columnDefs.indexWhere(columnDef => columnDef.columnName == columnName)
    
       val compositeVisualModel = new CompositeVisualModel()
       if (i != -1) {
         val valuesSet = tabularModel.content.map(row => row(i))
         compositeVisualModel.addVisualModel("Data Cloud",
           new JavascriptVisualModel("execute", Option.apply(valuesSet)))
       } else {
         compositeVisualModel.addVisualModel("Data Cloud",
           new JavascriptVisualModel("execute", Option.apply("No results")))
       }
       compositeVisualModel.addVisualModel("Hello World",
         new JavascriptVisualModel("simpleFunction", Option.apply("")))
       compositeVisualModel
     }

   • JavascriptVisualizationRuntime extends SparkRuntime.
   • The example overrides createVisualResults to insert custom code. For your customization, keep the parameters as defined.
   • CompositeVisualModel() is the core of the example, creating and defining the visualization. CompositeVisualModel means that you can have several "tabs" in the output. This example creates a visualization with two tabs: one with a "word cloud" and one with simple text that says "Hello World."
   • Within CompositeVisualModel, the example creates a JavascriptVisualModel using the following structure.

     case class JavascriptVisualModel(
               functionName:String,
               data: Option[Any]) extends VisualModel

   • The JavaScript code executes the function simpleFunction.
   • The example returns CompositeVisualModel.

   Before editing the example JavaScript code, check the signature class of the operator to ensure that the metadata includes useJavascript=True. JavaScript code does not run without this setting.

3. In the JavaScript file, define the functions called from the Scala code.

define([
    "dojo/dom-construct",
    "dojo/dom-geometry"
], function(domConstruct, domGeo) {
    return {
        /**
         * @param outpane The enclosing html <div> in which the Javascript code should place elements into
         * @param visualData Data specified by the custom operator in createVisualResults() when creating the JavascriptVisualModel
         */
        execute: function(outpane, visualData) {
            var geo = domGeo.position(outpane);
            var canvas = domConstruct.create('canvas', {width: geo.w, height: geo.h}, outpane);
            var context = canvas.getContext('2d');
              // Trivial visualization example - all elements in visualData (an array) are randomly positioned in the canvas with random colors
            for (var i = 0; i < visualData.length; i++) {
                context.fillStyle = 'rgb('+Math.floor(Math.random() * 255)+','+Math.floor(Math.random() *
                    255)+','+ Math.floor(Math.random() * 255)+')';
                context.fillText(visualData[i], 100 + (Math.random() * (geo.w - 200)), 100 + (Math.random() * (geo.h - 200)));
            }
        },
        simpleFunction: function(outpane, visualData) {
            var span = domConstruct.create('span', {}, outpane);
            span.innerHTML = "Hello World"
        }
    };
});