Creating the Utils Object

Now we create a Utils object to hold some utility functions and constants that we uses across the different classes in our operator.

    Procedure
  1. Create an object called DatasetGeneratorUtils:
    object DatasetGeneratorUtils {
    }
  2. Within this object, add the following:
    • A constant for the parameter key to use for input later in this tutorial.
    • A function to get the output schema of our result DataFrame.
    /**
    	* The key for the parameter for the "Number of Things"
    	*/
    	val numberRowsParamKey = "number"

    This variable is referenced in our GUI Node and the Spark job, in the next section.

  3. Add a function that defines the output schema for our results.

    This function is used at design time (while you are configuring the parameters on the operator, before running it). In this instance, create a tabular schema from a sequence of column definitions (a string and an int).

    def getOutputSchema(params: OperatorParameters) : TabularSchema = {
        // create an return a tabular schema object
          TabularSchema(Seq(
            ColumnDef("Thing", ColumnType.String), // the first field in our output dataset
            ColumnDef("Number", ColumnType.Int))  // the second field in our output dataset
          )
      }