Creating the Utils Object

Now we create a Utils object to hold some utility functions and constants that we will use across the different classes in our operator.

1. Create an object called DatasetGeneratorUtils:

   object DatasetGeneratorUtils {
   }

2. Within this object, we add two things: a constant for the parameter key we will use for input later in this tutorial, and a function to get the output schema of our result DataFrame.

   /**
    * The key for the parameter for the "Number of Things"
    */
   val numberRowsParamKey = "number"

   This variable will be referenced in our GUI Node and the Spark job, in the next section.

3. Next, you must add a function that defines the output schema for our results. This is used at design time (while you are configuring the parameters on the operator, before running it). In this instance, we create a tabular schema from a sequence of column definitions (a string and an int).

   def getOutputSchema(params: OperatorParameters) : TabularSchema = {
       // create an return a tabular schema object
         TabularSchema(Seq(
           ColumnDef("Thing", ColumnType.String), // the first field in our output dataset
           ColumnDef("Number", ColumnType.Int))  // the second field in our output dataset
         )
     }

Copyright © 2021. Cloud Software Group, Inc. All Rights Reserved.