alpine.conf Spark Settings

The following settings are added to the Spark submission. You can edit Spark settings, Team Studio-specific Spark settings, and YARN settings. All of these values can be found and edited in the file alpine.conf. Any Spark tuning you define at the operator level takes precedence.

Spark parameters at the datasource level

As of version 6.2, you can specify any Spark parameter at the datasource configuration level in Team Studio. However, any Spark parameter you define in alpine.conf or at the operator level takes precedence.

Configure connection parameters

  • Spark Values
    The following settings are added to the Spark submission in alpine.conf, where you can edit them if you choose. While these can be used to define some overall settings, any Spark tuning you define at the operator level takes precedence.
  • Team Studio-Specific Spark Values
    Team Studio uses the following settings to determine the bulk of the Spark settings (if they are not set manually at the operator level).
  • YARN Configuration Values
    To read information about your cluster, we access the YARN API connected to it. If we cannot contact the YARN API (usually due to security or ports being blocked), we use the following default values for your cluster size.
Related reference