TIBCO Spotfire® Service Python Installation and Administration

Troubleshooting Spotfire Service for Python

If you have problems with the Spotfire Service for Python, review these tips.

Problems with the Dockerfile on a Linux node manager
After the service runs, you can view the Dockerfile that the Spotfire Service for Python writes. You can find the file in the root service directory (for example, /opt/tibco/tsnm/<server version_#>/nm/services/Python service Linux-<version_#_ID>/dockerfile/Dockerfile.
Try test building the Docker image in the environment before starting the Spotfire Service for Python.
Note: Building an image takes time, so it can take a few minutes for the web UI to display a possible image build failure. If a build failure occurs, the retry mechanism is triggered automatically.
Problems with the startup script
Check your script line endings.
Important: Remember that for any script you write, the line endings must be appropriate for the operating system where Spotfire Service for Python runs. Many text editors can perform end-of-line (EOL) conversion.
Empty String columns in exported SBDF causes an error
When you export an SBDF from Python (for example, as output for a data function), and your output contains an empty column, you can encounter the following error:
spotfire.sbdf.SBDFError: cannot determine type for column 'EmptyString'; all values are missing

This error occurs because the Spotfire data function environment cannot determine the proper Spotfire type to export the data as if all values in the column are missing (in other words, Python's None, NumPy's nan, or Panda's NA or NaT values).

To resolve this issue, edit your data function to use the helper function set_spotfire_types.

Example

import pandas as pd
import spotfire

df = pd.DataFrame(inp)
spotfire.set_spotfire_types(df, {'EmptyString': 'String'})
Memory usage
The maximum memory needed to run a Python script can exceed the javaOptions setting, depending on the size of the data, the Python packages used, and how the Python script is written.
Note: The javaOptions setting controls Java memory only, and does not control memory allocated in the Python runtime. (Memory allocated by Java is typically very small, which makes it possible to set a very low value for the initial heap size, such as Xms64M.)
If you encounter memory issues,
  • Run your script using a stand-alone Python engine to gauge its memory usage. You should see no difference between the memory needed for running your script in a stand-alone Python engine and running under the Spotfire Service for Python.
  • If you are running a containerized service, set a limit on the memory usage for the engine process. See the example under ram.limit in containerized_configuration.html#GUID-61DD110C-DA0E-4D16-9719-6AFE64CFB1A4.
  • Use utility functions such as memAvailable to find, report, and then reallocate available memory.