Tibco

Integration of Machine Learning as a Service in TIBCO ModelOps

Contents

Overview

Integration of Machine Learning as a Service (MLaaS) lets TIBCO ModelOps to call REST-based external model endpoints for scoring. This integration uses python code in the Python Runner to support the functionality.

  • TIBCO ModelOps does not spin up any services in the cloud providers. Instead, it connects to the given URL endpoints that are running and available when the pipelines are created and run.
  • The external models are already trained and ready for scoring.
  • The user provides endpoint information for any customer-specific endpoints.
  • For customer-specific endpoints, the user provides input and output schema information for the services. User also defines how to translate from the endpoint output to the output schema.

Functionality

The following functionalities of Machine Learning as a Service are integrated with TIBCO ModelOps:

  • TIBCO ModelOps now calls web based external predefined model endpoints for scoring.
  • Users can now use Python Runner to support this functionality since no specific separate runner is now required.
  • Users get the following six predefined samples models to get users started and for demos
    • Text Sentiment Analysis
      • Amazon Comprehend
      • Azure Cognitive Services Text Analytics
      • Google Language Services
    • Image Tagging Analysis
      • Amazon Rekognition
      • Azure Vision API
      • Google Computer Vision

Architecture

The following diagram shows how MLaaS integration fits in TIBCO ModelOps and the larger environment.

The python code running in the Python Runner is used to call out to external model services that are hosted in the cloud. The score step is similar to other scoring steps. However, instead of the scoring done locally, the scoring is sent out to external services.
The python code uses the specific python libraries for the various services, which manage the lower-level REST calls.

Security

TIBCO ModelOps calls out to external web services that are secured by various authentication methods.

  • Authentication of model endpoints is provided by users via:
    • Username/password
    • Client ID and secret
    • JSON Service account file (e.g. Google)
    • TIBCO ModelOps OIDC integrated using services like AWS Cognito and Azure AD

The method of authentication and authentication information is specified by the users in the scoring flow.

Predefined Public Models

The predefined models cover the services exposed by Amazon, Azure, and Google. Most of the predefined public models used are free but do require some level of authentication to track the service user. The following services are supported in this version of TIBCO ModelOps:

For these services, there are predefined Python scripts, which need authentication and endpoint information from the user. The Python client libraries are used for the various services instead of doing direct REST calls. The input and output schemas are already defined.

Samples for Predefined Models

The samples for the predefined models are delivered as maven projects that include the TIBCO ModelOps artifacts, metadata, and documentation. The samples use the TIBCO Streaming GitHub repository.

Modifying Imported Samples

To modify imported samples:

  1. Import MLaaS sample project and obtain authentication information for service.
  2. Modify Python script with authentication details.
  3. Update the scoring flow to use the Sandbox model.
  4. Update the scoring pipeline to use sandbox flow.
  5. Approve the scoring pipeline and deploy it.
  6. Use the provided test script to generate results.