VALUES TO BE EXTRACTED FROM THIS STEP

• subscription_id
• DNS_ZONE

1. Subscription ID

   Your subscriptions should be listed in the portal, and for each, the subscription ID is listed

   []

   Assign to the variable name subscription_id the subscription ID you want to use. This variable is required in an upcoming command line directive. The subscription Id is a hash-type value such as : 79fd09g4-3345-4975-3057-d048c402ba89

2. Microsoft Entra ID (ME-ID) ACCESS

   From the home area of the Azure portal, select Microsoft Entra ID If you have ME-ID access, you should expect to see results such as:

   []

   If you do not have ME-ID access, you should expect to see results such as:

   []

3. DNS Zone (DNS Zone)

This zone will be the suffix appended to all hosts/urls created in your project. Your organization may have one or more. Within the Azure portal, search for “DNS zones to see if your organization has one.

Assign to the variable name DNS_ZONE the DNS zone you want to use. If you have none yet, you can create one. TIBCO DNS Zone Example: streamingaz.tibcocloud.com In this example streamingaz.tibcocloud.com is subzone of the parent domain tibcocloud.com.

1. Enable Oauth2

This file provides instructions for enabling OAuth2 during the installation process. For enabling OAuth2, refer to the Microsoft Entra ID documentation at the following link: Microsoft Entra ID

This section of the documentation provides the callback URL for Microsoft Entra ID. The callback URL is used for OAuth2 authentication.

Example: “fqdn”: “mms-server.lkvaks.streamingaz.tibcocloud.com/oauth2/callback”

ADDITIONAL VALUES REQUIRED FOR THIS STEP

• group

Assign to a variable named group some value. You can name your group ($CLUSTER_GROUP) however you wish, keeping in mind the naming conventions above in step 2.

NAMING CONVENTION TIP: The MMS Engineering team uses a convention where the name and the resource-group name are identical. This affords the user the benefit of needing to remember one less name when entering any given command. For instance, the command for cleanup takes both the name and the resource-group name:

az aks delete \

--yes \

--name=<Name> \

--resource-group=<ResourceGroupName>

Then you could then pass the same name for both <Name> and <ResourceGroupName>:

az aks delete \
    –-yes \
    -–name=prodmms \
    –-resource-group=prodmms

• location

  Assign to location your region. Choose the location – this is the region your organization is hosted in Azure. You may choose your organization’s region based on locality and network latency, or because you have sovereignty requirements for data centers. Your organization’s default location is based on the closest Microsoft Azure region available. Example: eastus. For a list of all geographies, See https://azure.microsoft.com/en-us/global-infrastructure/geographies/#geographies

Once you have the group and location variables set, issue the following command:

az group create \
        --name=$CLUSTER_GROUP \
        --location=$CLUSTER_LOCATION \
        --output=table

Output of the above command is a table listing the location and the specified group name:

ADDITIONAL VALUES REQUIRED FOR THIS STEP

• acr

  Assign to acr the name of your Azure Container Registry(ACR). You can name your ACR as you wish, keeping in mind the naming conventions above in step 2.

Create the Azure Container Registry. This command will create the registry and in doing so, create a registry ID. You must supply the group name ($CLUSTER_GROUP) from the previous command. Once you have the acr variable set, issue the following command:

az acr create \
        --name=$AZURE_CONTAINER_REGISTRY \
        --resource-group=$CLUSTER_GROUP \
        --sku=basic \
        --output=table

Output of above command is a table:

VALUES TO BE EXTRACTED FROM THIS STEP

• acr_registry_id

az acr show \
            --name=$AZURE_CONTAINER_REGISTRY \
            --query=id \
            --output=tsv

Assign the output of the above command to the parameter acr_registry_id.

ADDITIONAL VALUES REQUIRED FOR THIS STEP

• AZURE_WINPASSWORD
• AZURE_WINUSER
• AUTHORIZEDIP

The AZURE_WINPASSWORD and AZURE_WINUSER values set the admin credentials for any Windows Server containers created on the cluster and must meet Windows Server password requirements. If you don’t specify the windows-admin-password parameter, you will be prompted to provide a value.

Example: AZURE_WINPASSWORD=P@ssw0rd1234567!

Example: AZURE_WINUSER=azureuser

• cluster

  You can name your cluster as you wish, keeping in mind the naming conventions described in [registering the application] (#register-application). Failure to adhere to this convention will lead to an error such as this when the $CLUSTER_NAME parameter is used to name a DNS domain:

  *a DNS-1123 subdomain must consist of lower case alphanumeric characters, ‘-’ or ‘.’, and must start and end with an alphanumeric character (e.g. ‘example.com’, regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')

  Authorize IP Address: The command authorizedIP=$(curl ifconfig.me) dynamically retrieves your machine's public IP address. This is essential for securely configuring access to the AKS cluster. By using this IP address in the –api-server-authorized-ip-ranges parameter, you ensure that only your current machine or network can access the Kubernetes API server. This step enhances security by restricting unauthorized access while simplifying the setup process for users. If you need to allow additional machines or networks to access the cluster in the future, you can update the –api-server-authorized-ip-ranges setting to include their public IP addresses. This can be done through the Azure CLI or the Azure portal. For more details on managing authorized IP ranges, refer to the official Azure documentation.

  authorizedIP=$(curl ifconfig.me)
    echo "Ipaddress: ${authorizedIP}"
    

  Assign values to AZURE_WINPASSWORD, AZURE_WINUSER, AUTHORIZEDIP, and cluster. Once you have those values set, issue the following command:

az aks create \
            --resource-group=$CLUSTER_GROUP \
            --service-principal=$SERVICE_PRINCIPAL_ID \
            --client-secret=$CLIENT_SECRET_KEY \
            --name=$CLUSTER_NAME \
            --max-pods=200 \
            --node-count 1 \
            --enable-cluster-autoscaler \
            --api-server-authorized-ip-ranges ${authorizedIP} \
            --min-count=1 \
            --max-count=5 \
            --no-ssh-key \
            --windows-admin-password=$AZURE_WINPASSWORD \
            --windows-admin-username=$AZURE_WINUSER \
            --vm-set-type=VirtualMachineScaleSets \
            --node-vm-size=Standard_B8ms  \
            --network-plugin=azure \
            --attach-acr=$acr_registry_id \
            --output=table

This command will display a message such as

Running . .

And takes several minutes. When it completes, you will see a message such as:

VALUES TO BE EXTRACTED FROM THIS STEP

• nodegroup

Execute these commands to get the node group:

az aks get-credentials \
        --overwrite-existing \
        --resource-group=$CLUSTER_GROUP \
        --name=$CLUSTER_NAME
kubectl get node \
        -ocustom-columns=name:metadata.name,image:status.nodeInfo.osImage,cpu:status.capacity.cpu,memory:status.capacity.memory,status:.status.conditions[-1].type

Output from above command should be akin to this:

az aks show \
        --resource-group=$CLUSTER_GROUP \
        --name=$CLUSTER_NAME \
        --query=nodeResourceGroup \
        -o tsv

Output from above command should be akin to:

Assign the output of the above ‘show’ command to nodegroup

Note: If your AKS (Azure Kubernetes Service) cluster is not enabled with public access, you may need to whitelist your IP address to ensure connectivity. This is crucial for accessing the AKS cluster from your local machine or external services. For detailed steps on retrieving and authorizing your IP address, refer to the Authorize IP Address section above. This step is essential for secure and successful communication with your AKS cluster.

ADDITIONAL VALUES REQUIRED FOR THIS STEP

• ELASTIC_PW
• GIT_SERVER_PW
• NEXUS_SERVER_PW
• PROMETHEUS_PW
• SCORING_ADMIN_PW
• ARTIFACT_MANAGEMENT_PW
• Oauth2 server details
• External DNS details

In order to avoid clear text passwords, Kubernetes provides a Secrets facility. So prior to installation, Kubernetes Secrets have to be created to contain the passwords required by MMS.

Assign each of the names above to some value, then issue the following commands to set up the secrets.

NOTE: Without secrets installed, the [helm installation] (#install-helm-chart) step will be in a wait mode and eventually time out.

Clear out any old secrets that may exist with this series of delete secret commands:

    kubectl delete secret git-server --namespace mms
    kubectl delete secret nexus-server --namespace mms
    kubectl delete secret prometheus-server --namespace mms
    kubectl delete secret scoring-admin --namespace mms
    kubectl delete secret oauth2 --namespace mms
    kubectl delete secret externaldns-config --namespace mms
    kubectl delete secret artifact-management --namespace mms
    kubectl delete secret artifact-management-env --namespace mms

Create new secrets:

    kubectl create secret generic elasticsearch-es-elastic-user \
        --from-literal=elastic=$ELASTIC_PW \
        --namespace mms --dry-run=client --output=yaml 2>/dev/null > secret.yaml
    kubectl apply --filename secret.yaml
    kubectl create secret generic git-server \
        --from-literal=streaming-web=$GIT_SERVER_PW \
        --namespace mms
    kubectl create secret generic nexus-server \
        --from-literal=admin=$NEXUS_SERVER_PW \
        --namespace mms
    kubectl create secret generic prometheus-server \
        --from-literal=admin=$PROMETHEUS_PW \
        --namespace mms
    kubectl create secret generic scoring-admin \
        --from-literal=admin=$SCORING_ADMIN_PW \
        --namespace mms
    kubectl create secret generic artifact-management \
        --from-literal=admin=$ARTIFACT_MANAGEMENT_PW \
        --namespace mms
    kubectl create secret generic artifact-management-env --namespace mms

NOTE: The Elasticsearch password is limited to alphanumeric, “.”, “_”, “~”, and “-” characters, i.e. it must conform to this regular expression (‘^[a-zA-Z0-9._~-]+$’)).

NOTE: The prometheus-server secret is only used if you are not using OAuth2 authentication; if using OAuth2, this secret need not be created. It should consist of name value pairs where each name is a user name, and each value is the corresponding password. These are used to secure the Prometheus ingress with basic authentication.

Auto update of public DNS is possible by enabling the (external DNS)[https://github.com/kubernetes-sigs/external-dns] component - without this the DNS will need to be updated manually. External DNS on azure requires a secret containing azure credentials that has permissions to update DNS (tenant id, subscription id, dns resource group and client secret.

kubectl create secret generic externaldns-config --from-literal=azure.json="{ \"tenantId\": \"$TENANT_ID\", \"subscriptionId\": \"$subscription_id\", \"resourceGroup\": \"$dns_group\", \"aadClientId\": \"$SERVICE_PRINCIPAL_ID\", \"aadClientSecret\": \"$CLIENT_SECRET_KEY\" }" --namespace mms

The oauth2 secret depends on the type of authentication server used.

For Azure, the authentication server administrator should supply the Azure tenant id ($AZURE_TENANT_ID), app id ($AZURE_APP_ID) and client secret ($AZURE_CLIENT_SECRET). The secret is created with:

kubectl create secret generic oauth2 \
        --from-literal=TENANT_ID=$TENANT_ID \
        --from-literal=CLIENT_ID=$SERVICE_PRINCIPAL_ID \
        --from-literal=CLIENT_SECRET=$CLIENT_SECRET_KEY \
        --namespace mms

For Cognito, the authentication server administrator should supply the Cognito region ($COGNITO_REGION), pool id ($COGNITO_POOL_ID), client id ($COGNITO_CLIENT_ID), client secret ($COGNITO_CLIENT_SECRET), and the domian ($COGNITO_DOMAIN). The secret is created with:

kubectl create secret generic oauth2 \
        --from-literal=REGION=$COGNITO_REGION \
        --from-literal=POOL_ID=$COGNITO_POOL_ID \
        --from-literal=CLIENT_ID=$COGNITO_CLIENT_ID \
        --from-literal=CLIENT_SECRET=$COGNITO_CLIENT_SECRET \
        --from-literal=DOMAIN=$COGNITO_DOMAIN \
        --namespace mms

For Okta, the authentication server administrator should supply the Okta client id ($OKTA_CLIENT_ID), client secret ($OKTA_CLIENT_SECRET), the domain ($OKTA_DOMAIN), and the authorization server id ($OKTA_AUTH_SERVER). The secret is created with:

    kubectl create secret generic oauth2 \
        --from-literal=CLIENT_ID=$OKTA_CLIENT_ID \
        --from-literal=CLIENT_SECRET=$OKTA_CLIENT_SECRET \
        --from-literal=DOMAIN=$OKTA_DOMAIN \
        --from-literal=AUTH_SERVER_ID=$OKTA_AUTH_SERVER \
        --namespace mms

ADDITIONAL VALUES REQUIRED FOR THIS STEP

• name

  This is NOT the display name of the application registered at the beginning of this process. Rather, this is usually set to be the same as $CLUSTER_NAME

• subscription_id

  See the prerequisite steps above if you have not yet set this value.

• domain

Assign to this the combination of existing parameters: $CLUSTER_NAME.$DNS_ZONE

• subdomain

  This is an optional variable where you can specify the desired subdomain name. If it's not defined, the default value will be ‘streaming-web’.

• network_exposure

  Assign to this the string “ingress” (without the quotes)

• oauth2_server_type

  One of azure, cognito, or okta to use an external Oauth2 authentication server.
  If you are NOT using oauth, then you should NOT set the oauth parameter (–set aks.oauth2=$OUTH2_SERVER_TYPE) while installing the helm chart.

Optional: If you wish to display the helm chart, run this command:

helm show values kubernetes-installer-1.4.0.tgz

Assign each of the names above to an appropriate value, then issue the following command:

helm upgrade \
        --install mms $MMS_HOME/mms/kubernetes-installer-1.4.0.tgz \
        --atomic \
        --set cloud=aks \
        --set aks.externalDNS=azure \
        --set aks.containerRegistry=$AZURE_CONTAINER_REGISTRY.azurecr.io \
        --set aks.containerUsername=$SERVICE_PRINCIPAL_ID \
        --set aks.containerPassword=$CLIENT_SECRET_KEY \
        --set aks.azureTenantId=$TENANT_ID \
        --namespace mms \
        --set aks.networkExposure=$NETWORK_EXPOSURE \
        --set aks.ingressDomain=$DOMAIN \
        --set aks.ingressSubdomain=$subdomain \
        --set aks.oauth2=$OUTH2_SERVER_TYPE \
        --timeout 10m0s

The above command will produce a series of lines of output, including a thank you, and ending with a note for how to track the progress of the installation pipeline. See [monitoring the installation] (#monitor-installation) for more details.

Delete all completed TaskRuns in the namespace

tkn tr list --namespace mms -o name | sed 's#taskrun.tekton.dev/##' | xargs -I {} tkn tr delete {} --namespace mms -f

Delete all completed jobs in the namespace

kubectl delete jobs --all --namespace mms

If you want to keep some tasks or jobs for debugging purposes, you can use the following command to keep specific TaskRuns & jobs:

List all TaskRuns in the specified namespace, filter out the one to keep, and delete the rest

tkn tr list --namespace mms -o name | sed 's#taskrun.tekton.dev/##' | grep -v -E "$taskrun1_to_keep|taskrun2_to_keep" | xargs -I {} tkn tr delete {} --namespace mms -f

Replace ‘job1-to-keep’ and ‘job2-to-keep’ with the names of the jobs you want to keep

kubectl get jobs --namespace mms -o jsonpath='{.items[*].metadata.name}' | tr ' ' '\n' | grep -v -E '^(job1-to-keep|job2-to-keep)$' | xargs -I {} kubectl delete job --namespace mms {}

VALUES TO BE EXTRACTED FROM THIS STEP

• ingress_lb_ip

ADDITIONAL VALUES REQUIRED FOR THIS STEP

• DNS_RESOURCE_GROUP

DNS_RESOURCE_GROUP is the name of the resource-group which was used to create a DNS zone. The resource group name must be unique within the Azure subscription Example: DNS_RESOURCE_GROUP=sbazclouddnszone.

Issue the following command:

kubectl get \
        service/nginx-ingress-ingress-nginx-controller \
        -o jsonpath='{.status.loadBalancer.ingress[0].ip}' \
        --namespace mms

You should see output such as this:

Example: set ingress_lb_ip=52.186.40.167

Once your ingress_lb_ip parameter is set, clear out old record-sets by issuing the following command:

az network dns record-set a delete \
        --yes \
        --resource-group=$DNS_RESOURCE_GROUP \
        --zone-name=$DNS_ZONE \
        --name=mms.$CLUSTER_NAME

Assign a value to DNS_RESOURCE_GROUP and issue the following command to create new record-set:

az network dns record-set a add-record \
        --resource-group=$DNS_RESOURCE_GROUP \
        --zone-name=$DNS_ZONE \
        --record-set-name=mms.$CLUSTER_NAME \
        --ipv4-address=$INGRESS_LB_IP

You should see output such as this:

If API v1 integration is required to Team Studio, the mms-1 host should also be added to DNS in a similar way:

az network dns record-set a delete \
        --yes \
        --resource-group=$DNS_RESOURCE_GROUP \
        --zone-name=$DNS_ZONE \
        --name mms-1.$CLUSTER_NAME
az network dns record-set a add-record \
        --resource-group=$DNS_RESOURCE_GROUP \
        --zone-name=$DNS_ZONE \
        --record-set-name=mms-1.$CLUSTER_NAME \
        --ipv4-address=$INGRESS_LB_IP