/
Configuring Kyvos Native Compute Cluster from Kyvos Manager

Configuring Kyvos Native Compute Cluster from Kyvos Manager

You can change from an external compute cluster to Kyvos Native for processing semantic models through Kyvos Manager on the Compute Cluster page:

Important

  • From Kyvos 2025.3 onwards, you can process data on a dedicated compute cluster on all clouds.

  • Switching from Kubernetes, shared Query Engine, or External Compute (EMR, Dataproc, and Databricks) to a dedicated compute cluster is supported. You can switch it from Kyvos Manager on the Compute Cluster page.

  • If using a shared Kubernetes cluster with a dedicated user node pool, ensure that the user node pool has dedicated taints. refer to the Adding Taints and Configure Tolerations in Kyvos worker pods section.

  • For shared K8s cluster, provide the compute namespace that has already been created.

  • You can optionally use the K8s cluster on a shared basis (AWS, Azure, and GCP).

  • You can now increase or decrease the Kubernetes node pool Maximum Compute Server Count. This applies only to a dedicated node pool.

  • If you have deployed Kyvos with the Kyvos Native compute cluster, you cannot switch to the External Compute cluster to process semantic models.

Prerequisites for Kubernetes cluster to be used with Kyvos

The following are the prerequisites for the K8s cluster to be used with Kyvos.

  • The Kubernetes cluster must be in the same region in all cloud environments.

  • For GCP, the Kubernetes must be in the same project.

  • For Azure, the Kubernetes cluster is supported even in a different Resource Group.

Note

To fetch the Kubernetes cluster from a different resource group, ensure that Kyvos Managed Identity must have reader on that Kubernetes.

  • For shared K8s cluster, provide the compute namespace already created. Currently, only in the case of a dedicated (non-shared) K8s cluster will the system attempt to create a comput namespace with the provided name on Azure Enterprise.

Configure Kyvos Native Cluster configurations

To configure the Compute cluster, perform the following steps.

  1. Click the cluster name > Kyvos and Ecosystem > Compute Cluster on the navigation pane to view this information.

  2. The Compute Cluster page displays the two tabs:

    • Kyvos Native: To process the semantic model using Kyvos native, click this tab to select the containerized Kubernetes, Shared Query Engines, or Dedicated Compute as a compute cluster.

    • External: Click this tab to view the External compute cluster details.

      image-20250331-091145.png

  3. Enter the details as:

  • Kyvos Native

Changing the default compute configurations

Note

By default, each semantic model is processed using the default compute type. If you want to override the compute type for a particular semantic model, change the value for the kyvos.process.compute.type property from the advanced semantic model property.

To process all of your semantic models using a different compute type, you can change the default compute configurations for all semantic models at once, rather than modifying each one individually. This approach eliminates the need to configure the property for each model separately.

You can Change the Node Pool Instance Type for Kyvos Enterprise (AWS, Azure, and GCP).

  • This functionality is only available for dedicated user node pool.

  • The option to delete the older node pool is available only for dedicated (non-shared) user node pool. If node pool deletion permissions are there, after selecting node pool deletion, it will be deleted automatically to avoid manual deletion of node pool from external to system.

If you are not deleting the older node pool, ensure to remove the taints from older node pool. If you want to configure new taints and toleration, then refer the Adding Taints and Configure Tolerations in Kyvos worker pods

  • The instance type configured for a node pool must be associated with a single instance type, even if multiple instance types are supported for the node pool in any cloud environment.

  • Changing the instance type for a Kubernetes (K8s) cluster node pool should only be supported for the node pool where Kyvos jobs will execute. Therefore, there will be no support for changing the instance type of any other node pool not configured in Kyvos. For Azure, only the user node pool will support instance type changes, while the system node pool will not support such changes.

  • For Azure, following permissions are required to change node pool instance type.

    • "Microsoft.ContainerService/managedClusters/agentPools/write",

    • "Microsoft.ContainerService/managedClusters/agentPools/read",

    • "Microsoft.ContainerService/managedClusters/agentPools/delete"

  • To support K8s cluster node pool instance type, change, add below required permissions to AWS Automated and Wizard based deployment IAM role.

    • Create Node Group

      "eks:CreateNodegroup"
      "eks:TagResource"
      "iam:GetRole"
      "ec2:DescribeKeyPairs"

    • Delete Node Group

      "eks:DeleteNodegroup"

To change the default compute cluster for the Kyvos semantic model, perform the following steps.

  1. Click the cluster name > Kyvos and Ecosystem > Compute Cluster on the navigation pane to view this information.

  2. On the Compute Cluster page, click the Click to change link on the right of the page. The following dialog box is displayed.

    image-20241127-073612.png

  3. Select the required Default Compute Type from the list.

  4. Enter your Kyvos Manager password.

  5. Click Save. The default compute cluster is configured.

Creating a dedicated compute cluster

You can create a dedicated compute cluster either by downloading the JSON file or manually create a compute cluster.

For AWS

Supported dedicate compute instance types

  • Graviton: m6g.4xlarge

  • Non graviton: m5.4xlarge

  • Disk size: 250GB

Case 1: By downloading the ‘PostCreationDedicatedComputeAWS.json’ template

Note

If you use ARM architecture instances (Graviton) for Kyvos, you must also use them for dedicated compute, similarly for non-Graviton instances.

The following permission must be manually added to the Lambda role. You can remove the permissions once the external dedicated compute is created from the stack.

  • In the case of post-configuration, the following permissions are required on the lambda role to create a separate autoscaling group.

    • CloudFormation:DescribeStacks: This is used to get stack tags.

    • ec2:DescribeLaunchTemplates: It is used to describe a launch template.

    • ec2:CreateLaunchTemplateVersion: This is used to create a new version of a launch template, as tags are not attached to the autoscaling group. Hence, provide tags using the lambda function.

    • ec2:ModifyLaunchTemplate: This is required to modify an existing launch template.

Once the above-mentioned permissions are added to the Lambda role, create a stack using the PostCreationDedicatedComputeAWS.json JSON file available in the AWS installation Files folder, which will create a dedicated auto-scaling group. After creating the stack, configure the dedicated compute from the Compute Cluster page using Kyvos Manager.

  1. Download the PostCreationDedicatedComputeAWS.json JSON file available in the AWS installation Files folder.

  2. On the AWS CloudFormation Console, Create Stack > With New Resources (Standard) option.

  3. Step 1: Specify template

  4. In the Prerequisite - Prepare template, select the Choose an existing template option.

  5. In the Specify template,  select the Upload a template file option to upload your CloudFormation template using the Choose File button.

  6. Click NEXT.

  7. Step 2: Specify stack details

  8. Enter a Stack name. Stack name can contain letters (A-Z and a-z), numbers (0-9), and dashes (-). The Kyvos cluster will be deployed in this name, and you cannot edit the name after deployment.

  9. The Parameters area displays the configurations defined in your template. It also allows you to enter custom values to create or update a stack.

Parameter

Description

Parameter

Description

Subnets

Select the subnet to be attached to the Autoscaling Group. You must select two subnets from different availability zones.

Ec2IamRole

Enter the name of the EC2 IAM Role that will be attached to Autoscaling group instances.

NOTE: Use the same role which is used for KM Instance.

LambdaIamRole

Enter the name of the lambda IAM role that will be attached to Autoscaling group instances. Use the same role which is used for Kyvos Manager Instance.

InstanceSecurityGroupOtherVMs

Enter the ID of the security group attached to the BI Server.

BucketName

Enter the name of S3 bucket used in Kyvos deployment.
NOTE: Kyvos does not support dot in the S3 bucket name.

DedicatedComputeMaxNodes

The maximum desired dedicated compute cluster nodes.

CustomPrefixEc2

Enter the prefix you want to append to the names of EC2 instances of Autoscaling group used with Kyvos. The prefix can be up to 20 characters long and must begin and end with a word character.

PemFile

A key pair consisting of a public key (stored by AWS), and a private key file (stored by user). Together, they allow you to connect to your instance securely. Use the same key which was used in Kyvos deployment.

KyvosWorkDir

Enter the work directory path used in the Kyvos deployment.

Copyright Kyvos, Inc. 2025. All rights reserved.