This section details how to configure the Bring Your Own LLM (BYO-LLM) service in Kyvos to integrate a custom Large Language Model with Kyvos Dialogs for conversational analytics and MDX generation.

You can configure the "Bring Your Own LLM Service" feature in Kyvos. This feature enables AI/BI developers to integrate their custom Large Language Models (LLMs) service into the Kyvos platform for Kyvos Dialogs to work.

This framework potentially enables Kyvos Dialogs to use any LLM service for Conversational Analytics and MDX based KPI Generation.

This document covers steps to upload, configure, and validate your LLM service integration code.

Prerequisites

Ensure you have the following before starting:

Kyvos Manager Access: Required for uploading and configuring files.
Custom LLM Implementation: A JAR file having code to integrate with custom LLM service for all the operations that Kyvos supports.
Metadata JSON: A JSON file having all the required details for LLM service configuration and properties.
API Keys and Credentials Required for connecting to your LLM service.

Configuring Java Code And Metadata for Kyvos-LLM Integration

Step 1: Implement the Callback interfaces

GenAICallbackService Interface: To enable communication between Kyvos and your custom LLM service, you must implement the necessary methods defined in the GenAICallbackService interface.
The following methods need to be implemented:

public interface GenAICallbackService {

/**

This method is used to generate sample questions which will be displayed in copilot screen. It has arguments as user prompt which contains prompt and key elements which are marked on semantic model.
GenAIResponse contains a JSON array which has questions and its explanation.
@param userPrompt
@param genAIConnection
@return
@throws IOException
@throws InterruptedException
@throws GenAiServiceException

*/
GenAIResponse generateSampleQuestions(String userPrompt, GenAIConnection genAIConnection) throws IOException, InterruptedException, GenAiServiceException;

/**

This method is used to get list of semantic model ids having user prompt as semantic model details and NL text through which we get similar id's
@param userPrompt
@param genAIConnection
@return
@throws GenAiServiceException
@throws IOException

*/
GenAIResponse getSemanticModelsBasedOnNLText(String userPrompt, GenAIConnection genAIConnection) throws GenAiServiceException, IOException;

/**

This method is used to determine the type for natural language query, as Kyvos Copilot supports both business queries and command executions.
It helps determine the user's requested natural language query and decide how to execute it.
@param userPrompt
@param systemPrompt
@param genAIConnection
@return
@throws GenAiServiceException
@throws IOException

*/
GenAIResponse getNlQueryType(String userPrompt, String systemPrompt, GenAIConnection genAIConnection) throws GenAiServiceException, IOException;

/**

This method is used to execute commands like viz settings or other metadata commands supported (other then business queries) in Kyvos copilot.
@param userPrompt
@param systemPrompt
@param genAIConnection
@return
@throws GenAiServiceException
@throws IOException

*/
GenAIResponse executeCommand(String userPrompt, String systemPrompt, GenAIConnection genAIConnection) throws GenAiServiceException, IOException;

/**

This method is used to generate summary title based on spark SQL generated and result data after execution of SQL. It accepts user prompt as argument.
@param userPrompt
@param connection
@return
@throws GenAiServiceException
@throws IOException
@throws InterruptedException

*/
GenAIResponse generateSummaryTitle(String userPrompt, GenAIConnection connection) throws GenAiServiceException, IOException, InterruptedException;

/**

This method is used to generate summary and key insights based on spark SQL generated and result data after execution of SQL. It accepts user prompt and system prompt as argument.
@param userPrompt
@param systemPrompt
@param genAIConnection
@param jsonResponse
@return
@throws GenAiServiceException
@throws IOException

*/
GenAIResponse generateSummary(String userPrompt, String systemPrompt, GenAIConnection genAIConnection, boolean jsonResponse) throws GenAiServiceException, IOException;

/**

This method is used to get SQL from NL text passed by user. It takes user prompt and system prompt as arguments and return a JSON containing SQL.
@param userPrompt
@param systemPrompt
@param genAIConnection
@return
@throws GenAiServiceException
@throws IOException

*/
GenAIResponse getSQLFromNLText(String userPrompt, String systemPrompt, GenAIConnection genAIConnection) throws GenAiServiceException, IOException;

/**

This method is used to get SQL from NL text passed by use in case the first generated SQL fails it reattempts to get SQL from LLM. Here user gets sqlRetryJson as
an extra argument which contains last generated SQL and exception.
@param userPrompt
@param systemPrompt
@param genAIConnection
@param sqlRetryJson
@return
@throws GenAiServiceException
@throws IOException

*/
GenAIResponse getSQLFromRetry(String userPrompt, String systemPrompt, GenAIConnection genAIConnection, String sqlRetryJson) throws GenAiServiceException, IOException;
}

GenAICallbackService Interface: To integrate the custom embedding, you need to implement the GenAIEmbeddingCallbackService interface

public interface GenAIEmbeddingCallbackService {

/**

This core method is used to generate embeddings.
Implementation of this method should include below basic implementations:
1. input can be a text for whose embedding should be generated and returned in the form of text only.
2. input can be a List<String> for whose embeddings need to be generated for each column. A Map<String, Vector<Double>> has to be returned containing column names and embeddings.
3. input can be list of RAG records where it contains all information related to columns of a semantic model. User needs to return same List<RAGRecords> with adding a embedding against a new entry "embeddingColumn"
If any input is missed flow can be impacted.
@param input
@param embeddingColumn
@param genAIConnection
@param logResponse
@return
@throws IOException
@throws GenAiServiceException

     */
    Object processEmbeddings(Object input, String embeddingColumn,
                          GenAIConnection genAIConnection, boolean logResponse) throws IOException, GenAiServiceException;
}

Important

Both the interfaces should be implemented in a single class which will be required to configure as an endpoint on Kyvos Manager.
For example: public class BedrockCallbackServiceImpl implements GenAICallbackService, GenAIEmbeddingCallbackService { }

Step 2: Creating the category.xml file for connection

Create a category.xml file that holds details about all the configuration needed to integrate your custom LLM service with Kyvos. Using this file, Kyvos Manager will create a connection UI specific to this LLM service.
The category.xml file must use the following:

<CATEGORIES>
    <CATEGORY NAME="LLM">
        <PROVIDER NAME="aws-bedrock" DISPLAY_TEXT="AWS Bedrock" ALLOW_ADD_CONNECTION="true" ALLOW_MULTIPLE_CONNECTIONS="true" TYPE="READ">
            <PROVIDERDETAILS version="" ISADVANCEPROPERTY="FALSE">
                <PROVIDERINFO>
                    <URLPARAMS>
                        <PARAM TYPE="TEXT">
                            <NAME>
                                <![CDATA[kyvos.connection.name]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblConnectionName]]>
                            </DISPLAYNAME>
                        </PARAM>
                        <PARAM TYPE="PASSWORD">
                            <NAME>
                                <![CDATA[aws_bedrock_access_id]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[AWS Bedrock Access ID]]>
                            </DISPLAYNAME>
                            <DESC>
                                <![CDATA["The property provides the access id for accessing aws services."]]>
                            </DESC>
                        </PARAM>
                        <PARAM TYPE="PASSWORD">
                            <NAME>
                                <![CDATA[aws_bedrock_secret_key]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[AWS Bedrock Secret Key]]>
                            </DISPLAYNAME>
                            <DESC>
                                <![CDATA["The property provides the secret key for accessing aws services."]]>
                            </DESC>
                        </PARAM>
                        <PARAM TYPE="TEXT">
                            <NAME>
                                <![CDATA[aws_region_llm]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[AWS Region LLM]]>
                            </DISPLAYNAME>
                            <DESC>
                                <![CDATA["The property provides the region for llm"]]>
                            </DESC>
                        </PARAM>
                        <PARAM TYPE="TEXT">
                            <NAME>
                                <![CDATA[model]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[Model]]>
                            </DISPLAYNAME>
                            <DESC>
                                <![CDATA["TThe property specifies the name of the GenAI Large Language Model (LLM) used to generate the output."]]>
                            </DESC>
                        </PARAM>
                        <PARAM TYPE="COMBO">
                            <NAME>
                                <![CDATA[is_model_fine_tuned]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblIsModelFineTuned]]>
                            </DISPLAYNAME>
                            <DESC>
                                <![CDATA[genai.descIsModelFineTuned]]>
                            </DESC>
                            <DEFAULTVALUE>
                                <![CDATA[false]]>
                            </DEFAULTVALUE>
                            <PREDEFVALUES>
                                <PREDEFVALUE>
                                    <DISPLAYNAME>
                                        <![CDATA[genai.lblYes]]>
                                    </DISPLAYNAME>
                                    <VALUE>
                                        <![CDATA[true]]>
                                    </VALUE>
                                </PREDEFVALUE>
                                <PREDEFVALUE>
                                    <DISPLAYNAME>
                                        <![CDATA[genai.lblNo]]>
                                    </DISPLAYNAME>
                                    <VALUE>
                                        <![CDATA[false]]>
                                    </VALUE>
                                </PREDEFVALUE>
                            </PREDEFVALUES>
                        </PARAM>
                        <PARAM TYPE="MULTISELECTCOMBO">
                            <NAME>
                                <![CDATA[applicableUsecases]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblUsage]]>
                            </DISPLAYNAME>
                            <DESC>
                                <![CDATA[genai.descUsage]]>
                            </DESC>
                            <DEFAULTVALUE>
                                <![CDATA[CONVERSATIONAL_AI]]>
                            </DEFAULTVALUE>
                            <PREDEFVALUES>
                                <PREDEFVALUE>
                                    <DISPLAYNAME>
                                        <![CDATA[genai.lblMdxGeneration]]>
                                    </DISPLAYNAME>
                                    <VALUE>
                                        <![CDATA[CALCULATIONS]]>
                                    </VALUE>
                                </PREDEFVALUE>
                                <PREDEFVALUE>
                                    <DISPLAYNAME>
                                        <![CDATA[genai.lblConversationalAI]]>
                                    </DISPLAYNAME>
                                    <VALUE>
                                        <![CDATA[CONVERSATIONAL_AI]]>
                                    </VALUE>
                                </PREDEFVALUE>
                                <PREDEFVALUE>
                                    <DISPLAYNAME>
                                        <![CDATA[genai.lblNaturalLanguageSummary]]>
                                    </DISPLAYNAME>
                                    <VALUE>
                                        <![CDATA[NATURAL_LANGUAGE_SUMMARY]]>
                                    </VALUE>
                                </PREDEFVALUE>
                            </PREDEFVALUES>
                        </PARAM>
                        <PARAM TYPE="BOOLEAN">
                            <NAME>
                                <![CDATA[defaultConnectionConversationalAI]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblDefaultConnectionForConversationalAnalytics]]>
                            </DISPLAYNAME>
                            <DEFAULTVALUE>
                                <![CDATA[false]]>
                            </DEFAULTVALUE>
                        </PARAM>
                        <PARAM TYPE="BOOLEAN">
                            <NAME>
                                <![CDATA[defaultConnectionMdxGeneration]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblDefaultConnectionForMDXGeneration]]>
                            </DISPLAYNAME>
                            <DEFAULTVALUE>
                                <![CDATA[false]]>
                            </DEFAULTVALUE>
                        </PARAM>
                        <PARAM TYPE="BOOLEAN">
                            <NAME>
                                <![CDATA[defaultConnectionNaturalLanguageSummary]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblDefaultConnectionForNaturalLanguageSummary]]>
                            </DISPLAYNAME>
                            <DEFAULTVALUE>
                                <![CDATA[false]]>
                            </DEFAULTVALUE>
                        </PARAM>
                        <PARAM TYPE="COMBO">
                            <NAME>
                                <![CDATA[allow_values_for_nlp_generation]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblAllowSendingDatatoLLM]]>
                            </DISPLAYNAME>
                            <DESC>
                                <![CDATA[genai.descAllowSendingDatatoLLM]]>
                            </DESC>
                            <DEFAULTVALUE>
                                <![CDATA[true]]>
                            </DEFAULTVALUE>
                            <PREDEFVALUES>
                                <PREDEFVALUE>
                                    <DISPLAYNAME>
                                        <![CDATA[genai.lblYes]]>
                                    </DISPLAYNAME>
                                    <VALUE>
                                        <![CDATA[true]]>
                                    </VALUE>
                                </PREDEFVALUE>
                                <PREDEFVALUE>
                                    <DISPLAYNAME>
                                        <![CDATA[genai.lblNo]]>
                                    </DISPLAYNAME>
                                    <VALUE>
                                        <![CDATA[false]]>
                                    </VALUE>
                                </PREDEFVALUE>
                            </PREDEFVALUES>
                            <APPLICABLE_FOR_SM>
                                <![CDATA[true]]>
                            </APPLICABLE_FOR_SM>
                            <SEMANTIC_MODEL_SECTION>
                                <![CDATA[NLSummary]]>
                            </SEMANTIC_MODEL_SECTION>
                        </PARAM>
                        <PARAM TYPE="MULTISELECTCOMBO">
                            <NAME>
                                <![CDATA[generate_content]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblGenerateContent]]>
                            </DISPLAYNAME>
                            <DESC>
                                <![CDATA[genai.descGenerateContent]]>
                            </DESC>
                            <DEFAULTVALUE>
                                <![CDATA[TITLE,OVERVIEW,KEY_INSIGHTS]]>
                            </DEFAULTVALUE>
                            <PREDEFVALUES>
                                <PREDEFVALUE>
                                    <DISPLAYNAME>
                                        <![CDATA[genai.lblTitle]]>
                                    </DISPLAYNAME>
                                    <VALUE>
                                        <![CDATA[TITLE]]>
                                    </VALUE>
                                </PREDEFVALUE>
                                <PREDEFVALUE>
                                    <DISPLAYNAME>
                                        <![CDATA[genai.lblOverview]]>
                                    </DISPLAYNAME>
                                    <VALUE>
                                        <![CDATA[OVERVIEW]]>
                                    </VALUE>
                                </PREDEFVALUE>
                                <PREDEFVALUE>
                                    <DISPLAYNAME>
                                        <![CDATA[genai.lblKeyInsights]]>
                                    </DISPLAYNAME>
                                    <VALUE>
                                        <![CDATA[KEY_INSIGHTS]]>
                                    </VALUE>
                                </PREDEFVALUE>
                            </PREDEFVALUES>
                        </PARAM>
                        <PARAM TYPE="TEXT">
                            <NAME>
                                <![CDATA[max_rows_summary]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblMaxRowsSummary]]>
                            </DISPLAYNAME>
                            <DEFAULTVALUE>
                                <![CDATA[1000]]>
                            </DEFAULTVALUE>
                            <DESC>
                                <![CDATA[genai.descMaxRowsSummary]]>
                            </DESC>
                            <APPLICABLE_FOR_SM>
                                <![CDATA[true]]>
                            </APPLICABLE_FOR_SM>
                            <SEMANTIC_MODEL_SECTION>
                                <![CDATA[NLSummary]]>
                            </SEMANTIC_MODEL_SECTION>
                        </PARAM>
                        <PARAM TYPE="TEXT">
                            <NAME>
                                <![CDATA[prompt_token_limit]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblInputPromptTokenLimit]]>
                            </DISPLAYNAME>
                            <DEFAULTVALUE>
                                <![CDATA[16000]]>
                            </DEFAULTVALUE>
                            <DESC>
                                <![CDATA[genai.descInputPromptTokenLimit]]>
                            </DESC>
                        </PARAM>
                        <PARAM TYPE="TEXT">
                            <NAME>
                                <![CDATA[max_completion_tokens]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblOutputPromptTokenLimit]]>
                            </DISPLAYNAME>
                            <DEFAULTVALUE>
                                <![CDATA[2048]]>
                            </DEFAULTVALUE>
                            <DESC>
                                <![CDATA[genai.descOutputPromptTokenLimit]]>
                            </DESC>
                        </PARAM>
                        <PARAM TYPE="TEXT">
                            <NAME>
                                <![CDATA[max_retry_count_for_sql_execution]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblMaxRetryCount]]>
                            </DISPLAYNAME>
                            <DEFAULTVALUE>
                                <![CDATA[2]]>
                            </DEFAULTVALUE>
                            <DESC>
                                <![CDATA[genai.descMaxRetryCount]]>
                            </DESC>
                        </PARAM>
                        <PARAM TYPE="TEXT">
                            <NAME>
                                <![CDATA[similarity_threshold]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblSummaryRecordsThreshold]]>
                            </DISPLAYNAME>
                            <DEFAULTVALUE>
                                <![CDATA[0.1]]>
                            </DEFAULTVALUE>
                            <DESC>
                                <![CDATA[genai.descSummaryRecordsThreshold]]>
                            </DESC>
                        </PARAM>
                        <PARAM TYPE="TEXT">
                            <NAME>
                                <![CDATA[temperature]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblLLMTemperature]]>
                            </DISPLAYNAME>
                            <DEFAULTVALUE>
                                <![CDATA[1]]>
                            </DEFAULTVALUE>
                            <DESC>
                                <![CDATA[genai.descLLMTemperature]]>
                            </DESC>
                        </PARAM>
                    </URLPARAMS>
                    <ADVANCEPROPERTY>
                        <FILENAME/>
                    </ADVANCEPROPERTY>
                </PROVIDERINFO>
            </PROVIDERDETAILS>
        </PROVIDER>
    </CATEGORY>
    <CATEGORY NAME="EMBEDDING">
        <PROVIDER NAME="aws-bedrock" DISPLAY_TEXT="AWS Embedding Provider" ALLOW_ADD_CONNECTION="true"
              ALLOW_MULTIPLE_CONNECTIONS="true" TYPE="READ">
            <PROVIDERDETAILS version="" ISADVANCEPROPERTY="TRUE">
                <PROVIDERINFO>
                    <URLPARAMS>
                        <PARAM TYPE="TEXT" READONLY="true">
                            <NAME>
                                <![CDATA[kyvos.connection.name]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[Connection.lblConnectionName]]>
                            </DISPLAYNAME>
                        </PARAM>
                        <PARAM TYPE="PASSWORD">
                            <NAME>
                                <![CDATA[aws_bedrock_access_id]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[AWS Bedrock Access ID]]>
                            </DISPLAYNAME>
                            <DESC>
                                <![CDATA["The property provides the access id for accessing aws services."]]>
                            </DESC>
                        </PARAM>
                        <PARAM TYPE="PASSWORD">
                            <NAME>
                                <![CDATA[aws_bedrock_secret_key]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[AWS Bedrock Secret Key]]>
                            </DISPLAYNAME>
                            <DESC>
                                <![CDATA["The property provides the secret key for accessing aws services."]]>
                            </DESC>
                        </PARAM>
                        <PARAM TYPE="COMBO">
                            <NAME>
                                <![CDATA[embedding_model_name]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblModel]]>
                            </DISPLAYNAME>
                            <DESC>
                                <![CDATA[genai.descModel]]>
                            </DESC>
                            <DEFAULTVALUE>
                                <![CDATA[amazon.titan-embed-text-v2:0]]>
                            </DEFAULTVALUE>
                            <PREDEFVALUES>
                                <PREDEFVALUE>
                                    <DISPLAYNAME>
                                        <![CDATA[Amazon Titan Text V2]]>
                                    </DISPLAYNAME>
                                    <VALUE>
                                        <![CDATA[amazon.titan-embed-text-v2:0]]>
                                    </VALUE>
                                </PREDEFVALUE>
                            </PREDEFVALUES>
                        </PARAM>
                        <PARAM TYPE="BOOLEAN">
                            <NAME>
                                <![CDATA[defaultConnectionConversationalAI]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblDefaultConnectionForConversationalAnalytics]]>
                            </DISPLAYNAME>
                            <DEFAULTVALUE>
                                <![CDATA[false]]>
                            </DEFAULTVALUE>
                        </PARAM>
                        <PARAM TYPE="TEXT">
                            <NAME>
                                <![CDATA[prompt_token_limit]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblPromptTokenLimit]]>
                            </DISPLAYNAME>
                            <DESC>
                                <![CDATA[genai.descPromptTokenLimit]]>
                            </DESC>
                            <DEFAULTVALUE>
                                <![CDATA[8000]]>
                            </DEFAULTVALUE>
                        </PARAM>
                        <PARAM TYPE="TEXT">
                            <NAME>
                                <![CDATA[upper_threshold]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblSimilarityUpperThreshold]]>
                            </DISPLAYNAME>
                            <DESC>
                                <![CDATA[genai.descSimilarityUpperThreshold]]>
                            </DESC>
                            <DEFAULTVALUE>
                                <![CDATA[0.6]]>
                            </DEFAULTVALUE>
                            <APPLICABLE_FOR_SM>
                                <![CDATA[true]]>
                            </APPLICABLE_FOR_SM>
                            <SEMANTIC_MODEL_SECTION>
                                <![CDATA[NLQuery]]>
                            </SEMANTIC_MODEL_SECTION>
                        </PARAM>
                        <PARAM TYPE="TEXT">
                            <NAME>
                                <![CDATA[lower_threshold]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblSimilarityLowerThreshold]]>
                            </DISPLAYNAME>
                            <DESC>
                                <![CDATA[genai.descSimilarityLowerThreshold]]>
                            </DESC>
                            <DEFAULTVALUE>
                                <![CDATA[0.4]]>
                            </DEFAULTVALUE>
                        </PARAM>
                        <PARAM TYPE="TEXT">
                            <NAME>
                                <![CDATA[maximum_records_requested]]>
                            </NAME>
                            <DISPLAYNAME>
                                <![CDATA[genai.lblRAGMaxRecords]]>
                            </DISPLAYNAME>
                            <DESC>
                                <![CDATA[genai.descRAGMaxRecords]]>
                            </DESC>
                            <DEFAULTVALUE>
                                <![CDATA[50]]>
                            </DEFAULTVALUE>
                        </PARAM>
                    </URLPARAMS>
                    <ADVANCEPROPERTY>
                        <FILENAME/>
                    </ADVANCEPROPERTY>
                </PROVIDERINFO>
            </PROVIDERDETAILS>
        </PROVIDER>
    </CATEGORY>
</CATEGORIES>

Step 3: Structure of the zip file

Once have the JAR file (including java code to integrate your LLM service), create a zip file that must contain two folders at the root:

lib: Create a lib folder and place your main LLM JAR file and all supporting JAR files required for the LLM service to function.
conf: Create a conf folder and place the category.xml file inside it. This file must follow the required format and include necessary details like API keys, and other configurations.

Compress the folders

Ensure that the ZIP file does not include a parent folder.
For example, if the folder structure resides in my-llm/, the ZIP should only include the lib/ and conf/ folders directly, not my-llm/lib/ or my-llm/conf/.
Select the lib and conf folders.
Compress them into a ZIP file without including a parent directory.

Configuration

Upload the zip file from Kyvos Manager:

To integrate your LLM service into Kyvos, follow these steps

Log in to Kyvos Manager.
Navigate to the GenAI configuration page.
After configuring the LLM service, Kyvos Manager validates and stores the configuration in the designated folder on the Analytical Server engine.

Verification of the integration

Once your files are uploaded and the callbacks are implemented, perform the following tests:

Verify LLM Integration: Ensure that your LLM is successfully integrated with Kyvos by running test queries on Conversational Analytics Interface and in logs user can verify logging used in his interface using Kyvos Logger.
Check Metadata: Ensure the metadata is being correctly parsed and applied by Kyvos Manager.
Execute SQL Generation: Test the generation of SQL queries, summaries, and embeddings to verify they work as expected.

HTTP Status 429 – Too Many Requests

Configure the Bring Your Own LLM Service in Kyvos

Prerequisites

Configuring Java Code And Metadata for Kyvos-LLM Integration

Step 1: Implement the Callback interfaces

Step 2: Creating the category.xml file for connection

Step 3: Structure of the zip file

Compress the folders

Configuration

Verification of the integration

HTTP Status 429 – Too Many Requests