Load Step
Applies to: Kyvos Reporting
The Load step in a Query Object transfers transformed and extracted data into a target system such as a relational database, file system, HDFS, or NoSQL/columnar system.
Overview
Select a target connection from the list of existing Kyvos Reporting connections.
Configure target-specific properties for data loading.
Load step supports conditional operations when the target already exists (append, overwrite, skip).
Load failures generate a dump file to aid debugging.
Note
Ensure the target connection is not marked Read Only under Configure > Databases, or it will not be available for data load.
General Properties
Property | Description |
|---|---|
Target Connection | Select from preconfigured connections. “Default” uses the user’s default database connection. |
Table/File Name | Select or enter the name of the table or file to load data into. If it does not exist, it will be created. |
If Table/File Exists | Choose how to handle existing targets: - Append Data - Delete Existing Data - Skip Load - Drop and Create New - Truncate Existing Data |
Encoding | Select the encoding type (e.g., UTF-8, ASCII). |
Compression | Select compression type for file/network data (e.g., zip, gzip). |
Suffix Timestamp | Append a timestamp suffix to the output table/file name using a selected format. |
Data Properties
Property | Description |
|---|---|
If Record Exists | Determines record-level behavior: - Insert Only - Update Only - Update or Insert |
Ignore Empty Rows | If checked, skips empty rows while loading. |
Row and Column Properties (File or RDBMS Targets)
Property | Description |
|---|---|
Line Separator | Specify row delimiter for the file. |
Split After N Rows | Split the file after a specified number of rows. Blank or 0 disables splitting. |
Include Header | If checked, includes header row in the output. |
Enclosure | Enclosure character used around values (e.g., double quotes). |
Separator | Column delimiter used in the file. |
Fetch Columns | Fetches updated column list from the existing table. |
Remove All Columns | Clears all column definitions. |
HBase-Specific Properties
Property | Description |
|---|---|
Row Key | Select a field to act as the unique row key. |
Data Type | Specify the data type of the row key. |
Fetch Column Families | Fetch updated column families from the table. |
HDFS-Specific Properties
Property | Description |
|---|---|
Target File Name | Select or enter a file name with read/write/execute permissions on target directory. |
If File Exists | Choose action if the file already exists: - Append Data - Skip Load - Delete and Create New File |
Other supported file-level options, such as Encoding, Compression, Line Separator, Split After N Rows, Include Header, and Ignore Empty Rows, work similarly to RDBMS/file systems.
Error Handling
Property | Description |
|---|
Property | Description |
|---|---|
Error Threshold | Specifies the number of errors after which loading is aborted. For example, value |
When a Load step fails, Kyvos Reporting creates a CSV file containing failed records at:<KyvosReportingInstallPath>/KyvosReporting/reportengine/bin/<TargetTableName>_failed_records.csv
This location is controlled by the property LOAD_STEP_FAILS_DUMP in ReportEngine_Default.properties.
Execution and Scheduling
You can run and schedule Query Objects that include a Load step using Explorer.
The Query Object will execute the transformation logic and load data into the selected target.
Use scheduling options to run the load operation once or repeatedly.
Workflow Integration
The Load step can be followed by a Data Source step to treat loaded data as input.
Subsequently, a Format step can be used to define report design properties.