V10 Dataset Converter

The V10DatasetConverter class is part of the pyltm Python bindings and includes a tool for converting data in the V10 dataset to the new api objects. The package is in early development, and as of now, can only convert historical.h5 and tidsseriedata.h5 into api timeseries, and store them as external data format:

class V10DatasetConverter

This class handles exporting v10 datasets to API format. Requires a path to the origional v10 dataset.

__init__(path: str)

Initializes a new V10DatasetConverter instance.

Parameters:

path (str) – Path to the v10 dataset directory, as a str.

Example:

from pyltm import V10DatasetConverter
converter = V10DatasetConverter("path/to/dataset")

Methods

export_historical(externalDataType, externalDataFilename='', filename='historical.h5', scaleFactor=1.0)

Exports historical data with a specified data type, optional scaling factor, and filename. The default filename is historical.h5.

Parameters:

• externalDataType (ltmapi.ExternalDataType) – Type of external data.

• externalDataFilename (str, default="") – Optional filename for the external data output.

• filename (str, default="historical.h5") – The filename for the historical data. Defaults to “historical.h5”.

• scaleFactor (float, default=1.0) – A factor to scale the exported data.

Returns:

Exported data in JSON format.

Return type:

ltmapi.json

Example:

from pyltm.pyltm.datatypes import HDF5
converter.export_historical(HDF5, "output.h5", "custom_historical.h5", 0.5)

export_tidsseriedata(externalDataType, externalDataFilename='', filename='TidsserieData.h5')

Exports time series data with a specified data type and filename. The default filename is TidsserieData.h5.

Parameters:

• externalDataType (ltmapi.ExternalDataType) – Type of external data.

• externalDataFilename (str, default="") – Optional filename for the external data output.

• filename (str, default="TidsserieData.h5") – The filename for the time series data. Defaults to “TidsserieData.h5”.

Returns:

Exported data in JSON format.

Return type:

ltmapi.json

Example:

from pyltm.pyltm.datatypes import CSV
converter.export_tidsseriedata(CSV, "output_series.csv", "custom_tidsserie.h5")

export_exogenous_prices(externalDataType, externalDataFilename='', filename='exogenous_prices.h5')

Exports exogenous price data with a specified data type and filename. The default filename is exogenous_prices.h5.

Parameters:

• externalDataType (ltmapi.ExternalDataType) – Type of external data.

• externalDataFilename (str, default="") – Optional filename for the external data output.

• filename (str, default="exogenous_prices.h5") – The filename for the exogenous price data. Defaults to “exogenous_prices.h5”.

Returns:

Exported data in JSON format.

Return type:

ltmapi.json

Example:

from pyltm.pyltm.datatypes import JSON
converter.export_exogenous_prices(JSON, "output_prices.json", "custom_exogenous_prices.h5")

read_revisjonsplan_stas(revplan_stas_file: str)

Optional param revplan_stas_file:

Path to REVISJONSPLAN.STAS

Returns:

list of dicts

Example return data.

[
    {
        "module_number": 9995,
        "module_name": "upper_main",
        "station_name": "plant",
        "revisions": [
            {
                "start_week": 1,
                "end_week": 73,
                "revision_pump": 1e-05,
                "revision_plant": 170.0,
            },
            {
                "start_week": 74,
                "end_week": 99,
                "revision_pump": 1e-05,
                "revision_plant": 150.0,
            },
            {
                "start_week": 100,
                "end_week": 156,
                "revision_pump": 1e-05,
                "revision_plant": 0.0,
            },
        ],
    }
]

Example:

import pyltm

script_path = "/path/to/data"

def read_revisjonsplan_stas():

    converter = pyltm.V10DatasetConverter(script_path)
    print(converter.read_revisjonsplan_stas())

Example 2:

import pyltm

def read_revisjonsplan_stas(filename: str):

    converter = pyltm.V10DatasetConverter(".")
    print(converter.read_revisjonsplan_stas(filename))

Usage Example

# File: convert_v10.py
from pyltm import V10DatasetConverter
from pyltm.pyltm.datatypes import Internal, HDF5, SQLite, CSV


v10conv = V10DatasetConverter(<path to a v10 dataset>)

res = v10conv.export_historical(SQLite, <output path>/<filename>)

print(res)

$ python convert_v10.py
[{'1210-B': {'external_reference': {'type': 'SQLite', 'filename': '<path>/<filename>', 'path': 'historical_1210_B'}}}, ...}]

Here res is a python list of all the timeseries generated, where each element in the list is a dictionary with the information needed to make an external reference. When running export_historical, a file (multiple files in the case of CSV) is created in the output folder, and the data is being converted to fit the external refence format. Thus in input.json, one can include the reference, and run the simulation as expected.

Note, the export_historical, supports Internal datatype. In this case, the data is returned directly to python as timestamps and values. The user may then modify the timeseries as needed, and then store it into input.json, or any other suitable formats.