.. _dataset-converter:

=====================
V10 Dataset Converter
=====================

The `V10DatasetConverter` class is part of the `pyltm` Python bindings and includes a tool for converting data in the V10 dataset to the new api objects. The package is in early development, and as of now, can only convert historical.h5 and tidsseriedata.h5 into api timeseries, and store them as external data format:

.. py:class:: V10DatasetConverter

   This class handles exporting v10 datasets to API format. Requires a path to the origional v10 dataset.

   .. py:method:: __init__(path: str)

      Initializes a new `V10DatasetConverter` instance.

      :param path: Path to the v10 dataset directory, as a `str`.
      :type path: str

      **Example**::

          from pyltm import V10DatasetConverter
          converter = V10DatasetConverter("path/to/dataset")

   Methods
   -------

   .. py:method:: export_historical(externalDataType, externalDataFilename="", filename="historical.h5", scaleFactor=1.0)

      Exports historical data with a specified data type, optional scaling factor, and filename. The default filename is `historical.h5`.

      :param externalDataType: Type of external data.
      :type externalDataType: ltmapi.ExternalDataType
      :param externalDataFilename: Optional filename for the external data output.
      :type externalDataFilename: str, default=""
      :param filename: The filename for the historical data. Defaults to "historical.h5".
      :type filename: str, default="historical.h5"
      :param scaleFactor: A factor to scale the exported data.
      :type scaleFactor: float, default=1.0
      :return: Exported data in JSON format.
      :rtype: ltmapi.json

      **Example**::

          from pyltm.pyltm.datatypes import HDF5
          converter.export_historical(HDF5, "output.h5", "custom_historical.h5", 0.5)

   .. py:method:: export_tidsseriedata(externalDataType, externalDataFilename="", filename="TidsserieData.h5")

      Exports time series data with a specified data type and filename. The default filename is `TidsserieData.h5`.

      :param externalDataType: Type of external data.
      :type externalDataType: ltmapi.ExternalDataType
      :param externalDataFilename: Optional filename for the external data output.
      :type externalDataFilename: str, default=""
      :param filename: The filename for the time series data. Defaults to "TidsserieData.h5".
      :type filename: str, default="TidsserieData.h5"
      :return: Exported data in JSON format.
      :rtype: ltmapi.json

      **Example**::

          from pyltm.pyltm.datatypes import CSV
          converter.export_tidsseriedata(CSV, "output_series.csv", "custom_tidsserie.h5")

   .. py:method:: export_exogenous_prices(externalDataType, externalDataFilename="", filename="exogenous_prices.h5")

        Exports exogenous price data with a specified data type and filename. The default filename is `exogenous_prices.h5`.

        :param externalDataType: Type of external data.
        :type externalDataType: ltmapi.ExternalDataType
        :param externalDataFilename: Optional filename for the external data output.
        :type externalDataFilename: str, default=""
        :param filename: The filename for the exogenous price data. Defaults to "exogenous_prices.h5".
        :type filename: str, default="exogenous_prices.h5"
        :return: Exported data in JSON format.
        :rtype: ltmapi.json

        **Example**::

            from pyltm.pyltm.datatypes import JSON
            converter.export_exogenous_prices(JSON, "output_prices.json", "custom_exogenous_prices.h5")

.. py:method:: read_revisjonsplan_stas(revplan_stas_file:str)

    :optional param revplan_stas_file: Path to REVISJONSPLAN.STAS
    :type revplan_stas_file: string/path
    :return: list of dicts

    Example return data.

    .. code-block:: python

        [
            {
                "module_number": 9995,
                "module_name": "upper_main",
                "station_name": "plant",
                "revisions": [
                    {
                        "start_week": 1,
                        "end_week": 73,
                        "revision_pump": 1e-05,
                        "revision_plant": 170.0,
                    },
                    {
                        "start_week": 74,
                        "end_week": 99,
                        "revision_pump": 1e-05,
                        "revision_plant": 150.0,
                    },
                    {
                        "start_week": 100,
                        "end_week": 156,
                        "revision_pump": 1e-05,
                        "revision_plant": 0.0,
                    },
                ],
            }
        ]

    .. :ref:`Revisjonsplan.stas-reader`

**Example**:

.. code-block:: python

    import pyltm

    script_path = "/path/to/data"

    def read_revisjonsplan_stas():

        converter = pyltm.V10DatasetConverter(script_path)
        print(converter.read_revisjonsplan_stas())

**Example 2**:

.. code-block:: python

    import pyltm

    def read_revisjonsplan_stas(filename: str):

        converter = pyltm.V10DatasetConverter(".")
        print(converter.read_revisjonsplan_stas(filename))

.. .. toctree::
..    :maxdepth: 1
..    :glob:

..    revisjonsplan_stas.ipynb

Usage Example
-------------

.. code-block:: python

    # File: convert_v10.py
    from pyltm import V10DatasetConverter
    from pyltm.pyltm.datatypes import Internal, HDF5, SQLite, CSV


    v10conv = V10DatasetConverter(<path to a v10 dataset>)

    res = v10conv.export_historical(SQLite, <output path>/<filename>)

    print(res)

.. code-block:: console

    $ python convert_v10.py
    [{'1210-B': {'external_reference': {'type': 'SQLite', 'filename': '<path>/<filename>', 'path': 'historical_1210_B'}}}, ...}]


Here res is a python list of all the timeseries generated, where each element in the list is a dictionary with the information needed to make an external reference.
When running export_historical, a file (multiple files in the case of CSV) is created in the output folder, and the data is being converted to fit the external refence format.
Thus in input.json, one can include the reference, and run the simulation as expected.

Note, the export_historical, supports Internal datatype. In this case, the data is returned directly to python as timestamps and values. The user may then modify the timeseries as needed, and then store it into input.json, or any other suitable formats.