ReadMe for CombiValais

Created by Alain Foehn on April 28 2020

Last modification by Alain Foehn on April 28 2020

Table of Contents

• Code preparation
• Folder Data
• Interpolation methods
• API Inputs/Outputs
• Local stations data

Code preparation

To make the code work, the following steps must be carried out :

• Provide a folder CombiValais\Data (see point Folder DATA)
  • Configure adequatly the files Networks.csv and Sources.csv
• Generate a folder MODELS with a computation on an hour with sufficient wet stations.
• Configure the files Ini\API-research.ini, Ini\Parameters.ini and Ini\FTP.ini (if required) (copy them from subfolder Templates and add/modify required information)
• If in INI\parameters.ini config_auto = FALSE (not real-time computation), configure a config.ini file (example in INI\Templates\config) ; if config_auto = TRUE, a config.csv will be generated automatically based on the INI\Templates\config-auto.csv (to be modified in INI\Templates\config-auto.csv !). See below for the methods' structure.
• Configure a minerve-d3\global_config.ini file (to define Output and Temporaty folders ; template given in minerve-d3\Templates\Global_config_Templates.ini).

To run the code,

• Define in the first lines of CombiValais\Combivalais.R the correct folder where your minerve-d3 folder is (use line 5).
• Select all the code of CombiValais\Combivalais.R and run the code.

Explanation about the folder Data

The folder Data must contain two sub-folders :

• Data\Data
• Data\SIG

The subfolder Data\Data must contain the following items

• A file networks.csv, with two columns : column 1 gives the ID network, column 2 the Network name (ID ; Network name) :

	ID;Name
	1;MCH
	2;METEOGROUP

• A file Sources.csv, which defines the sources from which data will be acquired, with 4 columne (ID ; Source name, minimum number of data used for the hourly agregation for precipitation, minimum number of data used for the hourly agregation for precipitation)

	ID;Name;NminP;NminT
	21;MétéoSuisse;6;3
	35;MeteoGroup;1;1

Remark : in the present implementation, stations are associated to networks and sources, on Source potentially being linked to many networks.

The subfolder Data\Data must contain the following items

• a shapefile (with all associated files) named BASIN_NAME-ligne : shapefile of line giving the contour of your catchment
• a shapefile (with all associated files) named BASIN_NAME-surface : shapefile of polygon giving the surface of your catchment
• a shapefile (with all associated files) NouvellesRegions_1500m.shp : giving a division of your basin into regions for analysis purpose (not really used anymore but has bot been removed yet).

For the two first shapefiles, BASIN_NAME can be replaced by your real basin name but it must correspond to the name given in the column BASIN_NAME of the config.csv (respectively config-auto.csv) file.

Remark : The code contains a certain number of comments with TODO : some of them might represent an obstacle when applying the code on a different case study. Be aware of this...

Explanation about the available computation methods.

The following methods are available :

• Inverse Distance Weighting : IDW arguments : interpolation
• Regression Inverse Distance Weighting: RIDW arguments : linear_model,interpolation
• Regression Kriging : RK arguments : transformation,linear_model,variogram,interpolation
• Regression Co-kriging RCK arguments : transformation,linear_model_1,variogram_1,interpolation_1,linear_model_2,variogram_2,interpolation_2

The arguments for each methods are explained hereafter. The notion of groups of networks is explained just after.

• Interpolation
  • This refers to the groups of networks to be used for interpolation (respectively interpolation_1 and interpolation_2 for RCK where primary and secondary variables are considered).
  • This concerns all methods.
• Linear model
  • This refers to the groups of networks to be used for the linear regression computation between radar values and station values (respectively linear_model_1 and linear_model_2 for RCK where a primary and secondary variables are considered).
  • This concerns the three Regression methods (RIDW, RK and RCK).
• Transformation
  • This refers to a pre-transformation of the observed data (from both the radar and ground stations) before the entire computation. Possible values : 'sqrt' or 'none'.
  • This concerns the two kriging methods (RK and RCK).
• Variogram
  • This refers to the groups of networks to be used for the computation of the variogram (respectively variogram_1 and variogram_2 for RCK where a primary and a secondary variables are considered that requires the computation of a linear model of co-regionalization).

The notion of group of networks allows to define which networks must be used for the different computation steps detailed just above. In the config file, 4 columns C1, C2, C3 and C4 allow to define 4 groups of networks. The groups can then be called as arguments for the different functions. Two examples are given hereafter :

• RCK(sqrt,C1-C2,C1-C2,C1-C2,C3,C3,C3-C4)
• IDW(C1-C2)

The numbers to provide as network refer to the products ID of the products in the spatial database. Networks ID can be grouped by separating them with "-" (example : 53-54). See also template files in minerve-d3\Combi-Valais\Ini\Templates\config for examples.

API INPUTS/OUTPUS.

This part provides the data structure required to use the CombiValais tool with other APIs than the one of the minerve-d3 project. The data structure of each argument of the INI API file is given.

• First, the inputs arguments are given. When indicated with {}, this indicates a variable that will be replaced. When not indicated with {}, this indicates fix arguments in the request.
• Second, the structure of the API output file is given.

List of functions :

• API_token_path
• API_request_raster
• API_request_values
• API_get_pixel_altitude
• API_get_precipitation

1. API_token_path

INPUT (arguments)

The following arguments must appear in the base request (modified during the code execution):

• {user}
• {login}

OUTPUT

content-type: application/json; charset=utf-8

Response body :

{
  "token": "rtJhbGciOiJIUzI1N..."
}

2. API_request_raster

INPUT (arguments)

The following arguments must appear in the base request (modified during the code execution):

• {productId}
• {STARTDATE}
• {ENDDATE}

OUTPUT

GeoTiff with the extension .gtiff

3. API_request_values

INPUT (arguments)

The following arguments must appear in the base request (modified during the code execution):

• {productId}
• {STARTDATE}
• {ENDDATE}
• {X_val}
• {Y_val}

OUTPUT

content-type: text/csv

Response body :

2018/01/01 00:00;0.00
2018/01/01 01:00;0.00
2018/01/01 02:00;0.10
2018/01/01 03:00;1.53
2018/01/01 04:00;0.33

4. API_get_pixel_altitude

INPUT (arguments)

The following arguments must appear in the base request (modified during the code execution):

• {prod_id}
• {X_val}
• {Y_val}

OUTPUT

content-type: application/json; charset=utf-8

Response body : ` 1126 `

5. API_get_precipitation

INPUT (arguments)

The following arguments must appear in the base request (modified during the code execution):

• {sourceId}
• {X_val}
• {Y_val}
• {NMIN}
• precipitation

OUTPUT

content-type: application/json; charset=utf-8

• example for two stations and three hours
• the line aggregationCount is not mandatory (selection is done in the request itself, not in the code)

Response body :

{
  "id": 153,
  "name": "Adelboden",
  "coordinate": {
    "e": 2609376,
    "n": 1148937,
    "altitude": 1320
  },
  "monitoringNetworkId": 34,
  "precipitations": [
    {
      "event": "2019-08-03T00:00:00.000Z",
      "value": 0,
      "aggregationCount": 6
    },
    {
      "event": "2019-08-03T01:00:00.000Z",
      "value": 0,
      "aggregationCount": 6
    }
  ]
},
{
  "id": 154,
  "name": "Aigle",
  "coordinate": {
    "e": 2560401,
    "n": 1130713,
    "altitude": 381
  },
  "monitoringNetworkId": 34,
  "precipitations": [
    {
      "event": "2019-08-03T00:00:00.000Z",
      "value": 0,
      "aggregationCount": 6
    },
    {
      "event": "2019-08-03T01:00:00.000Z",
      "value": 0,
      "aggregationCount": 6
    }
  ]
}

Local stations data

Instead of acquiring the hourly (precipitation and temperature) stations data through the API, it is possible to define the variables P_station_source and T_station_source in the Parameters.ini file as "local" and provide in the folder Combi-Valais\Data\Data csv files with the necessary data. The name of these .csv files must be given in the config.ini file (columns station_data_P and station_data_T)

The structure of the data file must be as follow :

"STATION_ABR";"STATION_NAME";"STATION_SOURCE";"DATE_TIME";"P";"x";"y";"Z"
1;Le Moléson;5;"01.01.2015 00:00";0.1;567723.00;155072.00;1974
1;Le Moléson;5;"01.01.2015 01:00";0;567723.00;155072.00;1974
1;Le Moléson;5;"01.01.2015 02:00";0;567723.00;155072.00;1974
1;Le Moléson;5;"01.01.2015 03:00";0;567723.00;155072.00;1974
1;Le Moléson;5;"01.10.2014 02:00";0;567723.00;155072.00;1974

"STATION_ABR";"STATION_NAME";"STATION_SOURCE";"DATE_TIME";"T";"x";"y";"Z"
1;Le Moléson;5;"01.10.2014 02:00";7.2;567723.00;155072.00;1974
1;Le Moléson;5;"01.10.2014 03:00";7;567723.00;155072.00;1974
1;Le Moléson;5;"01.10.2014 04:00";6.7;567723.00;155072.00;1974
1;Le Moléson;5;"01.10.2014 05:00";6.6;567723.00;155072.00;1974
1;Le Moléson;5;"01.10.2014 06:00";6.8;567723.00;155072.00;1974

Remark : For methods using radar data, these will have to be acquired throug an API.