#####README NutrietBalance############################################################
#                                                                                    #
#Author : M.P. Weeber (Deltares)                                                     #
#Date   : 28-05-2021                                                                 #
#                                                                                    #
#Based on report : 11206834-000-ZKS-0001_v0.3-Stoffenbalans Volkerak Zoommeer        #
######################################################################################


The python library used for making the substance balance shown in this report can be accessed through the OpenEarth repository:
https://svn.oss.deltares.nl/repos/openearthtools/trunk/python/applications/NutrientBalance

To access this repository you are required to create a user and password on https://oss.deltares.nl (see “Login > Register > Create a MyDeltares account” topright of your screen).

 


#Installing the environment
The python library is developed using a specific set of libraries and python installation. The ensure good results you should try to recreate this environment. In the folder “envs” you will find the file “py3nutbal.yml”. With this file you can install the required Python environment through Anaconda. If you are unfamiliar with this process, we advise you to follow this link:
https://towardsdatascience.com/getting-started-with-python-environments-using-conda-32e9f2779307

#Running existing scripts
In NutrientBalance there are five example scripts that make use of the “nutrientenbalans_tool.py”:

-    nutrientenbalans_script_debieten.py
      Translates the cumulative flows from “Water-en chloride balans Volkerak-Zoommeer” (Vergroesen, 2020) towards non-cumulative flows.

-    nutrientenbalans_scirpt_overzichten_combined.py
      Makes a combined overview of nitrogen, phosphorus and selenium. However, it became soon apparent that different data processing was needed for each of these substances. This balance results are not to be used.

-    nutrientenbalans_script_overzichten_combined_N&P.py
      Makes an overview of nitrogen and phosphorus and contains substance specific data processing. This balance results are not to be used for NH3 as that is supposed to be calculated from NKjeldahl after filtration (here a overestimation).

-    nutrientenbalans_script_overzichten_combined_KRWv.py
      Makes an overview of nitrogen and phosphorus based on the KRW-verkenner (WFD-explorer) results. Does not contain any data processing appart from making the balances.

-    nutrientenbalans_script_overzichten_combined_NH3.py
      Calculates specifically NH3. NH3 needs to be derived from N-Kjeldahl after filtration or else it gives an overprediction. Same data processes are used as in the script for N&P.

-    nutrientenbalans_script_overzichten_combined_Seleen.py
      Makes an overview of selenium and contains substance specific data processing (especially reusing the measurement values of the Dintel). 

For running the existing scripts you can use the following actions after installing Anaconda and the environment. Start the command prompt and navigate to the folder containing the scripts:

|-------------------------------------------------------------------------------------|
| activate [name of environment]                                                      |
| python [name of script]                                                             |
|-------------------------------------------------------------------------------------|
The script will now generate the output in the folder output.

#Running a new script
Before running a new script the data (for example for a new watersystem) the required data needs to be collected from the datasources for flows (waterbalance, Z-info, KRW-verkenner) and for substances (Water.info, WKP database, Z-info, Emissieregistratie, KRW-verkenner).

-    Substance data
-    Flow data
-    Source setup
-    Substance setup
-    Substance translation



#Functions in Nutrientenbalans_tool

The python library nutrientenbalans_tool.py consist of several 
functions that can help with preparing a visualizing the data provided.


##Load NutrientBalans

Add the following lines to your Python script. Make sure it is placed 
in the NutrientBalance folder:
|---------------------------------------------------------------------------------|
|import nutrientenbalans_tool                                                     |
|                                                                                 |
|nutbaltest = nutrientenbalans_tool.NutrientBalans()                              |
|---------------------------------------------------------------------------------|



Now you can use the functions described below. To read flow data 
for example type in your script:

|---------------------------------------------------------------------------------|
|nutbaltest.read_flow_data_csv(“input/Niet_Cumulatieve_Debieten_waterbalans.csv”) |
|---------------------------------------------------------------------------------|


##Reading and writing data

|Function name                     | Function description
|----------------------------------|--------------------------
|read_flow_data_csv()              | Read the provided flow data
|read_substance_data_csv()         | Read the provided substance data
|read_sources_setup_csv()          | Read the provided sources setup
|read_substance_setup_csv()        | Read the provided substance setup
|read_substance_translation_csv()  | Read the provided substance translation
|Read_data_overview_from_file()    | During the script data overviews can be stored, 
|                                  | this function allows to read them in again.
|Write_data_overview_csv           | During the script data overviews can be stored, 
|                                  | this function allows to write them.


##Converting flow

|Function name                                   | Function description
|------------------------------------------------|------------------------------------------------------
|change_flow_data_cummulative_to_noncummulative()| Transfers cumulative flow data to non-cumulative data
|change_flow_data_noncummulative_to_cummulative()| Transfers noncumulative flow data to cumulative data
|make_dummy_flow()                               | Makes a flow based on one or several values for sources without a (known) flow


##Flow, substance and Load operations

|Function name                     | Function description
|----------------------------------|------------------------------------------------------------------
|couple_flow_and_substances()      | Combines the flow and substances for the read data sources and provides a date per source for the time period considered
|combine_substances()              | Group substances based on group name and priority provided in the substance setup. Missing values will be filled.
|select_by_group_sources()         | Make a subset of the data overview based on the group name provided and the sources setup file. Continue to work with this subset.
|combine_sources()                 | Group sources based on group name in the sources setup file. Values will be added.
|translate_substances()            | Calculate substances based on substance translation file supplied. For existing substances missing values will be filled. For non-existing substances a new column will be made and values will be added.
|interpolate()                     | Interpolate the values supplied based on a method (currently block or lineair). Can be done for subsets of sources and parameters.
|calculate_load()                  | Calculates the loads for substances based on the supplied flow and substance concentration.
|calculate_cummulative()           | Calculates the cumulative value per day for each source and parameter combination.
|fill_missing_parameters_periods() | Fills missing parameters from a period range based on another period range in the same source.
|fill_missing_periods_with_value() | Fill a parameter and source combination with a fixed value for a specific period.
|fill_missing_parameters_sources() | Fill the parameters of a source with the same parameters of another source.


##Visualisation

|Function name                                              | Function description
|-----------------------------------------------------------|------------------------------------------------------------------
|make_overview_of_data_sources_per_parameter()              |Make a graph of one parameter with lines for each of the sources.
|make_overview_of_data_sources_availability_per_parameter() |Make a graph of one parameter with a scatterplot indicating data availability for each of the sources.
|make_overview_of_yearvalues_parameter()                    |Make a stacked bargraph per year for one parameter indicating the contribution of each of the sources (positive : IN, negative : UIT). When difference is True, make a bargraph showing the difference between IN and UIT.
|make_overview_of_stacked_parameter()                       |Make a stacked area plot for the full period for one parameter indicating the contribution of each of the sources (positive : IN, negative : UIT). When difference is True, make a stacked area plot showing the difference between IN and UIT.
|make_overview_of_data_sources_all_parameters()             |Loop for all parameters and use the function make_overview_of_data_sources_per_parameter()
|make_overview_of_data_sources_availability_all_parameters()|Loop for all parameters and use the function make_overview_of_data_sources_availability_per_parameter()
|make_overview_of_all_stacked_parameters()                  |Loop for all parameters and use the function make_overview_of_yearvalues_parameter()
|make_overview_of_all_yearvalues_parameters()               |Loop for all parameters and use the function make_overview_of_stacked_parameter()