Welcome to Data processing documentation of open_eGo¶

[Add text here]

Getting Started¶

Overview¶

How to install
How to run the dataprocessing package

Installation¶

Installation of latest release

` (sudo) pip3 install dataprocessing `

Installation (of developer version) via pip on linux systems as follows

` sudo pip3 install -e <path-to-data_processing-repo> `

Troubleshooting

On Windows¶

__Problem__: Installation of required package shapely fails. __Solution__: Install pre-build

wheel from [here](http://www.lfd.uci.edu/~gohlke/pythonlibs/#shapely)

Run¶

You can run data processing by calling command-line script

` ~$ ego_data_processing `

Open_eGo the project¶

Open Electricity Grid Optimization¶

The project open_eGo aims to develop a transparent, inter-grid-level operating grid planning tool to investigate economic viable grid expansion scenarios considering alternative flexibility options such as storages or redispatch.

Uniform grid planning is required for a successful energy transition. This involves the management of the German electricity grid with more than 800 different network operators and the resulting wide range of interests that sometimes stand at odds with the national economic objectives of the energy transition. However, there is currently no suitable grid planning tool that is able to consider optimum national economic use of the various flexibility options at the different levels. The current challenges of planning for grid expansion associated with the energy transition are answered by open_eGo.

In energy system analysis, models and input data are often handled restrictively. Such a lack of transparency impedes reproducibility and consequently also a proper interpretation of the results. Thus, in open_eGo we publish all our code on github under the Affero General Public License Version 3. The data we use as input, but also all our results will be published on the OpenEnergy Platform, in most cases under an Open Database License Version 1.

For the open_eGo project several python packages are developt which are feeded by the input data of the data processing.

ego.io¶

SQLAlchemy Interface to the OpenEnergy database (oedb).OEDB table ORM objects are defined here and small helpers for io tasks are contained. Learn more here.

Ding0¶

The DIstribution Network GeneratOr (Ding0) is a tool to generate synthetic medium and low voltage power distribution grids based on open (or at least accessible) data. Learn more here.

eDisGo¶

The python package eDisGo provides a toolbox for analysis and optimization of distribution grids. Learn more here.

eTraGo¶

The python package eTraGo provides a toolbox for Optimization of flexibility options for transmission grids based on PyPSA. Learn more here.

eGo¶

The python package eGo is a toolbox and application which connects and integrates the tools eTraGo and eDisGo in order to calcualte the overall economic optimum. Learn more here.

The text on this website is published under CC-BY 4.0.

Data processing scripts¶

Data processing¶

SQL-Scripts¶

Data Processing SQL-Scripts¶

Overview of scripts¶

dataprocessing/sql_snippets/ scripts¶

ego_dp_loadarea_census.sql¶

Loads from Census 2011 Include Census 2011 population per ha. Identify population in OSM loads.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_loadarea_consumption.sql¶

Allocate consumption to Loadareas

copyright: Flensburg University of Applied Sciences, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: IlkaCu, Ludee

ego_dp_loadarea_griddistrict_results.sql¶

Results for MV Griddistrict After finishing the Loadareas we can aggregate the results for the MV Griddistricts: Area of MV Griddistrict. Municipality (Gemeinden). Municipality parts (Gemeinde-Einzelteile). Municipality types. Population results. Loadarea results (Area, Free area, Share). Consumption results.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_loadarea_industry_consumer.sql¶

OSM Industry consumer Calculate specific electricity consumption per million Euro GVA for each federal state. Calculate the electricity consumption for each industry polygon. Identify corresponding bus for large scale consumer (lsc) with the help of ehv-voronoi.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: IlkaCu, Ludee

ego_dp_loadarea_landuse.sql¶

OSM landuse sectors Extract landuse areas from OpenStreetMap. Cut the landuse with German boders (vg250) and make valid geometries. Divide into 4 landuse sectors: 1. Residential 2. Retail 3. Industrial 4. Agricultural

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_loadarea_loadcut_griddistrict.sql¶

Cut Loadarea with MV Griddistrict Identify and exclude Loadarea smaller than 100m². Generate Centre of Loadareas with Centroid and PointOnSurface. Calculate population from Census 2011. Cut all 4 OSM sectors with MV Griddistricts. Calculate statistics like NUTS and AGS code. Check for Loadareas without AGS code.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_loadarea_loadcut_voronoi.sql¶

Cut Loadarea with MV Voronoi cells Identify and exclude Loadarea smaller than 100m². Generate Centre of Loadareas with Centroid and PointOnSurface. Calculate population from Census 2011. Cut all 4 OSM sectors with MV Griddistricts. Calculate statistics like NUTS and AGS code. Check for Loadareas without AGS code.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_loadarea_loadmelt.sql¶

Melt loads from OSM landuse and Census 2011 Collect loads from both sources. Buffer collected loads with with 100m. Unbuffer the collection with 100m. Validate the melted geometries. Fix geometries with error. Check again for errors.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_loadarea_loads.sql¶

OSM Loads from landuse Excludes large scale consumer. Buffer OSM urban sectors with 100m Unbuffer buffer with -100m

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_loadarea_peakload.sql¶

Peak loads per Loadarea Uses SLP parameters per sectors.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: gplssm, Ludee

ego_dp_loadarea_statistic.sql¶

Results and statistics for eGoDP data Substation, Loadarea, MV Griddistricts and Consumption. MV Griddistrict types. Municipality (Gemeinden). Calculate statistics for BKG vg250.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_lv_consumption_peakload.sql¶

LV Consumption and Peakload Update LV Griddistrict table by a. sectoral consumption in each LV Griddistrict b. sectoral peak load in each LV Griddistrict

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: gplssm

ego_dp_lv_griddistrict.sql¶

LV Griddistrict Create LV Griddistrict from MVLV Substation. Generate OSM landuse per sectors.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee, jong42

ego_dp_lv_substation.sql¶

MVLV Substation (ONT) Create a lattice (regular fishnet grid) with 360m. Create MVLV Substation from lattice centroid.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee, jong42

ego_dp_lv_substation_voronoi.sql¶

MVLV Substation Voronoi Voronoi polygons with Eucldean distance (manhattan distance would be better but not available in sql).

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee, jong42

ego_dp_mv_griddistrict.sql¶

MV GridDistricts Generate MV GridDistricts from municipalities and Voronoi cells. Each HVMV Substation receives one catchment area. Detailed description can be found in Hülk et. al. 2017.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_powerflow_assignment_generator.sql¶

Generators which were assigned to a specific substation prior to this script need to be transformed to a data structure suitable for powerflow calculation with tool developed and used in the open_eGo project. The following script transforms data from the powerplant mviews and adds some parameters according to the characteristics of the generators. To reduce the data volumn in the final table structure (see ego_dp_powerflow_hv_setup.sql) the generators are clustered according to their source, installed capacity, weather point and substation they are assigned to. Here a new and unique aggregate-ID (aggr_id) is assigned. In an interims stage all generators are converted to a format suitable for powerflow flow calculation seperately. This data can be accessed in table model_draft.ego_supply_pf_generator_single.

Information on generators which are assigned to a specific substation are transformed to a data structure which is suitable for PyPSA. This script creates the scenarios ‘Status Quo’, ‘NEP 2035’ and ‘eGo 100’ in the hv powerflow schema.

copyright: Flensburg University of Applied Sciences, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: IlkaCu, Ludee

ego_dp_powerflow_assignment_load.sql¶

Similar to generators in the previous script the data on loads are converted and clustered to fit the data structure needed for powerflow calculations. The electricity demand of small scale consumer and industrial large scale consumer is considered.

copyright: Flensburg University of Applied Sciences, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: IlkaCu

ego_dp_powerflow_assignment_otgid.sql¶

This scripts updates tables containing renewable power plants and conventional power plants with information on the otg_id of substations which the generator is assigned to. The otg_id and subst_id of the substations are matched in tables containing information on HV/MV substations and EHV substations.

Additionally the otg_id of offshore wind turbines is updated manually. The geometry of offshore wind power plants is matched with polygons representing a catchment area per relevant offshore grid connection point.

copyright: Flensburg University of Applied Sciences, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: IlkaCu

ego_dp_powerflow_assignment_storage.sql¶

Equivalent to the assignment of generators in this script storages are converted and clustered for all three scenarios considered in open_eGo.

copyright: Flensburg University of Applied Sciences, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: IlkaCu”, “lukasol

ego_dp_powerflow_assignment_unid.sql¶

All generators from the conventional and renewable power plant list are brought together in a central generator list. A unified id (un_id) is assigned to those generators listed. Information on the un_id is then added to the conventional and renewable power plant lists.

copyright: Flensburg University of Applied Sciences, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: IlkaCu

ego_dp_powerflow_create_pp_mview.sql¶

This script creates discrete materialized views (mview) for conventional and renewable power plants per scenario, resulting in six different mviews for the three main scenarios considered in open_eGo.

copyright: Europa-Universität Flensburg - ZNES
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: wolfbunke

ego_dp_powerflow_electrical_neighbour.sql¶

The electricity grid model extracted from osmTGmod is limited to the German territory. This script adds border crossing lines and corresponding buses and transformers to all neighbouring countries which have a direct electrical connection to the German grid.

copyright: Flensburg University of Applied Sciences, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: IlkaCu

ego_dp_powerflow_grid_future_scenarios.sql¶

The grid model which is used as an input for powerflow calculations and optimization in open_eGo is the same in all three scenarios ‘SQ’, ‘NEP 2035’ and ‘eGo100’. In the following script the grid model created for the ‘SQ’ scenario in the previous scripts is duplicated for the remaining two future scenarios

copyright: Flensburg University of Applied Sciences, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: IlkaCu

ego_dp_powerflow_hv_setup.sql¶

This script creates all tables which are needed for hv-powerflow calculations. The characteristics of those tables follow the structure of PyPSA’s input data.

copyright: Flensburg University of Applied Sciences, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: mariusves, IlkaCu, ulfmueller, Ludee, s3pp

ego_dp_powerflow_lopf_data.sql¶

Missing parameters necessary for calculating a linear optimal power flow (LOPF) are added to the existing data. This includes marginal costs per technology, which is composed of specific operating cost, fuel costs and CO2 costs according to renpass_gis, NEP 2014 scenario. In addition p_max_pu is set for all generators with variable dispatch based on p_max_pu = p_set / p_nom.

A further section of the script is used to insert extendable battery and hydrogen storages to all substations in the grid model. These have a initial installed capacity p_nom=0, which can be extended when executing an optimization (by calculating a LOPF).

copyright: Europa-Universität Flensburg, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: wolfbunke, lukasol

ego_dp_powerflow_osmtgmod_to_pypsa.sql¶

osmTGmod provides a model of the German EHV and HV grid based on OpenStreetMap. This script extracts bus and branch data provided by osmTGmod and inserts the grid model into the corresponding powerflow tables. Additionally some (electrical) properties for transformers are adjusted or added.

copyright: Flensburg University of Applied Sciences, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: ulfmueller, IlkaCu, mariusves

ego_dp_powerflow_timeseries_demand.sql¶

Aggregated load time series for neighbouring countries are based on rennpassG!S results and are added to the corresponding powerflow table. The load is equivalent in all three scenarios.

copyright: Flensburg University of Applied Sciences, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: ulfmueller, wolfbunke

ego_dp_powerflow_voronoi_weatherpoint.sql¶

voronoi with climatepoints

copyright: Flensburg University of Applied Sciences, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: IlkaCu, Ludee

ego_dp_structure_input_verification.sql¶

Input verification (eGoPP) Check the necessary input tables from eGo PreProcessing. Return version of input tables.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_substation_ehv.sql¶

EHV Substation Abstract EHV Substations of the extra high voltage level from OSM. This script abstracts substations of the extra high voltage level from openstreetmap data.

copyright: DLR Institute for Networked Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: lukasol, C. Matke, Ludee

ego_dp_substation_ehv_voronoi.sql¶

EHV Substation Voronoi Voronoi polygons with eucldean distance on EHV Substation. Manhattan distance would be better but not available in sql.

copyright: Flensburg University of Applied Sciences, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: IlkaCu, Ludee

ego_dp_substation_hvmv.sql¶

HVMV Substation Abstract HVMV Substations of the high voltage level from OSM. This script abstracts substations of the high voltage level from openstreetmap data. All substations that are relevant transition points between the transmission and distribution grid are identified, irrelevant ones are disregarded.

copyright: DLR Institute for Networked Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: lukasol, C. Matke, Ludee

ego_dp_substation_hvmv_voronoi.sql¶

HVMV Substation Voronoi Voronoi polygons with eucldean distance on HVMV Substation. Manhattan distance would be better but not available in sql.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_substation_id_to_generator.sql¶

Substation ID to Generator All powerplants (Conventional and Renewable) receive the corresponding Substation ID. Either the HVMV Substation ID (= MV Griddistrict ID) or the EHV Substaion ID. Identify corresponding subst_id for all power plants according to their voltage_level and geometry.

copyright: Flensburg University of Applied Sciences, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: IlkaCu

ego_dp_substation_otg.sql¶

Substation OTG-ID Script to assign osmTGmod-id (OTG) to substation.

copyright: DLR Institute for Networked Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: lukasol, C. Matke

ego_dp_versioning.sql¶

eGo Data Processing result data versioning Copy a version from model_draft to OEP schema

copyright: © Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_versioning_mviews.sql¶

eGo Data Processing result data versioning Copy a version of mvies from model_draft to OEP schema

copyright: Europa-Universität Flensburg
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: wolfbunke

ego_dp_versioning_overview.sql¶

eGo Data Processing overview Check all versioned tables

copyright: © Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

dataprocessing/sql_snippets/functions scripts¶

function_query_metadata.sql¶

Function select important parameters of a table and it’s metadata

NOT WORKING

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

dataprocessing/sql_snippets/rea scripts¶

ego_dp_conv.sql¶

Skript to allocate conventional power plants to loadareas

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_lattice_500m.sql¶

Lattice (regular point grid) with 500m Lattice on bounding box of Germany.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_lattice_50m.sql¶

Lattice (regular point grid) with 50m Lattice on bounding box of Germany.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_rea_lattice_per_area_500m.sql¶

Prepare 500m lattice Lattice on bounding box of Germany with 500m per area: wpa - points inside wind potential area la - points inside loadarea x - points inside wind potential area and loadarea out - points outside area

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_rea_lattice_per_area_50m.sql¶

Prepare 500m lattice Lattice on bounding box of Germany with 50m per area: la - points inside loadarea

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_rea_m1.sql¶

M1 biomass and solar to OSM agricultural Allocates “biomass” & (renewable) “gas” to OSM agricultural areas. The rest could not be allocated, consider in M4.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_rea_m2.sql¶

M2 wind farms Allocates “wind” turbines with voltage level 4 to WPA as wind farms. The rest could not be allocated, consider in M3.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_rea_m3.sql¶

M3 wind turbines to WPA Allocates “wind” turbines with voltage levels “5” & “6” to WPA. Also considers rest of M2. The rest could not be allocated, consider in M4.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_rea_m4.sql¶

M4 other and rest Allocates “wind” with voltage levels “5” & “6” to WPA. “solar ground” & “wind” ohne voltage & Rest M1-1 & Rest M1-2 & Rest M3. Also considers rest of M1-1, M1-2 and M3. There should be no rest!

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_rea_m5.sql¶

M5 LV to Loadarea Allocate “solar” with voltage levels “6” & “7” to Loadarea. There should be no rest!

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_rea_results.sql¶

Results and statistics for REA

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_rea_setup.sql¶

Setup tables for REA Skript to allocate decentralized renewable power plants (dea). Methods base on technology and voltage level. Allocate DEA outside of Germany to next HVMV Substation. Generate OSM farmyards.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_rea_wpa_per_mvgd.sql¶

Wind potential area (WPA) per MV-Griddistrict Cut WPA with MV-Griddistrict and make valid geometries.

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_wpa.sql¶

Skript to allocate decentralized renewable power plants (dea) Methods base on technology and voltage level Uses different lattice from setup_ego_wpa_per_grid_district.sql

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

Python-Scripts¶

[use sphinx doc strings]

Preprocessing¶

SQL-Scripts¶

Preprocessing SQL-Scripts¶

Overview of scripts¶

preprocessing/sql_snippets/ scripts¶

ego_dp_res_rea_by_scenario.sql¶

SQL Script that prepare and insert single renewable power plant data by a given scenario in order to create a high resulution allocation renewable energy expansion.

For the project open_eGo and the tools eTraGo, eDisGo and eGo the scenarios are named ‘Status Quo’ (2015), ‘NEP 2035’, ‘eGo 100’. Learn more about the scenario here scenarios here.

This script is divided into two parts:

Part I¶

Development of new renewable power plants by NEP 2035 scenario data

Part II¶

Development of new renewable power plants by ego 100% scenario data

Methodology¶

Both parts of the script work more or less with the same Methodology of an expansion and allocation of renewable energy plants. A full documentation of the used Methodology of the Renewable allocation can be found here.

copyright: Europa-Universität Flensburg, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: wolfbunke

ego_dp_structure_boundaries_vg250.sql¶

Setup borders Inputs are german administrative borders (boundaries.bkg_vg250) Create mviews with transformed CRS (EPSG:3035) and corrected geometries Municipalities / Gemeinden are fragmented and cleaned from ringholes (bkg_vg250_6_gem_clean)

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_structure_census.sql¶

Result tables for eGoDP

WARNING: It drops the table and deletes old entries when executed!

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee, IlkaCu

ego_dp_structure_osm_landuse.sql¶

analyse OSM landuse data

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_structure_scenariolog.sql¶

Setup scenario log table Creates a table to get inserts from other processed tables

WARNING: It drops the table and deletes old entries when executed!

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_dp_structure_versioning.sql¶

Result tables for eGoDP

WARNING: It drops the table and deletes old entries when executed!

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee, IlkaCu

ego_nep_2015_scenario_capacities.sql¶

Create and setup the table model_draft.ego_nep_2015_scenario_capacities for electrical scenario capacities of the Netzentwicklungsplan 2015, erster Entwurf per federal state in Germany.

copyright: Europa-Universität Flensburg, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: wolfbunke

ego_pre_slp_parameters.sql¶

Create table with assumptions and parameters on standard load profiles (SLP)

WARNING: It drops the table and deletes old entries when executed!

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: gplssm

ego_pre_voltage_level.sql¶

Set or adjust voltage_level according to installed capacity and technology of power plants.

copyright: Flensburg University of Applied Sciences, Centre for Sustainable Energy Systems
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: IlkaCu

oedb_setup_schema_structure.sql¶

Setup the OpenEnergy Database (oedb) schema structure

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

preprocessing/sql_snippets/boundaries scripts¶

ego_pp_gn250_metadata.sql¶

Metadata for gn250 tables Geographische Namen 1:250 000 - GN250

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_pp_vg250_metadata.sql¶

Metadata for vg250 tables

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

preprocessing/sql_snippets/openstreetmap scripts¶

ego_pp_osm_line_street_mview.sql¶

Extracted OSM streets from line

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: jong42, Ludee

preprocessing/sql_snippets/society scripts¶

ego_pp_destatis_zensus_import.sql¶

Import DESTATIS zensus 2011 table

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

ego_pp_destatis_zensus_metadata.sql¶

Import DESTATIS zensus 2011 table

copyright: Reiner Lemoine Institut
license: GNU Affero General Public License Version 3 (AGPL-3.0)
url: https://github.com/openego/data_processing/blob/master/LICENSE
author: Ludee

Python-Scripts¶

[use sphinx doc strings]

Scenario framework¶

open_eGo Scenarios¶

Three scenarios were defined and used for the intended power flow simulations. Apart from one status quo scenario representing the German electrical energy system in 2015, two future scenarios were defined employing exogenous assumptions. For Germany, the installed generation capacities of the status quo scenario were taken from the power plant list of the Open Power System Data project [opsd-conv], [opsd-res] (State: 01-01-2016). Whereas the 2035 scenario is based on publicly available information and methods of the Netzentwicklungsplan (NEP) Strom 2025, erster Entwurf [NEP2015] . Out of several NEP scenarios, the so-called “B1-2035” was chosen; it is characterized by a high renewable energy expansion and an increased share of natural gas [NEP2015]. The third scenario pictures a future electrical energy system powered to 100% from renewable energy and is mainly based on the 100% RES scenario of the e-Highway2050 - Modular Development Plan of the Pan-European Transmission System 2050 [ehighway2050]. In order to build a 100% energy system in Germany 13,GW of gas fired power plants were removed (see: [christ2017] and [FlEnS]).

Table Characterization of scenarios by key parameters shows the scenario specifications and significant characteristics of all three scenarios.

Characterization of scenarios by key parameters¶
	Status Quo	NEP 2035	eGo 100%
Share of RES in installed capacity	46,46%	70,02%	100,00%
Net electricity consumption (Twh)	506	506	506
Annual peak load (GW)	87,01	87,01	87,01
Share of renewable energy in el. Consumption	27,31%	65,80%	100,00%

For the calculation of the Share of renewable energy in el. consumption the assumption-a where used.

Table Installed generation capacities in GW for Germany and marginal costs for conventional generation in 2014€/MWh , divided by scenario and technology/fuel displays the resulting installed electrical capacities in Germany and marginal cost assumptions for each scenario divided by the different technologies and fuels.

Installed generation capacities¶
Technology		status quo		NEP 2035		eGo 100
	GW	EUR/MWh	GW	EUR/MWh	GW	EUR/MWh
Nuclear energy	12	4,68	0	5,48	0	–
Lignite	23,3	10,78	9,1	17,64	0	–
Hard coal	31,5	14,95	11	24,79	0	–
Natural gas	27,9	32,3	33,5	41,93	0	56,05
Oil	4,5	41,02	0,5	68,86	0	–
Waste	1,7	31,65	0	39,93	0	–
CHP $<$ 10 MW	0	23,96	8,2	31,11	0	31,63
Other conventional generation (mixed fuels)	2,6	31,65	2,4	39,93	0	–
Total conventional generation	103,5		64,7		0
Wind onshore		41,3		88,8		98,4
Wind offshore		5,6		18,5		27
Photovoltaic		38,5		59,9		97,8
Biomass		7,2		8,4		27,8
Hydro power		3,9		4,2		4,3
Other renewable generation		1,4		1,2		2,2
Total renewable generation		97,9		181		257,5
Pump storage	9,3		12,7		12,8
Total generation	210,7		258,4		270,3

The table

open_eGo Scenarios¶

For the project open_eGo been three scenarios choosen in order to

Status Quo¶

NEP 2035¶

[NEP2015]

Scenario definition of renpassG!S¶

eGo 100¶

References¶

[NEP2015]

Übertragungsnetzbetreiber Deutschland, Netzentwicklungsplan Strom 2025, Version 2015, 1. Entwurf, 2015. https://www.netzentwicklungsplan.de/sites/default/files/paragraphs-files/NEP_2025_1_Entwurf_Teil1_0_0.pdf

[coastdat-2]

coastDat-2 Hindcast model http://www.coastdat.de/data/index.php.en

[FlEnS]

Wolf-Dieter Bunke Martin Söthe Marion Christ Cord Kaldemeyer (Fl)ensburg (En)ergy (S)cenarios - open_eGo Scenarios for 2014/2035/2050.

https://osf.io/bpf36/

Methodology¶

Methodologies¶

[Intro]

Methodologies of Renewable Energy Plants¶

[Intro]

Renewable allocation¶

The Methodology of the expansion and allocation of renewable energy plants is done in two parts. The first part is the developent and expansion of renewable power palnts by a scenario input of installed capacities. The allocation based on the spacial level of municipalities. The second part allocated the surplus of power plants power plants on a high spacial resultion by white areas and other Polygon objects (see XXX).

Expansion of power plants by technology¶

Wind onshore¶

Text

Wind offshore¶

text

Solar¶

text

Hydro Power¶

run of river and reservoir

Pumped Storage¶

Biomass¶

text

geothermal¶

text

Load¶

References¶

[NEP2015]

(1, 2) Übertragungsnetzbetreiber Deutschland. (2015).: Netzentwicklungsplan Strom 2025, Version 2015, 1. Entwurf, 2015. https://www.netzentwicklungsplan.de/sites/default/files/paragraphs-files/NEP_2025_1_Entwurf_Teil1_0_0.pdf

[coastdat-2]

coastDat-2 (2017).: Hindcast model http://www.coastdat.de/data/index.php.en

[FlEnS]

Bunke, Wolf-Dieter, Martin Söthe, Marion Wingenbach, and Cord Kaldemeyer. 2018.: “(Fl)ensburg (En)ergy (S)cenarios - open_eGo Scenarios for 2014/2035/2050.” Open Science Framework. June 13. doi:10.17605/OSF.IO/BPF36.

[opsd-conv]

Open Power System Data. 2016.: Data provided by Open Power System Data - Data Package Conventional power plants, version 2016-10-27. Primary data from BNetzA Kraftwerksliste, Umweltbundesamt Datenbank Kraftwerke in Deutschland.

[opsd-res]

Open Power System Data. 2017.: Data provided by Open Power System Data - Data Package Renewable power plants, early version 2016-02-10. Primary data from BNetzA, BNetzA_PV, TransnetBW, TenneT, Amprion, 50Hertz, Netztransparenz.de, Postleitzahlen Deutschland, Energinet.dk, Energistyrelsen, GeoNames, French Ministery of the Environment, Energy and the Sea, OpenDataSoft, Urzad Regulacji Energetyki (URE)

[ehighway2050]

e-Highway2050. (2015).: e-HIGHWAY 2050 Modular Development Plan of the Pan-European Transmission System 2050 - database per country. Retrieved from http://www.e-highway2050.eu/fileadmin/documents/Results/e-Highway_database_per_country-08022016.xlsx

[christ2017]

Christ, M. (2017).: Integration sozial-ökologischer Faktoren in die Energiesystemmodellierung am Beispiel von Entwicklungspfaden für den Windenergieausbau in Deutschland (PhD Thesis). Europa-Universität Flensburg.

[BMWi]

text …

Footnotes¶

[1]	Statistic and report values (See: BMWi 2017, sheet 20 year 2014 [BMWi], NEP 2025 [NEP2015], own calculation for eGo 100%.

What’s New¶

See what’s new as per release!

Releases

Release v0.3.0 (December 18, 2017)
Release v0.3.1 (April XX, 2018)

Release v0.3.0 (December 18, 2017)¶

[Description]

Added features¶

Bug fixes¶

Other changes¶

Release v0.3.1 (April XX, 2018)¶

[Description]

Added features¶

Bug fixes¶

Other changes¶

change use of geom (original) to rea_geom_new
…

Data References¶

[BKG2016]

Bundesamt für Kartographie und Geodäsie (BKG): Verwaltungsgebiete 1:250.000 Staat - (STA) (VG250), 2016, Geodatenzugangsgesetz (GeoZG), <http://www.geodatenzentrum.de>

[BKG2016]

Bundesamt für Kartographie und Geodäsie (BKG): Verwaltungsgebiete 1:250.000 Kreise - (KRS) (VG250), 2016, Geodatenzugangsgesetz (GeoZG), <http://www.geodatenzentrum.de>

[BKG2016]

Bundesamt für Kartographie und Geodäsie (BKG): Verwaltungsgebiete 1:250.000 Gemeinden - (GEM) (VG250), 2016, Geodatenzugangsgesetz (GeoZG), <http://www.geodatenzentrum.de>

[RLI2016]

Reiner Lemoine Institut gGmbH: eGo dataprocessing - Loadarea, 2016, Open Data Commons Open Database License 1.0 (ODbL-1.0), <http://openenergy-platform.org/dataedit/view/demand/ego_dp_loadarea>

[RLI2016]

Reiner Lemoine Institut gGmbH: eGo dataprocessing - LV Distribution grid area, 2016, Open Data Commons Open Database License 1.0 (ODbL-1.0), <http://openenergy-platform.org/dataedit/view/grid/ego_dp_lv_griddistrict>

[RLI2016]

Reiner Lemoine Institut gGmbH: eGo dataprocessing - MV Grid district, 2016, Open Data Commons Open Database License 1.0 (ODbL-1.0), <http://openenergy-platform.org/dataedit/view/grid/ego_dp_mv_griddistrict>

[RLI2016]

Reiner Lemoine Institut gGmbH: eGo dataprocessing - HVMV Substation, 2016, Open Data Commons Open Database License 1.0 (ODbL-1.0), <http://openenergy-platform.org/dataedit/view/grid/ego_dp_mvlv_substation>

[RLI]

Reiner Lemoine Institut gGmbH: eGo hv powerflow - loads, Open Data Commons Open Database License 1.0 (ODbL-1.0), <http://openenergy-platform.org/dataedit/view/grid/ego_pf_hv_load_pq_set>

[RLI]

Reiner Lemoine Institut: eGo dataprocessing - SLP parameters, Open Data Commons Open Database License 1.0 (ODbL-1.0), <http://openenergy-platform.org/dataedit/view/scenario/ego_slp_parameters>

[Europa-University]

Europa-Universität Flensburg: eGo renewable power classes of wind, Open Data Commons Open Database License 1.0 (ODbL-1.0), <http://openenergy-platform.org/dataedit/view/supply/ego_power_class>

[Europa-University]

Europa-Universität Flensburg: Renewable feedin time series for eGo, Open Data Commons Open Database License 1.0 (ODbL-1.0), <http://openenergy-platform.org/dataedit/view/supply/ego_renewable_feedin>

[VerNetzen2015]

Europa-University VerNetzen: VerNetzen - Wind potential Area, 2015, Open Data Commons Open Database License 1.0 (ODbL-1.0), <http://openenergy-platform.org/dataedit/view/supply/vernetzen_wind_potential_areat>

[VerNetzen2015]

Europa-University VerNetzen: VerNetzen - Wind potential Area, 2015, Open Data Commons Open Database License 1.0 (ODbL-1.0), <http://openenergy-platform.org/dataedit/view/supply/vernetzen_wind_potential_areat>

LICENSE¶

Copyright (C) 2015-2018 open_eGo-Team (C) Flensburg University of Applied Sciences, Centre for Sustainable Energy Systems (C) Europa-Universität Flensburg, Centre for Sustainable Energy Systems (C) Reiner Lemoine Institut (C) DLR Institute for Networked Energy Systems”

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see https://www.gnu.org/licenses/.

Welcome to Data processing documentation of open_eGo¶

Getting Started¶

Overview¶

Installation¶

On Windows¶

Run¶

Open_eGo the project¶

Open Electricity Grid Optimization¶

ego.io¶

Ding0¶

eDisGo¶

eTraGo¶

eGo¶

Data processing scripts¶

Data processing¶

SQL-Scripts¶

Data Processing SQL-Scripts¶

Overview of scripts¶

Python-Scripts¶

Preprocessing¶

SQL-Scripts¶

Preprocessing SQL-Scripts¶

Overview of scripts¶

Python-Scripts¶

Scenario framework¶

open_eGo Scenarios¶

open_eGo Scenarios¶

Status Quo¶

NEP 2035¶

Scenario definition of renpassG!S¶

eGo 100¶

References¶

Methodology¶

Methodologies¶

Methodologies of Renewable Energy Plants¶

Renewable allocation¶

Load¶

References¶

Footnotes¶

What’s New¶

Release v0.3.0 (December 18, 2017)¶

Added features¶

Bug fixes¶

Other changes¶

Release v0.3.1 (April XX, 2018)¶

Added features¶

Bug fixes¶

Other changes¶

Data References¶

LICENSE¶

Indices and tables¶