Student information
MSc thesis topic: Data Driven Utility Route Planner: Maps4Cables
Alliander is a Distribution System Operator (DSO), responsible for a reliable, sustainable and affordable energy network for roughly one third of the Netherlands. Maintaining 93.226 kilometres of electricity cables and 41.755 kilometres of gas pipelines providing access to energy for 5.8 million customer connections. The growing economy, housing development, and general sustainability improvement, is creating an exponentially rising demand for electricity.
In the year 2021, Alliander laid 2027 kilometres of new electricity cables, 260 kilometres of new gas pipelines and replaced or created 1075 new mid voltage stations. The demand for energy is increasing faster than previously anticipated resulting in a rising demand for new energy infrastructure. The shortage of technical personnel and materials complicates the timely expansion of the energy network. This results in increasing waiting time for new customer connections on the energy network. This is referred to as transport scarcity or “transportschaarste” in Dutch, meaning that the energy network is unable to meet the rising power demand in a given area.
Determining the optimal placement for new energy infrastructure is a complicated and time consuming task. The problem of determining a new cable or pipeline route is referred to as “tracébepaling” at Alliander. It plays a key role in facilitating the energy transition as sub-optimal placement will negatively affect the affordability and reliability of the energy network.
A potential way forward is the use of (human-guided) reinforcement learning for the generation of new utility routes. An reinforcement learning (RL) agent is trained to generate a utility route. The reward function is supplemented with human feedback to speed up training of the model and to incorporate human preferences (Lin et al., 2020). This interactive form of RL has been applied in autonomous car navigation (Wu et al., 2021) and underwater autonomous vehicle navigation (Zhang et al., 2022).
Objectives
- Exploring the possibilities of (human-guided) reinforcement learning in utility route generation.
- Demonstrate implementation on a case study.
Literature
- Lin, J., Ma, Z., Gomez, R., Nakamura, K., He, B., & Li, G. (2020). A Review on Interactive Reinforcement Learning From Human Social Feedback. IEEE Access, 8, 120757–120765.
- Wu, J., Huang, Z., Huang, W., & Lv, C. (2021). Prioritized Experience-based Reinforcement Learning with Human Guidance for Autonomous Driving. IEEE Transactions on Neural Networks and Learning Systems, 35(1), 855–869.
- Zhang, Q., Cheng, C., Fang, Z., Jiang, D., He, B., & Li, G. (2022). Continuous Control for Autonomous Underwater Vehicle Path Following Using Deep Interactive Reinforcement Learning. 2022 International Conference on Machine Learning, Control, and Robotics (MLCR), 19–26.
Requirements
- Dutch proficiency preferred as much documentation is in Dutch
- Experience with Python or similar scripting languages
- Interest in reinforcement learning
Theme(s): Modelling & visualisation