Active network management for electrical distribution systems: problem formulation and benchmark

Active network management for
electrical distribution systems: problem
formulation and benchmark
Brussels, Belgium
1
Q. Gemine, D. Ernst, B. Cornélusse
Department of Electrical Engineering and Computer Science
University of Liège
2014 Dutch-Belgian RL Workshop

Motivations
Environmental concerns are driving the growth of
renewable electricity generation
Installation of wind and solar power generation resources
GREDOR - Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables
2
at the distribution level
Current fit-and-forget doctrine for planning and operating
of distribution network comes at continuously increasing
network reinforcement costs

Active Network Management
ANM strategies rely on short-term policies that
control the power injected by generators and/or taken
off by loads so as to avoid congestions or voltage
problems.
Simple strategy:
More advanced strategy:
3
Curtail the production of generators.
Move the consumption of loads to
relevant time periods.
Such advanced strategies imply solving large-scale
optimal sequential decision-making
problems under uncertainty.

Observations
4
Several researchers tackled this
operational planning problem.
They rely on different formulations of the
problem, making it harder for one
researcher to build on top of another
one’s work.
We are looking to provide a generic formulation of the
problem and a testbed in order to promote the
development of computational techniques.

Problem description
D
10 2
9 3
8
7 5
5
We consider the problem faced by a DSO willing to
plan the operation of its network over time, while
ensuring that operational constraints of its
infrastructure are not violated.
This amounts to determine over
time the optimal operation of a
set of electrical devices.
We describe the evolution of the system
by a discrete-time process having a time
horizon T (fast dynamics is neglected).
12
6
4
11 1

5
4
Control Actions
Control actions are aimed to directly impact the
power levels of the devices .
8
7
6
5
4
3
2
1
0
Time
P (MW)
Potential prod.
Modulated prod.
d 2 D
Figure 3: Curtailment of a distributed generator.
8
7
6
5
4
3
2
1
3
2
1
Curtailment instructions can
be imposed to generators.
We also consider that the DSO can modify the consumption of the flexible loads. constitute a subset F out of the whole set of the loads C ⇢ D of the network. An activation is associated to this control mean and flexible loads can be notified of activation up immediately preceding the start of the service. Once the activation is performed at consumption of the flexible load d is modified by a certain value during Td periods. these modulation periods t 2 [[t0 + 1; t0 + Td]], this value is defined by the modulation !Pd(t − t0). An example of modulation function and its influence over the consumption is presented in Figure 4.
1
0.5
0
ï0.5
9
8
7
6
5
4
3
2
1
GREDOR - Gestion des Réseaux Electriques de Distribution 1
Ouverts aux Renouvelables 6
0.5
ΔE+
9
8
7
0
Time
P (MW)
Potential prod.
Modulated prod.
We also consider that the DSO can modify the consumption of the flexible loads. constitute a subset F out of the whole set of the loads C ⇢ D of the network. An activation is associated to this control mean and flexible loads can be notified of activation up immediately preceding the start of the service. Once the activation is performed at consumption of the flexible load d is modified by a certain value during Td periods. these modulation periods t 2 [[t0 + 1; t0 + Td]], this value is defined by the modulation !Pd(t − t0). An example of modulation function and its influence over the consumption is presented in Figure 4.
0
Time
P (MW)
We also consider that the DSO can modify the consumption of the flexible loads. These loads
constitute a subset F out of the whole set of the loads C ⇢ D of the network. An activation fee
is associated to this control mean and flexible loads can be notified of activation up to the time
immediately preceding the start of the service. Once the activation is performed at time t0, the
consumption of the flexible load d is modified by a certain value during Td periods. For each of
these modulation periods t 2 [[t0 + 1; t0 + Td]], this value is defined by the modulation function
!Pd(t − t0). An example of modulation function and its influence over the consumption curve
is presented in Figure 4.
1 2 3 4 5 6 7 8 9
ï1
t − t0 (time)
ΔPd (kW)
ΔE−
ΔE+
(a) Modulation signal of the consumption (Td = 9).
0
T ime
P (kW)
Standard cons.
Modulated cons.
t0 t0 + Tf
(b) Impact of the modulation signal over the con-sumption.
Figure 4
Cost: Encurt [MWh] ⇥ Price

e
MWh
"
Flexibility service of loads can
be activated.
Cost: activation fee
Time-coupling effect.

Problem Formulation
The problem of computing the right control actions is
formalized as an optimal sequential decision-making
problem.
We model this problem as a first-order Markov
decision process with mixed integer and continuous
sets of states and actions.
st+1 = f(st, at,wt+1)
st, st+1 2 S
at 2 Ast
wt+1 ⇠ p(·|st)
GREDOR - Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables 7

System state st
The electrical quantities can be deduced from the
power injections of the devices.
Active power injections of loads and power level of
the primary energy sources of DG (i.e. wind and
sun).
in st
The control instructions of the DSO that affect the
current period and/or future periods are also stored in
the state vector.
Upper limits on production levels and the number
of active periods left for flexibility services
in st
st = (P1,t, . . . ,P|C|,t, irt, vt, P1,t, . . . , P|G|,t, s(f)
1,t , . . . , s(f)
|F|,t, qt)

⇠ pW(·|0
of hour in the day qt allows capturing the daily trends of the process. Given the hypothesis constant power factor for the devices, the reactive power consumption can directly be deduced
from Pd,t+1:
thus governed by relation
0 5 10 15 20 25 30
the behavior of consumers inevitably leads to uncertainty from the network. However, over a one day horizon, some peaks arise for example in the early morning and in the system evolution from a state st to a state st+v (1 m/s)
is described by the transition function state Transition st+1 depends, f : S in ⇥ st+addition As Figure Function
to the preceding 1 = ⇥f(5: st, W7! Power at,wt) curve , S ,
of a wind state, generator.
of the control (12)
actions of the realization of the stochastic Qq,t+1 = tan processes, !modeled d · Pd,t+1 .
as Markov processes. specifically, we have
2 W and such that it follows a conditional probability law pW(·|st). In order to define
possible realizations of a random process. The general evolution by relation
3.3.3 Solar irradiance and photovoltaic production
Like wind generators, the photovoltaic generators inherit their uncertainty in production from the uncertainty associated to their energy source. This source is represented by the level solar irradiance, which is the power level of the incident solar energy per m2. The irradiance is the stochastic process that we model while the production level is obtained by a deterministic
function of the irradiance and of the surface of photovoltaic panels. This function is simpler
that the power curve of wind generators and is defined as
function in more detail, we now describe the various processes that constitute it.
In Section 5, we describe a possible procedure to model the evolution of the consumption an aggregated set of residential consumers using relation (13).
f : S ⇥ As ⇥W7! S ,
Load consumption
is the set of possible realizations of a random process. The general evolution thus governed by relation
Curtailment instructions for
next period and activation
of flexible loads.
Set of possible realizations of
a random process, with wt ∈ W
3.3.2 Wind speed and power level of wind generators
The uncertainty about the production level of wind turbines is inherited from the uncertainty
about the wind speed. The Markov process that we consider governs the wind speed, which
is assumed to be uniform over the network. The production level of the wind generators then obtained by using a deterministic function that depends of the wind speed realization, function if the power curve of the considered generator. We can formalize this phenomenon vt+1 = fv(vt, qt, w(v)
uncertainty about the behavior of consumers inevitably leads to uncertainty about the
they withdraw from st+the 1 = network. f(st, However, at,wt) over , a one day horizon, some trends can
observed. Consumption peaks arise for example st+1 = in f(the st, early at,wt) morning , and in the evening for
that consumers it follows but at levels a conditional that fluctuate from probability one day to another and among consumers.
2 model W and the evolution such that of it the follows consumption a conditional Pg,of t = each ⌘g · surfg load probability · d irt 2 ,
C law by
In order law pW(pW(·|·st). |st). In order detail, function we in more now detail, describe we now the describe various the various processes processes that that constitute constitute it.
consumption
t is a component of wt ⇠ pW(·|st). The technique used in Section 5 to build from a dataset is similar to the one of the wind speed case.
{
where ⌘g is the efficiency factor of the panels, assumed constant and equal to 15%, while is the surface of the panels in m2 and is specific to each photovoltaic generator. The irradiance
level is denoted by irt and the whole phenomenon is modeled by the following Markov process:
t ) , Pg,t+1 = ⌘g(vt+1), 8g 2 wind generators ⇢ G , such that w(v)
is a component of wt ⇠ pW(·|st). The dependency of functions fd to the quarter
the day qt allows capturing the daily trends of the process. Given the hypothesis of
power factor for the devices, the reactive power consumption can directly be deduced
1:
Load consumption
follows a conditional
probability law
Pd,t+1 = fd(Pd,t, qt, wd,t) , (13)
irt+1 = fir(irt, qt, w(ir)
t ) , Pg,t+1 = ⌘g · surfg · irt+1, 8g 2 solar generators ⇢ G , such that w(v)
uncertainty about the behavior of consumers inevitably leads to uncertainty t is a component of wt ⇠ pW(·|st) and where ⌘g is the power curve of generator
level g. A they typical withdraw example of from power the network. Qq,curve = tan ⌘g(v) is However, illustrated over a one day horizon, some t+1 !.
in Figure 5. Like loads, the production
observed. of reactive Consumption power is obtained peaks using:
arise for d example · Pd,t+1 in the early morning and in the evening residential Section consumers 5, GREDOR we describe - Gestion but des a at possible Réseaux levels Electriques procedure that de fluctuate Distribution to model Ouverts from the aux evolution one Renouvelables day of to the another consumption and among of
9
consumers.
Qg,t+1 = tan !g · Pg,t+1 .
aggregated set of residential consumers using relation (13).
we model the evolution of the consumption of each load d 2 C by

0 = hn(e, f, P
n,t,Q
n,t) , (24)
and, in each link l 2 L, we have:
Reward Function
T he re wa rd fun cti on r : S ⇥ A s ⇥ S 7! R associates an
instantaneous rewards for each transition of the
system from a period t to a period t+1:
il(e, f) = 0. (25)
3.4 Reward function and goal
In order to evaluate the performance of a policy, we first specify the reward function r : S ⇥
As ⇥ S7! R that associates an instantaneous rewards for each transition of the system from a
period t to a period t + 1:
r(st, at, st+1) = −
X
g2G
max{0,
Pg,t+1 − Pg,t+1
4 }Ccurt
g (qt+1)
| {z }
curtailment cost of DG
−
X
d2F
a(f)
d,t Cflex
d
| {z }
activation cost
of flexible loads
− |"(s{tz+1})
barrier function
,
2]0; 1[ is the discount factor. Given that !t < 1 for t > 0, the further in time (26)
transition from period t = 0, the less importance is given to the associated reward. Because
Because the operation of a DN must be always be
ensured, we consider the return R over an infinite
trajectory of the system:
where Ccurt
g (qt+1) is the day-ahead market price pour the quarter of hour qt+1 in the day and
operation of a DN must be always be ensured, it does not seem relevant to consider finite number of periods and we introduce the return R as
Cflex
d is the activation cost of flexible loads, specific to each of them. The function " is a
barrier function that allows to penalize a policy leading the system in a state that violates the
operational limits. It is defined as
R = = lim
!tr(st, at, st+1) , "(st+1) =
R1 corresponds to the weighted sum of the rewards observed over an infinite trajectory Given that the costs and penalties have finite values and that the reward function sum of an infinite number of these costs and penalties, it exists a constant C such X
n2N
TX−1
n,t+1 + f2
n,t+1 − V 2
["(e2
n) + "(V 2
n − e2
n,t+1 − f2
n,t+1)]
+
X
T!1
t=0
l2L
"(|Il,t+1| − Il) , (27)
where en,1, fn,(n 2 N) and Il,(l 2 L) are determined from st+using equations (21)-

can also observe that, because st+1 = f(st, at,wt), it exists a function ⇢ : S ⇥A⇥aggregates functions f and r and such that
Optimal Policy
r(st, at, st+1) = ⇢(st, at,wt) , pW(·|st). Let ⇡ : S7! As be a policy that associates a control action system. We can define, starting from an initial state s0 = s, the expected ⇡ by
r(st, at, st+1) = ⇢(st, at,wt) , wt ⇠ pW(·|st). Let ⇡ : S7! As be a policy that associates a control action to each system. We can define, starting from an initial state s0 = s, the expected return policy ⇡ by
Let be a policy that associates a control
action to each state of the system, the expected return
of this policy can be written as:
TX−1
"
⇢(s, a,w) + !J⇡⇤(f(s, a,w))
, 8s 2 S . only take into account the space of stationary policies (i.e. that select an action !t⇢(st, ⇡(st),wt)|s0 = s} . by ⇧ the space of all the policies ⇡. For a DSO, addressing the operational described in Section 2 is equivalent to determine an optimal policy ⇧, i.e. a policy that satisfies the following condition
J⇡(s) = lim
E
!t⇢(st, ⇡(st),wt)|s0 = . T!1
s} ⇢(st, at,wt) = r(st, at, f(st, at,wt))
denote by ⇧ the space of all the policies ⇡. For a DSO, addressing the operational problem described in Section 2 is equivalent to determine an optimal policy ⇡⇤ among elements of ⇧, i.e. a policy that satisfies the following condition
TX−1
wt⇠pW(·|st){
t=0
J⇡⇤(s) ( J⇡(s), 8s 2 S, 8⇡ 2 ⇧. well know that such a policy satisfies the Bellman equation [9], which can be written
J⇡⇤(s) = max
a2As
E
w⇠pW(·|s)
⇡ : S7! As
Let ⇧
be the space of all stationary policies. Addressing
the operational planning problem of a DSO consists in
finding an optimal policy :
⇡⇤ 2 ⇧
J⇡(s) = lim
T!1
E
wt⇠pW(·|st){
t=0
J⇡⇤(s) ( J⇡(s), 8s 2 S, 8⇡ 2 ⇧. know that such a policy satisfies the Bellman equation [9], which can J⇡⇤(s) = max
E
"
⇢(s, a,w) + !J⇡⇤(f(s, a,w))
, 8s 2 S

Solution Techniques
We identified three classes of solution techniques that
could be applied to the operational planning problem:
• mathematical programming and, in particular,
multistage stochastic programming;
• approximate dynamic programming;
• simulation-based methods, such as direct policy
search or MCTS.

Figure 6: Test network.
We designed a benchmark of the ANM
problem with the goal of promoting
computational research in this complex
field.
The set of models and parameters that
are specific to this instance as well as
documentation for their usage are
accessible as a Matlab class at
www.montefiore.ulg.ac.be/~anm/ . 10 20 30 40 50 60 70 80 90
iteration k, parameter values ✓i (i 2 I(k)) are evaluated by simulating trajectories of the system
are associated to the policies ⇡✓i . The result of these simulation allows the selection of
parameter values ✓i (i 2 I(k+1)) for the next iteration. The goal of such an algorithm is
converge as fast as possible towards a parameter value ˆ✓⇤ that defines a good approximate
optimal policy ⇡ˆ✓⇤ .
Another subset of simulation-based methods is the Monte-Carlo tree search technique [26,
A each time-step, this class of algorithms usually rely on the simulation of system trajec-tories
to build, incrementally, a scenario tree that does not have a uniform depth. These are
previous simulations that are exploited to select the nodes of the scenario tree that have to
developed. When the construction of the tree is done, the action that is deemed optimal for
root node of the tree is applied to the system.
Test instance
this section, we describe a test instance of the considered problem. The set of models
parameters that are specific to this instance as well as documentation for their usage
accessible at http://www.montefiore.ulg.ac.be/~anm/ as a Matlabr class. It has been
developed to provided a black-box-type simulator which is quick to set up. The DN on which
instance is based is a generic DN of 75 buses [28] that has a radial topology, it is presented
Figure 6. We bound various electrical devices to the network in such a way that it is possible
gather the nodes of this network into four distinct categories:
20
15
10
5
0
−5
dD
−10
−15
−20
−25
Pd,t (MW)
X
each residential node is the connection point of a load that represents a set of residential
−30
T emps
!d ∈DPd( t) (MW)
Time

Figure 8: Power withdrawal scenarios of the devices. Negative values indicate that more power than what is consumed by loads.
65
60
55

Active network management for electrical distribution systems: problem formulation and benchmark

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to Active network management for electrical distribution systems: problem formulation and benchmark

Similar to Active network management for electrical distribution systems: problem formulation and benchmark (20)

Recently uploaded

Recently uploaded (20)

Active network management for electrical distribution systems: problem formulation and benchmark