Since the early 1990’s, the process of deregulation and the introduction of competitive markets have been reshaping the landscape of the traditionally monopolistic and government-controlled power sectors. In many countries worldwide, electricity is now traded under market rules using spot and derivative contracts. However, electricity is a very special commodity. It is economically non-storable, and power system stability requires a constant balance between production and consumption.
At the same time, electricity demand depends on weather (temperature, wind speed, precipitation, etc.) and the intensity of business and everyday activities (on-peak vs. off-peak hours, weekdays vs. weekends, holidays and near-holidays, etc.). On the one hand, these unique and specific characteristics lead to price dynamics not observed in any other market, exhibiting seasonality at the daily, weekly and annual levels, and abrupt, short-lived and generally unanticipated price spikes. On the other hand, they have encouraged researchers to intensify their efforts in the development of better forecasting techniques.
At the corporate level, electricity price forecasts have become a fundamental input to energy companies’ decision making mechanisms. As the California crisis of 2000–2001 showed, electric utilities are the most vulnerable, since they generally cannot pass their costs on to the retail consumers. The costs of over-/under-contracting and then selling/buying power in the balancing (or real-time) market are typically so high that they can lead to huge financial losses or even bankruptcy.
Extreme price volatility, which can be up to two orders of magnitude higher than that of any other commodity or financial asset, has forced market participants to hedge not only against volume risk but also a against price movements. Price forecasts from a few hours to a few months ahead have become of particular interest to power portfolio managers. A generator, utility company or large industrial consumer who is able to forecast the volatile wholesale prices with a reasonable level of accuracy can adjust its bidding strategy and its own production or consumption schedule in order to reduce the risk or maximise the profits in day-ahead trading.
A variety of methods and ideas have been tried for electricity price forecasting (EPF), with varying degrees of success. This review series aims to explain the complexity of the available solutions, with a special emphasis on the strengths and weaknesses of the individual methods.
What and how are we forecasting?
The electricity ‘spot’ price
Unlike most other commodity or financial markets, the electricity ‘spot market’ is typically a day-ahead market that does not allow for continuous trading. This is a result of system operators requiring advance notice in order to verify that the schedule is feasible and falls within transmission constraints. In a day-ahead market, agents submit their bids and offers for the delivery of electricity during each hour (or a shorter load period) of the next day before a certain market closing time. Thus, when dealing with the modelling and forecasting of intraday electricity prices, it is important to remember that, in most markets, prices for all contracts of the next day are determined at the same time using the same available information.
The genuine role of an organised market for electricity (like a power exchange or a power pool) is to match the supply and demand of electricity so as to determine the market clearing price (MCP). Typically, the MCP is established in an auction, conducted once per day, as the intersection between the supply curve (constructed from aggregated supply bids) and the demand curve (constructed from aggregated demand bids) or the system operator estimated demand (in one-sided auction markets, like in Australia or Spain), for each of the load periods. Buy (sell) orders are accepted in order of increasing (decreasing) prices until the total demand (supply) is met.
Note that bids with negative prices are allowed in many markets, potentially leading to negative prices when the demand is very low (the costs of shutting down and ramping up a power plant unit can exceed the loss from accepting negative prices) or the production from renewable sources is very high (most notably from wind).
When there is no transmission congestion, the MCP is the only price for the entire system. However, when there is congestion, locational marginal prices (LMP) or zonal clearing prices differ from the system price and from each other. For smaller and medium-sized markets (like the German EEX, Polish GEE, Scandinavian Nord Pool or Spanish OMEL), the system price is usually established, but for larger markets (like the North American PJM), zonal prices or prices for major market hubs are computed. Interestingly, transmission congestion itself can be predicted in the short-term for the South Norway (NO1) price area of the Nord Pool system.
Closer to real time
For very short time horizons before delivery, the (transmission) system operator (TSO, SO) operates the so-called balancing (or real-time) market. This technical market is used to price deviations in supply and demand from day-ahead or long-term contracts. The TSO needs to be able to call in extra production at very short notice, since the deviations must be corrected on a continuous basis in order to ensure system balance. It should be noted that the balancing market is not the only technical market; to minimise the reaction time in the case of deviations in supply and demand, the system operator runs an ancillary services market, which typically includes the down regulation service, the spinning and non-spinning reserve services, and the responsive reserve service. day-ahead, balancing and ancillary services markets serve different purposes and are complementary.
The patterns and characteristics of the prices of ancillary services differ considerably from those of day-ahead electricity prices, with the particular features of a lower price level, higher variability and more frequent and extreme spikes. The last feature in particular makes the prices for ancillary services more difficult to predict.
Finally, it should be noted that although we use the terms spot and day-ahead interchangeably here, the former need not necessarily refer to the day-ahead market. The European convention is to refer to the day-ahead price as the spot price. However, in the US, the term spot price is typically reserved for the intraday real-time market, while the day-ahead price is called the forward price. Nowadays, some markets in Europe (e.g., in the UK) also allow continuous trading for individual load periods, up to a few hours before delivery. With the shifting of volume from the day-ahead to balancing markets, the term spot is also being used more and more often in Europe to refer to the real-time market.
The average of the 24 hourly (or 48 half-hourly) prices is called the daily price, the daily spot price or the base-load price. The average of prices for the on-peak hours (typically 8 am to 8 pm) is called the peak-load price. These ‘daily’ price conventions generally refer to day-ahead prices. In single settlement real-time markets, the averages are computed for real-time prices.
It is customary to talk about short-, medium- and long-term electricity price forecasting, but there is no consensus in the literature as to what the thresholds should actually be.
Short-term EPF generally involves forecasts from a few minutes up to a few days ahead, and is of prime importance in day-to-day market operations. Medium-term time horizons, from a few days to a few months ahead, are generally preferred for balance sheet calculations, risk management and derivatives pricing. In many cases, evaluation is based not on the actual point forecasts, but on the distributions of prices over certain future time periods. As this type of modelling has a long-standing tradition in finance, an inflow of ‘finance solutions’ is observed readily.
Finally, the main objective of long-term EPF – with lead times measured in months, quarters or even years – is investment profitability analysis and planning, such as determining the future sites or fuel sources of power plants. As Ventosa remarks, capacity-investment decisions are the main variables, and unit-commitment decisions are usually neglected in this context. While similar tools and techniques can be used for short- and medium-term horizons, long-term horizons generally require a totally different approach.
Overview of modelling approaches
Nearly all of the review papers offer their own classifications of the various approaches that have been developed for analysing and predicting electricity prices. Some of them are better, some are worse, but all h ave many things in common. Without loss of generality, most take the classification of Weron as a starting point, with the main groups of models being:
- Multi-agent (multi-agent simulation, equilibrium, game theoretic) models, which simulate the operation of a system of heterogeneous agents (generating units, companies) interacting with each other, and build the price process by matching the demand and supply in the market.
- Fundamental (structural) methods, which describe the price dynamics by modelling the impacts of important physical and economic factors on the price of electricity.
- Reduced-form (quantitative, stochastic) models, which characterise the statistical properties of electricity prices over time, with the ultimate objective of derivatives evaluation and risk management.
- Statistical (econometric, technical analysis) approaches, which are either direct applications of the statistical techniques of load forecasting or power market implementations of econometric models.
- Computational intelligence (artificial intelligence-based, non-parametric, non-linear statistical) techniques, which combine elements of learning, evolution and fuzziness to create approaches that are capable of adapting to complex dynamic systems, and may be regarded as ‘intelligent’ in this sense.
Finally, it should be mentioned that many of the modelling and price forecasting approaches considered in the literature are hybrid solutions, combining techniques from two or more of the groups previously listed. Their classification is non-trivial, if indeed it is even possible.
Over the coming weeks we’ll be looking to cover – in depth – each of the main approaches to electricity price forecasting with discussions of the modelling, their strengths, weaknesses and how they are used in trading and risk management.If you’ve found this blog helpful and would like other topics covered, please feel free to drop me an email with suggestions. You’re welcome to subscribe using ‘Subscribe to Blog via Email’ section and this will get you the latest posts straight to your inbox before they’re available anywhere else