In April 2006, Standard & Poor's Ratings Services began to expand its analytical approach for assessing the trading risk management (TRM) practices of U.S. energy companies and to incorporate this analysis more formally into our ratings in this sector. The expansion focuses on the companies' risk management policies, infrastructure, and methodologies, or PIM. This analytical tool is one of three that we use to assess an energy company's trading risk position. The other two parts are our well established liquidity survey, which focuses on liquidity adequacy given certain market stresses, and our capital adequacy methodology, which measures the discrete risks of market, credit, and operational events and estimates capital needed for these exposures.
We have completed PIM reviews for 10 U.S. energy companies that have significant trading and marketing operations, i.e., companies that transact in the market daily, have open commodity positions, and use financial and/or physical transactions to hedge those positions.
We selected an initial set of firms to evaluate our PIM approach across a spectrum of U.S. energy companies, including large and small firms with a wide range of exposure to trading risk. The set included merchant power producers, pure financial trading companies, unregulated utilities, and oil and gas concerns, and the group was selected to enable a reasonable peer comparison of the results. We used a small sample size to assess the effectiveness of our approach before rolling it out to a broader group of companies.
We plan to use the results of these initial efforts to make modifications to the methodology and then identify and implement refinements to the approach to improve our analysis of overall risk management. We also expect to use our findings to benchmark this group of peers against the best practices of this sector. Some are outliers and may not be benchmarked against the merchant peers.
PIM Methodology Focuses On Three Areas
We assessed the TRM practices of the initial set of companies based on the PIM methodology defined in our report,
“Taking The "PIM" Approach When Assessing U.S. Energy Companies' Risk Management,” (published on RatingsDirect on April 21, 2006). Our analysis breaks down the PIM structure to evaluate three specific aspects of a company's risk management infrastructure:
Policies. For this segment, we focus on four key items: The stature of risk management in the company, an assessment of risk appetite, the risk control process, and how risk information is disclosed internally and externally. We examine the importance of risk management (RM) to the company, and how this importance is formally defined in the organizational structure in development of business strategy, risk appetite, and budgeting. Regarding risk appetite, we examine how the company establishes risk tolerance qualitatively and quantitatively and the strength of the process to approve new products. For the risk control process, we focus on how risk tolerance and RM responsibilities are defined in company policies, and how the risk limits and limit monitoring are set. Risk disclosure reveals the content and effectiveness of internal and external risk disclosure.
Infrastructure. In this dimension, we look at two primary functions: How risk data is captured and managed with the risk architecture and an assessment of back office functions. This assessment examines the degree of risk system integration, and robustness of the data recovery operations, and how the business continues under disruption events. The back office assessment is geared towards looking at structure, the experience of the key personnel running the operations, and the integrity of the data sources.
Methodologies. In this component, we assess the quality and variety of the company's valuation techniques and the way in which models are vetted and updated. We examine how the company determines its pricing models, how it captures the asset operating risks that differentiate the company from financial institutions, and the methods the company adopts to assess counterparty credit risk in its trading activities. In addition, we look at the extent to which the company performs backtesting for its value-at-risk (VaR) and other pricing models. We also examine the ways in which the companies attempt to measure operational risks--such as power plant outages or internal communication errors--and the potential financial risk that such events might create.
The PIM method that we used for our initial analysis was derived from that used by Standard & Poor's Financial Institutions Group to assess RM practices of large financial players exposed to types of trading and marketing risks. However, we modified the PIM approach for our evaluation of the energy sector to better capture the characteristics of the more narrow and unique field, composed mostly of utilities, energy merchants, and oil and gas companies.
We conducted our assessment based on analysis of company documentation related to the TRM and detailed interviews and presentations by senior management of the companies' risk management functions. We evaluated each company against criteria that we developed and consider to be the best practices for the industry, although these benchmarks may not be widely applied with in the energy sector.
For each of the specific risk management attributes, we assigned a qualitative score. The quality was considered weak if most of the criteria are not met, standard if about one-half to two-thirds are satisfied, strong if about two-thirds are achieved, and very strong if nearly all the criteria are satisfied.
The companies we selected are listed in the table below.
Companies Selected
Company
Rating
Black Hills Corp.
BBB-/Stable/--
Constellation Energy Group Inc.
BBB+/Negative/A-2
Dynegy Inc.
B/Stable/B-2
Edison Mission Energy
BB-/Stable/--
Exelon Corp.
BBB+/Watch Neg/A-2
Mirant Corp.
B+/Watch Neg/--
NRG Energy Inc.
B+/Stable/B-2
Reliant Energy Inc.
B/Positive/B-2
Sempra Energy
BBB+/Stable/A-2
TXU Corp.
BB/Watch Neg/--
Results Show Wide Ranging Policies And Practices
The results of the initial PIM analysis show that there is a wide range of RM arrangements to manage trading risk and that, generally, the more volume of trading that a company does, the more comprehensive the RM infrastructure. These results are different from those found for financial institutions, which show significantly less variability across the PIM spectrum. Following are summarized findings for specific aspects of the RM infrastructure that we focused on in our analysis. Our goal at this point is not to identify the overall risk scores of the companies but to point out the wide range of structures that these companies employ to manage trading related risks.
Policies
Stature of risk management.
There is a wide range of RM statures in our initial set of companies, with the overall average being standard. Very few companies meet nearly all of the best practice criteria for establishing a sound RM function, which is key to establishing business strategy and is more or less independent of the commercial side of the business. Most of the companies scored in the standard range while we viewed two as weak and two as very strong. There was a clear distinction among the large number of companies clustered in the standard category. Notably, there are companies that have a weak RM practice along with a significant exposure to trading operations, which introduces concern that business strategy and risk management aren't aligned.
Three companies meet the best practice structure of having the RM function report to the CEO, rather than to a lower level officer or committee. In about one-half of the companies, the Chief Risk Officer (CRO) reports to the CFO, while in one firm, the CRO reports to the treasurer. While reporting to the CFO introduces a potential conflict of interests with the commercial function, in some cases the CRO has dotted-line reporting to the board of directors. In some cases, the CRO holds dual functions, in one case also being the CFO and in another case being the treasurer. In most cases, the CRO is a part of the executive management team, with an active role in establishing the RM infrastructure throughout the organization. In a small number of companies, the RM function is relegated to a lower reporting position or stature within the company following the replacement of the CRO, likely due to conflicts between the RM and commercial considerations.
Generally, the CROs and their staffs are well experienced, and in many cases have been employed by their respective companies for several years. However, in some companies the CRO had limited experience with RM while in another, the CRO has no experience at all. In the case of board members, the consensus was that many lacked a background in RM.
Risk tolerance.
Overall, we view the companies' risk appetite to be in line with their business strategies. In most cases, the RM function plays a key role in establishing risk tolerance limits for allowed trading products, but there are cases where senior management has the ability to grant exceptions to limits without RM approval. In most cases, the RM policies provide clear tolerance limits and the trading products that are permitted, and, in better firms, the tolerances are laid out in more advanced metrics, such as risk adjusted return on capital (RaROC), impact on earnings, and revenue volatility.
Less consistent, however, are findings on the vetting process for new products or models. On the one hand, several companies have strong procedures to appraise their models that are well documented and in some cases have been used for many years. On the other hand, a small number of companies didn't have a formal vetting procedure or were viewed as murky given the information provided. Surprisingly, strong vetting procedures were adopted by companies that scored weak and very strong on RM stature. There is some argument that these procedures may not need to be as robust for firms that have a lower level of trading activity and trade around their assets rather than with proprietary products. However, one firm that lacks a history of building new trading products has a respectable procedural process to appraise them.
Risk control processes.
There is considerable variation on how each company formally controls its trading risk. Many companies employ detailed, versioned risk policy manuals that clearly establish the goal of the trading function, roles and responsibilities, trading limits, model vetting, and other items. In most cases, requirements exist to approve policies on a regular basis. However, the policies provided by some companies are light regarding detailed trading limits, requirements for period reviews, and other items. Among the particular features that separate the companies are the use of trading limits for the enterprise as a whole and the application of trading limits to each transaction. Also notable was that one company lacks the infrastructure to conduct stress testing despite being exposed to significant trading risks. Moreover, there is a wide range of stress testing that the companies perform: Generally the larger the company and the more they undertake non-asset based trading, the more stress and scenario testing they perform. The message here is that small companies may not have the resources to fully develop stress and scenario testing activities or are not willing to apply their limited resources fully to risk control.
One feature shared by all companies beyond the two strongest are that RM policies are fairly new, ranging from just a few years to less than one year. It may be that policies existed in a less formal manner than they do today, since many of these firms have been trading for a much longer period. Our work in the financial institution group on TRM suggests that it takes years for companies to develop robust risk control policies that approach best practices, which suggests that steady improvements in the policies of the energy companies are likely to occur.
Risk information disclosure.
In our assessment, we reviewed with senior management all of the internal and external reports related to risk management and evaluated the ability of senior management to efficiently use the data to meet the company's business goals and monitor risk positions. Overall, no company achieved a very strong assessment in this category, primarily due to poor external reporting that was limited to basic VaR and credit exposure. Such external reporting appears to be the industry norm and perhaps understandable from a competitive viewpoint. However, from the perspective of credit, shareholder interests, and adoption of best practices, we believe that energy companies must provide effective external disclosure of their risk appetite, control measures, and potential exposure of "tail risks" that goes beyond the minimum financial reporting standards.
Although external reporting is essentially de minimus, internal reporting is often quite robust, but the effectiveness of the reporting clearly differs among the companies. Based solely on this measure, we assessed most companies as standard, a few as strong, and a few as weak. All reporting involved substantial amounts of quantitative data, but not all provided effective qualitative data as well, which we view as important for proper communication with senior management and board members. Another deficiency we noted is the lack of data on the potential financial effects of operational risk to which nearly each company is exposed.
Methodology
Valuation techniques.
The assessment of valuation processes focuses on whether pricing models exist for all transactions and the methodology used to assess counterparty credit risk. We also look at the ways in which the companies assess risk, include VaR methods, and the types and frequency of stress and scenario testing they adopted to assess potential exposure. Only one company approached a very strong assessment on valuation techniques, and most other companies came out on the standard category.
All companies employ some form of VaR method to determine trading risk exposure, but the ability to identify VaR at all levels, transaction, commodity class, and consolidated, isn't universal. We found several to be weak in this discipline for various reasons. The companies that have consistently scored above average on the PIM survey tend to go well beyond basic VaR measured when assessing risk exposure, such as credit VaR, EBIT at risk (EBITaR) cash flow at risk, and RAROC. Pricing models for transactions are the norm.
A surprising finding is that many companies in our survey lack robust stress and scenario testing. These companies tend to have a limited level of trading activity and small amounts if any of proprietary trading. In addition, those firms with limited stress and scenario analytic capability tend to be on the smaller side. Given the potential for high volatility of commodity prices in energy markets, the lack of sold stress and scenario testing is problematic for understanding market and counterparty credit exposures.
Model vetting process.
Our TRM reviews focused on how companies validate and review their pricing models, and the results showed a wide range of approaches to this discipline. Very few firms showed strong to very strong model-vetting arrangements. Almost all companies reported that models are reviewed at some point in time, but few have firm requirements for period testing and evaluation. There is also wide use of independent curve validation. In addition, most of the companies we examined employ backtesting, but some of the companies don't backtest at all due primarily to system constraints. The integrity of backtesting data sets arose as an issue for some companies. Another problem for some of the firms that lack strong backtesting capability is their limited ability to achieve granularity at the transaction level. Overall, we view this area of model vetting as one that requires additional analytical assessment.
Infrastructure
Most of the companies employ an integrated risk infrastructure, but companies differed significantly in how they characterized their systems. Few provided detailed schematics that would provide senior management with a detailed understanding of the integrated path from the trading operations to the financial books. Some companies lack complete integration. For example, in some cases, a company's technology architecture does not handle all commodities and, as noted previously, some systems can't break down VaR at the transaction level. We did not review in depth the detailed plans for business continuity under disruption events.
Most of the companies have very experienced personnel managing and operating this part of the business, and many people have been performing the same operations at their respective companies for a long time. The more advanced firms have staff with significant experience with trading operations at large investment banks.
Next Steps
We are encouraged by this initial foray into TRM analysis, and, following some refinements to our PIM assessment approach, we will continue to include a trading risk assessment in the ongoing ratings process for all U.S.-based power companies with trading operations. We expect to expand the management section of the business risk profile beyond what is done currently, i.e., to factor in a company's risk appetite and the risk management infrastructure in place to mitigate related credit risks.
A satisfying byproduct of our examination of trading risk among these test firms was to inform our analysis on overall risk management practices-–beyond trading risk alone–-allowing us to form impressions of companies' approach to managing other, broader corporate risks. As a result, we expect to propose for comment later this year an expanded risk assessment approach that encompasses the broader risks to the firm, going from TRM to enterprise risk management (ERM) for U.S. power companies.
These criteria will build on our PIM (policies, infrastructure, and methodology) framework that was constructed to assess the trading operations of large utility firms. The structure and components of the framework that will be developed to assess the ERM practices of these utility firms will represent what we believe to be "sound practices," and aren't necessarily widely applied in the industry.
While we have historically viewed risk management practices of corporations from a more passive perspective, this enhanced analytic framework will reflect the evolving nature of risk management practices across the industry and provide us with a unified and consistent platform to assess the ERM practices of these organizations globally. As ERM is a dynamic and forever evolving discipline, our criteria will be revised and updated regularly to reflect our dialogue with companies and the evolving risk management practices within and across industries.
Risk governance and operational risk will be the driving factors in the analytical evolution from TRM to ERM, as it influences the quality and strength of risk management across the majority of firms, independent of their sector and size. In addition, we will assess the risk culture, risk appetite, and the ability and robustness of the firm to view its aggregate risks and make decisions based on a comprehensive risk-reward framework and the quality of its risk disclosure.
Operational risk is inherent in all the activities that a corporation pursues. Hence a default in the structural content of this component can lead to significant loss impacts. Embedded in this analysis will be an assessment of the robustness of the processes around a firm's practices that govern its insurance against business, legal, and reputation risk as well. In addition, we will assess other firm-specific areas of risk, including funding and liquidity.
Analytic services provided by Standard & Poor's Ratings Services (Ratings Services) are the result of separate activities designed to preserve the independence and objectivity of ratings opinions. The credit ratings and observations contained herein are solely statements of opinion and not statements of fact or recommendations to purchase, hold, or sell any securities or make any other investment decisions. Accordingly, any user of the information contained herein should not rely on any credit rating or other opinion contained herein in making any investment decision. Ratings are based on information received by Ratings Services. Other divisions of Standard & Poor's may have information that is not available to Ratings Services. Standard & Poor's has established policies and procedures to maintain the confidentiality of non-public information received during the ratings process.
Ratings Services receives compensation for its ratings. Such compensation is normally paid either by the issuers of such securities or third parties participating in marketing the securities. While Standard & Poor's reserves the right to disseminate the rating, it receives no payment for doing so, except for subscriptions to its publications. Additional information about our ratings fees is available at www.standardandpoors.com/usratingsfees.
