Ratings

It is well understood that VaR as a measure of market risk is a function of several underlying assumptions. Hence, to look at the reported trading VaR numbers of two institutions and comment on their relative risk appetite is not meaningful.

Most financial institutions use one of three measures for estimating VaR: parametric (or variance-covariance) VaR, Monte Carlo VaR, and historical VaR. Not surprisingly, Standard & Poor's has found through its TRM surveillance sessions that most institutions do not use parametric VaR or Monte Carlo VaR as an aggregate measure of risk. One must remember that parametric VaR assumes normality of the profit and loss distribution and requires an estimate of the volatilities (variances) and correlations (covariances) of the risk factors in the portfolio. Then, for a given confidence level and time horizon, the parametric VaR of the portfolio is a multiple of the portfolio's standard deviation, which is derived from the estimated matrix of variances and covariances. Parametric VaR is easy to compute, but in addition to the assumption of normality of returns, it has several limitations. The most important of these is its inability to capture the risk arising from the nonlinearity of the positions in the portfolio, or gamma risk. Nonlinearity is present in fixed-income positions (such as a bond) and in those positions with optionality. In addition, this approach requires an estimation of the volatilities and correlations of the risk factors. Obtaining a "true" estimate of the future volatilities and correlations (or variance-covariance (VCV) matrix) of the risk factors is nontrivial.

A second approach to estimating VaR is employing a Monte Carlo simulation technique. Under this approach, a future probability distribution for the relevant risk factors is assumed and randomly simulated several times over a given time horizon. The portfolio is re-evaluated under each simulation and a histogram of the profit and loss is obtained, from which the VaR is inferred. However, the Monte Carlo methodology is also dependent on the future VCV matrix and the distributional assumptions of the risk factors. In addition, the systems requirements to run a full-fledged Monte Carlo VaR engine are quite demanding, especially if the portfolio has a number of complex options and mortgages. Hence, regulators have resorted to the more practical and easily understood historical method of calculating VaR.

Briefly, historical VaR calculates a time series of historical changes in the relevant risk factors over a specified time period, applies those changes to the current levels of the risk factors, and then re-evaluates the portfolio to obtain a histogram of the profit and loss, from which the VaR is inferred. This approach assumes that the historical distribution will hold (or is a good proxy for the "true" distribution of returns) for the period over which VaR is being calculated. Also, as this approach is non-parametric, it does not require an estimate of the variances and covariances as these are already reflected in the historical time series. This approach is clearly simple to understand and, given the availability of historical data, is easy to compute.

Standard & Poor's has observed that the industry tends to use a range of values as inputs to an historical VaR computation. The key inputs are the length of the historical look-back period for the risk factors, the choice of the confidence interval, and the method for revaluing the portfolio given the set of historical scenarios. For all these key inputs, the range of values can vary significantly across institutions, making a meaningful comparison of the VaR estimate very difficult.

For example, Standard & Poor's has found that the length of the look-back period for the time series of risk factors used by a number of financial institutions could vary from as short as 250 days (the equivalent of one trading year) to more than 1,000 days (approximately four years). Institutions that weigh recent market volatility higher tend to use a shorter time series, while those that aim at having a relatively more stable VaR over time tend to use a longer time series for the risk factors.

There is no general consensus on whether the level of the confidence interval should be 99%, 98%, or 95%. The higher the confidence interval, the higher might be the VaR estimate, depending on the shape of the tail of the profit and loss distribution.

In addition, the methods employed for revaluing the positions over the set of historical scenarios vary across institutions. At one end of the spectrum are institutions that use an approximation (such as a delta or delta-gamma approach) to revalue the positions. This is analogous to approximating the value of a bond using either duration only or duration and convexity (to capture the curvature or non-linearity in the price-yield relationship of the bond). Also, one must remember that approximating the value of a bond by duration only can lead to a significant underestimation in value depending on the size of the yield change.

At the other end of the valuation spectrum are institutions that employ a full revaluation. To continue with the bond example, this is equivalent to present-valuing the cash flows from the bond at an appropriate discount rate. Between these approaches are some institutions that use a combination of these methods; an approximation for some products (usually the ones that are complex with a fair degree of non-linearity, and hence the ones that are most likely to produce the valuation errors) and a full revaluation for others. While the approximation techniques speed up the computational time for the VaR calculations, they do have the drawback of producing potential valuation errors depending on the degree to which the non-linearity inherent in the positions is accurately captured.

Given the range of possible assumptions behind estimating VaR, it is not always possible to use the reported VaR estimates as a common denominator for comparing the risk appetite across financial institutions. Further, VaR is only one aggregate measure of risk. VaR does not capture tail risk; it says nothing about what the worst case loss might be. Hence, it is critical for investors and analysts to look beyond VaR and request that institutions report on alternative risk measures as well.

Philippe Jorion and Nassim Taleb have debated the pros and cons of VaR as an appropriate measure of risk (5,6). It is well known that if markets suddenly experience severe turbulence, VaR will inevitably underestimate the risk and will only capture the sudden jump in volatility a few days down the road. In addition, the VaR for complex portfolios may be computationally challenging and may not clearly identify the risks inherent in the portfolio. For example, if one portfolio contains a single long position in an option and another a short futures position, VaR will indicate the same level of risk, even though the potential loss in the short futures position is arbitrarily high!

As highlighted above, calculating VaR can be quite challenging, as it depends on a number of assumptions employed by the user. In addition, from a systems perspective for large portfolios, the aggregate computation cannot be split into smaller subcomputations, as VaR is not additive by position or by risk factor. In other words, if a portfolio is split into smaller subportfolios, the VaR of the entire portfolio is not the sum of VaRs of the individual subportfolios. Further, if a portfolio is split by the relevant risk factors, such as foreign exchange (FX), equities (EQ), interest rates (IR), or commodities (COM), then the VaR of the entire portfolio is not the sum of the VaRs of the individual risk factors. That is, FX(VaR), plus EQ(VaR), plus IR(VaR), plus COM(VaR) is not equal to Portfolio(VaR).

One important reason why VaR is not a robust risk measure is that it does not always provide an accurate sense of the degree of diversification within a portfolio. This occurs because VaR lacks an important mathematical property referred to by Philippe Artzner et al. (1, 2) as subadditivity. A risk measure such as VaR is said to be subadditive if the VaR of the portfolio is less than or equal to the sum of the VaRs of its individual components. [Mathematically, if a portfolio is split into two subportfolios X and Y, then subadditivity implies that VaR(X(Y) ( VaR(X) ( VaR(Y)].

Intuitively, subadditivity is associated with the notion of risk reduction through diversification. If there is no diversification benefit in a portfolio (i.e., all components were perfectly correlated), then one would expect the VaR of the portfolio to be as much as the VaR of the individual components. On the other hand, if there were diversification, one would expect the VaR of the portfolio to be less than the VaR of the individual components. The risk in a diversified portfolio should be less than the risk in a portfolio that is not diversified. However, VaR does not guarantee this property, unless one assumes a Gaussian space (or that asset returns are normally distributed). Hence, the extent of diversification reported by firms based on a VaR measure can be misleading and incorrect! (See the Appendix of this article for an example that demonstrates how VaR might violate this property.) Artzner et al. have also demonstrated that because VaR lacks this property of being subadditive, it is not an appropriate metric for allocating capital or assessing risk-adjusted performance.

A risk measure that looks beyond VaR and into the tail of the distribution is expected shortfall (ES). While a 95% VaR is the minimum potential loss of the 5% worst-case scenarios on a portfolio over a given time horizon, ES on the other hand is the mean loss of the 5% worst-case scenarios on a portfolio over a given time horizon. ES looks beyond the quantile on the left tail of the distribution that is used to compute VaR. Hence, it is also referred to in the literature as "mean excess loss" or "tail conditional expectation." What differentiates this measure from VaR is that it captures tail risk.

More generally, ES is the average of the (1 – X%) worst-case outcomes stipulated by the calculated X% VaR estimate. For example, if a 99% VaR is calculated using 1,000 days of historical data (or outcomes), the ES is the average of the (1-99%) or 0.01 worst-case outcomes. In this case, since there are 1,000 outcomes, the ES is the average of the 10 (= 0.01 X 1,000) worst outcomes.

In addition to capturing tail risk, ES satisfies the property of subadditivity in that the ES of a diversified portfolio is always less than the ES of the portfolio's individual components. This makes this risk measure more robust than VaR in expressing diversification benefits. An important characteristic of ES is that it is aware of the shape of the conditional distribution of the worst-case scenarios beyond a specified quantile, while VaR (by definition) ignores this altogether and hence underestimates the potential risk. (See the Appendix for an example of why ES is a "coherent" and robust risk measure, and to that extent more appropriate for capital allocation decisions.)

There is very little additional work, either from a computational or from a systems perspective, that is required of an institution to calculate ES. It is not surprising that some institutions currently look at this measure in addition to their VaR. As a discipline leading to a high quality risk management practice, it makes sense for all institutions to adopt ES within their risk assessment framework. In addition, because of ES' simplicity, it is equally important for analysts and investors to request this information of institutions when assessing their market risk appetite and the robustness of their risk management practices.

Stress testing is an important risk management tool, and the quality and framework adopted by institutions around stress testing is an important criteria used by Standard & Poor's analysts in assessing the risk management practices of financial institutions. Broadly speaking, stress testing assesses the potential impact of historical or hypothetical events or specific movements in risk factors (sensitivity-type analysis) on a given portfolio. All institutions conduct some form of stress testing.

Stress testing evaluates the risks that VaR and ES may not capture, such as events with a very low probability of occurrence but with a significant impact for the firm. The primary objective of stress testing is to achieve an understanding of the risk profile of the firm that goes well beyond what can be described by VaR and ES. A fair amount of thought coupled with macroeconomic analysis goes into designing meaningful stress tests, whether they are historical, hypothetical, or sensitivity-type analysis. In addition, correlations and implied effect among risk factors need to be carefully assessed when designing these tests. Hence, the quality and framework around stress testing adopted by institutions is what adds value to the risk management process and complements measures such as VaR and ES.

For example, some of the most commonly run historical stress tests are the events of "October 1987," "Bond Markets of 1994," "Asian Crisis of 1997," "Long-Term Capital Management/Russian Crisis of 1998," and "Sept. 11, 2001." However, when an institution stress tests its portfolio against "Black Monday," what exactly does that mean? Do they apply the drop in equity markets as experienced on Oct. 19, 1987? Or do they use a time window around that date? Do they also apply the implied correlations of the bond markets as well? These questions, and similar ones, are important issues, as the outcomes will vary depending on how the stress test is designed. Designing meaningful stress tests is critical and, when they are performed correctly, they constitute a powerful tool that complements standard risk measures such as VaR and provides the institution and its shareholders with a deeper understanding of the firm's risk profile.

In conclusion, it should be clear for the reasons and examples cited above that VaR is only one measure of aggregate risk and that it is important to look beyond VaR when measuring market risk. ES and well-designed stress tests are excellent complements to VaR, as they capture tail risk and provide for a more meaningful assessment of the institution's risk profile.

[1] Carlo Acerbi, Dirk Tasche, "Expected Shortfall: A Natural Coherent Alternative to Value-at- Risk," (Milan, AbaxBank, 2001).

[2] Carlo Acerbi, Claudio Nordio, Carlo Sirtori, "Expected Shortfall as a Tool for Financial Risk Management" (Milan, AbaxBank, 2001).

[3] Philippe Artzner, Freddy Delbaen, Jean-Marc Eber, David Heath, "Thinking Coherently," RISK, November 1997, 68-71.

[4] Philippe Artzner, Freddy Delbaen, Jean-Marc Eber, David Heath, "Coherent Measures of Risk," Mathematical Finance, July 1999, 203-228.

[5] Philippe Jorion, "Value at Risk: The New Benchmark for Managing Financial Risk," (New York, McGraw-Hill, 2000).

[6] Nassim Taleb, "Fooled by Randomness: The Hidden Role of Chance in the Markets and in Life" (New York, Thomson Texere, 2001).

Artzner et al. have identified a risk measure to be "coherent" if it possesses certain mathematical properties or axioms.

One important property for coherence is subadditivity. To see how VaR may violate the subadditivity property, consider the following example described by Acerbi and Tasche. Suppose there are two bonds, A and B, with non-overlapping probabilities of default. Each has two different default states with recovery values of 70 and 90 and associated probabilities of 3% and 2%, respectively. In all other scenarios, the bonds redeem at 100. Table 1 below summarizes this information under five possible scenarios. For simplicity of exposition, assume that the initial value (or current market value) of the bonds and the portfolio of the two bonds is the expected value (EV) of the payoff given the assumed probability distribution. Taking the sum of the last three columns in Table 1, this can be calculated as being 98.9 for Bond A and Bond B and 197.8 for the portfolio of the two bonds.

Table 1 Bond Outcomes Under Five Scenarios
Scenarios	Probability	Bond A	Bond B	Portfolio(A+B)	EV of Bond A	EV of Bond B	EV of Portfolio (A+B)
1	0.03	70	100	170	2.1	3	5.1
2	0.02	90	100	190	1.8	2	3.8
3	0.03	100	70	170	3	2.1	5.1
4	0.02	100	90	190	2	1.8	3.8
5	0.9	100	100	200	90	90	180
EV—Expected value.

Table 2 below shows the profit and loss (P/L) for each bond and the portfolio under each scenario. From this table, it is easy to calculate the 95% VaR or 5% worst-case losses.

Table 2 Profit/Loss Distribution For Bonds And Portfolios Under Each Scenario
Scenarios	Probability	P/L Bond A	P/L Bond B	P/L for Portfolio (A+B)
1	0.03	(28.9)	1.1	(27.8)
2	0.02	(8.9)	1.1	(7.8)
3	0.03	1.1	(28.9)	(27.8)
4	0.02	1.1	(8.9)	(7.8)
5	0.9	1.1	1.1	2.2

The 95% VaR for each bond is 8.9 and 27.8 for the portfolio. Clearly, in this example VaR violates the property of subadditivity. (The portfolio VaR turns out to be greater than the sum of the individual VaRs for each bond). Based on VaR, diversification between these two bonds would be discouraged! Notice also that VaR understates the risk; it is the "best" of the worst-case scenarios and totally ignores tail risk or the risk beyond the percentile used to compute VaR. It seems natural, then, that investors should look at other measures beyond VaR in assessing the potential level of market risk within an institution.

To see why ES is a coherent (or robust) measure of risk in terms of satisfying the axiom of subadditivity, we can refer back to the bond example in Table 2. In looking at the 5% left tail of the distribution of P/L for Bond A, Bond B, and the portfolio of the two bonds, it can be seen that the ES for Bond A and Bond B is 18.9 while the ES for the portfolio of the two bonds is 27.8.

The first thing to note is that ES satisfies the axiom of subadditivity in that the ES of the portfolio is less than the sum of the ES of the individual bonds. In addition, one should note how VaR might underestimate the potential risk on the individual bonds. According to the 95% VaR measure (or 5% worst-case losses), the potential loss on the individual bonds is only 8.9, while ES estimates the loss at 18.9!