Quant Mashup - Portfolio Optimizer Supervised Portfolios: A Supervised Machine Learning Approach to Portfolio Optimization [Portfolio Optimizer]Standard portfolio allocation algorithms like Markowitz mean-variance optimization or Choueffati diversification ratio optimization usually take in input asset information (expected returns, estimated covariance matrix…) as well investor constraints and preferences (maximum asset weights, risk(...) Correlation-Based Clustering: Spectral Clustering Methods [Portfolio Optimizer]Clustering consists in trying to identify groups of “similar behavior”1 - called clusters - from a dataset, according to some chosen characteristics. An example of such a characteristic in finance is the correlation coefficient between two time series of asset returns, whose usage to partition a(...) Volatility Forecasting: HExp Model [Portfolio Optimizer]In this series on volatility forecasting, I previously detailed the Heterogeneous AutoRegressive (HAR) volatility forecasting model that has become the workhorse of the volatility forecasting literature1 since its introduction by Corsi2. I will now describe an extension of that model due to(...) The Mathematics of Portfolio Return [Portfolio Optimizer]Whether we manage our own investment assets or choose to hire others to manage the assets on our behalf we are keen to know how well our […] portfolio of assets is performing1 and the calculation of portfolio return is the first step in [that] performance measurement process1. Now, while the(...) Covariance Matrix Forecasting: Iterated Exponentially Weighted Moving Average Model [Portfolio Optimizer]In the previous post of this series on covariance matrix forecasting, I reviewed both the simple and the exponentially weighted moving average covariance matrix forecasting models, which are straightforward extensions of their respective univariate volatility forecasting models to a multivariate(...) The Return of Simple and Exponentially Weighted Moving Average Models [Portfolio Optimizer]In the initial post of the series on volatility forecasting, I described the simple and the exponentially weighted moving average forecasting models, that are both easy to understand and relatively performant in practice. Beyond (univariate) volatility forecasting, these two models are also widely(...) Capital Market Assumptions: Combining Forecasts for Improved Accuracy [Portfolio Optimizer]Capital market assumptions1 (CMAs) are forecasts of future risk/return characteristics for broad asset classes over the next 5 to 20 years produced by leading investment managers, consultants and advisors2. These forecasts are well-reasoned, analytically rigorous assumptions about uncertain future(...) Volatility Forecasting: HAR Model [Portfolio Optimizer]Among the different members of the family of volatility forecasting models by weighted moving average1 like the simple and the exponentially weighted moving average models or the GARCH(1,1) model, the Heterogeneous AutoRegressive (HAR) model introduced by Corsi2 has become the workhorse of the(...) Combating Volatility Laundering: Unsmoothing Artificially Smoothed Returns [Portfolio Optimizer]It is common knowledge that returns to hedge funds and other alternative investments [like private equity or real estate] are often highly serially correlated1. This results in apparently smooth returns that have artificially lower volatilities and covariations with other asset classes2, which in(...) Bootstrap Simulations with Exact Sample Mean Vector and Sample Covariance Matrix [Portfolio Optimizer]Bootstrapping is a statistical method which consists in sampling with replacement from an original data set to compute the distribution of a desired statistic, with plenty of possible variations depending on the exact context (non-dependent data, dependent data…). Because bootstrap methods are(...) Cluster Risk Parity: Equalizing Risk Contributions Between and Within Asset Classes [Portfolio Optimizer]The equal risk contribution (ERC) portfolio, introduced in Maillard et al.1, is a portfolio aiming to equalize the risk contributions from [its] different components1. Empirically, the ERC portfolio has been found to be a middle-ground alternative1 to an equally weighted portfolio and a minimum(...) Volatility Forecasting: GARCH(1,1) Model [Portfolio Optimizer]In the previous post of this series on volatility forecasting, I described the simple and the exponentially weighted moving average volatility forecasting models. In particular, I showed that these two models belong to the generic family of weighted moving average volatility forecasting models1,(...) Random Portfolio Benchmarking: Simulation-based Performance Evaluation in Finance [Portfolio Optimizer]As noted in Surz1, the question “Is [a mutual fund’s]2 performance good?” can only be answered relative to something1, typically by comparing that fund to a benchmark like a financial index or to a peer group. Unfortunately, these two methodologies are not without issues. For example, it is(...) Sparse Index Tracking: Limiting the Number of Assets in an Index Tracking Portfolio [Portfolio Optimizer]In the previous post, I introduced the index tracking problem1, which consists in finding a portfolio that tracks as closely as possible2 a given financial market index. Because such a portfolio might contain any number of assets, with for example an S&P 500 tracking portfolio possibly(...) Beyond Modified Value-at-Risk: Application of Gaussian Mixtures to Value-at-Risk [Portfolio Optimizer]In a previous post, I described a parametric approach to computing Value-at-Risk (VaR) - called modified VaR12 - that adjusts Gaussian VaR for asymmetry and fat tails present in financial asset returns3 thanks to the usage of a Cornish–Fisher expansion. Modified VaR, when properly used4, provides(...) Index Tracking: Reproducing the Performance of a Financial Market Index (and more) [Portfolio Optimizer]An index tracking portfolio1 is a portfolio designed to track as closely2 as possible a financial market index when its exact replication3 is either impractical or impossible due to various reasons4 (transaction costs, liquidity issues, licensing requirements…). In this blog post, after reviewing(...) Volatility Forecasting: Simple and Exponentially Weighted Moving Average Models [Portfolio Optimizer]One of the simplest and most pragmatic approach to volatility forecasting is to model the volatility of an asset as a weighted moving average of its past squared returns1. Two weighting schemes widely used by practitioners23 are the constant weighting scheme and the exponentially decreasing(...) Range-Based Volatility Estimators: Overview and Examples of Usage [Portfolio Optimizer]Volatility estimation and forecasting plays a crucial role in many areas of finance. For example, standard risk-based portfolio allocation methods (minimum variance, equal risk contributions, hierarchical risk parity…) critically depend on the ability to build accurate volatility forecasts1.(...) Correlation Matrix Stress Testing: Random Perturbations of a Correlation Matrix [Portfolio Optimizer]In the previous posts of this series, I detailed a methodology to perform stress tests on a correlation matrix by linearly shrinking a baseline correlation matrix toward an equicorrelation matrix or, more generally, toward the lower and upper bounds of its coefficients. This methodology allows to(...) Managing Missing Asset Returns in Portfolio Analysis: Backfilling through Residuals Recycling [Portfolio Optimizer]In a multi-asset portfolio, it is usual that some assets have shorter return histories than others1. Problem is, the presence of assets whose return histories differ in length makes it nearly impossible to use standard portfolio analysis and optimization methods… Estimating the historical(...) Simulation from a Multivariate Normal Distribution with Exact Sample Mean Vector and Sample Covariance Matrix [Portfolio Optimizer]In the research report Random rotations and multivariate normal simulation1, Robert Wedderburn introduced an algorithm to simulate i.i.d. samples from a multivariate normal (Gaussian) distribution when the desired sample mean vector and sample covariance matrix are known in advance2. Wedderburn(...) The Bogle Model for Bonds: Predicting the Returns of Constant Maturity Government Bond ETFs [Portfolio Optimizer]In his original 1991 article Investing in the 1990s1, John Bogle described a simple model to help investors setting reasonable expectations for long-term U.S. government bond returns. This model relies on what Bogle describes as the single most important factor in forecasting future total returns(...) The Single Greatest Predictor of Future Stock Market Returns, Ten Years After [Portfolio Optimizer]In his 2013 post The Single Greatest Predictor of Future Stock Market Returns, Jesse Livermore1 from the blog Philosophical Economics introduced an indicator to forecast long-term U.S. stock market returns and empirically demonstrated that it outperformed all the commonly used stock market valuation(...) The Gerber Statistic: A Robust Co-Movement Measure for Correlation Matrix Estimation [Portfolio Optimizer]The Gerber statistic is a measure of co-movement similar in spirit to the Kendall’s Tau coefficient that has been introduced in Gerber et al.1 to estimate correlation matrices within the Markowitz’s mean-variance framework. In this post, after providing the necessary definitions, I will(...) Corrected Cornish-Fisher Expansion: Improving the Accuracy of Modified Value-at-Risk [Portfolio Optimizer]Modified Value-at-Risk (mVaR) is a parametric approach to computing Value-at-Risk introduced by Zangari1 that adjusts Gaussian Value-at-Risk for asymmetry and fat tails present in financial asset returns2 through a mathematical technique called Cornish–Fisher expansion. Since its publication, mVaR(...) The Mathematics of Bonds: Simulating the Returns of Constant Maturity Government Bond ETFs [Portfolio Optimizer]With more than $1.2 trillion under management in the U.S. as of mid-July 20221, investors are more and more using bond ETFs as building blocks in their asset allocation. One issue with such instruments, though, is that their price history dates back to at best 20021, which is problematic in some(...) The Turbulence Index: Regime-based Partitioning of Asset Returns [Portfolio Optimizer]The turbulence index, introduced in the previous blog post, is a measure of statistical unusualness of asset returns popularized by Kritzman and Li1. It provides a way to measure how much the behavior of a group of assets differs from its historical pattern. In this post, based on the paper Optimal(...) The Diversification Ratio: Measuring Portfolio Diversification [Portfolio Optimizer]Continuing the series of blog posts on diversification indicators, I describe in this post a correlation-based measure of portfolio diversification called the diversification ratio, initially introduced by Yves Choueffaty and Yves Coignard in their paper Toward maximum diversification1 and later(...) The Informativeness: Measuring the Homogeneity of a Universe of Assets [Portfolio Optimizer]In this post, I will describe a measure of the homogeneity of a universe of assets, called the informativeness, introduced by Brockmeier et al.1 in their paper Quantifying the Informativeness of Similarity Measurements. After quickly going through the associated mathematics, I will present two(...) Trading Strategy Monitoring: Modeling PnL as Geometric Brownian Motion [Portfolio Optimizer]Systematic trading strategies have the unfortunate habit of exhibiting worse performances in real-life than in backtests, partially due to backtest overfitting1. Monitoring their behavior once they are deployed in production is then very important to be able to detect as early as possible any(...) Correlation Matrices Denoising: Results from Random Matrix Theory [Portfolio Optimizer]The estimation of empirical correlation matrices in finance is known to be affected by noise, in the form of measurement error, due in part to the short length of the time series of asset returns typically used in their computation1. Worse, large empirical correlation matrices have been shown to be(...) The Probabilistic Sharpe Ratio: Hypothesis Testing and Min Track Record Length [Portfolio Optimizer]In the first post of this series about the Sharpe ratio considered as a statistical estimator, I introduced a probabilistic framework to answer the question What is the probability that an estimated Sharpe ratio is statistically significantly greater than a reference Sharpe ratio? In this second(...) The Probabilistic Sharpe Ratio [Portfolio Optimizer]The Sharpe ratio1 is one of the most commonly used measure of financial portfolio performance, but because it is deeply rooted in mean-variance theory, its usage with return distributions deviating from normality (e.g. hedge funds, cryptocurrencies) is frequently questioned2. One solution to this(...) Bootstrap Simulation with Portfolio Optimizer: Usage for Financial Planning [Portfolio Optimizer]In statistics, a bootstrap method, also called bootstrapping, is a compute-intensive procedure that allows to estimate the distribution of a statistic through repeated resampling from a single observed sample of data1. Bootstrapping has several applications in quantitative finance, for example to(...) Computation of Theory-Implied Correlation Matrices [Portfolio Optimizer]In this short post, I will provide an overview of the TIC algorithm1 introduced by Marcos Lopez de Prado in his paper Estimation of Theory-Implied Correlation Matrices2, which aims to compute a forward-looking asset correlation matrix blending both empirical and theoretical inputs. I will also(...) The Effective Number of Bets: Measuring Portfolio Diversification [Portfolio Optimizer]Many different measures of portfolio diversification have been developed in the financial literature, from asset weights-based diversification measures like the Herfindahl Index1 to risk-based diversification measures like the Diversification Ratio of Choueifaty and Coignard2 to other more complex(...) Correlation Matrix Stress Testing: Shrinkage Toward the Lower and Upper Bounds of a Correlation Matrix [Portfolio Optimizer]I previously described on this blog an intuitive way of performing stress tests on a correlation matrix, which consists in shrinking a baseline correlation matrix toward an equicorrelation matrix12. A limitation of this method, though, is that it alters all the correlation coefficients of the(...) Mean-Variance Optimization in Practice: Subset Resampling-based Efficient Portfolios [Portfolio Optimizer]In a previous post, I introduced near efficient portfolios, which are portfolios equivalent to mean-variance efficient portfolios in terms of risk-return but more diversified in terms of asset weights. Such near efficient portfolios might be used to moderate the tendency of efficient portfolios to(...) Selecting a Stock Market Data (Web) API: Not So Simple [Portfolio Optimizer]I am sometimes asked if I recommend any stock market data (web) API for a personal use, especially because I mention Alpha Vantage in a couple of previous posts1. I will describe in this post part of the thought process and of the due diligence which led me to select this financial market data(...) The Absorption Ratio: Measuring Financial Risk, Part 2 [Portfolio Optimizer]In the previous post, I reviewed the turbulence index, an indicator of financial market stress periods based on the Mahalanobis distance, introduced by Chow and al.1 and Kritzman and Li2. In this post, I will review the absorption ratio, a measure of financial market fragility based on principal(...) The Turbulence Index: Measuring Financial Risk [Portfolio Optimizer]One of the challenges in portfolio management is the timely detection of financial market stress periods, typically characterized by an increase in volatility and a breakdown in asset correlations1. Chow and al.2 propose to detect such periods through the usage of the caste distance, a measure(...) Mean-Variance Optimization: Well Diversified (Near) Efficient Portfolios [Portfolio Optimizer]One well-known stylized fact of the Markowitz’s mean-variance framework is that, irrespective of the quality of the estimates of asset returns and (co)variances, efficient portfolios are concentrated in a very few assets1. From a practitioner’s perspective, this has always been a problem12. In(...) Ulcer Performance Index Portfolio Optimization [Portfolio Optimizer]The Ulcer Performance Index1 (UPI) is a portfolio reward-risk measure introduced by G. Martin2 similar in spirit to the Sharpe Ratio, but using the Ulcer Index (UI) as a risk measure instead of the standard deviation. In this blog post, I will present the mathematics behind the Ulcer Performance(...) The matrix effective rank: measuring the dimensionality of a universe of assets [Portfolio Optimizer]Quantifying how diversified is a universe of assets is an open problem in quantitative finance, partly because there is no definite formula for diversification1. Let’s make the (reasonable) assumption that the way assets are moving together within a universe is important for its diversification.(...) Beyond Hierarchical Risk Parity: Hierarchical Clustering-Based Risk Parity [Portfolio Optimizer]In a previous post, I introduced the Hierarchical Risk Parity portfolio optimization algorithm1. In this post, I will present one of its variations, called Hierarchical Clustering-Based Risk Parity, first described in Papenbrock2 and then generalized in Raffinot34 and in Lohre et al.5, from which(...) Hierarchical Risk Parity: Introducing Graph Theory and Machine Learning in Portfolio Optimizer [Portfolio Optimizer]In this short post, I will introduce the Hierarchical Risk Parity portfolio optimization algorithm, initially described by Marcos Lopez de Prado1, and recently implemented in Portfolio Optimizer. I will not go into the details of this algorithm, though, but simply describe some of its general ideas(...) Correlation Matrix Stress Testing: Shrinkage Toward an Equicorrelation Matrix [Portfolio Optimizer]Financial research has consistently shown that correlations between assets tend to increase during crises and tend to decrease during recoveries1. The recent COVID-19 market crash was no exception, as illustrated on Alvarez Quant Trading blog post Correlations go to One for both the individual(...) Residualization of Risk Factors: Examples and Pitfalls [Portfolio Optimizer]The most common approach to measuring portfolio (risk) factor exposures is linear regression analysis, which describes the relationship between a dependent variable - portfolio returns - and explanatory variables - factors - as linear. One of the outputs of this analysis are the partial regression(...) Replicating the J.P. Morgan Efficiente Index [Portfolio Optimizer]The J.P. Morgan Efficiente 5 Index is a tactical asset allocation strategy designed by J.P. Morgan based on a broad universe of 13 ETFs. This post will illustrate how to replicate this strategy with Google Sheets. Notes: A fully functional spreadsheet corresponding to this post is available here.(...) When a correlation matrix is not a correlation matrix and what can be done about it [Portfolio Optimizer]Estimating how individual assets are moving together is an important part of many financial applications1 and the most commonly used measure for this is the Pearson correlation. Unfortunately, for a variety of reasons, what sometimes appears to be a correlation matrix is actually not a valid(...)