Quant Mashup - Eran Raviv

Most popular posts – 2018 [Eran Raviv]

2019 is well underway. 2018 was personally difficult, so I am happy it’s behind us. Without further ado, here is what my analytics report shows to be the three most popular posts for 2018: – Create own Recession Indicator using Mixture Models (3:53 minutes average time on page) – Portfolio

*- 1 week ago, 13 Jan 2019, 09:24pm -*

Reproducible Finance with R - Book Review [Eran Raviv]

Reproducible Finance with R is a clever book, with modern treatment of classical concepts. Here below is what I liked- and disliked about the book. Back when I was practicing Judo, there was a guy in my group who mastered that one exercise (called Uchi Mata). He could go fighting 20 consecutive

*- 2 weeks ago, 5 Jan 2019, 12:13pm -*

Create own Recession Indicator using Mixture Models [Eran Raviv]

Broadly speaking, we can classify financial markets conditions into two categories: Bull and Bear. The first is a “todo bien” market, tranquil and generally upward sloping. The second describes a market with a downturn trend, usually more volatile. It is thought that those bull\bear terms

*- 1 month ago, 27 Nov 2018, 09:31pm -*

Price Movement Prediction [Eran Raviv]

Just finished reading the paper Stock Market’s Price Movement Prediction With LSTM Neural Networks. The abstract attractively reads: “The results that were obtained are promising, getting up to an average of 55.9% of accuracy when predicting if the price of a particular stock is going to go up

*- 3 months ago, 15 Oct 2018, 02:09am -*

Test of Equality Between Two Densities [Eran Raviv]

Are returns this year actually different than what can be expected from a typical year? Is the variance actually different than what can be expected from a typical year? Those are fairly light, easy to answer questions. We can use tests for equality of means or equality of variances. But how about

*- 3 months ago, 9 Oct 2018, 04:56pm -*

Visualizing Time Series Data [Eran Raviv]

This post has two goals. I hope to make you think about your graphics, and think about the future of data-visualization. An example is given using some simulated time series data. A very quick read. In visualization, like in programming, presenting or any other skill, there is much to learn. Also

*- 4 months ago, 17 Sep 2018, 10:52am -*

Market intraday momentum [Eran Raviv]

I recently spotted the following intriguing paper: Market intraday momentum. From the abstract of that paper: Based on high frequency S&P 500 exchange-traded fund (ETF) data from 1993–2013, we show an intraday momentum pattern: the first half-hour return on the market as measured from the

*- 5 months ago, 29 Jul 2018, 10:15am -*

R in Finance [Eran Raviv]

The yearly R in Finance conference is one of my favorites: 1. Titans of the R community are there every year. This year the founder of Rstudio (but much more really), JJ Allaire was a keynote speaker. He gave a talk about Machine Learning with TensorFlow and R. 2. Single track. I like everything,

*- 6 months ago, 27 Jun 2018, 10:47pm -*

Curse of dimensionality part 3: Higher-Order Comoments [Eran Raviv]

Higher moments such as Skewness and Kurtosis are not as explored as they should be. These moments are crucial for managing portfolio risk. At least as important as volatility, if not more. Skewness relates to asymmetry risk and Kurtosis relates to tail risk. Despite their great importance, those

*- 7 months ago, 20 Jun 2018, 12:59pm -*

Portfolio Construction with R [Eran Raviv]

Constructing a portfolio means allocating your money between few chosen assets. The simplest thing you can do is evenly split your money between few chosen assets. Simple as it is, good research shows it is just fine, and even better than other more sophisticated methods (for example Optimal Versus

*- 9 months ago, 10 Apr 2018, 10:26pm -*

Machine learning is simply statistics [Eran Raviv]

Note: I usually write more technical posts, this is an opinion piece. And you know what they say: opinions are like feet, everybody’s got a couple. Machine learning is simply statistics A lot of buzz words nowadays. Data Science, business intelligence, machine learning, deep learning, statistical

*- 10 months ago, 7 Mar 2018, 11:09am -*

Bitcoin exponential growth [Eran Raviv]

Is bitcoin a bubble? I don’t know. What defines a bubble? The price should drastically overestimate the underlying fundamentals. I simply don’t know much about blockchain to have an opinion there. A related characteristic is a run-away price. Going up fast just because it is going up fast. How

*- 11 months ago, 29 Jan 2018, 11:09am -*

Most popular posts – 2017 [Eran Raviv]

Writing this, I can’t believe how quickly the year 2017 has gone by. Also weird, we are already three weeks into 2018, unreal. Time flies when you’re having fun I guess. The analytics report shows that the three most popular posts for 2017 are: – Understanding False Discovery Rate (4 minutes

*- 1 year ago, 19 Jan 2018, 10:07am -*

R vs MATLAB - round 4 [Eran Raviv]

This is another comparison between R and MATLAB (Python also in the mix this time). In previous rounds we discussed the differences in 3d visualization, differences in syntax and input-output differences. Today is about computational speed. Spoiler alert: MATLAB wins by a knockout. A genuinely fair

*- 1 year ago, 6 Sep 2017, 11:35am -*

Visualizing Tail Risk [Eran Raviv]

Tail risk conventionally refers to the risk of a large and sharp draw down of the portfolio. How large is subjective and depends on how you define what is a tail. A lot of research is directed towards having a good estimate of the tail risk. Some fairly new research also now indicates that investors

*- 1 year ago, 8 Aug 2017, 05:50am -*

Lasso, Lasso, Lasso (and friends) [Eran Raviv]

LASSO stands for Least Absolute Shrinkage and Selection Operator. It was first introduced 21 years ago by Robert Tibshirani (Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B). In 2004 the four statistical masters: Efron, Hastie, Johnstone and

*- 1 year ago, 5 Jul 2017, 11:47pm -*

Density Estimation Using Regression [Eran Raviv]

Density estimation using regression? Yes we can! I like regression. It is one of those simple yet powerful statistical methods. You always know exactly what you are doing. This post is about density estimation, and how to get an estimate of the density using (Poisson) regression. The “go-to”

*- 1 year ago, 26 Jun 2017, 03:17am -*

Computer Age Statistical Inference [Eran Raviv]

If you consider yourself Econometrician\Statistician or one of those numerous buzz word synonyms that are floating around these days, Computer Age Statistical Inference: Algorithms, Evidence and Data Science by Bradley Efron and Trevor Hastie is a book you can’t miss, and now nor should you. You

*- 1 year ago, 5 Jun 2017, 09:09am -*

Random Books [Eran Raviv]

It seems like a very long while since my bachelor. Checking my bookshelf the other day I was thinking to flag some of those books which helped or inspired me along the way. Here they are in no particular order. Risk: Elements of Financial Risk Management Clear and to the point, 5 stars. Value at

*- 1 year ago, 4 Jun 2017, 02:31am -*

Shrinkage in statistics [Eran Raviv]

Shrinkage in statistics has increased in popularity over the decades. Now statistical shrinkage is commonplace, explicitly or implicitly. But when is it that we need to make use of shrinkage? At least partly it depends on signal-to-noise ratio. Introduction The term shrinkage, I think, is the most

*- 1 year ago, 11 May 2017, 02:50am -*

Machine Trading from @ChanEP - Book Review [Eran Raviv]

In trading and in trading-related research one could be quickly overwhelmed with the sea of ink devoted to trading strategies and the like. It is essential that you “pick your battles” so to speak. I recently finished reading Machine Trading, by Ernest Chan. Here is what I think about the book.

*- 1 year ago, 3 May 2017, 04:45am -*

Understanding False Discovery Rate [Eran Raviv]

False Discovery Rate is an unintuitive name for a very intuitive statistical concept. The math involved is as elegant as possible. Still, it is not an easy concept to actually understand. Hence i thought it would be a good idea to write this short tutorial. We reviewed this important topic in the

*- 1 year ago, 4 Apr 2017, 07:14am -*

Understanding K-Means Clustering [Eran Raviv]

Google “K-means clustering”, and you usually you find ugly explanations and math-heavy sensational formulas*. It is my opinion that you can only understand those explanations if you don’t need them; meaning you are already familiar with the topic. Therefore, this is a more gentle introduction

*- 1 year ago, 12 Mar 2017, 07:00pm -*

Outliers and Loss Functions [Eran Raviv]

A few words about outliers In statistics, outliers are as thorny topic as it gets. Is it legitimate to treat the observations seen during global financial crisis as outliers? or are those simply a feature of the system, and as such are integral part of a very fat tail distribution? I recently read a

*- 1 year ago, 19 Feb 2017, 10:11pm -*

Density Confidence Interval [Eran Raviv]

Density estimation belongs with the literature of non-parametric statistics. Using simple bootstrapping techniques we can obtain confidence intervals (CI) for the whole density curve. Here is a quick and easy way to obtain CI’s for different risk measures (VaR, expected shortfall) and using what

*- 1 year ago, 26 Jan 2017, 11:18am -*

Most popular posts - 2016 [Eran Raviv]

Another year. Looking at my google analytics reports I can’t help but wonder how is it that I am so bad in predicting which posts would catch audience attention. Anyhow, top three for 2016 are: On the 60/40 portfolio mix The case for Regime-Switching GARCH Most popular machine learning R packages

*- 2 years ago, 28 Dec 2016, 08:51am -*

Optimism of the Training Error Rate [Eran Raviv]

We all use models. We all continuously working to improve and validate our models. Constant effort is made trying to estimate: how good our model actually is? A general term for this estimate is error rate. Low error rate is better than high error rate, it means our model is more accurate. By far

*- 2 years ago, 5 Dec 2016, 09:38am -*

Central Moments [Eran Raviv]

Sometimes I read academic literature, and often times those papers contain some proofs. I usually gloss over some innocent-looking assumptions on moments’ existence, invariably popping before derivations of theorems or lemmas. Here is one among countless examples, actually taken from Making and

*- 2 years ago, 14 Nov 2016, 10:07am -*

Extreme Value Theory [Eran Raviv]

Extreme Value Theory (EVT) is busy with understanding the behavior of the distribution, in the extremes. The extreme determine the average, not the reverse. If you understand the extreme, the average follows. But, getting the extreme right is extremely difficult. By construction, you have very few

*- 2 years ago, 20 Sep 2016, 04:09am -*

Multivariate Volatility Forecast Evaluation [Eran Raviv]

The evaluation of volatility models is gracefully complicated by the fact that, unlike other time series, even the realization is not observable. Two researchers would never disagree about what was yesterday’s stock price, but they can easily disagree about what was yesterday’s stock volatility.

*- 2 years ago, 1 Sep 2016, 03:31am -*

Why bad trading strategies may perform well [Eran Raviv]

You probably know that even a trading strategy which is actually no different from a random walk (RW henceforth) can perform very well. Perhaps you chalk it up to short-run volatility. But in fact there is a deeper reason for this to happen, in force. If you insist on using and continuously testing

*- 2 years ago, 12 Aug 2016, 02:59am -*

Human significance, economic significance and statistical significance [Eran Raviv]

We are now collecting a lot of data. This is a good thing in general. But data collection and data storage capabilities have evolved fast. Much faster than statistical methods to go along with those voluminous numbers. We are still using good ole fashioned Fisherian statistics. Back then, when you

*- 2 years ago, 3 Jul 2016, 12:12pm -*

Forecast combinations in R [Eran Raviv]

Few weeks back I gave a talk in the R/Finance 2016 conference, about forecast combinations in R. Here are the slides.

*- 2 years ago, 13 Jun 2016, 10:23am -*

Most popular machine learning R packages [Eran Raviv]

The good thing about using open-source software is the community around it. There are very many R packages online, and recently CRAN package download logs were released. This means we can have a look at the number of downloads for each package, so to get a good feel for their relative popularity. I

*- 2 years ago, 16 May 2016, 11:21am -*

Forecast averaging example [Eran Raviv]

Especially in economics/econometrics, modellers do not believe their models reflect reality as it is. No, the yield curve does NOT follow a three factor Nelson-Siegel model, the relation between a stock and its underlying factors is NOT linear, and volatility does NOT follow a Garch(1,1) process,

*- 2 years ago, 3 May 2016, 02:34am -*

Measurement error bias [Eran Raviv]

What is measurement error bias? Errors-in-variables, or measurement error situation happens when your right hand side variable(s); your x in a y_t = \alpha + \beta x_t + \varepsilon_t model is measured with error. If x represents the price of a liquid stock, then it is accurately measured because

*- 2 years ago, 25 Apr 2016, 04:42am -*

The case for Regime-Switching GARCH [Eran Raviv]

GARCH models are very responsive in the sense that they allow the fit of the model to adjust rather quickly with incoming observations. However, this adjustment depends on the parameters of the model, and those may not be constant. Parameters’ estimation of a GARCH process is not as quick as those

*- 2 years ago, 5 Apr 2016, 12:30am -*

On the 60/40 portfolio mix [Eran Raviv]

Not sure why is that, but traditionally we consider 60% stocks and 40% bonds to be a good portfolio mix. One which strikes decent balance between risk and return. I don’t want to blubber here about the notion of risk. However, I do note that I feel uncomfortable interchanging risk with volatility

*- 2 years ago, 24 Mar 2016, 03:44am -*

ASA statement on p-values [Eran Raviv]

There are many problems with p-values, and I too have chipped in at times. I recently sat in a presentation of an excellent paper, to be submitted to the highest ranked journal in the field. The authors did not conceal their ruthless search for those mesmerizing asterisks indicating significance. I

*- 2 years ago, 14 Mar 2016, 05:03am -*

Multivariate volatility forecasting, part 6 - sparse estimation [Eran Raviv]

First things first. What do we mean by sparse estimation? Sparse – thinly scattered or distributed; not thick or dense. In our context, the term ‘sparse’ is installed in the intersection between machine-learning and statistics. Broadly speaking, it refers to a situation where a solution to a

*- 2 years ago, 15 Feb 2016, 01:12pm -*

Linear regression assumes nothing about your data [Eran Raviv]

We often see statements like “linear regression makes the assumption that the data is normally distributed”, “Data has no or little multicollinearity”, or other such blunders (you know who you are..). Let’s set the whole thing straight. Linear regression assumes nothing about your data It

*- 2 years ago, 26 Jan 2016, 08:58am -*

Curse of dimensionality part 1: Value at Risk [Eran Raviv]

The term ‘curse of dimensionality’ is now standard in advanced statistical courses, and refers to the disproportional increase in data which is needed to allow only slightly more complex models. This is true in high-dimensional settings. Here is an illustration of the ‘Curse of

*- 3 years ago, 17 Jan 2016, 10:35pm -*

Most popular posts – 2015 [Eran Raviv]

The top three for the year are: Out-of-sample data snooping Code for my yield curve forecasting paper Review of a couple of books I personally enjoyed the most writing a few words on ML estimation, and about those great statistical discoveries. Since the last post did not involve any code or images

*- 3 years ago, 4 Jan 2016, 08:15am -*

Present-day great statistical discoveries [Eran Raviv]

Some time during the 18th century the biologist and geologist Louis Agassiz said: “Every great scientific truth goes through three stages. First, people say it conflicts with the Bible. Next they say it has been discovered before. Lastly they say they always believed it”. Nowadays I am not sure

*- 3 years ago, 21 Dec 2015, 08:48pm -*

Orthogonal GARCH [Eran Raviv]

In multivariate volatility forecasting (4), we saw how to create a covariance matrix which is driven by few principal components, rather than a complete set of tickers. The advantages of using such factor volatility models are plentiful. First, you don't model each ticker separately, you can

*- 3 years ago, 7 Dec 2015, 01:19am -*

'predictions', 'forecasts' or 'projections'? [Eran Raviv]

Perhaps it is the different jargon used in different disciplines, not sure. But for some reason, the terms ‘predictions’, ‘forecasts’ and ‘projections’ are frequently used interchangeably. There should be at least some distinction, here is what I entertain: The word ‘predictions’

*- 3 years ago, 3 Dec 2015, 12:41am -*

Correlation and correlation structure (3), estimate tail dependence using regression [Eran Raviv]

What is tail dependence really? Say the market had a red day and saw a drawdown which belongs with the 5% worst days (from now on simply call it a drawdown): weekly SPY returns One can ask what is now, given that the market is in the blue region, the probability of a a drawdown in a specific stock?

*- 3 years ago, 11 Nov 2015, 01:47am -*

Multivariate volatility forecasting (4), factor models [Eran Raviv]

To be instructive, I always use very few tickers to describe how a method works (and this tutorial is no different). Most of the time is spent on methods that we can easily scale up. Even if exemplified using only say 3 tickers, a more realistic 100 or 500 is not an obstacle. But, is it really

*- 3 years ago, 20 Oct 2015, 09:34pm -*

Multivariate volatility forecasting (3), Exponentially weighted model [Eran Raviv]

Broadly speaking, complex models can achieve great predictive accuracy. Nonetheless, a winner in a kaggle competition is required only to attach a code for the replication of the winning result. She is not required to teach anyone the built-in elements of his model which gives the specific edge over

*- 3 years ago, 13 Oct 2015, 03:39am -*

Correlation and correlation structure [Eran Raviv]

This post is about copulas and heavy tails. In a previous post we discussed the concept of correlation structure. The aim is to characterize the correlation across the distribution. Prior to the global financial crisis many investors were under the impression that they were diversified, and they

*- 3 years ago, 21 Sep 2015, 02:38am -*