Reproducibility has emerged as a major issue in numerous fields of scientific research, ranging from psychology, sociology, economics and finance to biomedicine, scientific computing and physics. Many of these difficulties arise from experimenter bias (also known as “selection bias”):

While out hiking, I found this rock. Evidently it was created by aliens, as can be shown by a probability argument. The following table gives measurements made on the rock. The first two rows give the overall length and width of the rock. Each of the next six rows, after the first two, gives

Fifty years ago, Princeton economics professor Burton Malkiel published A Random Walk Down Wall Street. He boldly asserted that a blindfolded chimpanzee throwing darts could pick a stock portfolio that would do as well as one created by many expert practitioners in the field. At the time, Malkiel

Suppose, in a national TV newscast, instead of citing data, analysis and predictions from major government agencies, the weatherperson displayed a chart of recent temperatures, noting “trends,” “waves” and “breakout patterns.” Most of us would not have confidence in such a dubious and

A new paper, Causal factor investing: Can factor investing become scientific?, has been written by our esteemed colleague Marcos Lopez de Prado of Cornell University, Abu Dhabi Investment Authority and True Positive Technologies. In his 75-page preprint, Lopez de Prado argues that almost all journal

“Hedge funds” were pioneered some 70 years ago by Australian financier Alfred Winslow Jones. His idea was to combine a “long” position (i.e., one that profits if the securities go up in price), typically a set of growth stocks, with a “short” position (i.e., one that profits if the

In a previous Mathematical Investor blog, we presented data on actively managed fund versus passive fund performance over various time horizons, based on the February 2019 Morningstar Active-Passive Barometer report. These data showed, for instance, that only 12.6% of actively managed U.S. large

In several articles on this site (see, for instance, A and B), we have commented on the dangers of backtest overfitting in finance. By backtest overfitting, we mean the usage of historical market data to develop an investment model, strategy or fund, where many variations are tried on the same fixed

The present author, together with Marcos López de Prado, has just published the article How backtest overfitting in finance leads to false discoveries in Significance, a journal of the British Statistical Society. The published article is now available at the Significance (Wiley) website. This

Today, arguably more than ever before, the world is governed by the science of probability and statistics. “Big data” is now the norm in scientific research, with terabytes of data streaming into research centers from satellites and experimental facilities, analyzed by supercomputers. “Data

In a previous MathInvestor article, we mentioned how absurd it would be if someone offered predictions of stock or bond prices or cryptocurrency rates based on astrological signs. Consider for a moment that financial market prices are based on a confluence of many thousands of factors worldwide,

Recent public reports have underscored a crisis of replicability in numerous fields of science: In 2012, Amgen researchers reported that they were able to replicate fewer than 10 of 53 cancer studies. In March 2014, physicists announced with fanfare that they had detected evidence of gravitational

Suppose, during a nightly TV weather broadcast, that a reporter presented forecasts by persons, with no credentials in mathematical meteorology, who based their analysis on eyeballing a few charts and graphs. If anyone took such amateur forecasts seriously, when a severe storm was approaching,

Marcos López de Prado, whom we have featured in previous Math Scholar articles (see Article A, Article B and Article C), has been invited to present a keynote presentation at the ACM Conference on Artificial Intelligence in Finance, to be conducted virtually October 14-16, 2020. López de Prado is

It is no secret that artificial intelligence (AI) systems have made enormous strides in recent years, partly due to the adoption of Bayesian (probability-based) machine learning techniques rather than the rule-based techniques used until about 20 years ago. AI systems have advanced in lockstep with

Famed quantitative financial mathematician Marcos Lopez de Prado, who was recently featured as Master of the Robots by Bloomberg, testified today (6 December 2019) before the U.S. Congress, together with four other panelists. The topic for the panel, organized by the U.S. House Committee on

Gregory Zuckerman, author of The Greatest Trade Ever, has published a new book highlighting the life and work of Jim Simons, who, at the age of 40, walked away from a very successful career as a research mathematician and cryptologist to try his hand at the financial markets, and ultimately

Marcos Lopez de Prado, who was named Quant of the Year for 2019 by the Journal of Portfolio Management, is widely regarded as one of the leading quantitative mathematicians in today’s financial world. He currently ranks #1 among authors in the economics field on the SSRN research network, as

As we explained in an earlier Mathematical Investor blog, “target-date funds” are currently the rage in the finance world. The term refers to a mutual fund that targets a given retirement date, and then steadily shifts the allocation of assets from, say, a 80%/20% mix of stocks and bonds at the

Marcos Lopez de Prado, who was named “Quant of the Year” for 2019 by the Journal of Portfolio Management, and who has recently formed his own investment firm True Positive Technologies, was recently interviewed by KNect365, an organization that sponsors numerous conferences and other exchanges

In an interview published at the Enterprising Investor blog, Frank Fabozzi, a well-known researcher and author in the mathematical finance field, has sharply criticized the current state of academic economics and finance. Here are some highlights: The “rational models” constructed in economics

Suppose, in the weather forecast part of a local newscast, the person handling the weather displays a chart of recent temperatures in the local area, pointed out “trends” and “waves,” then mentions a “breakout pattern” from a recent temperature range. Most of us would not have much

Recent public reports have underscored a crisis of reproducibility in numerous fields of science. Here are just a few of recent cases that have attracted widespread publicity: In 2012, Amgen researchers reported that they were able to reproduce fewer than 10 of 53 cancer studies. In 2013, in the

Marcos Lopez de Prado, recently named 2019 Quant of the Year by the Journal of Portfolio Management, has released a presentation entitled The seven reasons most econometric investments fail. Lopez de Prado’s overall point is that many widely used econometric approaches in finance either rely on

Marcos Lopez de Prado, a member of Mathematicians Against Fraudulent Financial and Investment Advice (MAFFIA), has been named “2019 Quant of the Year” by Journal of Portfolio Management. Here are some excerpts from their press release: The Journal of Portfolio Management (JPM) has named Marcos

Jack Bogle, founder of Vanguard Funds and a life-long apostle of index investing, died on 16 January 2019. Vanguard CEO Tim Buckley summarized his career in these terms: “Jack Bogle made an impact on not only the entire investment industry, but more importantly, on the lives of countless

Spurious results are the norm Having done a healthy share of paper replications over the past decade, and having been consistently disappointed when the models or techniques broke down on data shortly after (or even before) the authors’ sample periods, I would say that data misuse is a gigantic

In recent years, machine learning techniques and big-data facilities have become quite popular in the finance and investment world. In the wake of this success, numerous machine learning researchers have decided to found their own asset management companies, hoping to capitalize on this trend. This

As we have noted in previous Mathematical Investor blogs (see this blog for instance), surprisingly few mutual funds beat their respective benchmark (typically some market index). Even fewer consistently beat their benchmarks year after year. A new report from S&P Dow Jones sheds light on this

In a provocative new book, Lost in Math: How Beauty Leads Physics Astray, quantum physicist Sabine Hossenfelder argues that the scientific world in general, and the field of physics in particular, has repeatedly clung to notions that have been rejected by experimental evidence, or has pursued

A new video has been produced by the Mathematicians Against Fraudulent Financial and Investment Advice (MAFFIA) group. It explains, in simple terms, how many of the financial strategies and funds available today are based on a statistically dubious foundation, typically rooted in selection bias

Bloomberg columnist Mohamed El-Erian recently lamented that the discipline of economics “is divorced from real-world relevance and has lost credibility.” Among the problems he mentions currently afflicting the field are the following: The proliferation of simplifying assumptions that lead to an

In this post we look at the one plot that proves that technical analysis (TA) is useless. TECHNICAL ANALYSIS AND HOROSCOPES As volatility has made a come back in recent months, investors have sought advice to asset managers. In many instances, such advice consists in technical indicators. Charles

Two of the most talked-about topics in modern finance are machine learning and quantitative finance. Both of these are addressed in a new book, written by noted financial scholar Marcos Lopez de Prado, entitled Advances in Financial Machine Learning. In this book, Lopez de Prado strikes a well-aimed

Index investing has grown significantly over the past 30 years. Back in 1990, few were even aware of the option for indexing, and options were limited mostly to a handful of conventional mutual funds tracking the U.S. S&P 500 index. In 1993, Boston’s State Street Global Advisors launched the

Many individual investors employ mutual funds as an alternative to direct ownership of stocks or bonds. Indeed, mutual funds have some advantages: Diversity: Even a single fund can encapsulate a large sector of the market. Peace of mind: One is less likely to stress out about sudden bad news

The “January effect,” in common with the “Halloween indicator” and “sell in May and go away”, is a catchy, get-rich-quick investment idea adored by financial commentators because it is so easy to explain to unsophisticated readers. It rests on the claim that the U.S. stock market

“Big data” is already a frequently-heard buzzword, both in the business analytics arena, but also in the field of high-performance scientific computing. Basically, “big data” encompasses the collection, processing, indexing and utilization of large-scale datasets. Some concrete examples

One of us (Marcos Lopez de Prado) has been interviewed on the topic of educational training in the finance field by Institutional Investor. A brief synopsis of this interview is below. The full article is HERE. How Universities Are Failing Finance Students With investment shops fighting over

As the old joke says, “math is what mathematicians do.” Somehow this simple tautology is lost in the dishonest world of finance Quantitative investing: A crisis waiting to happen In a recent WSJ article, Jason Zweig brilliantly summarizes the unbearable hype and hubris exhibited by some

In the past few years “smart beta” (also known as “alternative beta” or “strategic beta”) investments have grown rapidly in popularity. As of the current date (January 2017), assets in these investment categories have grown to over USD$500 billion, and are expected to reach USD$1

One of us (Marcos Lopez de Prado) has published the article Mathematics and economics: A reality check in the Journal of Portfolio Management. The article is open-access — there is no fee for viewing or downloading. Lopez de Prado argues that while economics is arguably one the most mathematical

Over USD$2 trillion is held in exchange-traded equity funds, just in the U.S., with hundreds of new funds added each year. Strategies vary from simple index-tracking funds to funds that follow sophisticated strategies (e.g., “smart beta”) designed to yield impressive results, based on backtests.

Mark Hulbert has compiled an interesting list of recent market panics: August 2015: Concerns about the Chinese economy and stock market led to panic selling, with the Shanghai index plunging 8.5% in one day. Soon after in the U.S., on August 24, 2015, the DJIA plunged over 1,000 points in just a few

According to the just-published 2016 Rich List of the World's Top-Earning Hedge Fund Managers by Institutional Investor's Alpha magazine, eight of the top ten earners fall into the "quant" category, and half of the 25 richest of the year are quants. The firms listed include the

The January barometer is the claim, often mentioned in financial circles, that the performance of the stock market in January is a reliable portend of its performance for the full year — as January goes, so goes the year. The term was first coined by Yale Hirsch in 1972. Many market analysts take

When a prophet speaketh, … if the thing follow not, nor come to pass, … the prophet hath spoken it presumptuously: thou shalt not be afraid of him.” [Deuteronomy 18:22]. In a December 2014 Math Investor blog, we assessed how 2014 market prophets had fared (answer: not very well). Thus with the

Reproducibility in scientific research In the past year or two, the reproducibility of research results in finance and economics has come under serious question. If it is any comfort, similar difficulties have emerged in numerous other scientific fields. In 2011, a team of Bayer researchers

The Social Science Research Network’s Econometrics: Mathematical Methods and Programming eJournal distributes working and accepted paper abstracts in the area of mathematical methods applied to econometrics. The journal maintains a list of the All Time Top Ten Papers of the journal, based on total

On April 21, 2015, the U.S. Department of Justice announced that it would press criminal charges against Mr. Navinder Singh Sarao, a 36-year-old small-time British day-trader. He is being blamed for nothing less than causing the “Flash Crash” of May 6, 2010, the second largest point swing