In this blog post, I will demonstrate how to work with stock price data using the DuckDB database management system in R. DuckDB is a fast and lightweight analytical database engine that is designed to work with various programming languages, including R. You can use Duck DB from the command line or

Towards the end of last year, we ran a couple of free Zoom webinars on: The Basics of Edge Extraction – the “trader smarts” of getting an edge Data Analysis for Traders – an interactive research session. Here are the recordings: Basics of Edge Extraction Data analysis for Traders The colab

dplyr 1.1.0 was a significant release that makes several common data operations more syntactically intuitive. The most significant changes relate to joins and grouping/aggregating operations. In this post we’ll look at the changes to joins. First, install and load the latest version of dplyr:

Over the last few years, a number of new market data providers have come online. They tend to have modern websites, broad coverage, and well-documented RESTful APIs. Their services are often priced very competitively – especially for personal use – and usually have generous free tiers. One such

In Australia, if you’re serious about getting the job done effectively and efficiently, you might say: “I’m not here to f*** spiders.” Many traders act like they are, indeed, here to f*** spiders. If you’re making soup, you first need a good stock. Stock isn’t exciting. Everyone has

Imagine you’re a relatively small, independent trader trying to turn trading from a hobby into a serious business. If that’s you, then there are a few concepts that will help you pick the right trades to get after. This is important because picking the right trades is most of the game. First,

rsims is a new package for fast, quasi event-driven backtesting in R. You can find the source on GitHub, docs here, and an introductory blog post here. Our use case for rsims was accurate but fast simulation of trading strategies. I’ve had a few questions about how I made the backtester as fast as

rsims is a new package for fast, realistic (quasi event-driven) backtesting of trading strategies in R. Really?? Does the world really need another backtesting platform…?? It’s hard to argue with that sentiment. Zipline, QuantConnect, Quantstrat, Backtrader, Zorro… there are certainly plenty

If you have some factor that you think predicts future stock returns (or similar) and you are making charts like below, then here are some tips… We’ll go through an example of trying to “time” SPX with the level of VIX. You get daily SPX index prices and daily VIX close data You align them

Many beginner traders don’t realize how variable the p&l of a high-performing trading strategy really is. Here’s an example… I simulated ten different 5 year GBM processes with expected annual returns of 20% and annualized volatility of 10%. (If you speak Sharpe Ratios, I’m simulating a

Broadly, there are three types of systematic trading strategy that can “work.” In order of increasing turnover they are: Risk premia harvesting Economically-sensible, statistically-quantifiable slow-converging inefficiencies Trading fast-converging supply/demand imbalances This post provides an

Earlier versions of Zorro used to ship with a script for converting market data in Zorro binary format to CSV. That script seems to have disappeared with the recent versions of Zorro, so I thought I’d post it here. When you run this script by selecting it and pressing [Test] on the Zorro

Several years ago, I wrote about some experimentation I’d done with data mining for predictive features from financial data. The article has had several tens of thousands of views and nearly 100 comments. I think the popularity of the article lay in its demonstration of various tools and modeling

I have been sharing examples of simple real-time trading research on my Twitter account. I do this kind of thing a lot in the training program of our trading group – and I’m sharing in the hope that it might also help a wider audience. Here’s a piece of analysis I did recently on a really

I’m a big fan of Ernie Chan’s quant trading books: Quantitative Trading, Algorithmic Trading, and Machine Trading. There are some great insights in there, but the thing I like most is the simple but thorough treatment of various edges and the quant tools you might use to research and trade them.

The S&P index committee recently announced that Tesla, already one of the biggest stocks listed in the country, would be included in the S&P 500. Here’s the press release: Due to TSLA’s size, it was widely expected to have entered the S&P 500 index much earlier – but S&P has

This post is a BONUS LESSON taken directly from Zero to Robot Master Bootcamp. In this Bootcamp, we teach traders how to research, build and trade a portfolio of 3 strategies including an Intraday FX Strategy, a Risk Premia Strategy and a Volatility Basis Strategy. If you’re interested in adding

I recently listened to a podcast about one of the earliest human civilizations – the ancient Sumerians. Apparently, our system of minutes, hours, and days has been with us since the time of these ancient people, who developed it based on a simple base-12 counting system: There are three joints in

I was very sad to learn that Quantopian is shutting down its community services. Quantopian’s efforts to bring quant finance outside of institutions was a genuine game-changer. The educational content was solid, the tech was excellent, and the QuantCon conferences were professional, well-run, and

You’ve probably noticed that there’s a US election on the horizon. This is an event of known uncertainty: a “known unknown” in the now immortal language of Donald Rumsfeld. In trading, we sometimes observe marginal pricing inefficiencies around these “known unknowns”. For example, ahead

Trading books set a low bar for the reviewer. 99% are full of facile feel-good advice (don’t fight the trend, always use a protective stop). The 1% that are useful tend to either be dry technical treatments (quants who don’t trade), or sporadically helpful insights from traders who make money

This post presents an analysis of the SPY returns process using the QuantConnect research platform. QuantConnect is a strategy development platform that lets you research ideas, import data, create algorithms, and trade in the cloud, all in one place. For this research, I’ve used their online

On Zero to Robot Master Bootcamp, we teach how to build a portfolio of three automated systematic trading strategies. One of them is a long term Risk Premia Harvesting strategy which trades asset class ETFs. ETFs are useful instruments for analysing long term (tradeable) performance of various asset

Why aren’t calls more expensive than puts for an asset which is more likely to go up than down? We have an asset trading at $100 for which the distribution of future returns is a known fact. It has annual returns described by a normal distribution with mean 5% and standard deviation 10%. This is,

In the world of Big Data, there are lots of tools and technologies to choose from. Choosing the “right” one depends on the things that you are building and the problems you are trying to solve. Trading firms have skilled teams that monitor and deploy data pipelines for their organisation and the

Holding data in a tidy format works wonders for one’s productivity. Here we will explore the tidyr package, which is all about creating tidy data. In particular, let’s develop an understanding of the tidyr::pivot_longer and tidyr::pivot_wider functions for switching between different formats of

Anyone that’s been around the markets knows that the monthly release of the United States Department of Labor’s Non-Farm Payrolls (NFP) data can have a tremendous impact, especially in the short term. NFP is a snapshot of the state of the employment situation in the US, representing the total

When data is too big to fit into memory, one approach is to break it into smaller pieces, operate on each piece, and then join the results back together. Here’s how to do that to calculate rolling mean pairwise correlations of a large stock universe. Background We’ve been using the problem of

When you’re working with large universes of stock data you’ll come across a lot of challenges: Stocks pay dividends and other distributions that have to be accounted for. Stocks are subject to splits and other corporate actions which also have to be accounted for. New stocks are listed all the

One way we can quantify a stock’s movement relative to the market index is by calculating its “beta” to the market. To calculate the beta of MSFT to SPY (for example) we: calculate daily MSFT returns and daily SPY returns align the returns with one another regress MSFT returns against SPY

In today’s article, we are going to take a look at rolling and expanding windows. By the end of the post, you will be able to answer these questions: What is a rolling window? What is an expanding window? Why are they useful? What is a Rolling or Expanding window? Here is a normal window. We use

Recently, we wrote about calculating mean rolling pairwise correlations between the constituent stocks of an ETF. The tidyverse tools dplyr and slider solve this somewhat painful data wrangling operation about as elegantly and intuitively as possible. Why did you want to do that? We’re building a

Working with modern APIs you will often have to wrangle with data in JSON format. This article presents some tools and recipes for working with JSON data with R in the tidyverse. We’ll use purrr::map functions to extract and transform our JSON data. And we’ll provide intuitive examples of the

In this post, we look at tools and functions from the field of digital signal processing. Can these tools be useful to us as quantitative traders? What’s a Digital Signal? A digital signal is a representation of physical phenomena created by sampling that phenomena at discrete time intervals. If

How might we calculate rolling correlations between constituents of an ETF, given a dataframe of prices? For problems like this, the tidyverse really shines. There are a number of ways to solve this problem … read on for our solution, and let us know if you’d approach it differently! First, we

Modern data science is fundamentally multi-lingual. At a minimum, most data scientists are comfortable working in R, Python and SQL; many add Java and/or Scala to their toolkit, and it’s not uncommon to also know one’s way around JavaScript. Personally, I prefer to use R for data analysis. But,

In this post, we’re going to show how a quant trader can manipulate stock price data using the dplyr R package. Getting set up and loading data Load the dplyr package via the tidyverse package. if (!require('tidyverse')) install.packages('tidyverse') library(tidyverse) First,

Every aspiring millionaire who comes to the markets armed with some programming ability has implemented a systematic Get Rich Quick (GRQ) trading strategy. Of course, they don’t work. Deep down even the greenest of newbies knows this. Yet, still, we are compelled to give them a try, just once,

In this post, we are going to construct snapshots of historic S&P 500 index constituents, from freely available data on the internet. Why? Well, one of the biggest challenges in looking for opportunities amongst a broad universe of stocks is choosing what stock “universe” to look at. One

If you want to make money trading, you’re going to need a way to identify when an asset is likely to be cheap and when it is likely to be expensive. You want to be a net buyer of the cheap stuff and a net seller of the expensive stuff. Thanks, Capitain Obvious. You’re welcome. How does this

There are 2 good reasons to buy put options: because you think they are cheap because you want downside protection. In the latter case, you are looking to use the skewed payoff profile of the put option to protect a portfolio against large downside moves without capping your upside too much. The

This post summarises the key lessons of the academic literature that has been published on pairs trading. The key themes are highlighted at the end of the page. Pair Trading Literature Review Gatev, Goetzmann, Rouwenhorst – “Pairs Trading: Performance of a Relative Value Arbitrage Strategy”

One of the things I’ve noticed from staring at the screen all day for the last few months is that most of the large negative returns in US stock indexes have come overnight. What do you mean by “overnight”? The core stock trading session for US stocks is between 9:30 am and 4 pm Eastern Time.

In today’s post we are going to be extracting CoT (Commitment of Traders) reports from the CFTC website using a pipeline built on Apache Airflow. What is CoT data? The CoT report is a weekly publication which reports the open positions of market participants in the U.S futures market. It’s

In the eye of the recent storm, with VIX up over 50, many traders were looking to “short the VIX” using products like TVIX. “Surely it’s going to coming back down?” Well yeah, it will, eventually, but that doesn’t mean that you can profitably short VIX products. First, some basics…

For simulating stock prices, Geometric Brownian Motion (GBM) is the de-facto go-to model. It has some nice properties which are generally consistent with stock prices, such as being log-normally distributed (and hence bounded to the downside by zero), and that expected returns don’t depend on the

Optimisation tools have a knack for seducing systematic traders. And what’s not to love? Find me the unique set of parameters that delivered the greatest return in my ten-year backtest. And do it in under five seconds. That’s certainly attractive. But do you want to hear something controversial?

One of the keys to running a successful systematic trading business is a relentless focus on high return-on-investment activities. High ROI activities include: Implementing new trading strategies within a proven framework. An example might be to implement a portfolio of pairs trades in the equity

To say we’re living through extraordinary times would be an understatement. We saw the best part of 40% wiped off stock indexes in a matter of weeks, unprecedented co-ordinated central bank intervention on a global scale, and an unfolding health crisis that for many has already turned into a

The vector autoregression (VAR) framework is common in econometrics for modelling correlated variables with bi-directional relationships and feedback loops. If you google “vector autoregression” you’ll find all sorts of academic papers related to modelling the effects of monetary and fiscal