Quant Mashup - Dekalog Blog

Simple Machine Learning Models on OrderBook/PositionBook Features [Dekalog Blog]

This post is about using OrderBook/PositionBook features as input to simple machine learning models after previous investigation into the relevance of such features. Due to the amount of training data available I decided to look only at a linear model and small neural networks (NN) with a single

*- 9 months ago, 8 Apr 2022, 12:56pm -*

Matrix profile: Using Weakly Labeled Time Series to Predict Outcomes [Dekalog Blog]

Back in May of this year I posted about how I had intended to use Matrix Profile (MP) to somehow cluster the "initial balance" of Market Profile charts with a view to getting a heads up on immediately following price action. Since then, my thinking has evolved due to my learning about the

*- 1 year ago, 5 Sep 2021, 10:00pm -*

Update on Recent Matrix Profile Work [Dekalog Blog]

Since my previous post, on Matrix Profile (MP), I have been doing a lot of online reading about MP and going back to various source papers and code that are available at the UCR Matrix Profile page. I have been doing this because, despite my initial enthusiasm, the R tsmp package didn't turn

*- 1 year ago, 27 May 2021, 11:28am -*

Market/Volume Profile and Matrix Profile [Dekalog Blog]

A quick preview of what I am currently working on: using Matrix Profile to search for time series motifs, using the R tsmp package. The exact motifs I'm looking for are the various "initial balance" set ups of Market Profile charts. To do so, I'm concentrating the investigation

*- 1 year ago, 28 Mar 2021, 10:24am -*

Heatmap Plot of Forex Temporal Clustering of Turning Points [Dekalog Blog]

Following up on my previous post, below is the chart of the temporal turning points that I have come up with. This particular example happens to be 10 minute candlesticks over the last two days of the GBP_USD forex pair. The details I have given about various turning points over the course of my

*- 1 year ago, 5 Feb 2021, 10:13am -*

Temporal Clustering Times on Forex Majors Pairs [Dekalog Blog]

In the following code box there are the results from the temporal clustering routine of my last few posts on the four forex majors pairs of EUR_USD, GBP_USD, USD_CHF and USD_JPY. This is based on 10 minute bars over the last year or so. Readers should read my last few previous posts for background.

*- 2 years ago, 31 Jan 2021, 08:04pm -*

Temporal Clustering on Real Prices, Part 2 [Dekalog Blog]

Below are some more out of sample plots for the Temporal Clustering solutions of the EUR_USD forex pair for the week just gone. The details of how these solutions are derived is explained in my previous post, Temporal Clustering on Real Prices. First is Tuesday's solution where the major (blue

*- 2 years ago, 2 Dec 2020, 09:44am -*

Temporal Clustering on Real Prices [Dekalog Blog]

Having now had time to run the code shown in my previous post, Temporal Clustering, part 3, in this post I want to show the results on real prices. Firstly, I have written two functions in Octave to identify market turning points and each function takes as input an n_bar argument which determines

*- 2 years ago, 24 Nov 2020, 09:37am -*

Temporal Clustering, Part 3 [Dekalog Blog]

Continuing on with the subject matter of my last post, in the code box below there is R code which is a straight forward refactoring of the Octave code contained in the second code box of my last post. This code is my implementation of the cross validation routine described in the paper Cluster

*- 2 years ago, 15 Nov 2020, 09:17pm -*

A Temporal Clustering Function, Part 2 [Dekalog Blog]

Further to my previous post, below is an extended version of the "blurred_maxshift_1d_linear" function. This updated version has two extra outputs: a vector of the cluster centre index ix values and a vector the same length as the input data with the cluster centres to which each datum has

*- 2 years ago, 10 Nov 2020, 10:16am -*

A Temporal Clustering Function [Dekalog Blog]

Recently a reader contacted me with a view to collaborating on some work regarding the Delta phenomenon but after a brief exchange of e-mails this seems to have petered out. However, for my part, the work I have done has opened a few new avenues of investigation and this post is about one of them.

*- 2 years ago, 20 Oct 2020, 10:55am -*

Candlestick Pattern Scanner Functions [Dekalog Blog]

Since my last currency strength candlestick chart post it seemed to make sense to be able to scan said charts for signals, so below is the code for two Octave functions which act as candlestick pattern scanners. The code is fully vectorised and self-contained, and on my machine they can scan more

*- 2 years ago, 16 Aug 2020, 12:48pm -*

Forex Intraday Seasonality [Dekalog Blog]

Over the last week or so I have been reading about/investigating this post's title matter. Some quotes from various papers' abstracts on the matter are: "We provide empirical evidence that the unique signature of the FX market seasonality is indeed due to the different time zones

*- 2 years ago, 15 Jul 2020, 08:55pm -*

More Work on RVFL Networks [Dekalog Blog]

Back in November last year I posted about Random Vector Functional Link (RVFL) networks here and here. Since then, along with my recent work on Oanda's API Octave functions and Market/Volume Profile visualisation, I have continued looking at RVFL networks and this post is an update on this

*- 2 years ago, 18 Jun 2020, 09:47pm -*

Downloading FX Pairs via Oanda API to Calculate Currency Strength Indicator [Dekalog Blog]

In the past I have posted a series of blog posts about a Currency Strength Indicator (here, here, here and here). This blog post gives an Octave function to use Oanda's API to download all the 10 minute OHLC data required to calculate the above strength indicators on the 10 minute time frame.

*- 2 years ago, 7 Jun 2020, 11:12pm -*

An Improved Volume Profile Chart with Levels [Dekalog Blog]

Without much ado, here is the code ## Copyright (C) 2020 dekalog ## ## This program is free software: you can redistribute it and/or modify it ## under the terms of the GNU General Public License as published by ## the Free Software Foundation, either version 3 of the License, or ## (at your option)

*- 2 years ago, 21 May 2020, 10:26am -*

A Volume Profile With Levels Chart [Dekalog Blog]

Just a quick post to illustrate the latest of my ongoing chart iterations which combines a levels chart, as I have recently been posting about, but with the addition of a refined methodology of creating the horizontal histograms to more clearly represent the volumes over distinct periods. The main

*- 2 years ago, 19 May 2020, 10:30am -*

A Comparison of Charts [Dekalog Blog]

Earlier in May I posted about Market Profile with some charts and video. Further work on this has made me realise that my earlier post should more accurately be described as Volume Profile, so apologies to readers for that. Another, similar type of chart I have seen described as a TPO chart (TPO

*- 2 years ago, 18 May 2020, 11:08am -*

Market Profile Chart in Octave [Dekalog Blog]

In a comment on my previous post, visualising Oanda's orderbook, a reader called Darren suggested that I was over complicating things and should perhaps use a more established methodology, namely Market Profile. I had heard of Market Profile before Darren mentioned it, but had always assumed

*- 2 years ago, 4 May 2020, 09:26am -*

Visualising Oanda's Orderbook [Dekalog Blog]

My earlier post of 26th March shows code to visualise the most recent instantaneous snapshot of Oanda's order book, realised as a horizontal bar chart superimposed over a price chart. Below is a screen shot of a different type of chart designed to show the historical order book, which is

*- 2 years ago, 24 Apr 2020, 10:30am -*

Generic Octave_Oanda_API Function [Dekalog Blog]

My last two posts have shown Octave functions that use the Oanda API to access and download data. In the first of these posts I said that I would post more code for further functions as and when I write them. However, on further reflection this would be unnecessary as the generic form of any such

*- 2 years ago, 15 Apr 2020, 10:13pm -*

First Octave Function using Oanda API [Dekalog Blog]

As part of my on-going code revision I have written my first Octave function to use the Oanda API. This is just a simple "proof of concept" function which downloads an account summary. ## Copyright (C) 2020 dekalog ## ## This program is free software: you can redistribute it and/or modify

*- 2 years ago, 6 Apr 2020, 09:31am -*

Some Basic Code Housekeeping [Dekalog Blog]

Since my last post, back in late November last year, I have been doing a few disparate things such as: improving the coding of some functions in R to use the Oanda API to automatically download data using cronjobs coding some Octave functions to plot/visualise the above data more work on Random

*- 2 years ago, 29 Mar 2020, 12:27pm -*

Preliminary Results from Weight Agnostic Training [Dekalog Blog]

Following on from my last post, below is a selection of the typical resultant output from the Bayesopt Library minimisation 3 3 2 2 2 8 99 22 30 1 3 3 2 3 2 39 9 25 25 1 2 2 3 2 2 60 43 83 54 3 2 1 2 2 2 2 0 90 96 43 3 2 3 2 2 2 2 43 33 1 2 3 2 3 2 2 0 62 98 21 2 2 2 2 2 18 43 49 2 2 2 3 2 4 1 2 0

*- 3 years ago, 4 Nov 2019, 09:05am -*

Weight Agnostic Neural Net Training [Dekalog Blog]

I have recently come across the idea of weight agnostic neural net training and have implemented a crude version of this combined with the recent work I have been doing on Taken's Theorem ( see my posts here, here and here ) and using the statistical mechanics approach to creating synthetic

*- 3 years ago, 30 Oct 2019, 07:44pm -*

Another Method of Creating Synthetic Data [Dekalog Blog]

Over the years I have posted about several different methodologies for creating synthetic data and I have recently come across yet another one which readers may find useful. One of my first posts was Creation of Synthetic Data, which essentially is a random scrambling of historic data for a single

*- 3 years ago, 13 Oct 2019, 07:15pm -*

Ideal Cyclic Tau Embedding as Times Series Features [Dekalog Blog]

Continuing on from my Ideal Tau for Time Series Embedding post, I have now written an Octave function based on these ideas to produce features for time series modelling. The function outputs are two slightly different versions of features, examples of which are shown in the following two plots,

*- 3 years ago, 1 Oct 2019, 10:59pm -*

Preliminary Test Results of Time Series Embedding [Dekalog Blog]

Following on from my post yesterday, this post presents some preliminary results from the test I was running while writing yesterday's post. However, before I get to these results I would like to talk a bit about the hypothesis being tested. I had an inkling that the dominant cycle period might

*- 3 years ago, 5 Sep 2019, 12:46pm -*

Process Noise Covariance Matrix Q for a Kalman Filter [Dekalog Blog]

Since my last post I have been working on the process noise covariance matrix Q, with a view to optimising both the Q and R matrices for an Extended Kalman filter to model the cyclic component of price action as a Sine wave. However, my work to date has produced unsatisfactory results and I have

*- 3 years ago, 23 Jun 2019, 09:37pm -*

Determining the Noise Covariance Matrix R for a Kalman Filter [Dekalog Blog]

An important part of getting a Kalman filter to work well is tuning the process noise covariance matrix Q and the measurement noise covariance matrix R. This post is about obtaining the R matrix, with a post about the Q matrix to come in due course. In my last post about the alternative version

*- 3 years ago, 6 Jun 2019, 02:13pm -*

Extended Kalman Filter, Alternative Version [Dekalog Blog]

Below is alternative code for an Extended Kalman filter for a sine wave, which has 4 states: the sine wave value, the phase, the angular frequency and amplitude and measurements thereof. I have found it necessary to implement this version because I couldn't adjust my earlier version code to

*- 3 years ago, 31 May 2019, 03:03pm -*

Extended Kalman Filter [Dekalog Blog]

In the code box below I provide code for an Extended Kalman filter to model a sine wave. This is a mashup of code from a couple of toolboxes I have found online, namely learning-the-extended-kalman-filter and EKF/UKF Tollbox for Matlab/Octave. The modelled states are the phase, angular frequency and

*- 3 years ago, 27 May 2019, 01:59pm -*

Tests of Constant and Variable Acceleration Model Kalman Filters [Dekalog Blog]

In my last post I said that this next post would report the results of tests on a Constant Acceleration model Kalman filter, and the results are: fail, just like the Constant Velocity model, so I won't bore readers with reporting the details of the failed tests. However, tests of a Variable

*- 3 years ago, 5 Apr 2019, 11:38am -*

Test of Constant Velocity Model Kalman Filter [Dekalog Blog]

Following on from my previous post, this post is a more detailed description of the testing methodology to test kinematic motion models on financial time series. The rationale behind the test(s) which are described below is different from the usual backtesting in that the test(s) are to determine

*- 3 years ago, 1 Apr 2019, 09:40am -*

Revisiting the Kalman Filter [Dekalog Blog]

Some time ago ( here, here and here ) I posted about the Kalman filter and recently I have been looking at Kalman filters again because of this Trend Without Hiccups paper hosted at SSRN. I also came across this Estimation Lecture paper which provides MATLAB code for the testing of Kalman filters

*- 3 years ago, 21 Mar 2019, 09:02am -*

Estimating the Bid-Ask Spread [Dekalog Blog]

Below I provide a vectorised Octave function to estimate the bid-ask spread from high, low and close prices according to "A Simple Way to Estimate Bid-Ask Spreads from Daily High and Low Prices," (Corwin and Schultz, 2012). The paper can be downloaded from one of the author's homepage

*- 4 years ago, 14 Dec 2018, 10:44am -*

A Bull Bear Background Plotting Function for Octave [Dekalog Blog]

As part of my recent research I have found it convenient to write another custom plotting function for Octave, which plots a single line price plot against a conditionally coloured background, e.g. two separate colours for bull and bear market regimes. Being able to plot like this avoids the

*- 4 years ago, 20 Oct 2018, 04:37am -*

"Black Swan" Data Cleaning [Dekalog Blog]

Since my last post I have been investigating training features that can be derived from my Currency Strength indicator as input for machine learning algorithms and during this work it was obvious that there are instances in the raw data that are Black Swan outliers. This can be seen in the chart

*- 4 years ago, 11 Oct 2018, 04:18pm -*

Update on Improved Currency Strength Indicator [Dekalog Blog]

Following on from my previous post I have now slightly changed the logic and coding behind the idea, which can be seen in the code snippet below Essentially the change simultaneously optimises, using Octave's fminunc function, for both the gold_x and all currency_x geometric multipliers

*- 4 years ago, 8 Jun 2018, 09:55am -*

An Improved Currency Strength Indicator plus Gold and Silver Indices? [Dekalog Blog]

In the past I have blogged about creating a currency strength indicator ( e.g. here, here and here ) and this post talks about a new twist on this idea. The motivation for this came about from looking at chart plots such as this, which shows Gold prices in the first row, Silver in the second and a

*- 4 years ago, 28 May 2018, 11:50am -*

Hidden Markov Modelling of Synthetic Periodic Time Series Data [Dekalog Blog]

I am currently working on a method of predicting/projecting cyclic price action, based upon John Ehlers' sinewave indicator code, and to test it I am using Octave's implementation of a Hidden Markov model in the Octave statistics package hosted at Sourceforge. Basically I measure the

*- 4 years ago, 3 Mar 2018, 01:36pm -*

Time Warp Edit Distance [Dekalog Blog]

Part of my normal routine is to indulge in online research for use useful ideas, and I recently came across An Empirical Evaluation of Similarity Measures for Time Series Classification, and one standout from this paper is the Time Warp Edit Distance where, from the conclusion, "...the TWED

*- 5 years ago, 11 Dec 2017, 10:14am -*

Candlestick Plotting Function for Octave [Dekalog Blog]

I have long been frustrated by the lack of an "out of the box" solution for plotting OHLC candlestick charts natively in Octave, the closest solution I know being the highlow plot function from the financial package ( which does not yet implement a candle function ) over at Octave

*- 5 years ago, 20 Nov 2017, 07:40am -*

Updating Historical Data Using Oanda's API and R [Dekalog Blog]

Following on from my previous post about downloading historical data, this post shows how previously downloaded data may be updated and appended with new, more recent data without having to re-download all the old data all over again. The main function to do this, HisPricesDates, downloads data

*- 5 years ago, 27 Oct 2017, 10:20am -*

Downloading Historical Data Using Oanda's API and R [Dekalog Blog]

It has been about 5 months since my last blog post and in this time I have been working away from home, been on summer holiday and spent some time mucking about on boats, so I have not been able to devote as much time to my blog as I would have liked. However, that has now changed, and this blog

*- 5 years ago, 22 Sep 2017, 10:16pm -*

Using the BayesOpt Library to Optimise my Planned Neural Net [Dekalog Blog]

Following on from my last post, I have recently been using the BayesOpt library to optimise my planned neural net, and this post is a brief outline, with code, of what I have been doing. My intent was to design a Nonlinear autoregressive exogenous model using my currency strength indicator as the

*- 5 years ago, 21 Apr 2017, 07:14pm -*

Preliminary Tests of Currency Strength Indicator [Dekalog Blog]

Since my last post on the currency strength indicator I have been conducting a series of basic randomisation tests to see if the indicator has better than random predictive ability. The first test was a random permutation test, as described in Aronson's Evidence Based Technical Analysis book,

*- 6 years ago, 8 Nov 2016, 08:33pm -*

Currency Strength Indicator [Dekalog Blog]

Over the last few weeks I have been looking into creating a currency strength indicator as input to a Nonlinear autoregressive exogenous model. This has involved a fair bit of online research and I have to say that compared to other technical analysis indicators there seems to be a paucity of pages

*- 6 years ago, 27 Oct 2016, 10:32am -*

Loading and Manipulating Historical Data From .csv Files [Dekalog Blog]

In my last post I said I was going to look at data wrangling my data, and this post outlines what I have done since then. My problem was that I have numerous csv files containing historical data with different date formats and frequency, e.g. tick level and hourly and daily OHLC, and in the past I

*- 6 years ago, 16 Sep 2016, 10:48am -*

Possible Addition of NARX Network to Conditional Restricted Boltzmann Machine [Dekalog Blog]

It has been over three months since my last post, due to working away from home for some of the summer, a summer holiday and moving home. However, during this time I have continued with my online reading and some new thinking about my conditional restricted boltzmann machine based trading system has

*- 6 years ago, 3 Sep 2016, 12:31pm -*