data&stuff

Loaded Words in Wordle

Why the 'Best' Wordle Seed Words Aren't Really the Best

Posted on February 10, 2022

Over the past few weeks, I noticed more and more green, yellow, and black/white grids posted on Facebook, and that’s when I discovered Wordle [1]. I was hooked - not so much in playing the game the human way, but in developing a system to try to play optimally. As... [Read More]

Tags: data science wordle optimisation simulation

Towards Open Options Chains Part V: Containerising the Pipeline

Posted on February 9, 2022

In this post, we move the entire stack onto Docker, a tool that enables us to package the solution and its dependencies into neat little things called container images. This will make the app portable, enable it to run consistently on different machines, and make it easier to run/stop the... [Read More]

Tags: data science finance options data engineering

Towards Open Options Chains Part IV: Building the DAG

Posted on February 2, 2022

In this post, we will build on our work in Part II: Foundational ETL Code and Part III: Getting Started with Airflow by converting our ETL pipeline into a Directed Acyclic Graph (DAG), which comprises the tasks and dependencies for the pipeline on Airflow. [Read More]

Tags: data science finance options data engineering

Towards Open Options Chains Part III: Getting Started with Airflow

Posted on January 26, 2022

In this post, we set up Apache Airflow as part of the preparation to convert our ETL code into a full-fledged data pipeline. [Read More]

Tags: data science finance options data engineering

Towards Open Options Chains Part II: Foundational ETL Code

Posted on January 19, 2022

In this post, we develop the essential code for extracting, transforming, and loading options data. We will need to retrieve data from the TD Ameritrade API, transform it, and load the data into the PostgreSQL database we created in Part I: Designing the Database. We will also run some checks... [Read More]

Tags: data science finance options data engineering

Towards Open Options Chains: A Data Pipeline for Collecting Options Data At Scale - Part I

Posted on January 13, 2022

Part I: Designing the Database Since September last year, I started trading options. Naturally, I wanted to use data to validate my strategies. But, it was tough getting hold of intra-day options data. I was surprised to learn that it is expensive as hell: a subscription for 30-minute data (quotes... [Read More]

Tags: data science finance options data engineering

Schooling: By the Numbers

Posted on August 5, 2021

Schooling: By the Numbers Joseph’s Schooling’s recent performance at the Tokyo Olympics sparked a huge reaction. Having won the 100m Butterfly at the Rio Olympics in 2016, it seemed that people expected Schooling to replicate his success in Tokyo. There were numerous snide remarks (most have been deleted) and expressions... [Read More]

Tags: data science sports

Deep Learning for Aircraft Recognition Part I: Building a Convolutional Neural Network (CNN) from Scratch

Posted on September 5, 2020

Computer Vision for the Military For the past few years, I’ve kept current on developments in machine learning (ML) through courses and interest groups. One thing I’ve noticed is that a lot of success stories were recycled from the business world. Although there are many potential military applications, especially for... [Read More]

Tags: aircraft recognition computer vision deep learning

Singapore COVID-19 Case Data

Posted on April 9, 2020

On Monday, I came down with flu (better now) and was issued a 5-day MC. Stuck at home, I thought it would be interesting to explore data on Singapore’s COVID-19 cases to see what insights could be drawn. This post contains some exploratory data analysis (EDA) on data from the... [Read More]

Tags: exploratory data analysis COVID-19

Machine Learning for Property Valuation

Posted on September 15, 2019

Discovering Property Valuation Recently, I discussed the property market with a friend who was a real estate agent. I became fascinated with the real estate market: the marketing, the negotiations, the incentives, and the contracting process. Of greatest interest to me was property valuation. I learned that property sellers used... [Read More]

Tags: data science real estate