ML-Experiment: My Doodling Style

There Are No Failures

Below is a Tensorflow project I worked on for a couple weeks. The final output is not impressive (I think)... however I did a few things for my very first time which I think are worth documenting:

The source code for this webpage can be found here, which includes the inference model, training playback, and MDN sampling functions. In addition, the Python notebook I used for training can be found here.

The Goal

I wanted to make a system that can copy how I doodle in my notebooks. Below are some pictures from my notebooks over the years, showing how I doodle.

You'll notice that, generally, I draw random lines, and I try not to overlap those lines. There are also some common angles and shapes that I subconsciously make.

I've made small programs in the past that try to achieve this programmatically, however it always looked a little too robotic.

Did It Work?

Not really, but I can say that after working on this model for a couple weeks, I learned quite a lot about how to make it work better.

I think one major thing that could have make it better was using training drawings that are NOT just my random doodles. Random doodles are hard for me (a human) to tell apart from just noise when a ML model is generating them...

Getting Some Data

To start with, I needed to create a system for recording my doodling style. I created a program that records my drawings in an HTML5 Canvas, and then drew for a couple hours while listening to podcasts.

You can see some of my drawings below. Press the "Random" button to see one of my doodles being drawn.

Training the Model

I used a Python notebook to train the models using Tensorflow 2.0-beta.

The training data (my drawings) were loaded and sent through the following processes:

  1. Compress total points, to keep same shape with less data
    • This allowed me to keep the final model as small as possible, while still being able to generate recognizable shapes
  2. Synthetically create new drawings (more data!), through:
    • Rotating each drawing four times at 90 degrees
    • Scaling each drawing down in size and transposing
  3. Creating training X/Y pairs, for both the simple autoencoders, and the RNN models

The final model is a combination of three sequential models:

  1. Convolutional Encoder
  2. LSTM
  3. Mixture Density Network

In the diagram below, I tried to demonstrate the sequence of how I trained each model independently. Training each sub-model separately seemed to help the training process learn faster and more accurately than training end-to-end.