Nature of Code / Final Project

TITLE: Case by Case

DESCRIPTION: Case by Case, my Nature of Code final project came from the idea to distinguish lowercase letters from uppercase letters for little kids to use. This was inspired by a parent-teacher meeting for my 3-year old. The teacher told me that my child was really good at identifying uppercase letters, but not lower case letters. When we read, we are deciphering many strings of lowercase letters, so I wanted to figure out a fun exercise to help him learn in preparation to learning how to read.

MAIN GOAL: To explore and understand the initial steps of letter and number recognition in a machine learning system using Shiffman’s Neural Network with p5 example of handwritten numbers and applying letters to his sketch.


 

PRESENTATION:

CASE BY CASE PRESENTATION

 



DOCUMENTATION:
Continued from last week’s Final Project: Step 1….

1) To distinguish between upper and lowercase letters, I needed to creating the handwritten letter dataset to add to Shiffman’s Neural Network.

ASCI_TEST

 

TEST WITH DECORATIVE LETTER

 


NEXT STEPS: To keep exploring this method and eventually build this kid’s app that could not only create testing data from what the child writes, but could also be a fun way for kid’s to practice writing their letters and identifying the letters case by case.

 

SOURCE CODE: 

Nature of Code / Final Project: Step 1

TITLE: Case by Case

DESCRIPTION: Nature of Code final project that distinguishes lowercase letters from uppercase letters for little kids to use. The idea was inspired by a children’s app called Endless Wordplay and from a parent-teacher meeting for my 3-year old. His teacher informed me that he was really good at identifying uppercase letters, but not lower case letters. In preparation towards the next steps of reading, deciphering lowercase letters is something that we needed to work on since most of reading is strings of lowercase letters.  And for myself, in effort to grasp the idea of neural networks and machine learning, I decided to work with Shiffman’s neural network of handwritten numbers using the MNIST database and apply letters to his sketch. Eventually, I would like to apply that model to identify letters and numbers in graphic illustrations, photos and different typefaces.

————

MOCKUP:


DOCUMENTATION:
1) In building upon Daniel Shiffman’s Neural Network example from Nature of Code, which was also based on Tariq Rashid’s Make Your Own Neural Network, I want to use a training set of handwritten letters to distinguish between upper and lowercase letters as an initial step. I eventually want to take photos or illustrations of letters and numbers, similar to pieces in 36 Days of Type below, and teach the neural network to identify the letter or number.

36 DAYS OF TYPE

 

2) Converting a test image of a number illustration into a bitmap file to add to the training set. Converted the test image into a 28 x 28 pixel greyscale image in Photoshop and then used Python to extract the pixel values from the photo illustration.

from PIL import Image
im = Image.open('um_000000.png')
pixels = list(im.getdata())

(SOURCE)

3) Creating the upper and lowercase training set, adding that to the data folder and build…

Nature of Code / Exercise 3: Datasets

DATASET
For this week’s assignment, I decided to think of a dataset that I would like to use with a supervised learning machine learning algorithm and want to concentrate on finding patterns and similarity within images. At first I looked at the Instagram API because I wanted to try to find ‘magazine covers’ from the feed without hashtags or geotags, but since the Instagram users own their images, there seems to be a lot of restrictions. So I remembered from another class that the Smithsonian API could also be used to explore the images and through that I found the Cooper Hewitt API, which seems to format the data as JSON files. Below is an example of how they used colors to classify the pieces in the museum.

 

Nature of Code / Exercise #1 / Binary Tree

full screen

EXERCISE #1Visual Binary Trees: I wanted to concentrate on something visual and understand binary trees in code. The samples above (heavily relying on Dan Shiffman’s video tutorial and source code on binary trees viz) are really basic playing with colors and shapes in p5 taking inspiration from Giorgia Lupi and Stefanie Posavec’s Dear Data graphs to help myself understand the mechanism of the code. I definitely need to work on this more to come up with something more original. Eventually, I’d like to model it after these neural illustrations done below by neuroscientist, Santiago Ramon y Cajal.