Icon ASCII : A Love Letter


Icon My Neural Network isn't working! What should I do?


Icon Phase-Functioned Neural Networks for Character Control


Icon 17 Line Markov Chain


Icon 14 Character Random Number Generator


Icon Simple Two Joint IK


Icon Generating Icons with Pixel Sorting


Icon Neural Network Ambient Occlusion


Icon Three Short Stories about the East Coast Main Line


Icon The New Alphabet


Icon "The Color Munifni Exists"


Icon A Deep Learning Framework For Character Motion Synthesis and Editing


Icon The Halting Problem and The Moral Arbitrator


Icon The Witness


Icon Four Seasons Crisp Omelette


Icon At the Bottom of the Elevator


Icon Tracing Functions in Python


Icon Still Things and Moving Things


Icon water.cpp


Icon Making Poetry in Piet


Icon Learning Motion Manifolds with Convolutional Autoencoders


Icon Learning an Inverse Rig Mapping for Character Animation


Icon Infinity Doesn't Exist


Icon Polyconf


Icon Raleigh


Icon The Skagerrak


Icon Printing a Stack Trace with MinGW


Icon The Border Pines


Icon You could have invented Parser Combinators


Icon Ready for the Fight


Icon Earthbound


Icon Turing Drawings


Icon Lost Child Announcement


Icon Shelter


Icon Data Science, how hard can it be?


Icon Denki Furo


Icon In Defence of the Unitype


Icon Maya Velocity Node


Icon Sandy Denny


Icon What type of Machine is the C Preprocessor?


Icon Which AI is more human?


Icon Gone Home


Icon Thoughts on Japan


Icon Can Computers Think?


Icon Counting Sheep & Infinity


Icon How Nature Builds Computers


Icon Painkillers


Icon Correct Box Sphere Intersection


Icon Avoiding Shader Conditionals


Icon Writing Portable OpenGL


Icon The Only Cable Car in Ireland


Icon Is the C Preprocessor Turing Complete?


Icon The aesthetics of code


Icon Issues with SDL on iOS and Android


Icon How I learned to stop worrying and love statistics


Icon PyMark


Icon AutoC Tools


Icon Scripting xNormal with Python


Icon Six Myths About Ray Tracing


Icon The Web Giants Will Fall


Icon PyAutoC


Icon The Pirate Song


Icon Dear Esther


Icon Unsharp Anti Aliasing


Icon The First Boy


Icon Parallel programming isn't hard, optimisation is.


Icon Skyrim


Icon Recognizing a language is solving a problem


Icon Could an animal learn to program?




Icon Pure Depth SSAO


Icon Synchronized in Python


Icon 3d Printing


Icon Real Time Graphics is Virtual Reality


Icon Painting Style Renderer


Icon A very hard problem


Icon Indie Development vs Modding


Icon Corange


Icon 3ds Max PLY Exporter


Icon A Case for the Technical Artist


Icon Enums


Icon Scorpions have won evolution


Icon Dirt and Ashes


Icon Lazy Python


Icon Subdivision Modelling


Icon The Owl


Icon Mouse Traps


Icon Updated Art Reel


Icon Tech Reel


Icon Graphics Aren't the Enemy


Icon On Being A Games Artist


Icon The Bluebird


Icon Everything2


Icon Duck Engine


Icon Boarding Preview


Icon Sailing Preview


Icon Exodus Village Flyover


Icon Art Reel




Icon One Cat Just Leads To Another

Data Science, how hard can it be?

Created on July 2, 2014, 9:16 p.m.

Lets write a poem, and lets do this thing the programmer way. I've got this big juicy dataset of poetry I recently acquired from err, somewhere or other. Lets whip out the statistics and see what we can do with it. Just a little data science. How hard can it be...

But where to start?

How about the length. lets be honest, no one likes a long poem. Just stick it in a novel if you've that much to say. But we have the data now. Lets find out exactly how long it should be...

word count

Okay, so maybe long poems aren't such a drain. It doesn't seem like there is much correlation between poem length and rating. Instead the rating measure just gets more unstable as the amount of data drops.

Either way, I'm still giving a fat zero to the longest poem in the data set "Ashtaroth - a Dramatic Lyric" by "Adam Lindsay Gordon" which is an immense 17680 words long. Literally nobody has time for that.

I'm liking the look of the other side of the spectrum. The shortest poem in our data set is "Reflection On A Wicked World" by "Ogden Nash" consisting of just the three following words:

Is obscurity.

Nice. Poetic angst at its finest.

So length doesn't seem to matter. We'll keep it short because y'know - can we really be bothered.

How about line length? In English weren't we taught all about iambic pentameter, getting a nice rhythm to the poem and all that. How long should each line be, and does being iambic (having a du-DUH rhythm) really matter to how good a poem is.

According to my English teacher if you're going for it poems that are 10 syllables long, with a nice iambic rhythm should be pretty sweet.

line length

Okay flat as a pancake. Screw that. Lets take a different approach.

Lets decide what to write about first. Topic is surely more important than length. Should it be Love? Ambition? Existentialism? Even...Data Science? (one can dream). This data set is tagged with topics for each poem. Lets check out the most popular topics and their average scores to get an idea - any idea.


Wait what? So the best poems are about mentors, moms, chocolate, television, and football? And since when was "concrete" a topic for poetry anyway? I found over 25 poems in the set with that label. I'm no English major, but I've seen some concrete in my time, and it has never inspired me to write a beautiful sonnet.

I'm starting to wonder if this is really a good dataset. Or more specifically, if the rating and topic tagging, which appears to have largely been done by bored American housewives (with a love of concrete), is really going to tell me anything insightful.

In fact I think I've come up with the perfect poem already - a maternal haiku. Lets stop here.

mom eating chocolate
football on television
concrete between us

Pretty deep eh? I Know.

A special mention goes out to the topic "depression", the only topic out of several hundred (including "concrete") which scored below the default 5.0 score. I guess you better face it. The people don't want to hear about your troubled childhood. Go back and write something about candy, or, America, or do you just hate freedom?!

So I guess that isn't going anywhere. But perhaps topic is linked to other things. My teacher always told me that iambic pentameter was like a heartbeat. It expressed passion, and power - love and joy. Lets take a look and see if she was right...


Ouch. Okay that was clearly a write-off.

Enough on the topics. What about Rhymes. Everyone loves a sweet rhyme. The only thing that beats a sweet rhyme is a sweet pun, and iambic-inning to get sick of this poetry business. Eh. Eh.

Lets take a look at how the rhyme schemes stack up.


Wow. I mean I always liked him, but I didn't expect my main guy ABCAACCADEADEA, the greatest rhyme scheme ever, to be right up there. Not to mention my buddy, ABBACCCCCDDCCC. Five rhymes in a row, that is just something special. That's like - you can't just step up and write that - it takes something special inside.

But seriously, it doesn't really seem like rhyme schemes have any sensible correlation to rating, even when normalized to take into account the number of poems with that scheme. But it is nice to know that people slightly prefer ABCB to ABCC ... I guess.

Clearly we're not going to be able to get informed of how to write a super sweet poem this way.

But I have one final awesome idea.

Lets get the data to write the poem for us. Lets roll out the big guns.

It's Markov Chain time.

Oh yeah. This is it. This is the deep-o-tron.

they mow to what was happy folk of that second
this trivial circumstance to dust mixed up from
over the green field of life is the strife
i wronged and aye and not for the cataract

And never for the cataract!

before that i tell
were stirred
to the image of green

the shamrock the flowers
all in furrowed mountains
in a violin in a merchant who bore
come when we came and
the sun
i said earth are spray
the earth my father will
the clash
oh music
the rickety ferry for many a phantom forest


A personal favourite - this epic tale - of feeling, and questioning.

a tree
i see
all felt

And this saucy verse about perfumed monkeys and bridal ass.

we sit and rather one leap out the ass in her bridal bower
by a joy and holy ground
suddenly she young and baulked his body give medicine on the earth
my softest voice ill fares the warm our southern line
now mere profession noble the perfumed monkeys on yonder house can it in the 
moon didst not moonbeams that wretched dead
conquers every birth
the last has been at whose lucid
nothing but that is the blast
you think not distant speaking ask ye your languid arms to their gear of

From which I am taking my new exclamation of surprise, "by a joy and holy ground!".

Unfortunatly I'm kinda stuck now. So I'll leave you with my final submission. But it's been fun folks - remember to tune in next time for "Data Science, how hard can it be?".

This poem I call "A Pale English Man on Holiday".

a man
your victory is all its fill her sobbing with happiness
three days passed and brown
github twitter rss