My Haunted Text Editor

Allow me to let you in on a secret.

You want to know how to annoy my wife?

Well, you for sure can usher her into the express lane to Grumpy Town by simply offering to recap your previous night's dream.

Really though, who could blame her? I mean, you have to admit, that sort of garbage is pretty freakin annoying.

Frankly, if as an adult you remain the least bit surprised by the quality of crap your subconscious chooses to belch up each night, not to mention, thinking that this might rise to the level of being remotely interesting to anyone else.... well then...let me tell you about my dreams last night.... Please, sit back while I attempt to recount it to you in excruciating detail.......

There I was, minding my own beezwax while we were all crammed into this tiny car, headed out on the highway, making our way to the Big Rock Candy Mountain...., You were reading the new Phone Book to us

Aaron, Abal........12 Hunkner Downs Circle .... (413) 555-8080
Aaron, Abraham.....701 Nowhere Place .......... (413) 555-5809
Aaron, Acer........111 You Get My Drift Ct .... (413) 55555555555555

And you were in the driver's seat.... Only you weren't driving...because......well...you had the phonebook in your hands... Isn't that Weird?.... And this talking goat was riding shotgun... only it wasn't a goat, but rather a chinchilla with a goatee.....Oh wait....Not a chinchilla, but a marmot...and I think it was singing the chorus to I Will Follow...only the words were different..."I feel hollow!!"...at least I think that was what it was singing...I recall that it might have had a very thick chinchillean accent...not so sure now.... anyway the important thing is... or at least seemed to be at the time ... that it was wearing a Pavement T-Shirt, but it had a picture of Jennifer Herrema from the Royal Trux on it....which made it a valuable collector's item... and a kid from the nearby college radio station was trying to take it from me..chinchilla and all...I tired to run, but I found that I could barely lift my feet.... and that's when all my teeth fell out. Just then, Ash from Polvo showed up with a 16mm print of Shakes the Clown in hand... He asked if anyone had a moviola, and we scattered to the far corners of the earth trying to dig one up..... Did I mention the stuff about Nick Cave, Debbie Harry riding dinosaurs yet?

Aaron, Ackbar .... 818 Yougottabekiddingme TPK ..(413) 00000000000hhhhh Pleasssseee!

Anyways, as you can see I am now the asshole! And if you had any common decency left in the empty shell that I have just reduced you to, you should politely tell me that you have somewhere else to be, and make for the nearest exit post haste.

But here's the thing..... before you go, let me ask you just one question.

What if it wasn't a person who was presently recounting their weird nonsensical dreams to you? What if instead, it was a mysterious voice from the ether spewing this strange foggy nonsense before your very eyes? Foggy nonsense that was loosely based on the scant few words you had, moments earlier, typed into an unassuming text editor. Foggy nonsense that at the push of a button, mysteriously manifested itself on your screen. Would you find more wonder in these sort of rantings? Could you be intrigued by a bot? One tenuously lilting within a nonsensical haze, blurring the lines between the coherent, the spooky, and the maniacal?

Would that change your opinion of the dreaded dream journal shared?

Well, ever since I stumbled across this blog post a few years back by the author Robin Sloan, I have been able to answer this question for you. And as far as I'm concerned, the answer is a resounding "Yes!"

The project Sloan put together basically combines a few separate tools to create a bot that does just this sort of thing.

The parts:

torch-rnn:

What little I knew about machine learning back in 2016 was that there was something called Tensorflow that wouldn't run on my Mackbook Air. Book closed. That was until I read Sloan's blogpost and found out there were other tools available that can do some of the machine learning heavy lifting and that might actually run on at least one of my computers after all.

The engine of Sloan's project is torch-rnn. An LSTM (Long Short Term Memory) and RNN (Recurrent Neural Network) tool that can learn from reading huge chunks of text.

What does it learn? Well, how to write sentences that more often than not make some sort of coherent sense.

torch-rnn-server:

So Sloan then took the torch-rnn package and added a couple of bells and whistles to it. Basically creating an api service (torch-rnn-server) that runs in the background on your local machine. When called with an http request, and given the proper parameters, the api then returns a selection of texts generated from any text which was seeded to it. If no text is seeded to it?
Well then it just wings it, more or less.

rnn-writer:

The final piece is rnn-writer, a plugin for the Atom text editor that reads back a certain amount of characters preceding the text prompt and, using torch-rnn-sever generates predictive text based on that content using a model created by torch-rnn.

Are you still with me?

Well if you are, thanks for bearing with this rambling. And please let me tell you how much fun I've had over the past couple of years installing these tools on my home computers, work machines, and even playing around with them on a Raspberry PI or two.

Once the service is running in the background, and a text document is opened in the Atom text editor, I can spend hours generating predictive text completions to the words already on the screen.

With the press of the tab button, there is a slight delay before a block of words pour out to the right of what I have just typed. Then with the up and down arrows I can cycle through a seemingly infinite selection of suggested completions.

In order to produce anything even slightly coherent, it takes a bit of curatorial flex, cycling through these options before selecting the most fitting completion.

The crazy thing is, torch-rnn builds these suggested sentences not word by word, but rather character by character. That is to say, it doesn't know for example that "The Laser was pointed at the aliens" are separate words that fit together to tell a story, but rather that T-h-and-e often start a sentence together, followed by a space, which often has the letters l-a-s-e-r proceeding, again followed by a space... etc etc.... And how does it know this? Well because Sloan took the time to train torch-rnn on a dataset he compiled of pulp science fiction magazines pulled from the internet archive

It takes a lot of processing power and time to produce a model that can churn out coherent content.

I've tried training the RNN on a model of my own using an old MacBook pro that has an ancient nvidia gpu that could be harnessed to accelerate the learning. However, even with this acceleration the process was still excruciating slow and the corpus I built was a textbook example of what not to do when training a neural network.

Basically I set out to confuse the hell out of the learning tools. I'm happy to inform you that I succeeded magnificently in my efforts.

My data set consisted of a whole bunch of Gutenberg project texts as well as other sources from authors such as Joyce, Eliot, Pynchon, Vonnegut, O'Connor, Wharton, Melville and so on..... Suffice it to say that the resulting model was barely able to spew out a coherent sentence, not to mention only at a rate of about once in every 100 attempts did that content not include at least one word of gibberish.

But using the pre-trained SciFi model provided by Sloan, I have had a lot of fun slapping together little stories, poems and robot-dream journals that are now scattered about hard drives in my home, the cloud, and various other places of work over the past few of years.

At a previous job for instance, our daily standups were conducted in Slack. I would append each of my daily entries with a sentence or two of robot-blather that eventually made up this sort of serialized story; a selection from which I've posted to this listed feed.

Another entry in this feed is a more curated tale, again built with the rnn-writer plugin. However, in this case, the story has been composed with a bit more human intervention. Segueing from fragment to fragment, I tried here to use the rnn-output at the skeleton of the tale, but used my own editorial content to create a bit of continuity that the RNN cannot deliver. My goal was not to write any award winning narrative, but rather to nudge the text into a more readable selection, while still maintaining that dream-journal weirdness in the end.

I think that's the main attraction. Whereas my expectations are much higher for any human ranting I might be subjected to an any given day, the mere fact that a neural network can come close to emulating the annoying conversation of a well intentioned, but misguided human is the selling point here.

There already are much more effective neural networks out there that an generate completely readable text. I guarantee you that you have read more than a few news articles, sports reports, weather predictions and have even been suckered into falling for click-bait, all composed by an algorithm posing as a sentient meat sack.

In recent weeks there has been a lot of press attention given to the new GPT-2 neural network tool. I would bet even some of that reporting was bot assisted. But the toolset is being hailed as nearly indiscernible from human generated content, when flexed at it's full capacity. The hype has been that it is so efficient that the developers have decided not to release the full powered version of their work, out of fear for the potential mayhem such a tool could produce in the hands of bad actors.

But even crippled, this tool is pretty astounding.

Take this for example: I fed one of the weirder sentences found in one of the 'stories' I had previously generated with rnn-writer into a GPT2 driven text engine available on line. What follows is the completion:

In the round mirror the robot could see Frances walk over and lick water off the stranger's face. But Frances soon realised they were trying to steal her drink and the robot decided to run away, chasing it through the air.

"The robot couldn't see through, it had to get in there," said Jochen Wüstmer, head of the Robot Safety department at the Swiss Federal Robotics Office.

Instead of just running towards Frances to kiss him, the robot ran alongside her – and when Frances started to push against it, the robot turned and smashed into Frances, breaking her hand in the process.

"We decided to go into rescue mode: to go into rescue mode because the robot could not move itself properly," said Wüstmer.

Fris and Jochen Wüstmer: Swiss team have one robot for the whole world to worry about Read more

Frances told reporters the robot was unable to communicate properly with her partner during the assault, so when Jochen took over as team manager, the robot now lives with Frances and will remain behind her apartment in the Swiss canton of Rheims.

The incident happened on 22 March, while Frances was getting ready for work at a Swiss grocery store, where she was looking after the children of one of her employees, according to reports.

Even though it still has the dream-journal quality, the continuity is much more impressive than any text generated by torch-rnn. This new learning model now has a modicum of an attention span. And what I found particularly interesting about this example, is how the text generated seems to bounce between a literary and a journalistic style.

I may have to scrape and scrounge my pennies now in order to purchase a more powerful CUDA capable machine. Then I might just try to train GPT-2 with my trick-the-robot corpus of sundry and vastly different authors. What might be the result? I'm hoping it would generate exquisite-corpse like text, as if a folded page was passed from Flannery O'Connor to Albert Camus, to Kurt Vonnegut, to T.S. Eliot.

My imagined result:

"I don't believe in anything. I want to stay as close to the edge as I can without going over. Out on the edge you see all kinds of things you can't see from the center. I assure you that it's better to sleep with a sober cannibal than a drunk Christian, for only those who will risk going too far can possibly find out how far one can go while Talking of Michelangelo. "

I'm presently waiting on the delivery of a new AI driven Generative Text project of Sloan's which uses the new GPT-2 learning model in conjunction with some other bits of software which he has tinkered together. The crux of this project is that everyone who orders a copy gets a unique printed work based on their answers to a survey, some intermediate software developed by Sloan, and the ranting of the GPT-2 having added a corpus of fantasy fiction to flavor it's existing model built on scraped internet text.

Well, anyhoo, before you leave, why not watch a short film that was written using a torch-rnn like tool trained on a bunch of scifi scripts.

Sunspring | A Sci-Fi Short Film Starring Thomas Middleditch

Because: The Internet!

Also, I highly recommend reading Robin Sloans novels, Mr Penumbra's 24 Hour Bookstore and Sourdough. Both are delightful reads. A bit Sci-Fi, a bit Mystery, yet they never lean so heavily on genre tropes as to not primarily just be fun and charming stories in their own right.