In the first segment of this two-part series on computational creativity, we described computational creativity as a multidisciplinary endeavor. Without the influence of fields such as psychology, cultural studies, and art, it is likely that creative artificial intelligence would not be possible. One of the most essential aspects that all of these fields share is the use of narrative; they all tell their own stories. According to a paper published by Michael Mateas and Phoebe Sengers, telling stories helps us to make sense of the world. We are all trying to find our purpose; we want the events in our lives to have meaning. Authors David Blair and Tom Meyer call this “human ability to organize experience into narrative form: Narrative Intelligence.” Artificial intelligence research converges around narrative intelligence; if we want to understand artificial intelligence and computational creativity, we must understand the importance of narratives. In this article, we take a detailed look at programs from Google, Automated Insights, and Narrative Science in order to explore the ways in which companies implement narrative intelligence. Only then can we take an in-depth look at the artist and programmer Gene Kogan, whose work reveals what the combination of art and technology can produce.
Google’s DeepMind: Narrative Computers and Psychedelic Paintings
Since the creation of DeepMind in 2010, Google has been using artificial neural networks that mimic the human visual brain in order to create a computer capable of recognizing and recreating images. In order for them to create these networks, the team had to begin with the human brain–with psychology. According to Mateas and Sengers, in psychology, narrative is 
considered “a way in which humans make sense of the world.” In Jerome Burner’s study of narrative psychology, he argues that “humans make sense of intentional action by assimilating it into narrative structures.” Thus, the more we understand the ways in which humans build and understand narratives, the more accurately we can program our computers. If we wish to create realistic and successful models of creative artificial intelligence, we must make sure we program our software based on what it can “assimilate to narrative.”
Using Psychology and Narratives to Create Paintings
The Google team has built algorithms that can learn for themselves “directly from raw experience or data.” As the computer works, its complex neural network is operating quietly below the surface. This network is helping the computer make sense of the world; it allows the computer to form narratives and “mimic the way a brain finds patterns in objects.” By constructing narratives from the data provided, the software forms its own–albeit incomplete– understanding of the world. This allows it to make sense of its surroundings and “perform well across a wide variety of tasks straight out of the box.” What we see from this complex software is the final product: an unusual picture. If we wish to understand how and why the computer created this particular image, we must backtrack; we must understand the way Google’s software forms its narratives.
Training: Teaching a Computer How to Create
In order to train their software, the Google team presents their computer with an image. The algorithms the computer has previously learned help it produce what writer Kimberly Mok calls “a final interpretation, an ‘answer’ delivered by neurons that determine what the image shown best depicts.” This final answer is largely based on what the computer has previously learned; it is based on a collection of narratives.
These narratives are found in Google’s original software–its source code
called “DeepDream.” The team presents the computer with an image and tells it that “whatever [it] sees, [they] want more of it.” The computer’s higher level neurons control the abstraction of the image. If these neurons, for instance, think something looks like a bird, the final image will contain many birds. According to Mok, “this creates a feedback loop: if a cloud looks a little like a bird, the network will make it more like a bird… until a highly detailed bird appears, seemingly out of nowhere.” Mok explains how a feedback loop produces “over-interpretations” that are “abstracted, fractalized fusions of previously learned features.” The computer interprets the original image through its neural network and then merges this interpretation with its experiences and past narratives in order to form the final image. This results in an amazing amalgamation of art and technology. Oftentimes, the pictures the computer creates resemble hallucinations, like those one may experience during “psychedelic fever dreams.” These fractalized images may look as though they came out of the mysterious depths of human consciousness, but they are truly the production of artificial intelligence and a team of engineers who boldly combine art and technology.
The Capabilities of Google’s Software: From Psychology to History
Though it does not have the same abilities as a human artist, the DeepDream software can deal with abstractions; it can turn an image into whatever it thinks the shape resembles. In fact, the computer is at an advantage: it is not slowed down by the ego, which constantly second-guesses itself and over-thinks. In this way, Google’s DeepDream is capable of creating psychologically complex images without the impediments of true humanity.
The Google DeepMind team hasn’t just made a computer that can create images–they have made a computer that tells a story. Through the final image, we can see the ‘thought process’ of the computer; we can see which parts of the picture it focused on and which parts it edited. The DeepMind software creates images that are more than pieces of art; they are stories documenting revolutionary technological advancements. Through studying stories, we can continue to improve artificial intelligence.
Google’s software can even use artificial neural networks to create images in the style of famous artists. In a way, this is like going back in time; by using the style of past artists, computers can bring history to life. By looking at and reproducing an artist’s style–how they use color, shape, lines, and brushstrokes–we may even discover more about the psychology of that artist and what led them to create a certain kind of art.
Narrative Intelligence: Turning a Computer into a Writer
Creative software can do more than just recreate images–it can write stories. The more successful artificial intelligence becomes at forming, interpreting, and implementing narratives, the more possibilities arise. If we analyze and recreate the properties of narratives, we can forge “new artificial intelligence and machine learning techniques.”
If we want to make robots capable of writing stories, we must have an understanding of the way literary studies and technology overlap and interact. According to Mateas and Sengers, literary studies analyzes the countless perspectives on “story, narrative, and their function in our culture.” Each perspective “involves a novel way of thinking about narratives and its place in human experience that can be tapped for work in narrative intelligence.” After all, it is th
e human condition that fuels our interest in artificial intelligence; many create technology in order to understand and improve human cognition. The folklorist and scholar Vladimir Propp understood this connection between the human condition, narratives, and technology. Propp broke down stories into “morphemes,” or “analyzable chunks,” and “identified narratemes–narrative units– that comprised the structure of many stories.” The morphemes and narratemes can combine to create different sequences. Propp’s analysis has “served as inspiration for many AI researchers.” If one can understand the common themes in narration, one can generate new stories and better understand narrative systems.
Analogical Story Merging
An example of an algorithm that requires an understanding of literary studies is ASM, or Analogical Story Merging. This machine learning algorithm, created by professor Mark A. Finlayson, is capable of extracting “culturally-relevant plot patterns from sets of folk tales.” From a given set of stories, this algorithm can derive a series of patterns. These patterns help to form and interpret other stories.
Automated Insights: Creating Stories Through Algorithms
Artificial intelligence experts have written code that can in turn teach a computer how to write both nonfiction and fiction stories. The computers take the data presented and use algorithms to produce stories. At Automated Insights Inc., an artificial intelligence platform called Wordsmith can “generate human-sounding narratives from data.” Wordsmith’s pieces vary in style; its news articles are often “dry and efficient in the same way as human-authored stories,” but it can also write fantasy football match recaps with the “snark-laden personality you’d expect from an overworked sports journalist–even including jokes and slang.”
How it Works: Natural Language Processing
Wordsmith uses natural language processing–just like we do with Claire–in order to turn data into narratives. It is the task of a natural language generation system to plan and merge information in order to generate text. The system generates natural language from a knowledge 
base–a database that allows one to share and manage knowledge. Next, algorithms in Wordsmith’s software are used to transform whatever data has been entered into words. Wordsmith is the only open natural language generation application program interface (API). This means that anyone can use their system to generate content; all you pay for is what you end up using. Simply check out the Wordsmith API documentation to get started.
Advantages of Natural Language Processing: Improved Efficiency and Productivity
In the past, the Associated Press was only able to produce around three hundred stories per quarter, which, according to Automated Insights, left “thousands of potential company earnings stories unwritten.” In addition, many stories–especially corporate earnings recaps– are tedious to write. New York Magazine’s Kevin Roose recalls “pulling numbers off press release[s], copying them into pre-written outlines[s], fixing headline[s], and publishing as quickly as possible.” Not only does narrative intelligence increase the the speed of production, but the overall process is more efficient; there is no need to waste money on writing services and there are less overall errors. In the automation of earnings reports alone, Automated Insights has “freed up about twenty percent of the time that [they previously] spread throughout the staff.” From a single narrative design and set of data, billions of pieces can be written. Automated Insights boasts that “your expertise and knowledge of your users can be built into Wordsmith so that the choices you instinctively make when you write are done at scale.” This means that Wordsmith will actively learn how to better produce narratives for your company. Because of this creative artificial intelligence, the Associated Press now produces almost 2,700 quarterly earnings stories, which is over twelve times the number that could be written manually.
Narrative Science’s Artificial Intelligence Quill
Like Automated Insights, Narrative Science created their product in order to solve a major problem: the team wanted to give people a “fast and simple way to understand data.” Narrative Science understands that it is a waste to “invest billions sourcing and aggregating data, spreadsheets and dashboards as mechanisms to understand data.” The stories that the data tells are what truly matter; they teach a company about “their operations, their business, [and] their life.” The Narrative Science team concludes the discussion of their product and process by asking a rhetorical question: “what better way to tell stories than through language?” To solve this problem, the team created Quill.
How it Works: Analyze, Generate, and Inform
Quill’s software uses an advanced natural language generation platform in order to interpret data and create narratives. The software can “immediately add value to data by identifying the most relevant information and relaying it through professional, conversational language.” Like Wordsmith, Quill uses natural language generation in order to interpret data. Narrative Science splits
Quill’s job into three steps: analyze, generate, and inform. First, Quill identifies which facts will form the foundation of a given narrative. In order to do this, Quill determines what matters the most to a business by using a business’ “ rules to identify thresholds, drivers, trends and relationships.” Then,
taking into account a company’s communication goals, business rules, and stylistic preferences, Quill generates the narrative by using natural language generation software. Quill can then apply natural language, which makes the narrative indistinguishable from a human-written piece. Finally, a company can “increase the value of data by fulfilling the tailored information requirements of all audiences.” Quill informs the public based on a company’s particular target audience and goals. This allows companies to personalize their communication.
Advantages: Changing How We Approach Data
Nick Beil, Chief Operating Officer at Narrative Science, believes that we have lost sight of why we are aggregating data. Companies often feel secure knowing that they have a great deal of data. Unfortunately, they don’t have the tools to understand this data. As a result, most companies have a great deal of data that goes unused.
Quill reminds us that the reason we collect data is because we want to answer questions about our businesses. Not only do we need answers, but we need explanations–we need to know where these answers come from. Kristian Hammond, the Chief scientist at Narrative Science, believes that we derive power from these explanations. According to Hammond’s video about Quill, the “notion of narrative analytics is a set of tools and techniques where the analysis of data is driven by a company’s communication objectives.” Before a company starts analyzing data, they must first decide what it is they want to say. Every algorithm the company uses must then be driven by their established communication goals. The Narrative Science team hopes that, by helping companies make better decisions, Quill will create a more informed and intelligent world.
Gene Kogan: Creating Narratives Through Style Transfers
Programmer and artist Gene Kogan understands that technology cannot stand on its own. He enjoys exploring many different fields– even creative writing. In an interview, Kogan told our writers that: “code and creative writing can overlap a lot, at least for me and many of the people I know doing it. A lot of code writing for me is exploratory… and creative writing is much the same, and I am doing a little bit of both these days, and they help each other a lot.” Kogan understands that programming and writing are more similar than it would seem; both require editing, simplifying, and clarifying. When we write narratives, we learn that there are a plethora of ways in which a topic can be approached and explored; this allows us to find unique ways of expressing ourselves. If we practice writing narratives, we can become more creative and open-minded. Despite what many think, working with technology can require a great deal of creativity.
What are Style Transfers and How Do They Work?
In a recent project, Kogan chose to create artwork through style
transfers. This technique recomposes images in the style of other images. Kogan credits the paper A Neural Algorithm of Artistic Style as inspiration. This paper discusses implementing “neural representations to separate and recombine content and style of arbitrary images, providing a neural algorithm for the creation of artistic images.” As we discuss in part one of “Computational Creativity,” there is a “striking similarity” between artificial and biological ne
ural networks. This similarity “offers a path forward to an algorithmic understanding of how humans create and perceive artistic imagery.” With his strong understanding of the interdisciplinary nature of computational creativity, Kogan is able to create impactful artwork.
In order to create his pictures, Kogan picks a particular image; for one transfer, he chose a video of Picasso painting on glass in 1937. The next step is to think of a way to restyle the image. Kogan chose to restyle the video of Picasso by using his work from his Blue, African, and Cubist periods. Then, using code written by Justin Johnson, he creates a new image. The result is a beautiful amalgamation of history and art. Kogan describes his process in depth here.
Narration in Kogan’s Style Transfers
Kogan’s style transfer pieces are alive. Not only do they literally move, but they breathe humanity. Each picture has its own narrative–its own complex history. Kogan could not produce such beautiful images without an understanding of narrative processes. As Finlayson states, an understanding of narration is “critical to achieving a complete grasp of human cognition.” Kogan taps into humanity by turning a picture into a journey through time. He tells us that “an engagement with humanities and the social sciences may help you code with more intention and expression.” Through combining coding with narration, Kogan demonstrates how technology and art can coexist and collaborate.
Kogan’s Other Projects
Sometimes, if you are dedicated and passionate, the most complicated topics can become manageable and exciting. This is how Kogan writes his codes; to him, it is an “exploratory” process. He tries to find things that interest him and then “sort of thinks through the code,” he says. Kogan’s projects let us see how his mind works; we get to witness and learn from his exploratory thought processes.
“Exploring the Latent Space of Chinese Handwriting”
In this project, Kogan was inspired by Alec Radford’s work to explore the latent space that makes up Chinese handwritten characters. Kogan created his images with a “deep convolutional generative adversarial network (DCGAN)” that he trained on a database of Chinese characters. DCGAN is a type of convolutional neural network which Kogan states 
can learn “an abstract representation of a collection of images.” After training, DCGAN can generate samples that are similar to the originals. This beautiful project is a story that has come to life; each character flows into the next and creates “an impression of imaginary characters which are interpolated from in between the real ones, perhaps corresponding to semantically intermediate concepts.” Kogan’s project highlights the artistry of Chinese characters; symbols and ideas flow into one another. This gives us an idea of the way Chinese people think–the way they group concepts together. In this project, Kogan took what was already a narration–a series of characters–and turned it into a deeper, more complex story that explores the connectivity of ideas and images in Chinese culture.
“Color of Words”
In 2011, Kogan created this project with the goal of discovering “the implicit colors of words, phrases, and concepts.” In order to do this, Kogan selected a set of words to put through a Google image search. Then, he analyzed the color distribution of the first hundred images on Google. After this, Kogan re-synthesized these images using a self-organizing map (SOM), which he describes as “a machine learning algorithm for clustering and visualizing data.” Kogan then took these Google images and fit the Gaussian mixture model to the distribution of pixel colors. Check out more about Kogan’s process and see the final images here.
Robot Shakespeare
In 2013, Kogan created , a robot who generates a “pseudo-Shakespeare” tweet every thirty minutes. For this project, Kogan states that he used a “Hidden Markov Model–which you can read up on here–doing a random walk on the bard’s word-to-word transition probabilities.” Kogan’s creation brings a piece of history to life; Robot Shakespeare combines the story of the most famous writer of all time with the story of modern technology. The narrative of Shakespeare’s life becomes intertwined with the “life” of the robot Shakespeare; their stories create a dialogue between history and modern technology. If you want to run your own Robot Shakespeare, follow Kogan’s detailed instructions on GitHub.
The Future of Narrative Intelligence: Improving Our Society
With so many brilliant people creating new ways for robots to improve our society, it’s hard to imagine how artificial intelligence will improve in even just ten years. Many of us have hopes, questions–and even fears–about what the future will hold.
So, Will These Creative Robots Replace Humans?
The writer Richard Moss assures his readers that robot writers such as Quill will not put anyone out of a job. Instead, these robots can do “grunt work, the stuff that nobody likes to do but is necessary to the job, like writing cookie-cutter earning reports.” Robots have different strengths than humans do; they can work together to achieve a goal without stepping on one another’s toes, so to speak. Moss believes that a robot’s strength is that it can “take mountains of raw structured data, which humans find difficult to parse and understand, and translate it all into clear [paragraphs]…that get across the core ideas or statistical highlights.” Robots will write stories that are simple, but necessary. Companies can easily change the style that their robot writes in by “tweaking a few variables.”
Michael Cook, a PhD student and research associate at Goldsmiths College Computational Creativity Group, explains that computers do not have the same talents as humans. While we often forget or get tired, emotional, and confused, computers are very good at being unemotional and considering each option equally. People “are prone to fatigue and conscious or unconscious biases.” This can cause us to quit projects too early, without fully exploring every one of our options. Artificial intelligence can show us what we may have overlooked.
Cook believes that “AI will invent genres that humans could never have possibly conceived of…one day people will steal ideas from software, not because they want the fame or the pride, but because it’s the current mobile trend and it’s too good not to steal.” Though Cook calls this a “cynical and sad aspect of the future,” he also likes that, some day, artificial intelligence will get the validation it deserves.
What Computational Creativity Can Teach Us: Possibilities for the Future
Left and right, companies are choosing creative artificial intelligence. The perfect amalgamation of art and technology can lead to unimaginable success. According to Richard Moss, since the introduction of narrative intelligence, the Associated Press has seen its quarterly output of computer-generated articles “incre
ase tenfold–from 300 to 3,000–since adopting its automatic prose generator.” Creative artificial intelligence expands the possibilities for art and technology. These robots are artists in their own rights; they collaborate with their human creator and produce original pieces. Narrative intelligence teaches us the value of shifting our perspective. Often times, when we choose a career, we end up neglecting everything that we consider to be a part of a different discipline. Narrative intelligence reminds us that some of the most successful fields require an interdisciplinary knowledge. The importance of being creative is paramount; creativity is not restricted to a certain discipline. No matter our career path, if we remain creative and open–and always search for the narratives hidden in everything around us– we have a better chance at becoming successful.
Want to Learn More About Computational Creativity?
Check out part one of this article, “The True Potential of Computational Creativity: Technology and Humanity,” in which we take a look at the work of the late Harold Cohen and explore the connection between technology and humanity in depth.