The day an algorithm started painting…

I’ve been reading and writing a lot lately on the future of work, specifically about how many jobs may no longer exist in the future because of automation. This means that algorithms and robots might be able to do the work more efficiently and cheaply than any human alternative. I’ve started to see some examples of this in the news. One was a robot that was able to inspect bridge infrastructure and alert someone if it found the concrete pillar of a bridge was crumbling. The robot boat could travel up and down a river, inspecting the underwater structures in ways that had been previously handled by human divers. There are other examples of robots that use magnets to cling to a bridge, allowing them to crawl over the structure and inspect it for defects. Both of these are great examples of robots replacing tedious and potentially dangerous jobs that are currently being done by people.

For many people researching the future of work this isn’t such an issue. It makes sense that these kinds of jobs will become a hybrid of people and machines working together to solve a problem. In some cases it might mean a machine can completely take over the job.

There has always been a sense of security that the more “human” jobs would be free from any encroachment from machines. Those were the more complex jobs that required creativity and complex problem solving. It seemed like a good partnership, machines could handle the tedious tasks and we could tackle the real problems of the world.

Then I heard about “The Next Rembrandt” project. The project analyzed all of Rebrandt’s paintings using 3D scanning that could captured the brushstrokes and the painting itself,  pixel by pixel. When the team was done it asked the algorithm to take what it knew about a real Rembrandt:


Credit: H. O. Havemeyer Collection, Bequest of Mrs. H. O. Havemeyer, 1929

and used it to create a completely new Rembrandt using 3D printing to replicate the colour and brushstroke texture of an original painting. It then told the algorithm to take what it had learned and paint something. They gave it some parameters, the program should painting a portrait of a 30 to 40 year old Caucasian male with facial hair, he should be wearing dark clothing and have a collar around his neck and a hat on his head. The program then took all this information and returned this:


Credit: The Next Rembrandt

It seems I was a bit smug about what machines wouldn’t be able to tackle in the future. Art has traditionally been considered an exclusively human domain. On the surface it would seem that that domain has been breached. In truth, it hasn’t happened yet. The machines that created this painting were carefully directed by human programmers and designers. It didn’t decide that it needed to become a painter because of some intrinsic need, it was following orders. It managed to do this in an original and unpredictable way which is most important take away from this project. If humans are able to design these kinds of machines we are going to be able to experiment and innovate in much more interesting and accelerated ways. The real thrill from this kind of experiment is projected it forward to the next challenge. Imagine an algorithm being feed decades of diagnostic information on cancer, all of the available pharmaceutical and medical techniques available and then being told “cure cancer”. Initially it won’t be that expansive, more likely the command will be “cure my cancer” but you can start to see how these tools can begin to actually change the way we tackle problems.

The Shallowing Hypothesis


Storytellers are challenged with capturing the essence of a story and delivering it to their audience in a convincing way. Story elements might include a mood, a subtle emotion or a moment of shock and surprise. Transmedia authors have the same challenges but they face an additional challenge in picking the appropriate channel to deliver their story. Even after an author has crafted a story he or she needs to understand who is listening to those channels and how to engage them. Some recent research on text messaging out of the University of Winnipeg seems to indicate that heavy users of texting channels are becoming shallower. When you think about how carefully a story element needs to be crafted to reach an audience this can have some meaningful implications.

Read blog post…

“Defiance”: A Big Transmedia Gamble

“Defiance” is a new science fiction television series/ massively multiplayer online role-playing game (MMORPG) that will be released on the SyFy cable network channel April 15th. Another television series and video game set in a dystopian future that has earth invaded by aliens and destroyed by an environmental disaster isn’t an original approach. You only have to look to a host of television shows like “Falling Skies”, films like “Independence Day” or video games like “Gears of War”. The originality, and the risk, comes from the creation of a story world that spans both the television series and the online video game in real time.

Read Blog…

Prometheus: The Second Screen Experience

1Prometheus, the latest film from Ridley Scott has arrived on Blu-ray. This isn’t really news but one notable addition to the release is a second screen app that provides an interactive and immersive viewing experience.  This review looks at how the app was designed and what the second screen experience looks like for the film. If you are interested in understanding the app development market for the second screen or how the second screen might impact narrative in linear forms such as film and television, this will probably be interesting. Otherwise, you might as well get back to looking for more leaked material on the sequel to Prometheus that is apparently already in the works. For those without a background on the recent and rapid evolution of the second screen here’s a bit of an introduction.

Introduction to the Second Screen

There is a considerable amount of hype around the second screen experience for television and films. The basic idea is that we are fast becoming a culture that simultaneously consumes both the big screen of our big living room flat screen as well as the small screen of our tablets and smart phones. The recent numbers from Nielsen indicate that 40% of us are using tablets or smartphones while watching TV. Those numbers are being used by a lot of app developers to justify the creation of a new breed of apps, the second screen experience for your television viewing. The justification is a bit of flawed logic, most of us are multi-tasking while we’re on that second device. Although some people might be looking up an actor on IMDB or checking sports stats of the team currently playing, most of us are checking email, Facebook or browsing online content that isn’t related to what we’re watching on our TV. Most of us live in an extremely busy world where our mobile devices allow us to be in constant communication with our social network and information sources. It isn’t anything new, the clichéd vision of someone reading a paper and eating breakfast has been replaced with that same person eating breakfast while reading news feeds on their iPad.  We’ve always tried to juggle multiple activities, the second screen has just become another tool for us to engage in our many activities. That proviso in mind, watching film and television is one of those many activities we engage in every week. The developer that figures out how to get a viewer to watch a second screen that is related to the big screen in the viewer’s living room is looking at a potential gold mine in marketing and advertising space.

The Prometheus Second Screen App

First impressions on the app was that it has some great design that maintained the look and feel of the film itself. It was great to look at throughout the entire second screen experience.

I have no idea what motivated the actual design. I haven’t found any published reasons for creating the app for Prometheus. The app is full of free content such as deleted scenes, documentaries and production art. The motivation doesn’t seem to have anything to do with actually monetizing the second screen in order to provide the creative property with an additional revenue stream. Some second screen apps try to create extra income through the sale of additional content or advertising space n the screen. The Prometheus second screen app is free for download from the iTunes store and it doesn’t have any advertising. There is one link to the Fox store that lets you buy the Blu-ray collector set but that is the only attempt get any money from the app user. It seems most likely that Ridley Scott and Twentieth Century Fox were motivated by three things:

1. Providing an incentive to buy the Blu-ray and not download a pirated copy

2. Creating an innovative and richer viewing experience for Prometheus fans

3. Experimenting with second screen experiences and new models of narrative

Prometheus Second Screen App Home Page

It is difficult to guess at which motivation was stronger. I’m sure someone who is a real fan of the film will buy the 4 disk 3D special collector’s edition of the film and get more extra content, documentaries, deleted scenes and production material than they could possibly get from a second screen experience. I doubt that a second screen app is going to stop anyone from illegally downloading the film but that is only an opinion. I suspect that Scott, and more specifically Fox wants to be in this space because despite the hype there might be something to second screen that is going to differentiate their films and provide another reason for people to purchase their products.

The features in the app are mostly there to provide additional information about the movie, the artwork and people involved in the project. It has a passive archive of content that can be accessed through the app. It also has a number of interactive elements and a remote control features that allows you to control a Wi-fi enabled Blu-ray player. The interaction with the Blu-ray player lets you access alternate scenes that can be played on your television during the movie itself. A good review of the features can be seen in this YouTube summary of the app.

I’ll walk through a number of the features but I’ve added a diagram that breaks down the architecture of the app:

Prometheus Second Screen app-4
Prometheus Second Screen App Architecture

You can follow along as I move through and review the features.

Home Page

The home page was a simple navigation screen that let you dive into the extra content and synchronize the second screen experience with your Blu-ray viewing experience. It also gives you a standard help screen and the ability to access the Fox store in case you decided it was time to go purchase a collectors edition of the film or any additional paraphernalia.


The archive will look familiar to anyone who has gone through the bonus features on a DVD or Blu-ray. It has a pretty standard collection of content that included:

  • Pre-vis – Some visuals done during pre-production on the film
  • The Art of Prometheus -Original concept art
  • Screen test -Screen tests from the actors in the film
  • Unit photography -Still photography from the film production
  • Time -lapse photography -Time lapse sequences of constructing some of the more elaborate sets
  • Marketing Gallery -A collection of posters and visuals created for film marketing
Prometheus Second Screen App Archive Screen

There material is interesting to look at if you’re the kind of person who likes to go through all the bonus features on a Blu-ray. I imagine that you would prefer to be viewing the material directly off the Blu-ray in HD on a much larger screen with higher resolution. Overall it is kind of interesting but hardly a compelling reason to get into a second screen experience.


The Navigation bar is common to both the archive and the sync experience. It provides access to a number of features like live chat, social media and other online sites such as IMDB so you can go look up additional information about the film and the people involved in it.

Navbar Social Features

The social aspects of movie viewing have always been compelling. After all, how many of us  still go to the movie theater just to see a film? Most of us are there as an activity with friends. It shouldn’t be surprising that the Prometheus app has social features built into its architecture. I didn’t see a lot of activity tracked through the app but it had the standard Twitter and Facebook publishing options. Get Glue is a newer entry into the social media market. They have a narrower niche than Twitter and Facebook, providing a social network for film, television and sports viewers.

Prometheus Second Screen App Social Screen


Probably the most interesting feature for me was the ability to sync the app with the actual movie viewing experience. This is the most innovative feature within the app although it requires a fair amount of technology in a home entertainment system to make it work. You need to have a Wi-fi network that connects both your second screen (iPhone, iPad, tablet, whatever you’ve got) with your Blu-ray player. You also need to have Wi-fi capability with both your second screen device and your Blu-ray player. I used a Sony Wi-fi Blu-ray, a Sony Bravia and an iPhone. If you don’t have all the necessary technology it is possible to do a manual sync. This will sync the content on your television with your second screen device. If you use the manual option you miss a lot of the most interesting features in the app though, I’ll cover those next.

Second screen app Sync screen

As you go through the film new content becomes available on the second screen. You can look at content such as storyboards and concept art but whenever an alternate scene comes along you can “flip” it from your second screen to your big screen. You will then start watching a different version of the film on the big screen until the alternate scene runs though and puts you back into the original film. The Prometheus app has the ability to act like a remote control. It is using this functionality to communicate with the Blu-ray player, telling it to go find the alternate scene on the disk itself and start playing it. It is like a custom playlist for the different chapters of content on the disk, the second screen app itself isn’t sending any content to the big screen itself. If you think about how many Gigabytes of data are in a single Blu-ray scene it becomes pretty obvious why the app was designed this way. The first example was the opening scene from the film when a lone engineer sacrifices himself to create new life on an empty planet.

Prometheus Engineers alternate scene
Alternate scene of the Engineers creating life on a planet in Prometheus

This scene was shot, went to post-production but it was later edited out by Scott. You also have the chance to post a screen shot of the moment to Twitter or Facebook. It is a great way to push out the film to a person’s social group, a nice way to get your audience to do some film marketing for you. 

There are some film clips that you can view directly on the second screen. These are small videos such as mini-documentaries that explain some of the features of the film. The Prometheus app sends a message to the Blu-ray player to stop playing the main film so that you can view the shorter video. Once the short video is finished the main movie will resume.


Mini documentaries which seem to be focused on answering a lot of the questions that people had about the original Alien franchise. Unfortunately they didn’t do much to clear up a lot of the confusion that people had about the most recent film in the franchise, Prometheus itself.


Now it’s time to forget about the technology. It is pretty impressive but what does it do to the narrative or storycraft of the film? Up until now the only way you’d experience a different version of a film’s narrative was waiting until the extended director’s cut of a film was released. Often the director’s cut would change the pacing and emphasis on the story itself. These director’s cuts were carefully edited so that the music and visuals would make sense to the viewer. It should still have the effect of luring the viewer into the story for the duration of the film. And that is where the second screen app seems to be most disruptive to one of the major goals of a filmmaker.

Although it might seem that the second screen app allows the viewer to make their own version of the film the reality was much different. The constant scanning back and forth from the large screen to the small screen was distracting. I used the app on my second viewing of Prometheus, if I had tried it on my first viewing I don’t think I would have lasted long. The activity tended to disrupt the flow of the film for me. For anyone who writes screenplays this should be setting off some alarms. In Riding the Alligator, Pen Densham talks about narratives creating a trance for their audience. Having a story that can achieve this is critical to an audience getting lost for 90 minutes in your film. The way dialogue and cinematography combine is critical to getting your audience into that flow. The second screen provided a great way of getting into the production and creative processes of Ridley Scott but they did so at the cost of a cohesive narrative.

One of the tensions with a lot of new technologies and media is whether or not you hand over control of the narrative to the audience. When you want them to have more control over the way the narrative unfolds you sacrifice cohesion of the narrative for control. In other words the audience member becomes more active and can control their experience. In the traditional model the audience is passive, sitting back and experiencing a very cohesive (at least they hope) narrative. The Prometheus second screen app is a great example of how the approaches of control versus cohesion can conflict in a narrative.


Much like the movie, the second screen app was visually striking but not very cohesive. It had a lot of great material and technically had great execution. I think it was great that Fox tried this out and I hope this is just the start of experimenting in the space. I still think that we’re at the hype phase of second screen, there is a lot of over-enthusiasm that is leading to some disillusionment when the audience gets a hold of the results. There is going to be some realism that eventually sinks into the use of the new technology.

Audiences probably aren’t ready for most of the things that second screen technologies can provide. Many of them have very traditional expectations on the movie viewing experience.  Most of them are interested in being a passive participant in a film experience. This isn’t just an old school mentatlity.  If Pen Densham, and a lot of other screenwriters, are right, we’re biologically hardwired to lose ourselves in narrative. It is an experience that lets us escape into a story. Disrupting that experience with a second screen will affect the flow of the story and will probably affect how we judge the success of the story. There will always be some at the bleeding edge that are going to demand these interactive experiences but I believe the majority of viewers are going to give it a pass. This doesn’t mean that the technology has no where to go, it just means we haven’t figured out how to make it part of a narrative experience yet, that might take some time. Hopefully Fox will continue to experiment until the technology matures and reaches its true potential, many companies give when early hype doesn’t meet expectations. This technology still has the potential to have a significant impact.

The Relationship Between Social Media Buzz and TV Ratings | Nielsen Wire

An interesting Neilsen report on the relationship between social media buzz and tv ratings. One of the biggest hurdles facing most Transmedia projects is understanding the value proposition of those kinds of projects. They can incur a substantial production cost for creative property and many traditional producers don’t understand their value. Like all Transmedia you need to pick your channel carefully. In this case the strongest correlation between social media buzz and television ratings was in the younger demographics (12-17 and 18-34). It is an important consideration as any show aiming at a older demographic wouldn’t be likely to have much of an impact on television ratings. Rather than being a failure, it would be an example of wrong transmedia channel for that demographic. That also brings up another good point though, not all programming needs to consider transmedia channels for their property. The hype around transmedia and second screen is considerable at the moment but the most intelligent decision might be to look through the hype and understand if it is even the best tool for your project at all.


The Relationship Between Social Media Buzz and TV Ratings | Nielsen Wire.

Short-Form Video and The Future Of Marketing

There is an increasing belief that the future of most content on the internet is video. The general prediction is that by 2020 internet video will overtake broadcast TV. Cable is expensive and there is an increasing amount of free content available on the internet. This doesn’t mean that all of it will be long format programs that run for an hour. The YouTube generation has increasingly demanding short form video for their entertainment. Some companies have taken that format and started to create marketing campaigns around it. Red Bull has been one of the most successful, not only in building a brand around short videos but being success and telling stories in just a few minutes.

Read more…

Why Short-Form Video Is The Future Of Marketing | Fast Company.

via Why Short-Form Video Is The Future Of Marketing.

How your audience is using their mobile device

In an article on how-to develop apps for film and television I made it clear that the context or situation of app usage can have a strong influence on how users interact with it. If you take a look at the Nielson report here you can see the trends. It basically comes down to this:

  • Tablets70% use them while watching television, 57% while in bed
  • e-readers – 35% use them while watching television, 61% while in bed
  • Smart phones68% use them while watching television, 51% while in bed

Obviously tablets and smart phones are heaviliy integrated into people’s lives while e-readers are mostly about catching up on a book before bed.

This might seem like a strong support for the “second screen” movement that is currently sweeping the app world. This is the idea that you can provide a second screen through a mobile device that will engage users in a creative property beyond the television screen they are already watching. To really understand if this is real or hype you need to look at another study by Nielson. Although this study has daily usage of mobile devices in front of a television at around 40% (quite a bit lower than the previous study). Now, what are they doing with that second screen?

  • 60% – Checking their email
  • 46% – Surfing the web for content unrelated to the television program
  • 30% – Looking up anything related to the TV program
  • 19% – Looking up something they saw on an advertisement on the TV program

So the majority of the time a mobile user is attending to other aspects of their life that have nothing to do with the television program they are watching. Why this is happening is still unknown. It may be that time starved individuals are multi-tasking all the time. It may be that the apps that go with those television programs haven’t met their expectations yet. No matter what the reason it is an important consideration before you invest in app development for a creative property.

New B.C. film centre promises high return, includes forward thinking approaches to education in Transmedia

Capilano University in Vancouver, B.C. has opened up a new $31-million Nat and Flora Bosa Centre for Film and Animation. The facility is 69,000-square-foot facility and comes with $7 million in new equipment. The new facility is designed to align with the university’s new degree program in motion picture production, as well a new two-year visual effects program and the existing animation program.

The new curriculum is going to include marketing and transmedia. Bosa Centre instructor and filmmaker David Hauka said for members of the “transmedia generation,” keeping ahead of the curve in a changing industry means acquiring a variety of skills beyond making movies.

Alberta’s Spillwall and Kairos Transmedia studio connect

It looks like Kairos Transmedia and its partner Spillwall, have inked a new deal with Film Alberta Studio in Edmonton, Alberta. The new partnership will create content aimed at the youth market and combine traditional production with evolving social media technologies. The deal is probably a sign of how most production companies are going to need to evolve and grow.  The partnership combines content production, marketing and distribution in a single company. It is a great idea as a lot of the independent producers I work with on the film side do the content production side very well but haven’t got a clue about the marketing and distribution. It ends with a lot of great content sitting around, looking for a home. The government funding structure for a lot of creative content just perpetuates the model so that trend is likely to continue. The Kairos/ Spillwall deal is an intelligent solution to the problem, it remains to be seen if it is the right solution but it is definitely one to watch. Check out the news release.

BBC Click – Story on Transmedia

A nice summary of transmedia as a strategy for getting narrative noticed by consumers that was done by BBC’s program “Click”. It looks at how digital has opened up a whole new audience for a lot of creative industries. It also addresses the challenges. A lot of creators need to know their storyworld in far more detail than ever before and be able to pick the right media channel to broadcast stories and characters to a potential audience.  Click here.