TransPose algorithm writes the soundtrack to novels

in books on 2014-03-20 21:00 (#3GN)

The automatic analysis of sentiment in text is fast changing the way we interpret and interact with words. On Twitter, for example, researchers have begun to gauge the mood of entire nations by analysing the emotional content of the tweets people generate.

In the same way, other researchers have started to measure the emotional temperature of novels by counting the density of words associated with the eight basic emotions of anticipation, anger, joy, fear, disgust, sadness, surprise and trust.

The next step, obviously was to write an algorithm that measures that emotional temperature throughout full length novels, and generate a musical soundtrack to accompany the text.

Interesting research, or pointless? Time will tell, but energy and money is increasingly being used to judge moods and allow software or equipment to react accordingly. The true value of this research might not become apparent until sometime in the future, even if it is only used so your phone can sing you a soothing song after you receive a nasty email from your boss.

there is something different (Score: 3, Insightful)

by rocks@pipedot.org on 2014-03-20 22:42 (#QJ)

about live performances and recorded performances even by the same musicians. While I think it is really cool that we can automate music making, I still believe for now that there will always be something vital and imperfect about natural and live music-making.

Part of it is that while there is a standard vocabulary for associating particular musical patterns with emotions, we are also always playing with these patterns to shift our associations. So I'm wondering if the algorithmic approach will be able to have this kind of exploration and growth element added and, if so, how?

Re: there is something different (Score: 3, Interesting)

by danieldvorkin@pipedot.org on 2014-03-21 03:04 (#QN)

I think you're right, but I also think a lot of the people looking for soundtracks won't care. Big Hollywood studios will always be able to pay someone like John Williams to compose the score for feature films, but a TV producer on a tight budget who wants distinctive music for this week's episode might weclome software that would compose music from the script.

Presumably the algorithm can be altered as musical tastes change. It's interesting how a lot of movies and TV shows from, say thirty or more years ago don't look nearly as out-of-date as they sound .

Re: there is something different (Score: 4, Funny)

by zafiro17@pipedot.org on 2014-03-21 09:31 (#QS)

I'm skeptical about algorithms being able to do something as subtly complex as this. As for Hollywood movies, aren't they already being written and acted by computers and robots anyway? (Keanu Reeves, whoever wrote the Diehard series: I'm looking at you). I mean, same crap, over and over and over. I think the entire romantic comedy genre has been coopted by a bit of software only marginally more complex than emacs' psychiatrist.

Re: there is something different (Score: 2, Interesting)

by rocks@pipedot.org on 2014-03-21 12:21 (#QZ)

I was thinking that one of the big challenges for scores accompanying Hollywood movies is getting climaxes in the music to match the cutting/editing of the film sequences and action. I wonder if right now film composers actually use a hybrid system whereby they develop the theme, variations and scores for the music, but they use algorithms to find best fits for the music to the video?

Re: there is something different (Score: 3, Informative)

by zafiro17@pipedot.org on 2014-03-21 15:39 (#R8)

Why not just line up the soundtrack so the climax falls where you want it, and then work backwards the length of time you require and add a fade in? Place climax, include 3 seconds earlier plus 0.5 second fade in. Badda boom, badda bing.

Re: there is something different (Score: 2, Interesting)

by rocks@pipedot.org on 2014-03-21 15:47 (#RA)

Good point, I've used that approach in some simple videos of my own, in fact.

I think I also remember watching a documentary about John Williams where he conducts the orchestra while watching the final film to actually lay down the music in sync with the video, but obviously this takes more financial muscle to pull off.

Re: there is something different (Score: 1)

by sleazyridr@pipedot.org on 2014-03-24 19:20 (#SJ)

I couldn't agree with you more. I love a lot of the computer generated music from a technological perspective, but there is no substitute to a man with a guitar who can read the emotion of the audience and change they way he plays to keep them all hyped up.