If the recent UK general election taught us anything, it’s that data based on intent is about as reliable as a Greek IOU.

The behavioural findings of the NOP/ Mori exit poll nailed the result as the polling stations closed because people were collared in the moment. But six weeks of prior polls fooled pundits and politicos alike.

We also found out – if we didn’t know it already – that social media is an echo chamber.

Social sentiment seemed emphatically pro-Labour round my way; people stayed schtum about their intentions only to sheepishly admit after the fact, when a stampede to the Tories became apparent.

Data can, however, help people decide what telly programmes to make. Netflix algorithms revealed that millions of us enjoyed watching anything with Kevin Spacey, the original British production of House Of Cards and the films of David Fincher – so a successful remake was born.

Predictions are possible, as long as you have data that draws on real behaviour. With the benefit of a billion breadcrumb trails, digital tech gives creatives the ability to see hitherto hidden proclivities in the audience and make entertainment that appeals.

So we’re on the cusp of a massive influx of data into storytelling, held back only by proper caution over privacy.

Planners have used data to lead creative ideas for years. But most good planners know data often lies. As Mark Earls exposed in Herd, people don’t know what they will do, and misremember what they have done.

We act because of others and aren’t the rational, autonomous beings of our fantasies. Digital consumption data reveals that we are not the paragons of virtue and good taste we’d like to think we are.

If a gym’s clients claim to go three times a week but our data sources tell us most of them don’t make it twice, it’s clear that our creative approach to those people should play not on “because you’re active” but “because you want to be active”.

Likewise, if a bank reckons it can address what keeps people awake at night, it won’t find out by asking the question in the daytime. Search data integrated with insights into mood from social feeds and behavioural data will reveal more than a room of people bribed with £50 and the promise of free snacks.

In both cases, the increased power of data does not mean creativity loses importance; the strategy just provides a more defendable jumping-off point.

Data and creativity are one, and you can really see it in service design and the world of tech start-ups, where the best data scientists are as creative as the leap-of-faith entrepreneurs. Data, after all, doesn’t give you the answers so much as show you where to look for them. Insight is the child of rigour and imagination.

My fear is that originality in the long run might suffer. Data makes the ideas economy more efficient, but is there a law of diminishing returns here? It’s not hard to picture a data-driven, mash-up-fixated commissioning model moving in ever decreasing circles.

And TV ads certainly seem less surprising than ever. But, given the right data and the right talent, the outcome needn’t be shit. Commercial creativity can benefit just as the “over the top” broadcasters have.

We just need faith in both “sides”. We must use data to hedge our bets, but not be afraid to make them.

In TV land, there’s the odd worrying portent. Last autumn, Netflix announced that it is backing four new Adam Sandler comedies. This is democracy at work. With entertainment, as with politics, you get the content your data deserves.