It sometimes seems we're being deluged with data.
Wave upon wave of news and messages.
Submerged by step counts.
Constantly bailing out to make room for more.
We buy it, surf it, occasionally drown in it
and with modern technology quantify ourselves and everything else with it.
Data is the new currency of our time.
Data has become almost a magic word for...anything.
Crime and lunacy and literacy and religion and...drunkenness.
You name it, somebody was gathering information about it.
It offers the ability to be transformationally positive.
It's, in one sense, just the reduction in uncertainty.
So what exactly is data?
How is it captured, stored, shared and made sense of?
The engineers of the data age are people that most of us have never heard of,
despite the fact that they brought about
a technological and philosophical revolution,
and created a digital world that the mind boggles to comprehend.
This is the story of THE word of our times...
..how the constant flow of more and better data has transformed society...
..and is even changing our sense of ourselves.
I can't believe this is my life now.
So come on in, because the water's lovely.
My name is Hannah Fry,
I'm a mathematician, and I'd like to begin with a confession.
I haven't always loved data.
The truth is mathematicians just don't really like data that much.
And for most of my professional life I was quite happy sitting in
a windowless room with my equations, describing the world around me.
You can capture the arc of a perfect free kick or the beautiful
aerodynamics of a race car.
The mathematics of the real world is clean and ordered and elegant,
everything that data absolutely isn't.
There was one moment that helped to change my mind.
It was in 2011 when I came across a little game that a teenage Wikipedia
user called Mark J had invented.
Now, Mark noticed that if you hit the first link in the main text of any
Wikipedia page and then do the same for the next page, a pattern emerges.
So the page for data, for example,
links from "set" to "maths"
to "quantity" to "property" and then "philosophy",
which after a few more links will loop back onto itself.
Now, the page "egg" ends up in the same place,
and even that famously philosophical boyband One Direction will take you
all the way through to "philosophy",
although you have to go through "science" to get there.
The same goes for "fungi", or "hairspray",
"marmalade", even "mice", "dust" and "socks".
It was a very strange finding and it called for some statistics.
Another Wikipedia user, Il Mare,
wrote a computer program to try and investigate this phenomenon.
Now, he discovered, amazingly, that for almost 95% of Wikipedia pages,
you will end up getting to "philosophy" eventually.
Now, that's pretty cool, but how did it change my mind about data?
Well, the pattern that Mark J discovered and the data that was captured and analysed,
it revealed a hidden mathematical structure,
because Wikipedia is just a network with loops and chains hidden all over the place
and it's something that can be described beautifully using mathematics.
For me this was the perfect example of how there are two parallel universes.
There's the tangible, noisy, messy one,
the one that you can see and touch and experience.
But there's also the mathematical one, where I think the key to our
understanding lies.
And data is the bridge between those two universes.
Our understanding of everything from cities to crime,
global trade,
migration and even disease...
..it's all underpinned by data.
Take this for example.
Rural Wiltshire and a dairy farm
gathering data from its cows wearing
pedometers.
We can't be out here 24-7.
The pedometers help us to have our eyes and ears everywhere.
It turns out when cows go into heat
they move around a lot more than normal.
Constant monitoring of their steps and some background mathematics
reveal the prime time for insemination.
We'll be able to look at the data
and within 24 hours there'll be a
greater chance of getting her in calf.
Data-driven farming is now big business,
turning a centuries-old way of life into precision science.
Pretty much every industry you can think of
now relies on data.
We all agree that we are undergoing
a major revolution in human history.
The digital world replacing the analogue world.
A world based on data,
they are made of codes rather than a world made of biological or physical
data, that is extraordinary.
Why philosophy at this stage?
Because when you face extraordinary challenges,
the worst thing you can do is to get close to it.
You need to take a long run-up.
The bigger the gap, the longer the run-up.
And the run-up is called philosophy.
In the spirit of taking a long run-up,
we'll start with the word itself.
"Data" is originally from the Latin "datum",
meaning "that which is given".
Data can be descriptions...
..counts, or measures...
..of anything...
..in any format.
It's anything that when analysed becomes information,
which in turn is the raw material for knowledge,
the only true path to wisdom.
Look at the data on data.
And before the scientific and industrial revolution,
the word barely gets a look in, in English.
But then it starts to appear in print
as scientists and the state gather,
observe and create more and more of it.
This arrival of the age of data would change everything.
Industrial Revolution Britain.
For Victorians,
booming industry and the growth of major cities were changing both the
landscape and daily life beyond recognition.
Into this scene stepped an unlikely man of numbers,
William Farr,
one of the first people to manage data on an industrial scale.
William Farr had a quite unusual upbringing,
in that he was actually the son of a farm labourer but who had managed to
get a medical education, which was really very unusual for someone of his class.
Farr very quickly became absorbed in the study of statistics.
He was particularly interested, as you might expect for somebody with medical training,
in public health, life expectancy and about causes of death.
For anyone interested in statistics,
there was only one place to be.
Somerset House in London was home to
the General Register Office,
where, in 1839,
Farr found his dream job.
From up there in the north wing, William Farr, the apothecary,
medical journalist and top statistician,
would really rule the roost.
Now, this place was almost like a factory.
Here, they would collect, process and analyse
vast amounts of data.
So in would come the census returns, the records of every single birth,
death and marriage in the country, and out would come the big picture,
the usable information that could help
inform policy and reform society.
I think it's sometimes difficult for us to remember just how little people knew
in the early 19th century about the changes that Britain was going through.
So when Farr did an analysis of population density and death
rate, he was able to show that life expectancy in Liverpool
was absolutely atrocious.
It was far, far worse than the surrounding areas.
This came as a surprise to a lot of people who believed that Liverpool,
a coastal town, was actually quite a salubrious place to live.
At Somerset House, Farr spearheaded a revolution in the systematic
collection of data
to uncover the real picture of this changing society.
Its scale and ambition was described in a newspaper at the time.
"In arched chambers of immense strength and extent
"are, in many volumes, the
"genuine certificates of upwards of 28 million persons
"born into life, married or passed into the grave."
Here, every person was recorded equally.
A revolutionary idea.
"Here are to be found the records of nonentities,
"side-by-side with those once learned in the law
"or distinguished in
"literature, art or science."
But what really motivated William Farr was not just data collection,
it was the possibility that data gathered could be analysed to help
overcome society's greatest ill.
Cholera was probably the most feared of all of the Victorian diseases.
The terrifying thing was that you could wake up in the morning and feel absolutely fine,
and then be dead by the evening.
Between the 1830s and the 1860s,
tens of thousands died in London alone.
The control of infectious diseases like cholera,
which no-one fully understood,
became the greatest public-health issue of the time.
However great London might have looked back then,
it would have smelled absolutely terrible.
At that point, the Victorians didn't have really a great way of disposing
of human waste, so it would have flowed down the gutters
into open sewers
and out into the Thames.
Now, the city smelt so bad that it
was pretty plausible that the foul air
was responsible for carrying the disease.
Farr collected a huge range of data during each
cholera outbreak to try to
identify what put people most at risk from the bad air.
He used income-tax data to try and measure the affluence of the different
boroughs that were affected by cholera.
He asked his friends at the Royal Observatory to provide data on the
temperature and climatic conditions.
But the one that he thought was most convincing was about the topography.
It was about the elevation above the Thames.
Using the data, Farr suggested a mathematical law of elevation.
Its equations described how cholera mortality falls the higher you live
above the Thames.
Now, he published his report in 1852,
which the Lancet described as one of
the most remarkable productions of
type and pen in any age and country.
The only problem was that Farr's work,
although elegant and meticulous,
was fundamentally flawed.
Farr stuck to the prevailing theory
that cholera was spread by air.
Such is the power of the status quo.
But in 1866,
5,500 people died in just one square mile of London's East End,
and that data made Farr change his mind.
When Farr came to write his next report,
the data told a different story
which proved the turning point in
combating the disease.
The common factor among those who died was not elevation or air
but sewage-contaminated drinking water.
With this new report,
Farr may seem to have contradicted
much of his own work, but I think that
this is the perfect example of what data can do.
It provides that bridge essential to scientific discovery,
from theory to proof, problem to solution.
Good data, even in huge volumes,
does not guarantee that you will arrive at the truth.
But, eventually, when the weight of the data tips the balance,
even the strongest-held beliefs can be overcome.
Of course, it was the weight of the data itself which,
with the dawn of the 20th century,
was becoming increasingly hard to manage.
Data stored long form in things like census ledgers could take the best
part of a decade to process,
meaning the stats were often out of date.
When you're dealing with figures like these, it's one thing.
But when you're counting the population like this it's quite a different matter.
A deceptively simple solution got what's now called the
information revolution under way,
encoding data as holes punched in cards.
These cards are passed over sorting machines,
each of which handles 22,000 cards a minute.
By the 1950s, data processing and simple calculations were
routinely mechanised,
laying the groundwork for the next generation of
data-processing machines.
They would be put to pioneering work
in a rather unlikely place.
In a grand London dining hall, a group of men and women,
many in their 80s and 90s, have gathered for a special work reunion.
At its peak, their employer, J Lyons,
purveyor of fine British tea and cakes,
had hundreds of tea shops nationwide.
There are hundreds of items of food.
All these in a varying quantity each day are delivered to a precise
timetable to the tea shops.
These people aren't former J Lyons bakers or tea-shop managers.
They were hired for their mathematical skills.
Lyons had a huge amount of data which has to be processed,
often very low-value data.
So, for example, the transaction from a tea shop
would be a cup of tea.
But each one had a voucher and had to be recorded,
and had to go to the accounts
for business reasons and for management reasons.
Every calculation you did, not only you had to do it twice,
but you had to get it checked by someone else as well.
The handling of these millions and millions of pieces of data,
the storage of that data, are the key of the business problem.
The Lyons team took the world by surprise when, in 1951,
they unveiled the Lyons Electronic Office,
or Leo for short.
At this point, only a handful of computers existed,
and they were used solely for scientific and military research,
so a business computer was a radical reimagining of what this brand-new
technology could be for.
Each manageress has a standing order depending on the day of the week.
She speaks by telephone to head office, where her variations are
taken quickly onto cards.
What the girl hears, she punches.
The programme is fed first,
laying down the sequence for the multiplicity of calculations Leo will perform.
It was the first
opportunity to process large volumes
of clerical work,
take all the hard work out of it,
and put it on an automatic system.
Before Leo, working out an employee's pay
took an experienced clerk eight minutes,
but with Leo that dropped to an astonishing one and a half seconds.
It was all so exciting
because we were breaking new ground the whole time.
Absolutely everything which we did has never been done before.
By anybody anywhere.
I don't think we realised the kind of transformation we were part of.
The post-war years saw a boom in the application of this new computing
technology.
Leo ran on paper, tape and cards,
but soon machines with magnetic tape and disks were developed,
allowing for greater data storage and faster calculations.
As more businesses and institutions adopted these new machines,
application of mathematics to a whole host of new,
real-world challenges took off.
And the word "data" went from relatively obscure to ubiquitous.
"Data" has become almost a magic word for anything.
The truth is that it is a kind of interface
today between us and the rest of the world.
In fact, between us and ourselves,
we understand our bodies in terms of data,
we understand society in terms of data,
we understand the physics of the universe in terms of data.
The economy, social science, we play with data,
so essentially it is what we interact with
most regularly every day.
Data underpins all human communication,
regardless of the format.
And it was the desire to communicate effectively and efficiently that led
to one of the most important academic papers of the 20th century.
A mathematical theory of communication
has justifiably been called the Magna Carta for the information age.
It was written by a very young and bright employee of Bell Laboratories,
the American centre for telecoms research that was founded by one of
the inventors of the telephone, Alexander Graham Bell.
Now, this paper was written by Claude Shannon in 1948 and it would
effectively lay out the theoretical framework for the data revolution
that was just beginning.
Those that knew him described
Shannon as a lifelong puzzle solver and inventor.
To define the correct path it registers the information in its memory.
Later, I can put him down in any part of the maze that he has already explored
and he will be able to go directly to the goal without making a single
false turn.
During World War II he worked on data-encryption systems,
including one used by Churchill and Roosevelt.
But at Bell Labs,
Claude Shannon was trying to solve the very civilian problem of noisy
telephone lines.
# There's a call, there's a call
# There's a call for you
# There's a call on the phone for you. #
In that analogue world of 20th-century phones,
your speech was converted into an electrical signal using a handset
like this and then transmitted down a series of wires.
The voice signals would travel along the wire,
be detected by the receiver at the other end and then be converted back
into sound waves to reach the ear of whoever had picked up.
The problem was,
the further the electrical signal travelled down the line,
the weaker it would get.
PHONE LINE CRACKLES Eventually you couldn't
even hear the conversation for the amount of noise on the line.
And you could boost the signal but it would mean boosting the noise, too.
Shannon's genius idea was just as simple as it was beautiful.
The breakthrough was converting speech
into an incredibly simple code.
ON PHONE: Hello?
First the audio wave is detected, then sampled.
Each point is assigned a code of ones and zeros
and the resulting long string of digits can then be sent down
the wire with the zeros as brief low-voltage signals
and ones as brief bursts of high voltage.
From this code, the original audio can be cleanly reconstructed and
regenerated at the other end.
ON PHONE: Hello?
Shannon was the first person to publish the name
for these ones and zeros,
the smallest possible pieces of information,
and they are called bits or binary digits,
and the real power of the bit and the mathematics behind it
applies way beyond telephones.
They offered a new way for everything,
including text and pictures, to be encoded as ones and zeros.
The possibility to store and share data digitally in the form of bits
was clearly going to transform the world.
If anyone has to be identified as
the genius who developed the
foundational science of mathematics
for our age, that is certainly Claude Shannon.
Now, one thing has to be clarified,
the theory developed by Shannon is about data transmission and it has
nothing to do with meaning, truth, relevance,
importance of the data transmitted.
So it doesn't matter whether the zero and one represent
an answer to, "Heads or tails?", or to the question, "Will you marry me?",
for a theory of information is data anyway and if it is a 50-50 chance
that you will or will not marry me or that it is heads or tails,
the amount of information, the Shannon information,
communicated is the same.
Shannon information is not information like you or I might think about it.
Encoding any and every signal using just ones and zeros is a pretty
remarkable breakthrough.
However, Shannon also came up with a revolutionary bit of mathematics.
That equation there is the reason you can fit an entire HD movie on a
flimsy bit of plastic or the reason why you can stream films online.
I'll admit, it might not look too nice, but...
don't get put off yet, because I'm going to explain how this equation
works using Scrabble.
Imagine that I created a new alphabet
containing only the letter A.
This bag would only have A tiles inside it
and my chances of pulling out an A tile would be one.
You'd be completely certain of what was going to happen.
Using Shannon's maths,
the letter A contains zero bits of what's called Shannon information.
Let's say then I got a little bit more creative, but not much,
and had an alphabet with two letters, A and B,
and equal numbers of both in this bag.
Now my chances of pulling out an A are going to be a half
and each letter contains one bit of Shannon information.
Of course, when transmitting real messages,
you'll use the full alphabet.
But English,
as with every other language, has some letters that are used
more frequently than others.
If you take a quite common letter like H,
which appear about 5.9% of the time,
this will have a Shannon information of 4.1 bits.
And incidentally, a Scrabble score of four.
Of course, there are some much more exotic and rare letters,
like Z, for instance, which appears about 0.07% of the time.
That gives it 10.5 bits
and Scrabble score of ten.
Bits measure our uncertainty.
If you're guessing a three-letter word and you know this letter is Z,
it gives you a lot of information about what the word could be.
But if you know it's H,
because it is a more common letter with less information,
you're more uncertain about the answer.
Now if you wrap up all that uncertainty together,
you end up with this, the Shannon entropy.
It's the sum of the probability of
each symbol turning up times the
number of bits in each symbol.
And this very clever bit of insight
and mathematics means that the code
for any message can be quantified.
Not every letter, or any other signal for that matter,
needs to be encoded equally.
The digital code behind a movie like this one of my dog, Molly,
for example,
can usually be compressed by up to 50%
without losing any information.
But there's a limit.
Compressing more might make it easier to share or download,
but the quality can never be the same as the original.
DOG BARKS
You can't really overstate the impact that Shannon's work
has had, because without it we wouldn't have JPEGs or Zip files
or HD movies or digital communications.
But it doesn't just stop there, because while the mathematics of
information theory doesn't tell you anything about the meaning of data,
it does begin to open up a possibility
of how we can understand ourselves
and our society, because pretty much anything and everything can be
measured and encoded as data.
We say that signals flow through human society,
that people use signals to get things done,
that our social life is, in many ways,
the sending back and forth of signals.
So what is a signal?
It's, in one sense, just the reduction in uncertainty.
What it means to receive a signal is to be less uncertain than you were
before and so, another way to think of measuring or quantifying signal
is in that change in uncertainty.
Using Shannon's mathematics to quantify signals
is common in the world of complexity science.
It's rather less familiar to historians.
I love maths, I love its precision, I love its beauty.
I absolutely love
its certainty, and that, Simon can bring that mathematical worldview,
that mathematical certainty to what I work with.
The reason behind this remarkable marriage between history and science
is the analysis of the largest single body of digital text
ever collated about ordinary people.
It's the Proceedings of London's Old Bailey,
the central criminal court of England and Wales,
which hosted close to 200,000 trials between 1674 and 1913.
There are 127 million words of everyday speech
in the mouths of orphans and women and servants and ne'er-do-wells,
of criminals, certainly,
but also people from every rank and station in society.
And that made them unique.
What's exciting about the Old Bailey and the size of the dataset,
the length and magnitude of it,
is that not only can we detect a signal,
but we are able to look at that signal's emergence over time.
Shannon's mathematics can be used to capture the amount of information in
every single word,
and like the alphabet, the less you expect a word,
the more bits of information it carries.
Imagine that you walk into a courtroom
at the time and you hear a single word,
the question we ask is how much information does that word carry
about the nature of the crime being tried?
You hear the word "the".
It's common across all trials and so gives you no bits of information.
Most words you hear are poor signals of what's going on.
But then you hear "purse".
It conveys real information.
Then comes "coin",
"grab" and "struck".
The more rare a word, the more bits of information it carries,
the stronger the signal becomes.
One of the clearest signals that we see in the Old Bailey,
one of the clearest processes that comes out,
is something that is known as the civilising process.
It's an increasing sensitivity to, and attention to, the
distinction between violent and nonviolent crime.
If, for example, somebody hit you and stole your handkerchief,
in the 18th-century context, in 1780,
you would concentrate on the handkerchief.
More worried about a few pence worth of dirty linen than the fact that
somebody just broke your nose or cracked a rib.
The fact that 100 years later, by 1880,
every concern, every focus, both in terms of the words used in court,
but also in terms of what people were brought to court for,
focus on that broken nose and that cracked rib,
speaks to a fundamental change in how we think about the world
and how we think about how social relations work.
Look at the strongest word signals for violent crime across the period.
In the 18th century, the age of highwaymen,
words relating to property theft dominate.
But by the 20th century,
it's physical violence itself and the impact on the victim
that carry the most weight.
That notion that one can trace change over time
by looking at language and how it's used,
who deploys it in what context,
that I think gives this kind of work its real power.
There are billions of words, there's all of Google Books,
there's every printed newspaper,
there is every speech made in Parliament,
every sermon given at most churches.
All of it is suddenly data and capable of being analysed.
The rapid development of computers in the mid 20th century
transformed our ability to encode, store and analyse data.
It took a little longer for us to work out how to share it.
This place is home to one of the most
important UK scientific institutions,
although it's one you've probably never heard of before.
But since the 1900s, this place has advanced all areas of physics,
radio communications, engineering, materials science, aeronautics,
even ship design.
NPL, the National Physical Laboratory,
in south-west London is where the first atomic clock was built
and where radar and the Automatic Computer Engine, or Ace, were invented.
The Ace computer was the brainchild of Alan Turing,
who came to work here right after the Second World War.
Now, Turing's contributions to the story of data are undoubtedly vast,
but more important for our story is another person who worked here with
Turing, someone who arguably is even less well known than this place,
Donald Davies.
Davies worked on secret British nuclear weapons research during the war...
..later joining Turing at NPL,
climbing the ranks to be put in charge of computing in 1966.
As well as the new digital computers,
Davies had a lifelong fascination with telephones and communication.
His mother had worked in the Post Office telephone exchange,
so even when he was a kid,
he had a real understanding of how these phone calls were routed and
rerouted through this growing network,
and that was the perfect training for what was to follow.
What was Donald Davies like, then?
He was a super boss because he was very approachable.
Everybody realised he'd got huge intellect but not difficult with it.
Very nice guy.
Davies' innovation was to develop, with his team,
a way of sharing data between computers, a prototype network.
Donald had spotted that there was a need to connect computers together
and to connect people to computers, not by punch cards or
paper tape or on a motorcycle, but over the wires,
where you can move files or programs, or
run a program remotely on another computer,
and the telephone network is not really suited for that.
In the pre-digital era,
sending an encoded file along a telephone line meant that the line
was engaged for as long as the transmission took.
So the opportunity here was because we owned the site,
78 acres with some 50 buildings, we could build a network.
Davies' team sidestepped the telephone problem
by laying high-bandwidth data cables before instituting a new way of
moving data around the network.
The technique he came up with was packet switching,
the idea being that you take whatever it is you're going to send,
you chop it up into uniform pieces, like having a standard envelope,
and you put the pieces into the envelope and you post them off and they go
separately through the network and get reassembled at the far end.
To demonstrate this idea,
Roger and I are convening NPL's
first-ever packet-switching data-dash...
..which is a bit more complicated than your average sports-day event.
The course is a data network.
There are two computers, represented
here as the start and finish signs.
Those computers are connected by a
series of network cables and nodes.
In our case, cables are lines of
cones and the connecting nodes are Hula Hoops.
Having built it, all we need now are some willing volunteers.
And here they are.
NPL's very own apprentices.
So welcome to our packet-switching sports day.
We've got two teams, red and blue.
'Both teams are pretending to be data
'and they're going to have to race.'
You're going to start over there
where it says "start", kind of obvious,
and you're trying to get through to the end as quickly as you possibly
can. You can't just go anywhere,
you have to go through these hoops to get to the finish line,
these little nodes in our network.
You're only allowed to travel along the lines of the cones,
but only if there's nobody else along that line.
All clear? OK, there is one catch.
All of you who are in the red team,
we are going to tie your feet together.
So you've got to travel round our network as one big chunk of data.
Those of you who are in blue, you are allowed to travel on your own,
so it's slightly easier.
'The objective is for both teams to deposit their beanbags
'in the goal in the right order, one to five.'
EXCITED CHATTER
Get in the hoop! Get in the hoop!
Bring out your competitive spirit here.
We've got packets versus big chunks of data.
I'm going to time you. Everyone ready?
OK, over to you, Roger.
TOOT!
Remember, you can't go down the route until it's clear.
'The red and blue teams are exactly the same size,
'let's say five megabytes each.
'But their progress through the network is clearly very different.'
THEY LAUGH
OK, blues, you took 13 seconds, pretty impressive.
Reds, 20 seconds.
That's a victory for the packet switchers.
Well done, you guys! Well done, you guys.
The impact that packet switching has had on the world, I mean,
it sort of came from here and then spread out elsewhere.
It did indeed, we gave the world packet switching, and the world,
of course, being America, they took it on and ran with it.
This little race, Donald Davies' packet switching,
was adopted by the people that would go on to build the internet,
and today, the whole thing still runs on this idea.
Let's say I want to e-mail you a picture of Molly.
First, it will be broken up into over 1,000 data packets.
Each one is stamped with the address of where it's from and where it's
going to, which routers check to keep the packets moving.
Regardless of the order they arrive,
the image is reassembled, and there she is.
This is quite a cool thing, right,
that you've got one of the original creators
of packet switching right here and you can ask him...
Every time you're like... Well, do anything, really.
"Why is my internet running so slowly?"
- THEY LAUGH - Don't ask me!
We've come a very long way in just a few decades.
Around 3.4 billion people now have access to the internet at home
and there are around four times the number of phones
and other data-sharing devices online,
the so-called Internet of Things.
Just by being alive in the 21st century
with our phones, our tablets, our smart devices, all of us are
familiar with data.
Really embrace your inner nerd here, because every time you wander around
looking at your screen, you are gobbling up and churning out
absolutely tons of the stuff.
Our relationship with data has really changed -
it's no longer just for specialists, it's for everyone.
There's one city in the UK that's putting the sharing and real-time
analysis of data at the heart of everything it does -
Bristol.
Using digital technology, we take the city's pulse.
This data is the route to an open, smart, liveable city,
a city where optical, wireless and mesh networks
combine to create an open, urban canopy of connectivity.
Taking the pulse of the city under a canopy of connectivity
might sound a bit sci-fi, or like something from a broadband advert.
But if you just hold on to your cynicism for a second,
because Bristol are trying to build a new type of data-sharing network for its citizens.
There's a city-centre area which now has next-generation
or maybe the generation after next
of superfast broadband and then that's coupled to a Wi-Fi network, as well.
The question is, what can you do with it?
We would have a wide area network of very simple Internet of Things
sensing devices that just monitor a simple signal like air quality
or traffic queued in a traffic jam.
Once you've got all this network infrastructure,
you can get an awful lot, a really huge amount of data
arriving to you in real time.
What's happening here is a city-scale experiment
to try and develop and test
what's going to be called the programmable city of the future.
It relies on Bristol's futuristic network,
vast amounts of data from as many sensors as possible
and a computer system that can simulate
and effectively reprogram the city.
The computer system can intervene.
It could reroute traffic and we can actually radio out to individuals,
so maybe they get a message on their smartphone
or perhaps a wrist-mounted device,
saying, "If you have asthma, perhaps you should get indoors."
Once you create that capacity for anything and everything in the city
to be connected together,
you can really start to re-imagine how a city might operate.
We are starting to experiment with driverless cars and, in order for
driverless cars to work,
they have to be able to communicate with the city infrastructure.
So, your car needs to speak to the traffic lights,
the traffic lights need to speak to the car, the cars to speak to each other.
All of that requires a completely different set of infrastructure.
Of course, as the amount of data a city can share grows,
the computing power needed to do something useful with it must grow, too.
And for that, we have the cloud.
For example, imagine trying to analyse all of Bristol's traffic data,
weather and pollution data on your home computer.
It could take a year.
Well, you could reduce that to a day by getting 364 more computers,
but that's expensive.
A cheaper option is sharing the analysis with other computers over the internet,
which Google worked out first, but they published the basics
and now free software exists to help anyone do the same.
Big online companies rent their spare computers
for a few pence an hour.
So, now anyone like me or you
can do big data analytics quickly for a few quid.
Such computing power is something we could never have dreamt of
just a few years ago, but it will only fulfil its potential
if we can share our own data in a safe and transparent way.
If Bristol Council wanted to know where your car was at all times
but could use that information to sort of minimise traffic jams,
how would you feel about something like that?
Er, I'm not sure if I'd particularly like it.
I think it is up to me where I leave my car.
I understand the idea of justifying it with all these great other ideas,
but I still probably wouldn't like it very much.
If they are using it for a better purpose, then yeah,
but one should know how they are using it and why they'll be using it, for what purpose.
I'd like to imagine a world in which all the data that was retained
was used for the greater good of mankind,
but I can't imagine a circumstance like that
in the world that we have today.
We live in a modern society, where if you don't
let your data out there, not in the public domain,
but in a secure business domain,
then you can't take part in society, really.
Unsurprisingly, people are pretty wary about what happens to their data.
We need to be careful that civil liberties are not eroded,
because otherwise the technology is likely to be rejected.
I think it's an area where us as a society have yet to sort of fully
understand what the correct way forward is
and therefore it is very much a discussion.
It's not a lecture, it's not a code,
it's one where we are co-producing and co-forming these sorts of rules
with people in the city, in order to sort of help us work out what the
right and wrong things to do are.
It will be intriguing to watch Bristol grapple with the technological
and ethical challenges of being our first data-centric city.
In all these contexts, Internet of Things...
..new forms of health care, smart cities,
what we're seeing is an increase in transparency.
You can see through the body, you can see through the house,
you can see through the city and the square, you can see through society.
Now, transparency may be good.
It's something that we may need to handle carefully in order to extract
the value from those data to improve
your lifestyle, your social interactions,
the way in which your city works and so on.
But it also needs to be carefully handled, because it's touching
the ultimate nerve of what it means to be human.
So how much data should you give away?
Traffic management is one thing but when it comes to health care,
the stakes, the risks and benefits are even higher.
And in Bristol, with a project called Sphere,
they're pushing the boundaries here, too.
The population is getting older, and an ageing population needs
more intense health care, but it's very difficult to pay for that health care
in institutions, paying for nurses and doctors.
So, the key insight of the Sphere team was that it's now possible
to arrange, in a house, lots of small devices
where each device is monitoring a simple set of signals
about what's going on in that house.
There might be monitors for your heart rate or your temperature,
but there might also be monitors that notice, as you're going up and down stairs,
whether you're limping or not.
They've invited me to go and spend a night in this
very experimental house, but unfortunately, I'm not allowed to tell you where it is.
The project is a live-in experiment and will soon roll out
to 100 homes across Bristol.
It's a gigantic data challenge, overseen by Professor Ian Craddock.
So, that's one up there, then?
Yes, that's one of the video sensors
and we have more sensors in the kitchen.
We have another video camera in the hall and some environmental sensors,
and a few more in here.
The house can generate 3-D video,
body position, location and movement data
from a special wearable.
How much data are you collecting, then?
So, when we scale from this house to 100 houses in Bristol,
in total we'll be storing over two petabytes of data for the project.
Lord. So, on my computer at home, I don't even have a terabyte hard drive
and you're talking about 20,000 of those.
Yes. I mean, you know, the interaction of people with their environment
and with each other is a very complicated and very variable thing
and that's why it is a very challenging area,
especially for data analysts,
machine learners, to make sense of this big mass of data.
I'm happy to find out that the research doesn't call
for cameras in the bedroom or bathroom,
but I do have to be left entirely on my own for the night.
The very first thing I'm going to do is pour myself a nice bloody big
glass of wine. There we go.
So, that nice glass of wine that I'm enjoying isn't completely guilt-free,
because I've got to admit to it to the University of Bristol.
I have to keep a log of everything I do,
so that the data from my stay can be labelled with what I actually got up to.
In this way, I'll be helping the process of machine learning,
teaching the team's computers how to automatically monitor things like
cooking, washing and sleeping,
signals in the data of normal behaviour.
In the interests of science.
'I was also asked to do some things that are less expected.'
Oh! I spilled my drink.
'The team need to learn to detect out-of-the-ordinary behaviour, too,
'if they want to, one day, spot specific signs of ill health.'
Right, I'm going to run this back to the kitchen now.
It's a fairly strange experience.
I think the temperature sensors, the humidity sensors,
the motion sensors, even the wearable
I don't have a problem with at all.
For some reason the body position is the one that's getting me.
On the flipside, though, I would go absolutely crazy to have this data.
This is the most wonderful... My goodness me.
Everything you could learn about humans. It would be so brilliant.
One thing I wanted to do was to do something completely crazy
just to see if they can spot it in the data. Just to kind of test them.
OK, ready?
I can't believe this is my life now.
'Anyone can get the data from my stay online if they fancy trying to find
'my below-the-radar escape.
'The man in charge of machine learning, Professor Peter Flach,
'has the first look.'
Between nine and ten, you were cooking.
Correct.
Then you went into the lounge. You had your meal in the lounge.
You know what? I ate on the sofa.
And you were watching crap television.
I was watching crap television?
I've been found out.
We didn't switch the crap-television sensor on. That's not on here, but OK.
- So, you were in the lounge sort of until 11:30. - Correct.
Then you went upstairs, there's a very clear signal here.
- And then, from then on, there isn't a lot of movement. - I was in bed.
So, I guess you were in bed.
Sleeping.
Normal activities, like cooking or being in bed, are relatively
straightforward to spot.
But what about the weird stuff?
This is yesterday, again.
I can see it. I can see the moment.
- You can see the moment? - I can see it, yeah.
There's something happening here which is sort of rather quick.
You've been in the lounge for quite a while and then, suddenly,
there's a brief move to the kitchen here
and then very quick cleaning up in the lounge.
- I wasted good wine on this experiment. - Good wine?
Humans are extraordinarily good at spotting most patterns.
For machines, the task is much more challenging,
but, once they've learned what to look for,
they can do it tirelessly.
I suppose, in the long run,
if you are going to scale this up to more houses,
you can't have people sifting through these graphs trying to find...
I mean, you have to train computers to do them.
You have to train computers to do them. One challenge that we are facing is that our models,
our machine learning classifiers and models, need to be robust
against changes in layout, changes in personal behaviour,
changes in the number of people that are in a house.
And maybe we are wildly optimistic about what it can do,
but we are in the process of trying to find out what it can do,
at what cost, at what...
..invasion into privacy, and then we can have a discussion about whether,
as a society, we want this or not.
If this type of technology rolls out,
machines will be modelling us in mathematical terms
and intervening to help keep us healthy in real time -
and that's completely new.
It's true that our fascination with machine, or artificial, intelligence
is as old as computers themselves.
Claude Shannon and Alan Turing both explored the possibilities
of machines that could learn.
But it's only today,
with torrents of data and pattern-finding algorithms,
that intelligent machines will realise their potential.
You'll hear a lot of heady stuff about what's going to happen when we
mix big data with artificial intelligence.
A lot of people, understandably, are very anxious about it.
But, for me, despite how much the world has changed,
the core challenge is the same as it always was.
It doesn't matter if you are William Farr in Victorian London trying to
understand cholera or in one of Bristol's wired-up houses,
all you're trying to do is to understand patterns in the data
using the language of mathematics.
And machines can certainly help us to find those patterns,
but it takes us to find the meaning in them.
We should be worried about what we're going to do with these smart technologies,
not about the smart technologies in themselves.
They are in our hands to shape our future.
They will not shape our futures for us.
In the blink of an eye, we have gone from a world where data,
information and knowledge belonged only to the privileged few,
to what we have now, where it doesn't matter if you're trying to work out
where to go on holiday next or researching the best cancer treatments.
Data has really empowered all of us.
Now, of course, there are some concerns about big corporations
hoovering up the data traces that we all leave behind in our everyday lives,
but I, for one, am an optimist as well as a rationalist
and I think that if we can marshal together the power of data,
then the future lies in the hands of the many and not just the few.
And that, for me, is the real joy of data.
MUSIC: Good Vibrations by The Beach Boys
Subtitles by Ericsson