April 20, 2018 at 04:04PM
To say that AI is big data is to overstate things a bit. And yet, without big data, AI wouldn’t be where it is today. In the last few decades, the two technologies have advanced in lock-step. Largely because without big data, however clever the AI programmers were, they couldn’t get past the theoretical stage.
Mainly, this is down to what big data is used for. Through data, it is possible to train AI and thereby give them the opportunity to learn things. The more data is available, the more opportunities there are to learn and the more nuanced the lessons become.
Different ways to process data
This is fundamentally different from how we learn things, of course. In comparison to modern technology, we aren’t very good at processing data. According to a MIT study, we only process data at 60 bits per second. Instead, we use rules of thumb, assumptions and pattern recognition to quickly draw conclusions from the limited amount of data we can actually process.
Computers generally work in reverse. They first need to crunch huge amounts of data and only at the end are able to draw conclusions. After that, those conclusions can then be turned into operative rules which can be let loose on further data sets.
They are therefore learning the relationships between different attributes. Naturally, as time progresses one step can follow the next, with a framework which is established today capable of offering a ratchet by which future AI software can then be given a head start over what we’re building right now.
In effect, AI will teach itself the heuristics that we already have inbuilt but then at a hugely increased velocity from how evolution did it. That means, the data we’re providing needs to be exponentially greater in a given time frame for the right lessons to be learned.
The relationship is just as important in reverse
Of course, it isn’t just AI that benefits from big data. The relationship holds true in reverse as well. As AI gets better at understanding the numbers, they will be able to tease relationships out of dataset which we ourselves would never have found.
Here, the non-biased way that AI can go to work is a huge advantage. It is simply going to look at what numbers correlate, whether it makes sense or not. In fact, the word ‘sense’ doesn’t have any meaning for an AI. They don’t (yet) engage in this kind of higher level thinking.
For us it’s different. Our rules of thumb are in built and can’t easily be turned off. Hence the large number of biases that color how we interpret the world.
Better together
Ultimately, that means that it’s all greater than the sum of its parts. And with that, I don’t just mean AI and big data, but also the human element. Each brings a unique element to the table that unlocks the power of the other two. Humans produce the big data, AI explores the relationships between the numbers and people can then in turn apply their higher-level thinking to drawing conclusions about what that will all mean.
Take out one of the pieces, and the relationship largely breaks down. We struggle to make sense of big data sets. AI can’t yet produce reasons why. And without big data, we can’t give AI the meat it needs to establish the relationships out there.
So really, it’s not that surprising that AI was so long in coming. The pieces simply didn't place before.