Deep learning and the Era of Artificial Intelligence

> Unsupervised Learning > Home

By

Nicholas Hindley BSc(Hons), BA

PhD Student, ACRF Image X Institute

The birth of Artificial Intelligence (AI)

It was 1956. “My Fair Lady” had just hit Broadway, IBM was developing the first hard disk and Eurovision was about to hold its inaugural song contest. Things were humming along well in the wake of the war and the world was just catching up to the resulting technological advancements. Foremost among these new technologies was the computer and, in the quiet town of Hanover, New Hampshire, a scientific flurry of activity was coming to a head.

The great computer scientist, John McCarthy saw that once disparate areas of inquiry were coalescing into a burgeoning new field and something had to be done. He arranged a conference, comprising a 2-month study of the top minds in complexity theory, linguistics and cognitive science, to settle the matter. Ever the formal affair, this was not an occasion for idle chatter. Rather, these men would:

“proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it”.

This gave birth to the research discipline of artificial intelligence (AI). 

Deep learning vs machine learning vs AI

Since the 1950s, AI has progressed in leaps and bounds. It has become pervasive in contemporary society. Terms ranging from deep learning to machine learning to neural nets have entered popular parlance, and so it will be useful for us to gain some basic understanding of what they mean.

Despite rapid advancements in AI over the past six decades, the bounds of AI research have changed very little since McCarthy’s decree. By and large, AI is still focused on describing the precise steps involved in all forms of human intelligence so that they can be mechanically carried out by machines. 

Now, when those machines require little or no human intervention we call this “machine learning” – and since statistical inference often plays a pivotal role in the way these machines function, this phenomenon also goes by the name “statistical learning”.

Indeed, it is statistical processing that makes these techniques for learning so powerful in today’s data-driven society. But “deep learning” is machine learning’s overachieving baby brother. It can do things like image classification and speech recognition because, while it too makes decisions on the basis of statistical inference, the way it does so is ingenious. It uses, what’s called a “neural net”.

So, what is a neural net?

Combining our improved understanding of the workings of the human brain, with Turing’s vision of reducing all aspects of intelligence to a sequence of logical steps, Warren McCulloch and Walter Pitts devised a computational model for neuronal processing.

McCulloch and Pitts assumed that, since any given neuron only ever occupies an excitation or inhibition state, any neural event could be completely described by using a string of 1s and 0s. Considering the human brain as an immensely complex biological network of neurons, they modelled feats of intelligence using a corresponding network of logical operations.

This concept was described as an “artificial neural network” or “neural net” and, even today, it is comprised of just four simple parts:

1. Neurons – these take in a number and, depending on its value, outputs some different number.
2. Connections – these describe the way in which neurons interact and affect one another.
3. Propagation functions – these modulate how outputs from one neuron become inputs to the next.
4. Learning rules – these dictate how the connections between neurons are modified over time to produce the desired output from any given input.

And that’s it!

In one number, out another

In summary, we’ve learnt so far that:

• Deep learning is a subset of machine learning which is a subset of AI;
• Machine learning uses an array of statistical techniques to emulate feats of intelligence;
• Neural nets are the statistical technique of choice for deep learning and these have shown particular promise in tasks, like image classification, previously thought to be the exclusive domain of human beings.

Now, technically-speaking, the power of neural nets is underpinned by what’s called “The Universal Approximation Theorem”. 

Essentially, this states that any real-valued function can be approximated by a sufficiently sophisticated neural network.

That’s an astounding fact!

There is a concrete statement of mathematics that says: so long as we have enough neurons and connections between them, we can discover any process (within certain limits) that turns one number into another. 

The problem is: the theorem doesn’t tell us how to arrange these neurons and connections. This represents a central problem. Indeed, much of the seminal work in deep learning has involved monumental efforts of trial-and-error. But the push to understand its inner workings continues…

Powerful black boxes

Neural nets are fraught with mystery. We know that they function as universal approximators. But we don’t quite understand how to build them.

We know what is minimal for their construction. But we don’t quite know what is optimal. 

And we know what the black box is trying to do. But we don’t quite know how they’re doing it.

This last point bears emphasis: while we know that neural nets are composed of neurons, connections, propagation functions and learning rules, precisely how these things interact to achieve their remarkable results is another question entirely.

AI into the future

Establishing a comprehensive understanding of the inner workings of neural nets is an important research task and until these processes are sufficiently understood, we must proceed with caution.

AI machines have achieved super-human abilities in certain tasks and are beginning to surpass us in a growing number of areas. Indeed, these black boxes have the potential to revolutionize much of the modern world.

Therefore, as we move into the Era of AI, we must ensure that we capitalise on the benefits while ensuring that we test rigorously, maintain close monitoring and exercise stringent control. Machines are playing an increasingly important role in addressing world’s most pressing problems – let’s ensure that AI safety doesn’t become one of them.

Nicholas Hindley, 17 April 2019

> Unsupervised Learning > Home