The 2024 Nobel Prize in Physics: Artificial Neural Networks and Machine Learning

The 2024 Nobel Prize in Physics was awarded to John J. Hopfield and Geoffrey E. Hinton for their “foundational discoveries and inventions that enable machine learning with artificial neural networks.”

This year’s Nobel laureates developed works based on physical methods that lay the groundwork for today’s powerful machine-learning technologies. Their studies help us understand how artificial intelligence and neural networks have evolved to their current state. John Hopfield designed a structure capable of storing and reconstructing information. Geoffrey Hinton developed a method that can independently discover features in data, which has become extremely important for large artificial neural networks.

Many people have experienced the ability of computers to translate between languages, interpret images, and even engage in reasonable conversation. However, perhaps the lesser-known fact is that such technologies have played a significant role in research for a long time, particularly in areas like the classification and analysis of large datasets. In the past 15-20 years, the development of machine learning has exploded, and this technology has become synonymous with artificial intelligence.

Although computers are unable to think (yet), they can now mimic functions like memory and learning. The physicists who won this year’s Nobel Prize used fundamental physical concepts and methods that contribute to the development of such technologies to create systems that use network structures for information processing. As a result, machines have become capable of solving more complex and ambiguous problems.

The Difference of Machine Learning

Machine learning differs from traditional software. Traditional software operates by entering a specific dataset, where it is processed step by step, and a specific output is produced. This is similar to baking a cake, where you gather ingredients, process them according to a recipe, and finally have your cake. In machine learning, you do not give the computer specific instructions. Instead, it acquires the ability to solve new and more complex problems by training on examples, like its ability to recognise objects by interpreting an image.

Hopfield and Associative Memory

Artificial neural networks are systems where information is processed using the entire network structure, inspired by the biological structure of the brain. In artificial neural networks, the neurons of the brain are represented by nodes, which are strengthened or weakened through their connections (synapses). This fundamental principle is regarded as one of the main rules used in the training of neural networks. The network established by Hopfield consists of nodes connected by varying strengths of links. Each node stores a value— in Hopfield’s early works, this value is either 0 or 1, just like the pixels in a black-and-white image.

Hopfield compares the overall state of this network to the concept of energy in a physical system. The energy is calculated using the values of the nodes and the strength of the connections between them. The working principle of this network begins with transferring an image to the nodes by assigning black (0) or white (1) values. The network’s connections are adjusted to store this image in a “low energy” state.

When another image is subsequently presented to the network, the nodes are checked one by one. If a node’s value is changed (for example, if a black pixel is turned white) and this results in a decrease in the network’s energy, this change is made. This process continues until no further improvements can be made. Eventually, the network typically reproduces the original image it was trained on. This process can be seen as an optimization aimed at storing the image in a low-energy state.

Boltzmann Machines and Hinton’s Contribution

Geoffrey Hinton advanced artificial neural networks further with a model called the Boltzmann machine, which was developed based on the Hopfield network. The Boltzmann machine operates on principles derived from statistical physics and uses the Boltzmann equation, which is employed to explain the movements of gas molecules. The machine works based on this equation to process data and recognise patterns. The Boltzmann machine is a system that learns from examples. This machine is trained on the examples provided to it. After this training, when the machine encounters a new example, it can recognise it and discern whether it belongs to a particular category.

In the 1990s, many researchers lost interest in artificial neural networks, but Hinton continued his work and, in 2006, developed a pre-training method that utilised Boltzmann machines in layers. This method ensured that the network connections had a better starting point and optimised the performance of recognising elements in images.

The Future of Machine Learning

With their work since the 1980s, Hopfield and Hinton laid the groundwork for a revolution in machine learning. In the 2010s, machine learning experienced a significant boom, advancing further due to big data and increasing computational power. Deep neural networks and deep learning methods rapidly developed during this period.

As the application areas of machine learning rapidly evolve, ethical discussions are also gaining importance. Machine learning has begun to be used in fields such as calculating the structures of molecules and developing new materials by working with large datasets. Even in physical research, these technologies have become a significant aid in the analysis and processing of data.