AI guide

Machine learning

Most of today’s AI applications are based on the principle of machine learning. This refers to analytical methods based on teaching by example and improving results “through experience”.

A computational model of the phenomenon under consideration is built by feeding the system with well-prepared, specific training data. Based on the model, predictions, interpretations, or decisions can be made without the need for instructions or rules to be programmed into the model.

For example, filtering spam can be done using machine learning. By providing enough examples of messages that users have marked as spam, the model can learn to identify with a high degree of certainty which messages are spam. Identification is based on the details of the message, such as the title, the words used in the text, and the sender’s details. The example also illustrates some of machine learning essentials: data is usually human-marked or enriched, it is in high demand, and it can be collected using input from service users.

So, giving the computer the ability to learn new things from data opens up countless possibilities in these days of rich and varied data collection. Raw data is like the new oil. It can be refined for machine learning and can be used to generate new knowledge in ways that traditional statistical methods cannot, and at a scale that is not possible by humans alone.

Neural networks and deep learning

Artificial neural networks (ANNs) are computational systems that are inspired by the structure of the human and animal brain. In simple terms, they are network-like structures where the nodes of the network, or neurons, receive a series of inputs from previous neurons in the network, perform computations on them, and pass the results on to the next neurons. All connections between neurons have weighted values that describe the importance of different inputs in the neurons’ computations. The cumulative complex interaction of the weights determines the outcome of the whole calculation.

A neural network is trained by feeding data through it, knowing what the end result of the computation should be. The network’s weighted values will be gradually adjusted to bring them closer to the correct configuration for the final result. The process is repeated until the network is adapted to produce the desired outcome. This simple approach has proven to be very effective, but the downside is that it is difficult to justify the neural network results and interpret the reasons behind them. However, neural network models are not entirely “black boxes” that cannot be understood. Improving their interpretability is an important area for research, and there are ways to do this.

Initially, artificial neural networks were designed to solve problems in much the same way as the human brain, but as the field has evolved and research has progressed, machine learning constructs have diverged from their biological counterparts. Application-specific neural network architectures have proven effective in many difficult AI application domains, including an object or human recognition from images and video, speech recognition, natural language processing, and semantical analysis.

The term deep learning has recently been used in connection with many impressive advances in AI. It refers to the deep or multi-layered structure of the neural networks used and the more sophisticated interactions within the network, which is intended to facilitate the perception of more complex structures and the learning of longer causal relationships. Training such neural networks usually require large amounts of data and are very computationally intensive. The harnessing of GPUs (graphics processing units) to train neural networks ushered in a new era of deep learning, as the computation of large complex models became many times more efficient.

One important type of structure is the convolution neural network (CNN), which has revolutionized the way image data is perceived, and where certain layers of neurons are designed to efficiently identify more detailed and complex features in images, layer by layer, regardless of their location in the image. The second is the recurrent neural network (RNN), where a neural network is essentially given a longer-lasting internal memory, the ability to learn the interconnections between sequential data elements, such as a timeline of events.

Recently, the best results in language modeling and text comprehension have been achieved by building a transformer model from large amounts of data (e.g., Wikipedia, news sites, and discussion forums) that can be efficiently parallel-computed to interpret the text as a whole and pay attention to the context of words and sentences. One of the most notable of these is BERT, developed by Google for search engine use, which has also been trained by a team of researchers at the University of Turku using Finnish text data. Based on the same idea, OpenAI’s massive GPT-3 and the versions after that, capable of impressive performance, have shown that the limit of the benefits of significantly increasing the size of the model has not yet been reached.

One very interesting and new idea is to put two neural networks in competition with each other; one to produce new material like training data and the other to distinguish whether it is a real or artificially produced sample. Such generative adversarial networks (GANs), which are usually harnessed to generate images, learn side by side to become progressively better at their task. In the end, the generating party will, at best, learn to produce original images that are very similar to the example material but new and original, for example, of a human face, landscape, or piece of fine art.

Models that can create high-quality text, images, and other content based on the data they were trained on are called generative artificial intelligence.

Machine learning system based on supervised learning

The most common and typical machine learning system is based on supervised learning and aims at predictive modeling. The available data is rarely suitable for training the model as such. Firstly, it is searched for and cleaned of errors, deficiencies, inconsistencies, and repetition. Improving the quality of data may also require changes in the way it is collected or compiled from different sources.

Preparation of data

To train the model, a collection of separate records is created from the raw data, describing example cases of the phenomenon under consideration. In supervised learning, you need to know what you are predicting for each example case. From the features describing each case, a set of features is selected as input to the modeling. In order for the computer to perform the calculations needed to build the model, the data must be represented in numbers only. The inputs to a machine learning model are feature vectors, a series of numerical values representing the properties of example cases. It is also often necessary to scale, group, or combine these figures to achieve the best results.

For example, when predicting the price of an apartment, the raw data could be a repository of completed transactions, the records are individual transactions, the modeling attributes are data describing the apartment, and for all examples, the variable to be predicted is known, i.e., the completed transaction price. Information describing the apartment as a vector of characteristics: surface area in square metres, number of rooms, location in latitude and longitude, number for each type of house, etc.

The generated data is randomly split into two parts. Most of it is used as training data for model fitting and optimization. The rest of the data is reserved as test data to evaluate the model and see how generalizable it will be with unprecedented data, i.e., to avoid overfitting the model to the specifics of the training data.

It is worth investing in researching, improving, and pre-processing data. The old wisdom in the industry is rubbish in, rubbish out. A machine learning model can only be as good as the data used to build it. Even the finest neural network model is essentially just a derivative of the data used to train it, a condensed representation of the information it contains. It is, therefore, important to pay attention to how comprehensively and fairly the data used represents the full range and variability of the application domain and its characteristics.

Any bias in the training data is largely carried over as such to the results produced by the model. For example, a face recognition system taught only by images of Finns cannot work optimally worldwide. Often habits, personal preferences, and prejudices influence the decisions we make, but this need not be the case with AI.

Teaching and evaluating the model

There are several different machine learning algorithms that can be used to build a model, from straightforward decision trees and linear regression to deep and complex neural network structures. Different algorithms have their specificities and strengths, and it is not possible to say unequivocally that one algorithm is better than another in all cases. Depending on the application and the nature of the input data, different algorithms can produce different results, so it is important to try and compare several options. For this, it is worth starting by developing an infrastructure, a whole pipeline from start to finish, for inputting training data, training different models, evaluating the performance of the models, and interpreting and visualizing the results.

Model training, in simple terms, means finding optimal values for the internal parameters of the model, on the basis of which the model calculates the final result. The process is an optimization that seeks to minimize the loss function of the predictions produced by the model over the entire training data, which describes how far from the truth the predictions produced by the model are compared to the correct answers known from the training data. The performance of classification models can be measured by various observable metrics using test data, most commonly by calculating the accuracy, i.e., the proportion of correctly classified samples. In the case of imbalanced data, when identifying rare samples from a large mass, attention should be paid to the precision (positive predictive value) and recall (sensitivity) per category; to identify the trade-off between finding a category and misclassifying it. The harmonic mean (F1-score) can be used as an equal measure of both.

The first model to be trained is usually the most straightforward and simplest one possible to ensure that all other parts of the system work. The results obtained can also be used as a benchmark for experimenting with more complex algorithms that are better suited to the situation. The operation of different machine learning algorithms and the training process itself is governed by so-called hyperparameters, different settings, and basic assumptions that need to be adapted to different applications. Finding the best model for a new application requires finding both the most suitable algorithm and the most suitable hyperparameters. This step is a part systematic exploration of alternatives, part art, and sometimes more than science.

Once a well-performing model has been found through systematic search, accumulated experience, or trial and error, and its generalisability is confirmed by test data, the model can be used to start making predictions and decisions from new data. It is also important to feed the new data through exactly the same pre-processing steps as when creating the training and testing data. The performance of the production system should be monitored, the training data updated as necessary, and the model refined, especially as data sources and the environment change over time.