7 types of Layers you need to know in Deep Learning and how to use them

Layers, the basic concept that structure Deep Learning. What are the main types and how to use them ? That what we’ll find out.

In a Deep Learning algorithm the input data are called tensors, it is basically a matrix which can have many dimensions.

The layers allow to transform the input data into information that can be understood by the computer.

In this article we have chosen to gather the 7 main layers to explain their principles and in which context to use them.

Basic layer

In Deep Learning, a model is a set of one or more layers of neurons.

Each layer contains several neurons that apply a transformation on each element of the input tensor.

The main difference between the types of layers lies in the way the neurons behave.

The Dense layer is the basic layer in Deep Learning.

It simply takes an input, and applies a basic transformation with its activation function.

The dense layer is essentially used to change the dimensions of the tensor.

For example, changing from a sentence (dimension 1, 4) to a probability (dimension 1, 1): “it is sunny here” ➡ 0.9

This model tells us that there is a 90% probability that the sentence is positive.

Convolution Layer

The Convolution layer is a bit more complex.

It is mainly used for Computer Vision tasks : to analyze images.

The convolution is the fact of analyzing an image or a data by pieces.

For example, with an image of a dog, the model will be able to analyze the ears or the muzzle by pieces, and then associate it with the word “dog”.

In fact the convolution layer focuses on parts of the image and associates to the word “dog” all the most relevant parts that he finds.

This type of layer is particularly effective for classifying objects or animals.

Pooling layer

Pooling is used to reduce the size of the data.

Indeed, it compresses the input tensor to keep only the relevant information.

Like the Convolution layer, it works in pieces. However the Pooling layer will only keep the important information of this piece and delete the rest.

There are different Pooling types :

• Max Pooling, which takes the maximum value in a subset of data
• Average Pooling, which takes the average of the values in a subgroup of data

Recurrent Layer

Recurrent layers also called RNN are mainly used in NLP, for text processing.

The idea behind this layer is to allow the Deep Learning model to have memory.

In technical terms, the Recurrent layer will analyze the input data several times.

The classic example is to take a sentence “I’m going to the beach”. The RNN will analyze each word one after the other, keeping in memory the information of the previous word.

In concrete terms, this brings context to the understanding of the sentence.

It is therefore quite natural that Recurrent Layers are used for translation problems.

Normalization Layer

The Normalization layer is probably the easiest to understand !

After using a Deep Learning layer, the data is transformed. This often results in totally scattered values.

Since every number in the input tensor has been transformed, some may have a value of 3 and others 2867. This is a very scattered value scale and is not good for our Deep Learning model.

So the Normalization layer applies… normalization to this data.

To be more precise, this layer will make sure that the data has the same distribution.

We can imagine normalizing our data so that they all are in a scale from 0 to 1. For example, our 3 would become 0.04 and the 2867 would be 0.89.

Applying normalization to our data after each layer makes it easier for the model to learn by making less error.

Regularization layer

The regularization layer is also used to improve the learning of the model.

With normalization, we simplified the task of our model. With the Regularization, we will make it more complex.

Indeed, the Regularization, for example the Dropout layer, will transform some values of our tensor into 0.

This technique will increase the difficulty for our model but paradoxically it will make it more efficient.

In fact, by randomly transforming some values into 0, we force our model to be more efficient in solving the task we have set for it.

Attention layer

The Attention layer is surely the most advanced approach described in this article.

The principle of the Attention layer is to improve RNNs… and even to surpass them !

In a RNN, we keep in memory each word already analyzed to understand the next one.

The problem is that we analyze the sentence by going through it from beginning to end, but not in the opposite direction.

With Attention, we may analyze the relationship of every word of a sentence with the others to understand the links that unite the words between them.

It is therefore an improvement of the RNNs in the sense that Attention will allow us to extract more information from the same sentence !

In this article we have quickly gone through the main layers that exist in Deep Learning.

However, if you want to go deeper into these concepts, several article can lead you to it.

See you soon for more news on Machine Learning ! 😉