An Introduction to Perceptron

Perceptron is a single-layer neural network. It is also called a deep feedforward network, this means that it does not give any feedback to the neurons and the information only flows forward.

The perceptron tries to estimate a function that can map the input to the output. It is a binary classifier, which means the model decides the class of the input. For example, if you do an RT-PCR test, there will only be two outcomes, positive or negative. This is called binary classification.

Data types for perceptrons
Data Types

How does a perceptron model work?

Well, the perceptron model works as follows: 

It multiplies the input(x) with their respective weights(w) and adds the bias(b). It then sums products and passes them through an activation function, which then gives us the output. The Perceptron algorithm learns the weights for the input signals in order to draw a linear decision boundary. This enables you to distinguish between the two linearly separable classes +1 and -1.

The perceptron consists of 5 parts.

  • Input layer: Perceptron takes real values as its input. For example: if the perception is tasked with classifying Iris flowers(an open deep learning dataset), two inputs could be the length and width of flower petals.
  • Weights: The weights allow the perceptron to evaluate the relative importance of each of the outputs. Neural network algorithms learn by discovering better and better weights that result in a more accurate prediction.
  • Bias: A bias weight is added and multiplied by a constant equal to 1. An input’s bias value gives the ability to shift the activation function curve up or down. This is a technical step that makes it possible to move the activation function curve up and down, or left and right on the number graph. It makes it possible to fine-tune the numeric output of the perceptron.
  • Net sum: The input values are multiplied by the weights and summed up. to create one aggregate value that is fed into the activation function. The weights control the strength or the influence of a neuron on the output and the bias ensures that the neurons are activated even if the input is zero.
Y= Weight*Input + Bias
Weight and bias being added to an input
  • Activation Function: The activation function maps the input values to the required output values. It generates a classification decision. For example, input values could be between 1 and 100, and outputs can be 0 or 1. For more details see our guide on activation functions from our site.

The feedforward networks play a very important role in machine learning. For example, the convolutional networks are used for object recognition as a specialized type of feed-forward network.

Perceptron Learning Rule

Perceptron Learning Rule states that the algorithm would automatically learn the optimal weight coefficients. The input features are then multiplied with these weights to determine if a neuron fires or not.

The Perceptron receives multiple input signals, and if the sum of the input signals exceeds a certain threshold, it either outputs a signal or does not return an output. In the context of supervised learning and classification, this can then be used to predict the class of a sample.

Error in Perceptron: In the Perceptron Learning Rule, the predicted output is compared with the known output. If it does not match, the error is propagated backward to allow weight adjustment to happen.

Types of perceptron models

There are two types of perceptron models:

  1. Single-layer perceptron:

Single-layer perceptrons can learn only linearly separable patterns. A single-layered perceptron model consists of the components that have been mentioned above and the activation function is a threshold transfer function inside the model. A threshold transfer function is also called the unit step function. It is shown below:

unit step function
By Can.guven, CC BY-SA 3.0,

The main objective of the single-layer perceptron model is to analyze the linearly separable objects with binary outcomes(i.e. 0 or 1 outcomes). Mathematically, it can be expressed as follows:

The mathematical representation of unit step function
The mathematical representation of the unit step function
  1. Multi layer perceptron(MLP):

Multiple Hidden layers are used to find the nonlinearity of the data. This instruction is also called a feed-forward network. An MLP consists of at least three layers: an input layer, a hidden layer, and an output layer. Except for the input layer, each layer uses a nonlinear activation function like Sigmoid, ReLU, Softmax, etc. MLP utilizes a supervised learning technique called backpropagation for training. MLP has multiple layers and non-linear activation functions which distinguish it from a linear perceptron. Its main advantage is that it can distinguish data that is not linearly separable.

Refer to the following Multi-layer perceptron video for a better understanding of the concept:


Similar Posts

Leave a Reply

Your email address will not be published.