Let us get started with AI hardware Design

Introduction

Why learning the basics of AI hardware design can be beneficial?

1. Optimizing System Performance: Understanding AI hardware is pivotal for optimizing the performance of AI systems.
2. Cost-Effective Solutions: Knowledge of AI hardware enables informed decisions on cost-effective infrastructure.
3. Innovation and Customization: Proficiency in AI hardware empowers professionals to innovate and design custom solutions.
4. Interdisciplinary Collaboration: A shared understanding of AI hardware facilitates collaboration between hardware engineers and AI researchers.

If you want to learn the history of AI Hardware refer to the awesome lectures and the material offered by Prof. Mitesh M. Khapra on NPTEL’s Deep Learning course. Check it out!

Let us Understand the Neuron Deeply…

So, the neuron inside our brain takes inputs from different sense organs

Each sense organ is assigned a priority, which we define as weight in neurons w1, w2 and w3.

To recognize, whether the image is a logo of embed thread or not, our brain needs input in a visual format, inputs from taste buds are not needed so taste buds will be assigned weight 0.

The inputs are multiplied with weights and added to get the output.

Neuron takes input, they communicate with each other and based on the decision output is decided.

Let us now understand the activation function

Here b is the bias which we will discuss later..

Let us understand the threshold function,

So for x less than 0 we get output 0 and for greater than 0, we get output 1.

Sigmoid function

Here the answer we get based on probability values varies from 0 to 1, here we set a threshold value, which is 0.5 any value greater than 0.5 we get an as 1 and if less than 0.5 we get an answer 0.

RELU

This is a default function in neural networks.

So, how it works, it calculates z by multiplying weights with input.

so now it compares the value of z with 0, greater value between the two is assigned in output.

Like we got z as 5, now between 0 and 5, 5 is larger so the answer will be 5.

Let us understand Bias more deeply

Why do we really need bias…

for the above weights and inputs, we get the output as 0.

If we get output as 0 that means our neuron did not learn anything with this there will be no communication between the neurons so we add a bias to it.

Now we added a bias, we got a non-0 output

In a later article, we will discuss how bias value is decided.

The motive of this article was to brush up on some basics of neurons because in the next edition we will implement the neuron on the basis of this knowledge.