The Neuron Hardware Implementation-Verilog

In the Last edition we learnt how a Neuron works..

Here we will learn about Hardware Design of Neuron

The Weight Memory

reg [dataWidth-1:0] mem [numWeight-1:0];

That is why our weight memory code will be as such

    `ifdef pretrained
        initial
		begin
	        $readmemb(weightFile, mem);
	    end
	`else
		always @(posedge clk)
		begin
			if (wen)
			begin
				mem[wadd] <= win;
			end
		end 
    `endif
    
    always @(posedge clk)
    begin
        if (ren)
        begin
            wout <= mem[radd];
        end
    end 

Let us learn how the weights are loaded into neuron, provided that neuron is not pretrained

We will weight value from all 1s, So that when valid weight and correct layer and configuration of neuron is identified, Wddr will be incremented with this value will be stored from 0 address..

always @(posedge clk)
    begin
        if(rst)
        begin
            w_addr <= {addressWidth{1'b1}};
            wen <=0;
        end
        else if(weightValid & (config_layer_num==layerNo) & (config_neuron_num==neuronNo))
        begin
            w_in <= weightValue;
            w_addr <= w_addr + 1;
            wen <= 1;
        end
        else
            wen <= 0;
    end

Now , we will multiply myinput with wout

    
    always @(posedge clk)
    begin
        mul  <= $signed(myinputd) * $signed(w_out);
    end

Why did we added delay to input

Whenever the input comes, it should be multiplied with corresponding weight.

Our weight memory works sequentially,it needs one clock latency so input also needs some delay..

Now , we will add product of weight memory and input and add it with previous sum.

assign comboAdd = mul + sum

So, now with this 2 possibilities arise i.e overflow and underflow..

If the msb of multiply output and sum is 0 and combo add is 1 than it is overflow

than we will add 0 to msb of sum and rest bit will be made 0.

Similarly the case for underflow is made…

But above dont work for last multiplication, because for last multiply we need to add bias to multiply output

assign BiasAdd = bias + sum

  
    always @(posedge clk)
    begin
        if(rst|outvalid)
            sum <= 0;
        else if((r_addr == numWeight) & muxValid_f)
        begin
            if(!bias[2*dataWidth-1] &!sum[2*dataWidth-1] & BiasAdd[2*dataWidth-1]) //If bias and sum are positive and after adding bias to sum, if sign bit becomes 1, saturate
            begin
                sum[2*dataWidth-1] <= 1'b0;
                sum[2*dataWidth-2:0] <= {2*dataWidth-1{1'b1}};
            end
            else if(bias[2*dataWidth-1] & sum[2*dataWidth-1] &  !BiasAdd[2*dataWidth-1]) //If bias and sum are negative and after addition if sign bit is 0, saturate
            begin
                sum[2*dataWidth-1] <= 1'b1;
                sum[2*dataWidth-2:0] <= {2*dataWidth-1{1'b0}};
            end
            else
                sum <= BiasAdd; 
        end

This sum now goes to activation function

Activation function supports 2 functions,one is RELU and other is sigmoid

    generate
        if(actType == "sigmoid")
        begin:siginst
        //Instantiation of ROM for sigmoid
            Sig_ROM #(.inWidth(sigmoidSize),.dataWidth(dataWidth)) s1(
            .clk(clk),
            .x(sum[2*dataWidth-1-:sigmoidSize]), //send the sum value
            .out(out)
        );
        end
        else
        begin:ReLUinst
            ReLU #(.dataWidth(dataWidth),.weightIntWidth(weightIntWidth)) s1 (
            .clk(clk),
            .x(sum),
            .out(out)
        );
        end
    endgenerate

In next editions we will learn, even more deeper this was a overview of implementation of Neuron!

Stay tuned