Skip to main content
Bumped by Community user
Bumped by Community user
Bumped by Community user
Added the output I am getting for better context.
Source Link

Here's my output

$ python model.py
input dim:  (32, 2010)
num inputs:  2010
filter dim:  (3, 1)
filter size:  3
output dim:  (10, 2008)
output length:  2008
receptive field dim:  (3, 1)
output at 0[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 1[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 2[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 3[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 4[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 5[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 6[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 7[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 8[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 9[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 10[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 11[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 12[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 13[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 14[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 15[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 16[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 17[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 18[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 19[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 20[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 21[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 22[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 23[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 24[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 25[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 26[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 27[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 28[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 29[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (2, 1)
Traceback (most recent call last):
  File "C:\Users\maste_0c98yk4\OneDrive\Desktop\Projects\Natural Language Processing Model - Sentiment Analysis\model.py", line 89, in <module>
    model.train(X, labels, num_epochs=10, batch_size=32)
  File "C:\Users\maste_0c98yk4\OneDrive\Desktop\Projects\Natural Language Processing Model - Sentiment Analysis\model.py", line 44, in train   
    conv_output = self.conv_layer.forward(inputs)
  File "C:\Users\maste_0c98yk4\OneDrive\Desktop\Projects\Natural Language Processing Model - Sentiment Analysis\convolution.py", line 32, in forward
    self.output[:, i] = np.dot(receptive_field.T, self.conv_filter)
ValueError: shapes (1,2) and (3,1) not aligned: 2 (dim 1) != 3 (dim 0)
(tf) 

Expected behavior:

Expected behavior:

Here's my output

$ python model.py
input dim:  (32, 2010)
num inputs:  2010
filter dim:  (3, 1)
filter size:  3
output dim:  (10, 2008)
output length:  2008
receptive field dim:  (3, 1)
output at 0[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 1[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 2[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 3[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 4[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 5[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 6[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 7[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 8[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 9[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 10[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 11[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 12[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 13[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 14[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 15[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 16[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 17[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 18[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 19[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 20[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 21[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 22[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 23[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 24[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 25[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 26[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 27[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 28[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (3, 1)
output at 29[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
receptive field dim:  (2, 1)
Traceback (most recent call last):
  File "C:\Users\maste_0c98yk4\OneDrive\Desktop\Projects\Natural Language Processing Model - Sentiment Analysis\model.py", line 89, in <module>
    model.train(X, labels, num_epochs=10, batch_size=32)
  File "C:\Users\maste_0c98yk4\OneDrive\Desktop\Projects\Natural Language Processing Model - Sentiment Analysis\model.py", line 44, in train   
    conv_output = self.conv_layer.forward(inputs)
  File "C:\Users\maste_0c98yk4\OneDrive\Desktop\Projects\Natural Language Processing Model - Sentiment Analysis\convolution.py", line 32, in forward
    self.output[:, i] = np.dot(receptive_field.T, self.conv_filter)
ValueError: shapes (1,2) and (3,1) not aligned: 2 (dim 1) != 3 (dim 0)
(tf) 

Expected behavior:

Source Link

Issue with Convolutional Layer in Python: Getting All Zeros in Output and Terminating at a Certain Iteration

I'm currently working on implementing a convolutional layer in Python for a natural language processing model. However, I've encountered an issue with the convolutional layer that I can't seem to resolve.

The problem is twofold:

  1. Getting all zeros in the output: When I run the forward pass of my convolutional layer, I'm consistently getting all zeros in the output at every position. I've verified this by printing the values during execution. This is unexpected because I initialized the output array correctly.

  2. Terminating at a certain iteration: Additionally, the forward pass terminates abruptly at a certain iteration, specifically at the 30th iteration. I'm unable to figure out why the loop is exiting prematurely.

Here's a simplified version of my Conv1DLayer class:

import numpy as np

class Conv1DLayer:

    def __init__(self, num_filters, filter_size):
        self.num_filters = num_filters
        self.filter_size = filter_size
        self.conv_filter = np.random.randn(filter_size, 1)

    def loss(self, pred, target):
        # compute loss function
        return np.mean((pred - target) ** 2)

    def forward(self, inputs):
        self.inputs = inputs
        num_inputs = inputs.shape[1]
        output_length = num_inputs - self.filter_size + 1
        self.output = np.zeros((self.num_filters, output_length))
        # Convolution
        # input dim is basically the size of the vocabulary
        print("input dim: ", inputs.shape)
        print("num inputs: ", num_inputs)
        print("filter dim: ", self.conv_filter.shape)
        print("filter size: ", self.filter_size)
        print("output dim: ", self.output.shape)
        print("output length: ", output_length)
        for i in range(output_length):
            if i+self.filter_size > num_inputs:
                break
            receptive_field = inputs[i:i+self.filter_size, 1].toarray()
            print("receptive field dim: ", receptive_field.shape)
            self.output[:, i] = np.dot(receptive_field.T, self.conv_filter)
            print("output at " + str(i) + str(self.output[:, i]))
            self.output[:, i] = np.maximum(0, self.output[:, i])

        return self.output
    
    def backward(self, grad_outputs, learning_rate):
        grad_input = np.zeros(grad_outputs.shape)
        grad_filter = np.zeros(self.conv_filter.shape)

        for i in range(grad_outputs.shape[0]):
            for j in range(self.num_filters):
               receptive_field = self.inputs[i:i+self.filter_size]
               grad_input[i:i+self.filter_size] += self.conv_filter[:, j] * grad_outputs[i, j]
               grad_filter[:, j] += receptive_field * grad_outputs[i, j]

            # Update the weights
            self.conv_filter -= learning_rate * grad_filter

        return grad_input

I've tried various modifications to the code, including checking the shape of input arrays, adjusting the receptive field, and updating the loop condition, but I haven't been able to resolve the issue.

Expected behavior:

  1. The forward pass should compute the convolution correctly, producing non-zero values in the output.
  2. The loop should iterate over all valid positions without terminating prematurely.

My assessment I think the issue is very likely just the way I am using indices but neither chatgpt nor bard have been able to fix it so it might be something deeper.

Any kind of help will be much appreciated. Thanks