I'm trying to implement a Hopfield network that denoises images. Here's wikipedia for Hopfield networks that I used as reference: https://en.wikipedia.org/wiki/Hopfield_network I believe my fit2 function calculates the weights as described on wikipedia, and glauber_dynamics does the asynchronous updating. However, my images tend to converge to something that's not the same as any image and the energy function has several jumps for a lot of them.

Here's my implementation (simplified by deleting energy calculations and plotting):

class HopfieldNetwork:
    def __init__(self, N):
        self.N = N
        self.weights = None

    def fit2(self, X):
        self.weights = np.matmul(X.T, X) / self.N
        np.fill_diagonal(self.weights, 0)

    def glauber_dynamics(self, state, iters=4900*100):
        for j in range(iters):
            i = np.random.randint(0, self.N)
            M = np.dot(state, self.weights[i, :])
            state[i] = 1 if M >= 0 else -1
        return state

I use it on 20 images that are 70x70, and most of them converge to something that's not similar to any of these images.

Example of how I use this network on an image:

hn = HopfieldNetwork(70*70)
hn.fit2(flattened_images) # flattened_images.shape - (20, 4900)

noisy_image = flattened_images[0].copy()
noisy_image = np.where(np.random.rand(N) < 0.01, -noisy_image, noisy_image)
denoised_image = hn.glauber_dynamics(noisy_image.copy())
denoised_image = denoised_image.reshape(image_size)

I know it's possible for the network to converge to something "in between" the images but in my case there are also jumps in energy which are not supposed to happen.

The energy calculation:

    def calculate_energy(self, state):
        E = -0.5 * np.dot(state, np.dot(self.weights, state))
            return E

I have to implement it in this kind of asynchronous fashion. I can't find any mistake that could cause this issue. The images are simple and I tried to make them dissimilar to each other. I'd really appresciate a quick response that gives at least some clue.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.