0
$\begingroup$

I was trying to learn Fuzzy Cognitive Map by Active Hebbian Learning approach from here. What I have understand is that the model learns iteratively, at each step a new concept values enters and tune the weighs until the MSE score in output neurone is very small. I thaught that it is similar to stochastic gradient descent. But I don't see any convergence in output MSE value when a new input comes.

import numpy as np
import matplotlib.pyplot as plt


mean1 = [0, 0]
cov1 = [[1, 0], [0, 1]] 

x1, x2 = np.random.multivariate_normal(mean1, cov1, 10000).T
x1 = x1.reshape([len(x1),1])
x2 = x2.reshape([len(x2),1])


def multivariate_normal(x, d, mean, covariance):
    """pdf of the multivariate normal distribution."""
    x_m = x - mean
    return (1. / (np.sqrt((2 * np.pi)**d * np.linalg.det(covariance))) * 
            np.exp(-(np.linalg.solve(covariance, x_m).T.dot(x_m)) / 2))




def sigmoid(x):
    return 1/(1+np.exp(-x))




def FCM(W,X1,X2,y,lr,gamma):
    y1 = 0
    y2 = 0
    loss = []
    Loss = []
    for i in range(0,2000):
        
        A1 = np.dot(W[0],np.transpose(X1[i]))
        A1 = A1.astype('float')
        A2 = np.dot(W[1],np.transpose(X2[i]))
        A2 = A2.astype('float')  
        y1 = sigmoid(A1 + y[i][0])
        y2 = sigmoid(A2 + y[i][1])  
        temp = ((y1 - y[i][0])**2 + (y2 - y[i][1])**2)/2
        print(i)
        Loss.append(temp) 
        count = 0
        while (count < 1):  
            count += 1
            #print(temp)
            A1 = np.dot(W[0],np.transpose(X1[i]))
            A1 = A1.astype('float')
            A2 = np.dot(W[1],np.transpose(X2[i]))
            A2 = A2.astype('float')  
            y1 = sigmoid(A1 + y[i][0])
            y2 = sigmoid(A2 + y[i][1]) 
            temp1 = (1 - gamma)*W[0] + lr*X1[i]*y[i][0]           
            temp2 = (1 - gamma)*W[1] + lr*X2[i]*y[i][1]
            #print(temp1,temp2)
            W[0] = temp1/(np.sqrt(np.dot(temp1,np.transpose(temp1))+np.dot(temp2,np.transpose(temp2))))
            W[1] = temp2/(np.sqrt(np.dot(temp2,np.transpose(temp2))+np.dot(temp1,np.transpose(temp1))))  
            A1 = np.dot(W[0],np.transpose(X1[i]))
            A1 = A1.astype('float')
            A2 = np.dot(W[1],np.transpose(X2[i]))
            A2 = A2.astype('float')  
            y1 = sigmoid(A1 + y[i][0])
            y2 = sigmoid(A2 + y[i][1]) 
            temp = ((y1 - y[i][0])**2 + (y2 - y[i][1])**2)/2   
            loss.append(temp)     
            if count > 1:
                if loss[len(loss) - 1] > loss[len(loss) - 2]:
                    break
       # Loss.append(temp)        
    return [W,Loss]        



X = np.concatenate((x1,x2),axis = 1)  


"""
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_temp = X
X = sc.fit_transform(X)
"""


y = []   
for i in range(len(X)):
    if multivariate_normal(X[i],2,mean1,cov1) < 0.5:
        y.append([0,1])
    else:
        y.append([1,0])
        
y = np.array(y).astype('float')        

y1 = y[:,1].reshape([len(y[:,1]),1])
y2 = y[:,0].reshape([len(y[:,1]),1])
X1 = np.concatenate((X,y1),axis = 1)
X2 = np.concatenate((X,y2),axis = 1)



W = np.array([[0.2,-0.3,-0.4],[-0.1,0.7,-0.6]])




L = FCM(W,X1,X2,y,0.1,0.1)
W2 = L[0]
loss = L[1]

To check the method, I have used Bi-normal data, data points close to then mean and far from the mean are classified to distinct class. The two output nodes are either [1,0] or [0,1].The two attributes of Bi-normal and the second of output are taken as input states for first node. Similarly, The two attributes of Bi-normal and the another first of output are taken as input states for scond node. The loss when a new data comes have ploted for 2000 iteration, there is no indication of convergence of the weights of the map.enter image description here I expected that the loss will grow doen eventually like stochastic gradient descent.

$\endgroup$

1 Answer 1

1
$\begingroup$

There are many issues, it is hard to say which one applies to your case: DOCs values are impossible to research, FCM is poorly designed and the maps won't converge or the map is too dense. These are some of the ones I came across while testing our code.

We recently developed fcmpy library. Hopefully, you can find some answers there! You can compare your implementation with ours and see why the convergence doesn't work.
You can use it for building FCMs, simulating them, creating interventions as well as optimizing or generating them using machine learning.

Article about the library is available here: https://arxiv.org/abs/2111.12749v1

The package can be installed via pip https://pypi.org/project/fcmpy/

Git repo https://github.com/SamvelMK/FCMpy

cheers!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.