0
$\begingroup$

Problem Statement

The goal is to have the Gaussian Mixture Model has_prediction() function to execute after the Gaussian Mixture Model fix() function that will invoke methods on this custom expectation maximization class to ultimately return the cluster means.

However, the ValueError shape is an issue. Once this logic works, then the next step is to follow up with the next function call for retrieve the Gaussian Mixture Model cluster means of the model data (data1), based on the prediction() for (xy_matrix). The goal is to print the mean an enumerate the cluster means, e.g,

for i, mean in enumerate(cluster_means):
    print(f"Cluster {i+1} Mean: {mean}")

So the problem that I need to solve to move forward, all codes has been tested, this is the only error to fix. The question is that some shape of a matrix is not aligned, but I have another K-Means that implements with this exact dame data shapes (5400,2) and (1,1) and there is NO alignment problem in my K-Means code. So what is going wrong in my GMM fit() and expectation maximization algorithm.

My Attempt to Fix the Problem

My first attempt was to perform the following process: In my class fit() function, I added a correction, namely, this code: "self.covariances = np.array([np.eye(n_features) for _ in range(self.n_components)])" defined with the numpy.eye() function to handle the shape issue within the expectation maximization logic where it seems that the covariance matrix may need to be properly shape updated during the expectation maximization process. But even this attempt is not fixing the ValueError, so I am at a loss to think of new solutions.

Error

ValueError: could not broadcast input array from shape (2700,1) into shape (2700,)

Base Invocation Calls / Sequences

My invoke python calls for my object:

gmm = GaussianMixtureModel(n_components=2) gmm_fit = gmm.fit(data1) gmm_predict = gmm._has_prediction(xy_matrix)

My Python Gaussian Mixture Model and Expectation Maximization Code

My class code is listed:

class GaussianMixtureModel:
def __init__(self, n_components, max_iter=100, tol=1e-4):
    self.n_components = n_components
    self.max_iter = max_iter
    self.tol = tol

def fit(self, X):
    n_samples, n_features = X.shape
    np.random.seed(0)
    self.weights = np.ones(self.n_components) / self.n_components
    self.means = X[np.random.choice(n_samples, self.n_components, replace=False)]
    self.covariances = np.array([np.eye(n_features) for _ in range(self.n_components)])

    for _ in range(self.max_iter):
        expectation_max = self._expectation(X)
        self._maximization(X, expectation_max)
        if self._has_converged(X, expectation_max):
            break

def _expectation(self, X):
    exp_max = np.zeros((X.shape[0], self.n_components))
    print('_expectation: ', exp_max.shape)
    for k in range(self.n_components):
        diff = X - self.means[k]
        diff = np.expand_dims(diff, axis=-1)
        exponent = -0.5 * np.sum(np.matmul(diff.transpose(0, 2, 1), np.linalg.inv(self.covariances[k])) * diff, axis=1)
        # Ensure self.weights has the correct shape for broadcasting
        weights_k = np.squeeze(self.weights[k])
        exp_max[:, k] = weights_k * np.exp(exponent) / np.sqrt(np.linalg.det(self.covariances[k]) + 1e-6)
    exp_max /= exp_max.sum(axis=1)[:, np.newaxis]
    print('_expectation: ', exp_max.shape)
    return exp_max

def _maximization(self, X, responsibilities):
    Nk = responsibilities.sum(axis=0)
    self.weights = Nk / X.shape[0]
    self.means = np.dot(responsibilities.T, X) / Nk[:, np.newaxis]
    for k in range(self.n_components):
        diff = X - self.means[k]
        # Ensure diff has the correct shape for matrix operations
        diff = np.expand_dims(diff, axis=-1)
        # Compute the covariance matrix using the responsibility matrix
        self.covariances[k] = np.sum(responsibilities[:, k, np.newaxis, np.newaxis] * np.matmul(diff, diff.transpose(0, 2, 1)), axis=0) / Nk[k]
 
def _has_prediction(self, X):
    responsibilities = self._expectation(X)
    cluster_labels = np.argmax(responsibilities, axis=1)
    return cluster_labels

def _has_covariances(self):
    return self.covariances

def _has_mean(self):
    return np.mean(self.means, axis=0)

def _has_cluster_means(self, X):
    cluster_means = []
    for i in range(self.n_components):
        points = X[self._has_prediction(X) == i]
        if len(points) > 0:
            cluster_mean = np.mean(points, axis=0)
            cluster_means.append(cluster_mean)
    return cluster_means

def plot_fit_model_ellipses(self):
    for i in range(self.n_components):
        covariances = self.covariances[i]  # Taking only the first 2 dimensions
        v, w = np.linalg.eigh(covariances)
        v = 2.0 * np.sqrt(2.0) * np.sqrt(v)
        u = w[0] / np.linalg.norm(w[0])
        angle = np.arctan(u[1] / u[0])
        angle = 180.0 * angle / np.pi  # Convert to degrees
        ell = plt.matplotlib.patches.Ellipse(self.means[i], v[0], v[1], 180.0 + angle, color='red', alpha=0.2)
        ell.set_clip_box(plt.gca().bbox)
        ell.set_alpha(0.5)
        plt.gca().add_artist(ell)

Data Repo Shared

My Data Repo Test "data1" is computed difference:

array([[40],
       [42],
       [42],
       [52],
       [32],
       [23],
       [35],
       [38],
       [45],
       [41],
       [21],
       [33],
       [43],
       [34],
       [38],
       [38],
       [44],
       [49],
       [37],
       [42],
       [25],
       [38],
       [44],
       [38],
       [42],
       [44],
       [38],
       [23],
       [26],
       [41],
       [36],
       [37],
       [40],
       [32],
       [27],
       [33],
       [50],
       [51],
       [19],
       [36],
       [44],
       [41],
       [44],
       [44],
       [38],
       [24],
       [40],
       [43],
       [39],
       [41],
       [39],
       [35],
       [20],
       [44],
       [44],
       [45],
       [46],
       [45],
       [41],
       [41],
       [40],
       [46],
       [29],
       [45],
       [35],
       [48],
       [26],
       [40],
       [23],
       [42],
       [36],
       [39],
       [43],
       [42],
       [40],
       [22],
       [36],
       [39],
       [38],
       [23],
       [40],
       [32],
       [40],
       [ 8],
       [42],
       [38],
       [34],
       [31],
       [49],
       [34],
       [44],
       [42],
       [37],
       [47],
       [39],
       [39],
       [46],
       [40],
       [44],
       [42],
       [41],
       [49],
       [41],
       [33],
       [51],
       [38],
       [38],
       [35],
       [21],
       [33],
       [42],
       [39],
       [33],
       [35],
       [42],
       [38],
       [36],
       [37],
       [46],
       [27],
       [35],
       [44],
       [52],
       [39],
       [36],
       [38],
       [36],
       [40],
       [46],
       [30],
       [40],
       [39],
       [31],
       [40],
       [41],
       [27],
       [42],
       [37],
       [42],
       [40],
       [48],
       [42],
       [53],
       [26],
       [44],
       [47],
       [34],
       [35],
       [57],
       [42],
       [40],
       [47],
       [36],
       [30],
       [34],
       [47],
       [21],
       [36],
       [25],
       [11],
       [37],
       [42],
       [35],
       [16],
       [37],
       [52],
       [18],
       [40],
       [23],
       [38],
       [40],
       [45],
       [37],
       [50],
       [10],
       [34],
       [40],
       [40],
       [41],
       [40],
       [14],
       [16],
       [33]

My Data Repo "xy_matrix" for testing x, y, coordinates

array([[1377, 1417],
       [ 299,  341],
       [ 376,  418],
       [1390, 1442],
       [1887, 1919],
       [ 707,  730],
       [ 163,  198],
       [1068, 1106],
       [ 495,  540],
       [ 157,  198],
       [ 255,  276],
       [ 245,  278],
       [1776, 1819],
       [1240, 1274],
       [1067, 1105],
       [1242, 1280],
       [1811, 1855],
       [ 522,  571],
       [1318, 1355],
       [ 485,  527],
       [1571, 1596],
       [1743, 1781],
       [1600, 1644],
       [ 544,  582],
       [1247, 1289],
       [ 826,  870],
       [ 425,  463],
       [ 682,  705],
       [1585, 1611],
       [ 197,  238],
       [ 968, 1004],
       [1896, 1933],
       [1691, 1731],
       [ 494,  526],
       [ 895,  922],
       [1120, 1153],
       [1487, 1537],
       [1220, 1271],
       [ 447,  466],
       [1484, 1520],
       [1741, 1785],
       [ 805,  846],
       [ 187,  231],
       [1374, 1418],
       [1146, 1184],
       [1052, 1076],
       [ 692,  732],
       [1480, 1523],
       [ 796,  835],
       [ 519,  560],
       [ 206,  245],
       [ 646,  681],
       [1486, 1506],
       [1197, 1241],
       [1659, 1703],
       [ 415,  460],
       [ 274,  320],
       [1842, 1887],
       [ 981, 1022],
       [1826, 1867],
       [ 119,  159],
       [ 353,  399],
       [ 851,  880],
       [1767, 1812],
       [1282, 1317],
       [1199, 1247],
       [ 932,  958],
       [1584, 1624],
       [1336, 1359],
       [1091, 1133],
       [ 677,  713],
       [ 983, 1022],
       [1279, 1322],
       [ 136,  178],
       [ 786,  826],
       [1596, 1618],
       [ 791,  827],
       [1708, 1747],
       [1284, 1322],
       [ 231,  254],
       [1701, 1741],
       [1202, 1234],
       [ 465,  505],
       [ 866,  874]

The full list of the ValueError:

ValueError                                Traceback (most recent call last)
Cell In[178], line 89
     84             plt.gca().add_artist(ell)
     88 gmm = GaussianMixtureModel(n_components=2)
---> 89 gmm_fit = gmm.fit(disparity_matrix)
     91 gmm_predict = gmm._has_prediction(stars)
     92 cluster_means = gmm._has_cluster_means(disparity_matrix)

Cell In[178], line 17, in GaussianMixtureModel.fit(self, X)
     14 self.covariances = np.array([np.eye(n_features) for _ in range(self.n_components)])
     16 for _ in range(self.max_iter):
---> 17     expectation_max = self._expectation(X)
     18     self._maximization(X, expectation_max)
     19     if self._has_converged(X, expectation_max):

Cell In[178], line 31, in GaussianMixtureModel._expectation(self, X)
     29     # Ensure self.weights has the correct shape for broadcasting
     30     weights_k = np.squeeze(self.weights[k])
---> 31     exp_max[:, k] = weights_k * np.exp(exponent) / np.sqrt(np.linalg.det(self.covariances[k]) + 1e-6)
     32 exp_max /= exp_max.sum(axis=1)[:, np.newaxis]
     33 print('_expectation: ', exp_max.shape)

ValueError: could not broadcast input array from shape (2700,1) into shape (2700,)
```
$\endgroup$
2
  • $\begingroup$ This is a programming issue, and it should be posted at Stack Overflow instead of here. $\endgroup$
    – desertnaut
    Feb 12 at 12:50
  • $\begingroup$ thanks for the note on appropriate source: since the question was also about the correctness on a data science machine learning deep learning expectation maximization for a Gaussian Mixture Model algorithm, I debated which site to post the question, my thinking was that data science people may have an edge on what is wrong for the expectation maximization for a Gaussian Mixture Model algorithm to that is why I made a final decision to post here. Hope you understand my logic reasons for posting here. $\endgroup$ Feb 12 at 17:58

1 Answer 1

1
$\begingroup$

Hard to be sure, but it looks like your 'extra' dimension is coming from the exponent array. Can you try squeezing it, something like this:

exp_max[:, k] = weights_k * np.squeeze(np.exp(exponent)) / np.sqrt(np.linalg.det(self.covariances[k]) + 1e-6)

Also, the self.has_converged() method is undefined.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.