Questions tagged [expectation-maximization]

An optimization algorithm often used for maximum-likelihood estimation in the presence of missing data.

Filter by
Sorted by
Tagged with
0 votes
1 answer
29 views

GMM expectation maximization ValueError: could not broadcast input array from shape (2700,1) into shape (2700,)

Problem Statement The goal is to have the Gaussian Mixture Model has_prediction() function to execute after the Gaussian Mixture Model fix() function that will invoke methods on this custom ...
Data Science Analytics Manager's user avatar
2 votes
1 answer
220 views

Math of Gaussian Mixture Models & EM Algorithm

I'm having a hard time understanding the math behind GMMs. A GMM is a weighted sum over K different Gaussian components with parameters $\mu_k, \sigma_k, \pi_k$ From my understanding, the general ...
bosonphoton's user avatar
0 votes
1 answer
67 views

Searching optimum ranges for maximizing a variable

I am currently working on a project where I have been asked to analyze a dataframe with various time series. ...
Islen's user avatar
  • 1
1 vote
0 answers
64 views

How to derive Evidence Lower Bound in the paper "Zero-Shot Text-to-Image Generation"?

Can someone share the derivation of Evidence Lower Bound in this paper ? Zero-Shot Text-to-Image Generation The overall procedure can be viewed as maximizing the evidence lower bound (ELB) (Kingma &...
p1p13 's user avatar
  • 11
2 votes
0 answers
64 views

Which latent variable model is better to find hidden variable?

Currently, I am exploring the concept of latent variable for regression type datasets. I have gone through literature of few of ...
Mohammad Saad's user avatar
1 vote
0 answers
31 views

Active learning with mixture model cluster assignments - am I injecting bias here?

Suppose I have a dataset of people's phone numbers and heights, and I'm interested in learning the parameters $p_{girl}$, $p_{boy}=1-p_{girl}$, $\mu_{boy}$, $\mu_{girl}$, and overall $\sigma$ ...
goopy's user avatar
  • 76
1 vote
1 answer
66 views

How to find the feature regions where each label is the most expected when using decision trees?

Given a decision tree for classification for example this one: What is the way to find the feature domain (petal and sepal width and length) where a sample would most likely occur in the feature ...
lalaland's user avatar
  • 113
3 votes
1 answer
148 views

Why Gaussian mixture model uses Expectation maximization instead of Gradient descent?

Why Gaussian mixture model uses Expectation maximization instead of Gradient descent? What other models uses Expectation maximization to find best optimal parameters instead of using gradient descent?
star's user avatar
  • 1,461
1 vote
0 answers
497 views

E-step for EM algorithm for document clustering

I have a code for the E-step in the EM algorithm for Document Clustering in the version of hard-EM algorithm. I'm trying to implement the E-step for soft-EM algorithm. Here is my code for Hard-EM: <...
t.wangd's user avatar
  • 11
1 vote
0 answers
31 views

N-Gram Linear Smoothing

In slide 61 of the NLP text, to smooth out the n-gram probabilities, we need to find the lambdas the miximazies a probability to held-out set given in terms of M(λ1, λ2, ...λ_k). What does this ...
chikitin's user avatar
  • 153
1 vote
1 answer
82 views

Gaussian Mixture Models Clustering

When using the EM algorithm in Gaussian Mixture Models (GMM), in the E-step, we take each x set in the training dataset to calculate and update the "weight" and parameters of each Gaussian ...
YCCCCC's user avatar
  • 121
0 votes
1 answer
269 views

Best python library for training using Hidden Marov model with Gaussian Mixture

I would like to train my data using HMM- GMM (Baum Welch approach with gaussian Mixture) to find the best parameters suited for my data. Note : My data is ...
Mari's user avatar
  • 165
2 votes
0 answers
366 views

Does feature normalization improve performance of Hidden Markov Models?

For training a Hidden Markov Model (HMM) on a multivariate, continuous time series, is it preferable to scale the data somehow? Some pre-processing steps may be: Normalize to 0-mean and unit-variance ...
hazrmard's user avatar
  • 328
0 votes
1 answer
42 views

How would you describe cluster 2 from this output of a run of the EM program?

My description: Cluster 2 consists of 9511 instances, the age is around 42 (ranges between 29.7207 and 54.5257). Considering Age, Cluster 2 is very well separated from Cluster 1, with a distance of ...
Shroomy's user avatar
4 votes
1 answer
131 views

Can Expectation Maximization estimate truth and confusion matrix from multiple noisy sources?

Suppose we have $m$ sources, each of which noisily observe the same set of $n$ independent events from the outcome set $\{A,B,C\}$. Each source has a confusion matrix, for example for source $i$: $$...
Brendan Hill's user avatar
0 votes
1 answer
38 views

How to interpret the mean for output clusters for expected-maximization?

I am trying to cluster data using scikit's expectation-maximization. So I created two different data sets from a normal distribution which is I have shown in the graph below. The mean for each of the ...
Suhail Gupta's user avatar
1 vote
0 answers
1k views

Code or Package to cluster sequences (or time series) of different lengths based on HMM?

Is there any existing code or packages in Python, R, Java, Matlab, or Scala that implements the sequence clustering algorithms in any of the following 2 papers? 1) 'Clustering Sequences with Hidden ...
mflowww's user avatar
  • 111
2 votes
1 answer
44 views

Statistical machine translation word alignment for FR-ENG and ENG-FR: what is p(e) and p(f)?

I'm currently trying to implement this paper, but am struggling to understand some of the math here. I'm pretty sure I understand how to implement the E-step, but for the M-step, I'm confused on how ...
mudkipium's user avatar
0 votes
2 answers
42 views

EM clustering with missing and misspelling data

I am currently working on a project that requires me to cluster the unlabeled input. The records contain personal information such as name, DOB, height, sex, etc. We need to cluster the same person in ...
roy's user avatar
  • 17
0 votes
2 answers
2k views

Does K-Means' objective function imply distance metric is Euclidean

The objective/loss function of K-Means algorithm is to minimize the sum of squared distances, written in a math form, it looks like this: $$J(X,Z) = min\ \sum_{z\in Clusters}\sum_{x \in data}||x-z||^2$...
Logan's user avatar
  • 453
1 vote
0 answers
93 views

Hidden Markov Models: Linking states to labels after EM training

The tl;dr version first: I have the following problem: I implemented Baum Welch for ergodic HMMs. I do it like this: I pass the model two number C1 and ...
lo tolmencre's user avatar
3 votes
1 answer
116 views

How to compare the performance of different number of mixing components for EM algorithm?

I am reading about the EM (Expectation-Maximization) algorithm in a machine learning book. At the end remark of the chapter, the authors mentioned that we cannot decide the "optimality" of the number ...
San Yeung's user avatar
2 votes
0 answers
35 views

With EM algorithm, can you infer the location and variance of each "peak" in a pdf? Gaussian Mixture Models?

When I plot my data into bins, there is a frequency of data points per bin, which I can plot with a histogram. Based on this probability density function, I would like to find the maximum likelihood ...
ShanZhengYang's user avatar
3 votes
1 answer
2k views

Using EM (Expectation Maximization) algorithm for Training Logistic Regression

Is it possible to learn the weights for a logistic regression classifier using EM (Expectation Maximization)algorithm? Is there any instance reference?
BlueGirl's user avatar
  • 171