Questions tagged [expectation-maximization]

An optimization algorithm often used for maximum-likelihood estimation in the presence of missing data.

Filter by
Sorted by
Tagged with
4
votes
1answer
105 views

Can Expectation Maximization estimate truth and confusion matrix from multiple noisy sources?

Suppose we have $m$ sources, each of which noisily observe the same set of $n$ independent events from the outcome set $\{A,B,C\}$. Each source has a confusion matrix, for example for source $i$: $$...
3
votes
1answer
2k views

Using EM (Expectation Maximization) algorithm for Training Logistic Regression

Is it possible to learn the weights for a logistic regression classifier using EM (Expectation Maximization)algorithm? Is there any instance reference?
3
votes
1answer
68 views

Why Gaussian mixture model uses Expectation maximization instead of Gradient descent?

Why Gaussian mixture model uses Expectation maximization instead of Gradient descent? What other models uses Expectation maximization to find best optimal parameters instead of using gradient descent?
3
votes
1answer
84 views

How to compare the performance of different number of mixing components for EM algorithm?

I am reading about the EM (Expectation-Maximization) algorithm in a machine learning book. At the end remark of the chapter, the authors mentioned that we cannot decide the "optimality" of the number ...
2
votes
1answer
36 views

Statistical machine translation word alignment for FR-ENG and ENG-FR: what is p(e) and p(f)?

I'm currently trying to implement this paper, but am struggling to understand some of the math here. I'm pretty sure I understand how to implement the E-step, but for the M-step, I'm confused on how ...
2
votes
0answers
45 views

Which latent variable model is better to find hidden variable?

Currently, I am exploring the concept of latent variable for regression type datasets. I have gone through literature of few of ...
2
votes
0answers
203 views

Does feature normalization improve performance of Hidden Markov Models?

For training a Hidden Markov Model (HMM) on a multivariate, continuous time series, is it preferable to scale the data somehow? Some pre-processing steps may be: Normalize to 0-mean and unit-variance ...
1
vote
1answer
49 views

Gaussian Mixture Models Clustering

When using the EM algorithm in Gaussian Mixture Models (GMM), in the E-step, we take each x set in the training dataset to calculate and update the "weight" and parameters of each Gaussian ...
1
vote
0answers
39 views

How to derive Evidence Lower Bound in the paper "Zero-Shot Text-to-Image Generation"?

Can someone share the derivation of Evidence Lower Bound in this paper ? Zero-Shot Text-to-Image Generation The overall procedure can be viewed as maximizing the evidence lower bound (ELB) (Kingma &...
1
vote
0answers
25 views

Active learning with mixture model cluster assignments - am I injecting bias here?

Suppose I have a dataset of people's phone numbers and heights, and I'm interested in learning the parameters $p_{girl}$, $p_{boy}=1-p_{girl}$, $\mu_{boy}$, $\mu_{girl}$, and overall $\sigma$ ...
1
vote
1answer
41 views

How to find the feature regions where each label is the most expected when using decision trees?

Given a decision tree for classification for example this one: What is the way to find the feature domain (petal and sepal width and length) where a sample would most likely occur in the feature ...
1
vote
0answers
188 views

E-step for EM algorithm for document clustering

I have a code for the E-step in the EM algorithm for Document Clustering in the version of hard-EM algorithm. I'm trying to implement the E-step for soft-EM algorithm. Here is my code for Hard-EM: <...
1
vote
0answers
26 views

N-Gram Linear Smoothing

In slide 61 of the NLP text, to smooth out the n-gram probabilities, we need to find the lambdas the miximazies a probability to held-out set given in terms of M(λ1, λ2, ...λ_k). What does this ...
1
vote
0answers
959 views

Code or Package to cluster sequences (or time series) of different lengths based on HMM?

Is there any existing code or packages in Python, R, Java, Matlab, or Scala that implements the sequence clustering algorithms in any of the following 2 papers? 1) 'Clustering Sequences with Hidden ...
1
vote
0answers
81 views

Hidden Markov Models: Linking states to labels after EM training

The tl;dr version first: I have the following problem: I implemented Baum Welch for ergodic HMMs. I do it like this: I pass the model two number C1 and ...
1
vote
0answers
32 views

With EM algorithm, can you infer the location and variance of each "peak" in a pdf? Gaussian Mixture Models?

When I plot my data into bins, there is a frequency of data points per bin, which I can plot with a histogram. Based on this probability density function, I would like to find the maximum likelihood ...
0
votes
2answers
1k views

Does K-Means' objective function imply distance metric is Euclidean

The objective/loss function of K-Means algorithm is to minimize the sum of squared distances, written in a math form, it looks like this: $$J(X,Z) = min\ \sum_{z\in Clusters}\sum_{x \in data}||x-z||^2$...
0
votes
2answers
36 views

EM clustering with missing and misspelling data

I am currently working on a project that requires me to cluster the unlabeled input. The records contain personal information such as name, DOB, height, sex, etc. We need to cluster the same person in ...
0
votes
0answers
137 views

Best python library for training using Hidden Marov model with Gaussian Mixture

I would like to train my data using HMM- GMM (Baum Welch approach with gaussian Mixture) to find the best parameters suited for my data. Note : My data is ...
0
votes
1answer
39 views

How would you describe cluster 2 from this output of a run of the EM program?

My description: Cluster 2 consists of 9511 instances, the age is around 42 (ranges between 29.7207 and 54.5257). Considering Age, Cluster 2 is very well separated from Cluster 1, with a distance of ...
0
votes
1answer
30 views

How to interpret the mean for output clusters for expected-maximization?

I am trying to cluster data using scikit's expectation-maximization. So I created two different data sets from a normal distribution which is I have shown in the graph below. The mean for each of the ...