# Tag Info

Accepted

### Parameterization regression of rotation angle

The second way, predicting $x=cos(\alpha)$ and $y=sin(\alpha)$ is totally okay. Yes, the norm of the predicted $(x, y)$ vector is not guaranteed to be near $1$. But it is not likely to blow up, ...
• 1,551

### CNN memory consumption

I will assume by C1, C2, etc, you mean convolutional layers, and by P1 ,...
• 2,011

### How are parameters selected in cross-validation?

Usually, the aim of K-fold cross-validation is to check how a model performs (both on average and how much it varies across folds) given some hyper-parameters setting. We then pick the "best"...
• 1,117
Accepted

### Why Huber loss has its form?

Let's check for continuity for the new proposed function. For convenience, let me call it $L_{\delta, c}$. $$\lim_{a \to \delta^+}L_{\delta, c}(a)=c\delta$$ and \lim_{a \to \delta^-}L_{\delta, c}(a)...
• 3,128

### How to calculate ideal Decision Tree depth without overfitting?

No! the best score on validation set means you are not in overfitting zone. As explained in my previous answer to your question, overfitting is about high score on training data but low score on ...
• 6,622

### generalized likelihood ratio test (GLRT)

Likelihood-ratio tests are a mainstay of classical hypothesis testing. The idea is to form the likelihoods of the two hypotheses under consideration, and choose the one with the highest likelihood if ...
• 10.5k

### CNN memory consumption

Maybe this link will give you an explanation on how to compute the memory usage of an arbitrary neural network. Bellow in the link is explained the memory usage of the VGGNet model. Click here and ...

### What is "oracle" in statistics?

"Oracle" refers to something that has access to the ground truth. It has the perfect information of which in practice, we rarely have some luxury. In such paper, you can typically see that it reflects ...
• 3,128

### Parameterization regression of rotation angle

Working with Cartesian coordinates works well as mentioned above. Yet, in my opinion, converting polar data to Cartesian creates dependencies between the X and Y coordinates that were not originally ...

### How to calculate ideal Decision Tree depth without overfitting?

Actually there is the possibility of overfitting the validation set. This because the validation set is the one where your parameters (the depth in your case) perform at best, but this does not means ...
• 1,774

### Which one first: algorithm benchmarking, feature selection, parameter tuning?

I just saw your question. It is COMPLETELY WRONG to do feature selection first and then tune the model using cross-validation. In elements of statistical learning and this blog post, it is clearly ...

### Paramaeter estimation in noisy conditions with Machine Learning, possible?

Statistical approach This question is more related to statistics than data science, perharps you would have better answers on Cross Validated. As you suggested in the question, a purely statistical ...
• 1,337
Accepted

### Can Expectation Maximization estimate truth and confusion matrix from multiple noisy sources?

I don't have a full answer to your question, but I wanted to help (and would love to know a complete answer). In a simpler case when you have one source $m=i=1$, it seems to me you are describing a ...
• 727

### Is probabilities mean of predicted class (RandomForest) a consistent estimator of class recall?

Unfortunately this doesn't work. To understand why, let's think about the perfect classifier and the maximally wrong classifier. The perfect classifier has 100% recall on any given class. If there are ...
1 vote

### Is number of Parameters a sufficient benchmark for measuring how much resource the end model will use?

Isnt measuring source usage of ML models an important subject? The efficiency of machine learning models is very important. Since machine learning is primarily an empirical field, hardware choices ...
• 22.1k
1 vote

### Is number of Parameters a sufficient benchmark for measuring how much resource the end model will use?

number of parameters linearly corresponds to FLOPS In general no, since FLOPS depends not only on # of parameters, but also on the computation required (e.g. model complexity). For example, say we ...
• 1,117
1 vote

### Using scipy.minimize to find the maximum likelihood estimates for multivariate gaussian

The problem actually was the problem. I have since learned that the covariance matrix was to be assumed as a constant and it is in fact only the mean which is to be determined. Removing this ...
• 31
1 vote

### Verifying my understanding of MLE & Gradient Descent in Logistic Regression

2) is Wrong. MLE stands for Maximum likelihood estimation. Not an analytical method.
• 1,117
1 vote
Accepted

### Why should MLE be considered in Logistic Regression when it cannot give a definite solution?

Maximum likelihood is a method for estimating parameters. Gradient descent is a numerical technique to help us solve equations that we might not be able to solve by traditional means (e.g., we can't ...
• 4,044
1 vote

### Why should MLE be considered in Logistic Regression when it cannot give a definite solution?

I think you are comparing apples and oranges here. Maximum likelihood is a the maximum value of your likelihood function, which somehow describes your data generation process. Specifically likelihood ...
• 563
1 vote
Accepted

### Understanding the likelihood function

$P(data|parameter)$ is not used in the sense of generating new data, but rather in the sense of how probable is that this data have been already generated by such parameters.
• 2,383
1 vote

### How to reverse engineer a logarithmic equation

You can determine the most likely values for $\log x$ and $\log y$ by linear regression since your relationship implies: $\log a = b.\log y + \log x$ So run regression on $b$ as input and $\log a$ as ...
• 563
1 vote
Accepted

For theory Tibshirani: The elements of statistical learning https://web.stanford.edu/~hastie/Papers/ESLII.pdf Also Andrew NG and other books from deeplearning.ai: Machine Learning Yearning https://...
• 329
1 vote

### Pearson correlation coefficient - is correlation estimator acceptable?

If the discrete variable has a lot of discrete values then it is almost the same as a continuous variable, because continuous variables are technically discrete due to the way how numbers are ...
• 1,299
1 vote

### Why Maximum Likelihood Estimation for normal distribution?

While it is true that we can compute the sample mean and sample standard deviation from a set of random variables, the Maximum Likelihood Estimation provides a formal framework for estimating the ...
• 411
1 vote

### Should I fit my parameters with brute force

Given that only have labels for a small subset of data, you should use unsupervised methods. You can cluster all the sensors data. Then see if there is a pattern to where the 100 bad sensors are and ...
• 22.1k
1 vote
Accepted

### Should I fit my parameters with brute force

If you don't know which of the 10,000 sensors are good and which are bad, the data from those 10,000 sensors is useless for training a regression line / classifier. You need labelled data, where you ...
• 3,401
1 vote

### CNN memory consumption

While training a convNet, total memory required include following: Memory for parameters Memory for the output of intermediate layers Memory for the gradient of each parameter Extra memory needed if ...
1 vote
Accepted

### Gibbs sampling in R

Here is the code I wrote to answer my question. It might not be the most efficient one but it works. Sharing is caring :) ...
• 353
1 vote
Accepted

### Variance in cross validation score / model selection

Since you want your model to be a general solution, you want to include all your data when building the final model. You are correct in saying that keeping the model with the best validation score in ...
• 1,459

Only top scored, non community-wiki answers of a minimum length are eligible