Skip to main content
19 votes
Accepted

Parameterization regression of rotation angle

The second way, predicting $x=cos(\alpha)$ and $y=sin(\alpha)$ is totally okay. Yes, the norm of the predicted $(x, y)$ vector is not guaranteed to be near $1$. But it is not likely to blow up, ...
David Dale's user avatar
  • 1,551
9 votes

CNN memory consumption

I will assume by C1, C2, etc, you mean convolutional layers, and by P1 ,...
StatsSorceress's user avatar
9 votes
Accepted

Knn distance plot for determining eps of DBSCAN

You take the last column of that matrix sort descending plot index, distance hope to see a knee (if the distance does not work well. there might be none)
Has QUIT--Anony-Mousse's user avatar
8 votes
Accepted

Which one first: algorithm benchmarking, feature selection, parameter tuning?

I assume you mean feature selection as feature engineering. The process I usually follow and I see some people do is Feature engineering Try a few algorithms, usually highly performant ones such as ...
Tu N.'s user avatar
  • 509
5 votes
Accepted

Why Huber loss has its form?

Let's check for continuity for the new proposed function. For convenience, let me call it $L_{\delta, c}$. $$\lim_{a \to \delta^+}L_{\delta, c}(a)=c\delta$$ and $$\lim_{a \to \delta^-}L_{\delta, c}(a)...
Siong Thye Goh's user avatar
5 votes

How to calculate ideal Decision Tree depth without overfitting?

No! the best score on validation set means you are not in overfitting zone. As explained in my previous answer to your question, overfitting is about high score on training data but low score on ...
Kasra Manshaei's user avatar
5 votes

How are parameters selected in cross-validation?

Usually, the aim of K-fold cross-validation is to check how a model performs (both on average and how much it varies across folds) given some hyper-parameters setting. We then pick the "best"...
lpounng's user avatar
  • 1,018
4 votes

generalized likelihood ratio test (GLRT)

Likelihood-ratio tests are a mainstay of classical hypothesis testing. The idea is to form the likelihoods of the two hypotheses under consideration, and choose the one with the highest likelihood if ...
Emre's user avatar
  • 10.5k
3 votes

CNN memory consumption

Maybe this link will give you an explanation on how to compute the memory usage of an arbitrary neural network. Bellow in the link is explained the memory usage of the VGGNet model. Click here and ...
Alexandru  Burlacu's user avatar
3 votes

Parameterization regression of rotation angle

Working with Cartesian coordinates works well as mentioned above. Yet, in my opinion, converting polar data to Cartesian creates dependencies between the X and Y coordinates that were not originally ...
Stav Bar-Sheshet's user avatar
3 votes

What is "oracle" in statistics?

"Oracle" refers to something that has access to the ground truth. It has the perfect information of which in practice, we rarely have some luxury. In such paper, you can typically see that it reflects ...
Siong Thye Goh's user avatar
2 votes

Which one first: algorithm benchmarking, feature selection, parameter tuning?

I just saw your question. It is COMPLETELY WRONG to do feature selection first and then tune the model using cross-validation. In elements of statistical learning and this blog post, it is clearly ...
Dhruv Mahajan's user avatar
2 votes

How to calculate ideal Decision Tree depth without overfitting?

Actually there is the possibility of overfitting the validation set. This because the validation set is the one where your parameters (the depth in your case) perform at best, but this does not means ...
Vincenzo Lavorini's user avatar
2 votes

Paramaeter estimation in noisy conditions with Machine Learning, possible?

Statistical approach This question is more related to statistics than data science, perharps you would have better answers on Cross Validated. As you suggested in the question, a purely statistical ...
Romain Reboulleau's user avatar
2 votes
Accepted

Can Expectation Maximization estimate truth and confusion matrix from multiple noisy sources?

I don't have a full answer to your question, but I wanted to help (and would love to know a complete answer). In a simpler case when you have one source $m=i=1$, it seems to me you are describing a ...
kevins_1's user avatar
  • 717
2 votes

Is probabilities mean of predicted class (RandomForest) a consistent estimator of class recall?

Unfortunately this doesn't work. To understand why, let's think about the perfect classifier and the maximally wrong classifier. The perfect classifier has 100% recall on any given class. If there are ...
Nicholas James Bailey's user avatar
1 vote

Why should MLE be considered in Logistic Regression when it cannot give a definite solution?

I think you are comparing apples and oranges here. Maximum likelihood is a the maximum value of your likelihood function, which somehow describes your data generation process. Specifically likelihood ...
Cryo's user avatar
  • 533
1 vote
Accepted

Understanding the likelihood function

$P(data|parameter)$ is not used in the sense of generating new data, but rather in the sense of how probable is that this data have been already generated by such parameters.
Nikos M.'s user avatar
  • 2,343
1 vote

How to reverse engineer a logarithmic equation

You can determine the most likely values for $\log x$ and $\log y$ by linear regression since your relationship implies: $\log a = b.\log y + \log x$ So run regression on $b$ as input and $\log a$ as ...
Cryo's user avatar
  • 533
1 vote
Accepted

Books about statistical inference

For theory Tibshirani: The elements of statistical learning https://web.stanford.edu/~hastie/Papers/ESLII.pdf Also Andrew NG and other books from deeplearning.ai: Machine Learning Yearning https://...
martin's user avatar
  • 329
1 vote

Pearson correlation coefficient - is correlation estimator acceptable?

If the discrete variable has a lot of discrete values then it is almost the same as a continuous variable, because continuous variables are technically discrete due to the way how numbers are ...
keiv.fly's user avatar
  • 1,259
1 vote

Why Maximum Likelihood Estimation for normal distribution?

While it is true that we can compute the sample mean and sample standard deviation from a set of random variables, the Maximum Likelihood Estimation provides a formal framework for estimating the ...
technik's user avatar
  • 391
1 vote

Should I fit my parameters with brute force

Given that only have labels for a small subset of data, you should use unsupervised methods. You can cluster all the sensors data. Then see if there is a pattern to where the 100 bad sensors are and ...
Brian Spiering's user avatar
1 vote
Accepted

Should I fit my parameters with brute force

If you don't know which of the 10,000 sensors are good and which are bad, the data from those 10,000 sensors is useless for training a regression line / classifier. You need labelled data, where you ...
D.W.'s user avatar
  • 3,351
1 vote

CNN memory consumption

While training a convNet, total memory required include following: Memory for parameters Memory for the output of intermediate layers Memory for the gradient of each parameter Extra memory needed if ...
Vijendra1125's user avatar
1 vote
Accepted

Variance in cross validation score / model selection

Since you want your model to be a general solution, you want to include all your data when building the final model. You are correct in saying that keeping the model with the best validation score in ...
Hobbes's user avatar
  • 1,439
1 vote

What is 'parameter convergence'?

Many ML and minimization tasks make use of an objective function. At each iteration, a parameter set to try is defined, and the objective function returns some scoring value that reflects how good, or ...
HEITZ's user avatar
  • 911
1 vote
Accepted

What is 'parameter convergence'?

A naive definition of Parameter convergence is when the weights or the values of the parameters reach a point asymptotically. What I mean is that, when your model training is not altering the ...
Hima Varsha's user avatar
  • 2,326
1 vote
Accepted

Gibbs sampling in R

Here is the code I wrote to answer my question. It might not be the most efficient one but it works. Sharing is caring :) ...
quant's user avatar
  • 353
1 vote

Gibbs sampling in R

In terms of writing this in R, here is an example I found: http://www.stat.purdue.edu/~zhanghao/MAS/handout/gibbsBayesian.pdf You could then iterate through different levels of α and select the level ...
Hobbes's user avatar
  • 1,439

Only top scored, non community-wiki answers of a minimum length are eligible