19
votes
Accepted
Parameterization regression of rotation angle
The second way, predicting $x=cos(\alpha)$ and $y=sin(\alpha)$ is totally okay.
Yes, the norm of the predicted $(x, y)$ vector is not guaranteed to be near $1$. But it is not likely to blow up, ...
- 1,551
9
votes
CNN memory consumption
I will assume by C1, C2, etc, you mean convolutional layers, and by P1 ,...
- 2,011
5
votes
How are parameters selected in cross-validation?
Usually, the aim of K-fold cross-validation is to check how a model performs (both on average and how much it varies across folds) given some hyper-parameters setting. We then pick the "best"...
- 1,117
5
votes
Accepted
Why Huber loss has its form?
Let's check for continuity for the new proposed function. For convenience, let me call it $L_{\delta, c}$.
$$\lim_{a \to \delta^+}L_{\delta, c}(a)=c\delta$$ and $$\lim_{a \to \delta^-}L_{\delta, c}(a)...
- 3,128
5
votes
How to calculate ideal Decision Tree depth without overfitting?
No! the best score on validation set means you are not in overfitting zone. As explained in my previous answer to your question, overfitting is about high score on training data but low score on ...
- 6,622
4
votes
generalized likelihood ratio test (GLRT)
Likelihood-ratio tests are a mainstay of classical hypothesis testing. The idea is to form the likelihoods of the two hypotheses under consideration, and choose the one with the highest likelihood if ...
- 10.5k
3
votes
CNN memory consumption
Maybe this link will give you an explanation on how to compute the memory usage of an arbitrary neural network. Bellow in the link is explained the memory usage of the VGGNet model. Click here and ...
3
votes
What is "oracle" in statistics?
"Oracle" refers to something that has access to the ground truth. It has the perfect information of which in practice, we rarely have some luxury. In such paper, you can typically see that it reflects ...
- 3,128
3
votes
Parameterization regression of rotation angle
Working with Cartesian coordinates works well as mentioned above. Yet, in my opinion, converting polar data to Cartesian creates dependencies between the X and Y coordinates that were not originally ...
2
votes
How to calculate ideal Decision Tree depth without overfitting?
Actually there is the possibility of overfitting the validation set.
This because the validation set is the one where your parameters (the depth in your case) perform at best, but this does not means ...
- 1,774
2
votes
Which one first: algorithm benchmarking, feature selection, parameter tuning?
I just saw your question. It is COMPLETELY WRONG to do feature selection first and then tune the model using cross-validation. In elements of statistical learning and this blog post, it is clearly ...
- 378
2
votes
Paramaeter estimation in noisy conditions with Machine Learning, possible?
Statistical approach
This question is more related to statistics than data science, perharps you would have better answers on Cross Validated. As you suggested in the question, a purely statistical ...
- 1,337
2
votes
Accepted
Can Expectation Maximization estimate truth and confusion matrix from multiple noisy sources?
I don't have a full answer to your question, but I wanted to help (and would love to know a complete answer). In a simpler case when you have one source $m=i=1$, it seems to me you are describing a ...
- 727
2
votes
Is probabilities mean of predicted class (RandomForest) a consistent estimator of class recall?
Unfortunately this doesn't work. To understand why, let's think about the perfect classifier and the maximally wrong classifier.
The perfect classifier has 100% recall on any given class. If there are ...
- 1,530
1
vote
Is number of Parameters a sufficient benchmark for measuring how much resource the end model will use?
Isnt measuring source usage of ML models an important subject?
The efficiency of machine learning models is very important. Since machine learning is primarily an empirical field, hardware choices ...
- 22.1k
1
vote
Is number of Parameters a sufficient benchmark for measuring how much resource the end model will use?
number of parameters linearly corresponds to FLOPS
In general no, since FLOPS depends not only on # of parameters, but also on the computation required (e.g. model complexity).
For example, say we ...
- 1,117
1
vote
Using scipy.minimize to find the maximum likelihood estimates for multivariate gaussian
The problem actually was the problem.
I have since learned that the covariance matrix was to be assumed as a constant and it is in fact only the mean which is to be determined.
Removing this ...
- 31
1
vote
Verifying my understanding of MLE & Gradient Descent in Logistic Regression
2) is Wrong. MLE stands for Maximum likelihood estimation. Not an analytical method.
- 1,117
1
vote
Accepted
Why should MLE be considered in Logistic Regression when it cannot give a definite solution?
Maximum likelihood is a method for estimating parameters.
Gradient descent is a numerical technique to help us solve equations that we might not be able to solve by traditional means (e.g., we can't ...
- 4,044
1
vote
Why should MLE be considered in Logistic Regression when it cannot give a definite solution?
I think you are comparing apples and oranges here. Maximum likelihood is a the maximum value of your likelihood function, which somehow describes your data generation process. Specifically likelihood ...
- 563
1
vote
Accepted
Understanding the likelihood function
$P(data|parameter)$ is not used in the sense of generating new data, but rather in the sense of how probable is that this data have been already generated by such parameters.
- 2,383
1
vote
How to reverse engineer a logarithmic equation
You can determine the most likely values for $\log x$ and $\log y$ by linear regression since your relationship implies:
$\log a = b.\log y + \log x$
So run regression on $b$ as input and $\log a$ as ...
- 563
1
vote
Accepted
Books about statistical inference
For theory Tibshirani: The elements of statistical learning
https://web.stanford.edu/~hastie/Papers/ESLII.pdf
Also Andrew NG and other books from deeplearning.ai:
Machine Learning Yearning
https://...
- 329
1
vote
Pearson correlation coefficient - is correlation estimator acceptable?
If the discrete variable has a lot of discrete values then it is almost the same as a continuous variable, because continuous variables are technically discrete due to the way how numbers are ...
- 1,299
1
vote
Why Maximum Likelihood Estimation for normal distribution?
While it is true that we can compute the sample mean and sample standard deviation from a set of random variables, the Maximum Likelihood Estimation provides a formal framework for estimating the ...
- 411
1
vote
Should I fit my parameters with brute force
Given that only have labels for a small subset of data, you should use unsupervised methods. You can cluster all the sensors data. Then see if there is a pattern to where the 100 bad sensors are and ...
- 22.1k
1
vote
Accepted
Should I fit my parameters with brute force
If you don't know which of the 10,000 sensors are good and which are bad, the data from those 10,000 sensors is useless for training a regression line / classifier. You need labelled data, where you ...
- 3,401
1
vote
CNN memory consumption
While training a convNet, total memory required include following:
Memory for parameters
Memory for the output of intermediate layers
Memory for the gradient of each parameter
Extra memory needed if ...
- 11
1
vote
Accepted
Gibbs sampling in R
Here is the code I wrote to answer my question. It might not be the most efficient one but it works. Sharing is caring :)
...
- 353
1
vote
Accepted
Variance in cross validation score / model selection
Since you want your model to be a general solution, you want to include all your data when building the final model. You are correct in saying that keeping the model with the best validation score in ...
- 1,459
Only top scored, non community-wiki answers of a minimum length are eligible