16

The second way, predicting $x=cos(\alpha)$ and $y=sin(\alpha)$ is totally okay. Yes, the norm of the predicted $(x, y)$ vector is not guaranteed to be near $1$. But it is not likely to blow up, especially if you use sigmoid activation functions (which are bounded by they nature) and/or regularize your model well. Why should your model predict a large value,...


9

I will assume by C1, C2, etc, you mean convolutional layers, and by P1 ,P2 you mean pooling layers, and FC means fully connected layers. We can calculate the memory required for a forward pass like this: One image If you're working with float32 values, then following the link provided above by @Alexandru Burlacu you have: Input: 50x50x3 = 7,500 = 7.5K C1: ...


9

You take the last column of that matrix sort descending plot index, distance hope to see a knee (if the distance does not work well. there might be none)


8

I assume you mean feature selection as feature engineering. The process I usually follow and I see some people do is Feature engineering Try a few algorithms, usually highly performant ones such as RandomForest, Gradient Boosted Trees, Neutral Networks, or SVM on the features. 2.1 Do simple parameter tuning such as grid search on a small range of ...


5

Let's check for continuity for the new proposed function. For convenience, let me call it $L_{\delta, c}$. $$\lim_{a \to \delta^+}L_{\delta, c}(a)=c\delta$$ and $$\lim_{a \to \delta^-}L_{\delta, c}(a)=\frac{\delta^2}2$$ If we want the function to be continuous, we would want $c$ to take value $\frac{\delta}2$. Now, let's check for diffentiability of $L_{\...


5

No! the best score on validation set means you are not in overfitting zone. As explained in my previous answer to your question, overfitting is about high score on training data but low score on validation. So to avoid overfitting you need to check your score on Validation Set and then you are fine. There is no theoretical calculation of the best depth of a ...


4

Likelihood-ratio tests are a mainstay of classical hypothesis testing. The idea is to form the likelihoods of the two hypotheses under consideration, and choose the one with the highest likelihood if their ratio is sufficiently large. Hypotheses come in two flavors: simple, and composite. Simple tests are those for which the hypothesis uniquely defines the ...


3

Maybe this link will give you an explanation on how to compute the memory usage of an arbitrary neural network. Bellow in the link is explained the memory usage of the VGGNet model. Click here and scroll down a bit))


3

Working with Cartesian coordinates works well as mentioned above. Yet, in my opinion, converting polar data to Cartesian creates dependencies between the X and Y coordinates that were not originally present in the data. For example, a robot's path decision model is more intuitive in polar coordinates than Cartesian. The dependency of the robot's velocity ...


2

Hey i just saw your question. It is COMPLETELY WRONG to do feature selection first and then tune the model using cross-validation. In elements of statistical learning and this blog post it is clearly mentioned that : CV method are unbiased only if all your model building is done inside the CV loop. So do feature selection inside the CV loop for parameter ...


2

Statistical approach This question is more related to statistics than data science, perharps you would have better answers on Cross Validated. As you suggested in the question, a purely statistical approach should work for sure (though it requires some assumptions). As i understand your problem, your prior knowledge is that $\alpha$ (or $\beta$) should be ...


2

I don't have a full answer to your question, but I wanted to help (and would love to know a complete answer). In a simpler case when you have one source $m=i=1$, it seems to me you are describing a scenario that resembles a hidden Markov model which uses a EM algorithm-based solution (Baum–Welch algorithm) – that is if you make the further hypothesis that a ...


1

$P(data|parameter)$ is not used in the sense of generating new data, but rather in the sense of how probable is that this data have been already generated by such parameters.


1

You can determine the most likely values for $\log x$ and $\log y$ by linear regression since your relationship implies: $\log a = b.\log y + \log x$ So run regression on $b$ as input and $\log a$ as output. The gradient will be $\log y$ and the intercept $\log x$


1

For theory Tibshirani: The elements of statistical learning https://web.stanford.edu/~hastie/Papers/ESLII.pdf Also Andrew NG and other books from deeplearning.ai: Machine Learning Yearning https://d2wvfoqc9gyqzf.cloudfront.net/content/uploads/2018/09/Ng-MLY01-13.pdf of course the applied machine learning books on computer languages: An introduction into ...


1

If the discrete variable has a lot of discrete values then it is almost the same as a continuous variable, because continuous variables are technically discrete due to the way how numbers are represented in computers (float64 for Python). The worst case is binary, but, in my experience, Pearson coefficient work well with binary and continuous data together. ...


1

Given that only have labels for a small subset of data, you should use unsupervised methods. You can cluster all the sensors data. Then see if there is a pattern to where the 100 bad sensors are and can generalize to the other sensors. If a majority of the 100 bad sensors are in the same region, you can label that cluster as bad and make a threshold based on ...


1

If you don't know which of the 10,000 sensors are good and which are bad, the data from those 10,000 sensors is useless for training a regression line / classifier. You need labelled data, where you know both the value of the features and the health of the sensor. Moreover, to be effective, you probably need your training set to contain both healthy ...


1

Actually there is the possibility of overfitting the validation set. This because the validation set is the one where your parameters (the depth in your case) perform at best, but this does not means that your model will generalize well on unseen data. That's the reason why usually you split your data into three set: train, validation and test. You train ...


1

"Oracle" refers to something that has access to the ground truth. It has the perfect information of which in practice, we rarely have some luxury. In such paper, you can typically see that it reflects a comparison with a model with perfect information. From Candes and Terry Tao's paper: "Even though $n$ may be much smaller than $p$, our estimator achieves ...


1

While training a convNet, total memory required include following: Memory for parameters Memory for the output of intermediate layers Memory for the gradient of each parameter Extra memory needed if you are using optimizer like Momentum, RMSprop, Adams etc Miscellaneous memory for implementation A good rough approximation is number of parameters x 3 x 4(if ...


1

Since you want your model to be a general solution, you want to include all your data when building the final model. You are correct in saying that keeping the model with the best validation score in the CV is overfitting. Including these inner random states help generalize your model, and since you have already tuned your model parameters using CV, you ...


1

Many ML and minimization tasks make use of an objective function. At each iteration, a parameter set to try is defined, and the objective function returns some scoring value that reflects how good, or bad, that parameter set is. Then the parameter set is altered, and the process repeats. So, when do you stop this process? You want to stop when the changes ...


1

A naive definition of Parameter convergence is when the weights or the values of the parameters reach a point asymptotically. What I mean is that, when your model training is not altering the parameter values(maybe less than epsilon-small values) it might be a good fit. For decision trees, I found this paper which explains rate of convergence and more. It ...


1

Here is the code I wrote to answer my question. It might not be the most efficient one but it works. Sharing is caring :) niter<-10000 N<-nrow(Y) T<-10 # I will take into consideration until t=10 to estimate my parametres and then I will forecast the rest values t=11,12,... etc result <- matrix(0,nrow = niter, ncol = 3) #Here I store 3 ...


1

In terms of writing this in R, here is an example I found: http://www.stat.purdue.edu/~zhanghao/MAS/handout/gibbsBayesian.pdf You could then iterate through different levels of α and select the level yielding the best results.


1

If you were fitting a large number of different models, and you had sequences of training data for each different model (with the params already known in those cases), then you might be able to use a recursive neural network (RNN) to provide param estimates for new data sequences. However, that does not seem to be your situation. As I understand your ...


1

You want to look at the spectral density to get an idea of the distribution of the frequency components. Check out the 'psd' package in r or the signal processing toolbox if you have access to Matlab.


1

I have already answered a similar question here. The process would be: Transformation and Reduction: Involves processes like transformations, mean and median scaling, etc. Feature Selection: This can be done in a lot of ways like threshold selection, subset selection, etc. Designing predictive model: Design the predictive model on the training data ...


Only top voted, non community-wiki answers of a minimum length are eligible