# Tag Info

10

It depends on the definition of accurate model, but in general the answer to your question 1) is No. Regarding your second question (based on results in the paper of Niculescu-Mizil & Caruana linked below): boosted trees and stumps - NO Naive Bayes - NO SVM - NO bagged trees - YES neural nets - YES You can test whether it is the case for your ...

7

By doing so, the joint distribution can be found easily by just multiplying the probability of each feature whilst in the real world they may not be independent and you have to find the correct joint distribution. It is naive due to this simplification.

6

When you have sensors, the values you receive change even if the signal that was recorded didn't change. This is one example of noise. When you have a model of the world, it abstracts from the real relationships by simplifying things which are not too important. To take into account for the simplification, you model the error as noise (e.g. in a Kalman ...

6

Both are literally the same. A Belief network is the one, where we establish a belief that certain event A will occur, given B. The network assumes the structure of a directed graph. The term Bayesian was coined after the name of Thomas Bayes.

5

Your problem could be solved either by direct numeric integration or by MCMC. Numeric integration can be performed most easily by scipy: import numpy as np import scipy.stats import scipy.integrate def weird_density(x): """ The function I want to sample """ return scipy.stats.lognorm.pdf(x, 1.0) def quad_quantile(fun, q, precision=1e-10, ...

5

First of all your question is about stemming words as mentioned in the other answer which can be found in any Python NLP library such as Spacy or NLTK. The other point to mention here is that despite the other answer, what libraries has as Stop Words list is not actually stop word! Do no remove them! In NLP stop words should be extracted based on working ...

5

When doing logistic regression you start calculating a bunch of probabilities $p_i$ and your target is maximize the product of those probabilities (as they're considered independent events). The higher the result of the product the better is your model. As we are dealing with probabilities we are multiplying numbers between 0 and 1, therefore, if you ...

4

I don't think there is a good way to do this for all models, however for a lot of models it's possible to get a sense of uncertainty (this is the keyword you are looking for) in your predictions. I'll list a few: Bayesian logistic regression gives a probability distribution over probabilities. MCMC can sample weights from your logistic regression (or more ...

4

You are looking at a Classification problem. Logistic regression, Decision trees, SVM. Any of the above can solve the job for you. But selecting the best model depends upon how good it is able to predict the test set. Use cross-validations and see which model can give you better accuracy. So split you test and train set accordingly. You don't expect the ...

4

Use chi-square test to check the goodness of fit to a specific distribution http://courses.wcupa.edu/rbove/Berenson/10th%20ed%20CD-ROM%20topics/section12_5.pdf

4

I do not think your formulation is correct. What you have described are just conditional distributions for each word in the sentence but not the joint conditional distribution, given a specific class. In your case, we have by Bayes rule: $$Pr(spam | X) \propto Pr(X | spam) \times Pr(spam) = Pr(you, won, lottery, for, 1million | spam) \times Pr(spam).$$ ...

3

You are correct that \begin{equation*} p(y|\mathbf{x})=\dfrac{p(\mathbf{x},y)}{p(\mathbf{x})}. \end{equation*} Similarly, we can write the joint probability $p(\mathbf{x},y)$ as follows: \begin{equation*} p(\mathbf{x},y)=p(\mathbf{x}|y)\cdot p(y) \end{equation*} From the above two equations, we obtain \begin{equation*} p(y|\mathbf{x})=\dfrac{p(\mathbf{x}|y)\...

3

Accuracy is measured in classification model by comparing the predicted labels to the actual known labels. The predicted labels are a function of both the predicted probabilities for each class and a predefined threshold(binary classification usually is 0.5) So if sample A got predict_proba of {0: 0.2, 1: 0.8} it will be labeled as 1(since 0.8 > 0.5). ...

3

I think it can be done by using this command at the time of prediction, giving example in R #To predict with probabilities testSet$pred_rf_prob<-predict(object,model_rf,testSet[,predictors],type='prob') To take average of the predictions: testSet$pred_avg<-(testSet$pred_rf_prob$Y+testSet$pred_knn_prob$Y+testSet$pred_lr_prob$Y)/3 This Link , might ...

3

I can't tell you for sure without you describing your calculation more or showing code, but my guess is you're not actually calculating the posterior probability here. I bet this is just the conditional likelihood, or at best the unnormalized posterior. Remember: the posterior calculation has a division component. Does yours? You're probably forgetting to ...

3

Your problem definition You have time series data which is used to measure the pressure using your sensor. You wish to identify when the pressure recordings are abnormal. This problem would be best solved using anomaly detection algorithms. But, there are so many ways that you can approach this problem. I would use a sliding window approach and use that as ...

3

Any Survival Function (1 minus the CDF) will have the desired property. Exponential is a potentially good candidate here, as it sometimes can be used to describe distances, but it's hard to say without more information. $$S(x) = \exp(-ax)$$ The parameter $a$ can be tuned or possibly estimated from the data. For reference, if $a = 1$ then you get, $$[0.... 3 I think brute force is the only method. There might be some complicated reductions but that would be outside the scope of the tutorial. Brute force isn't unreasonable here since there are only 46656 possible combinations. Even python should iterate through it in a couple of seconds. This is straight forward tree-search problem, where each node's values is ... 3 You can implement categorical cross entropy pretty easily yourself. It is calculated as$$ \text{cross-entropy} = -\frac{1}{n} \sum_{i=0}^{n} \sum_{j=0}^m \mathbf{y}_{ij} \log \hat{\mathbf{y}}_{ij}  where $n$ is the number of samples in your batch, $m$ is the number of classes, $\mathbf{y}_i$ is the one-hot target for example $i$, $\mathbf{\hat{y}}_i$ ...

3

If I had to calculate such a function, I would: Calculate the probability (not Z-score) for $x$ in a two-tailed test (https://en.wikipedia.org/wiki/One-_and_two-tailed_tests) for the probability that $x$ belongs to the distribution for both distributions. Then you have $p_s(x)$ and $p_c(x)$ for self-match/cross-match. Don't use probability $0.1\%$ to reject,...

3

This is a textbook multi-armed bandit problem where Morpheus needs to learn the correct policy about offering pills. As you’ve said the Neos are independent, and making the assumption that there is a best pill overall, we need an algorithm that will experiment with each of the pills to find out which one is most likely to be accepted. This is the same as ...

3

You can simulate one of these lines like this: library(ggplot2) X<-rnorm(1000, 1, sqrt(4)) f<-function(x) exp(-x*x) df<-data.frame(sample_size=seq_len(1000), sample_means=cumsum(f(X))/seq_len(1000)) ggplot(df, aes(x=sample_size, y=sample_means)) + geom_line() + theme_minimal() + labs(x=NULL, y=NULL)

2

No, you don't need to understand measure theory and real analysis to do machine learning in data science. However, it'd be hard to for you to read academic papers (eg: kernel methods) if you don't have the knowledge. Unless you want to be a mathematician or wish to pursue a Phd, you really don't need to know too much about the theories. In fact, most of the ...

2

Neither of real analysis or measure theory are necessary for data science. Most of the mathematics you need to know is at the undergraduate level and these courses should suffice: Calculus Linear Algebra Probability and Statistics Optimization Most of machine learning involved in data science (that deals with application vs creating novel machine ...

2

You might have mistaken the equation for maximum likelihood , and it should be like this : $p^y*(1-p)^{1-y}$ while y is your label , it should be 0 or 1 , 0 for negative and 1 for positive , and p is your sigmoid function value. so , in your case it should be : p(2.5)*(1 - p(0.3))*p(2.8)*p(0.5) Hopes this will help you , good luck !

2

It's hard to say without knowing the data. Often times the classes are imbalanced. This could cause your problems. So let's say that the "features are right" for 5% of your data and they belong to class $A$. You would expect the 5% to be classified correctly as $A$ almost all the time. But what about the remaining 95% in class $B$? The classifier learns ...

2

You are absolutely right to be skeptical about the results you are getting. I've been using boosted trees (not necessarily C5.0) for over 5 years, and I've never encountered results like what you've described here. Excluding the possibility that you simply had a typo in your code, I can think of two reasons: Low decision threshold: As is often the case in ...

2

Feature selection is one of the hardest and at the same time important step in creating successful models in general. For you case, you've some number of options aside from exhaustive search (which is doable depending on the number of features and training samples) that are greedy algorithms trying to come up some decent combination of features, namely; ...

2

Recommender Systems are a huge topic of its own right and goes without saying, with a lot of research going on. This book does a deep-dive into recommender systems and may not be something you want, but it's helpful as a reference. It seems like you were unsure what those terms mean. The Berekely AI Course covers most of these topics and their lectures are ...

Only top voted, non community-wiki answers of a minimum length are eligible