Parameters are set to default. This is not optimal :
1) Other parameters could give better results for a given model.
2) You can't conclude a model is better than another if you haven't set good parameters.
3) Without early stopping or penalty, you are likely to overfit. This is bad generally.
In real life, the code you have won't really work. Generally ...
You face three problems and here are my recommendations:
1. unbalanced classes
Logistic regression (unlike other methods) is very well capabable of handling imbalanced classes per se. There is the bias weight that shifts all the predictions around the correct mean. But it comes with some caveats mentioned in the paper below.
2. different class ...
In ML, all algorithms are useful depending on the dataset. Its naive to generalize an algorithm to be always better than the other.
In your case since you have only 90 examples, mlpclassifier couldn’t have trained proper as compared to logreg.
A general suggestion: if you post like rows of your dataset along with accuracy of the two algorithms, it would ...
There is not a reshape problem.
You need to transform your text in a set of features, say, vectorize it in the same way you created your dataset, in this case using TF-IDF.
Just prepare a query vector applying the same TF-IDF and will work.
It depends on your use case. if your use case is "prevent people from dying" or "find online customers" then any category that has marginal theoretical predictive capabilities should be analysed. If instead, your aim is to improve general wellness or brand awareness, then the category should be dropped. If your use case is rare, then rare categories will be ...
Indeed it's often a good idea to remove boolean features which are very rare, but the problem is that choosing a threshold by intuition is not necessarily optimal. Whenever possible the optimal value should be determined experimentally, and typically that should be possible for efficient methods such as log regression or SVM. The idea is simply to consider a ...
I will try and be as concise as possible. First, let's redefine the way you think about your data points. There can ever only be two types of visits in terms of time. Periodic and Non-Periodic. Let's call each visit an event. Some events could be related to chronic conditions where periodic visits are quite common. Some events could be related to flu, head ...
A regression is multivariate when you try to explain your y using more than one explanatory variable. Each coefficient will have to be interpreted as the impact of a given x, while keeping all other values constant.
It is univariate instead when it takes only one variable.
Because of this, you must run univariate models independently from the multivariate, ...
The performance measure should be related to the actual real world problem you face. (Imagine, you are a football trainer. You are seeking for young talents to introduce them to your very expensive training camp. It might be more important to you to prevent expenses on kids, that proof not to be as talented as hoped for, than reaching all hidden talents with ...
Monitor the metrics that you are attempting to optimize, and compute the change in those metrics after each epoch. Once those metrics stop improving according to certain criteria, then you can stop gradient descent. See this article for a way to keep track of the metric improvements.
As you can see from the error, it's that you're trying to predict using a subset of the features. There are a couple of duplicates (down to the code?), but perhaps without satisfactory answers:
Sklearn ValueError: X has 2 features per sample; expecting 11
Fixing this leads to a fundamental question of data ...
The error occurs because X_train and Y_train are pandas dataframes and you are trying to directly access them as arrays. There are two solutions:
Convert X_train and Y_train to arrays before using them as:
X_set, y_set = X_train.values, Y_train.values
Whenever you are accessing their values use iloc so everywhere replace X_set by X_set.iloc
I solved this by using torch.where to give the irrelevent entries a really big negative value, so that they vanish after an exp. I also took advantage of the log of CEL trick.
loss = torch.zeros(1).type(dtype)
states = torch.cat([states_[_] for _ in idc[:self.batch_size]]).type(dtype)
scores = torch.tensor([scores_[_] for _ in idc[:self.batch_size]]).type(...