# Tag Info

7

I think the easiest way to do this would be to have a decision tree where the final decision results in a linear formula. Setting aside whether this is actually easiest/best, this type of model does exist, usually called "model-based recursive partitioning". See e.g. https://stats.stackexchange.com/q/78563/232706 There are several packages in R for this: ...

6

Sklearn scaler works on feature/column (and thats why you want) Imagine if it did not. Than you would shift your mean and std in weird-determined-by distribution-of-the-whole-set-kind of way.

6

GridSearchCV is built around cross validation, but if speed is your main concern, you may be able to get better performance using a smaller number of folds. From the docs: class sklearn.model_selection.GridSearchCV(estimator, param_grid, scoring=None, n_jobs=None, iid='deprecated', refit=True, cv=None, verbose=0, pre_dispatch='2*n_jobs', error_score=nan, ...

5

I would suggest using spline regression. or some polynomial regression. Why? what you are basically approximating is the (almost) step-wise function. See here More info in here and a great background introduction in here.

4

Searching the source code of Sklearn for SimpleImputer (with strategy= "most_frequent"), the most frequent value is calculated within a loop in python, therefore that is the part of code that is so slow. In the source code of SimpleImputer there is also the comment that explains why they do not use the scipy.stats.mstats.mode, which is mush faster: scipy....

4

Sure, here is a skeleton in TF 2.0. import tensorflow as tf weights = tf.Variable(tf.random.normal(shape=(784, 10), dtype=tf.float64)) biases = tf.Variable(tf.random.normal(shape=(10,), dtype=tf.float64)) def logistic_regression(x): lr = tf.add(tf.matmul(x, weights), biases) return lr def cross_entropy(y_true, y_pred): y_true = tf.one_hot(...

3

You dont use self.k in your closest_n method (for starters). U dont use it at all, you just initialise the property. I would check sklearn implementation, just look under the hood here

3

A few things going on here: Your matrix is 100x100. So you have no degrees of freedom left in a linear model, which will cause $R^2=1$. See this post. You use random numbers. Thus, they should make little sense in terms of explaining your dependent variable (it's basically noise). Since Lasso "shrinks" parameters which are not useful (and none is useful ...

3

You mean first to second, not to third? In any case possible explanation: What are your parameters in decision tree. For example for different min_samples_split you can expect different GINI values. You got information gain values (very likely) calculated for all of the samples (rows) of your dataset, but thats not how decision tree calculates it (...

3

If the "cost" for experimenting is not really that big I suggest you take the time to experiment and take this as a learning opportunity and just try if it could actually work. There are many approaches to address class imbalance and setting class weight is one of them and the easiest to implement. Change loss function (for example to focal loss for ...

2

With a bit more details, let's look at the shape of what cv.split(...) will give you: from sklearn.datasets import load_iris from sklearn.model_selection import KFold X, y = load_iris(return_X_y=True) cv = KFold(n_splits=3) splits = list(cv.split(X, y)) print(len(splits)) print(len(splits[0])) print(splits) 3 2 [(array([ 50, 51, 52, 53, 54, 55, 56, ...

2

Pack it in a tuple. kfold.split(X, Y) returns an iterable of indizes, you wrap them in a tuple object for train test indizes.

2

Even after your update, I think Noah's hint to spline regression is the best way to approach the problem. Here is a brief example in R: # Generate data x <- -50:100 y <- 0.001*x^3 plot(x,y) df = data.frame(y,x) # Linear regression reg_ols=lm(y~.,data=df) pred_ols = predict(reg_ols, newdata=df) # GAM with regression splines library(gam) reg_gam = gam(...

2

From sklearn's documentation for the score function Returns the coefficient of determination R^2 of the prediction. R^2 is a measure of how well the variability of the data is explained by the model. So, at 0.97, the model is able to explain that really well. And yes, it is a measure of accuracy for regression models. However, you cannot construct ...

2

sklearn doesn't know that your feature is categorical; it's treating it as continuous, for which only splits of the form $x \leq \alpha$ are checked, so your second listed split candidate isn't actually a candidate. In general, sklearn doesn't support categorical variables (yet?), and you'll need to encode it differently (one-hot?) if you want different ...

2

By passing a callable for parameter scoring, that uses the model's oob score directly and completely ignores the passed data, you should be able to make the GridSearchCV act the way you want it to. Just pass a single split for the cv parameter, as @jncranton suggests; you can even go further and make that single split use all the data for the training ...

2

Problem is the way you're onehot encoding. Best practice for any type of encoding : You should train an estimator for Onehot encoding on the training data only, and when encoding test data, you should use the same estimator used on training data. Eg : sklearn.preprocessing.OneHotEncoder does this, and it has a parameter called : handle_unknown. ...

1

This method was researched in the 90s and I'm afraid that it wasn't very successful. Do a search on treed-regression.

1

A direct way to find the words which are the most representative of a class is to calculate the probability of the class given a word: $$p(c|w)=\frac{\#\{\ d\ |\ label(d)=c\ \land w\in d\}}{\#\{\ d\ |\ w\in d\ \}}$$ Ranking the words according to their probability $p(c|w)$ gives: highest values: the most correlated words for the class lowest values: the ...

1

I think here you must maintain the actual tf-idf and create corpus over it.. Assuming you already have lables for documents available. You can rum classification over it. Best classification I am anticipating for this problem would be naive bayes..

1

As Sergey has pointed out on the original question, going through the following link one finds the reference to the original article. So, we're going to proced by unraveling the components of each formula (explaining each of them) and then doing a proper interpretation of the splitting criterion that Jerome Friedmann proposed. 1. Probabilities $p_{k}$ ...

1

In R you can run a linear regression. Consider this "academic" minimal example: df = data.frame(c(3,5,2,7,5,3), c(1,0,1,0,1,0), c(0,1,1,0,1,0)) colnames(df) = c("A", "B", "C") df Take this data as an example: A B C 1 3 1 0 2 5 0 1 3 2 1 1 4 7 0 0 5 5 1 1 6 3 0 0 Now we can see how B and C describe A in the best way. reg = lm(A~B+C, data=df) summary(...

1

You can read the file line by line (or block by block), assuming it is in a format where that can be done, and keep track of the unique values of the category. Can handle multiple fields in 1 pass of the data instead of the 1 field like the below code. Of course the easiest way is to get more memory or move to a cluster. The line-by-line solution will not ...

1

Mutually exclusive classes are usually integers; in your case, win=0, draw=1, loss=2 (for instance). It is mostly a convention and is convenient for implementation.

1

Yes, you do not need to fit or train explicitly when using cross_validate(). Also see this example from the SKLearn documentation: from sklearn import datasets, linear_model from sklearn.model_selection import cross_validate from sklearn.metrics import make_scorer from sklearn.metrics import confusion_matrix from sklearn.svm import LinearSVC diabetes = ...

1

You sure can. One solution off the bat is to extend your estimator that takes object type variables. So what does that mean. Library that you said are all estimators in the sklearn form fit, predict methodology. So all you have to do is something as follows: > class modifiedTraf(oldTraf): > def __init__(self): > super(...

1

As promised, here you can find an example of how you could apply kfold cross validation for a defined convolutional neural network model, applied to an augmented dataset. You can find the code as a simple gist here It is done as follows: for a subset of the CIFAR10 images dataset, generate 3 augmented images (by applying horizontal_flip) per original image, ...

1

In addition to the comment made by Oxbowerce, you can reason about it as follows: in a real case, you would expect the distribution of your X_train data to be similiar to the X_test data, so applying the MinMaxScaler transformer to the X_test data fitted "only" on X_train means (or should mean) no actual difference compared to fitting it also with X_test; ...

1

If you would fit the scaler using x_test you would be using information from your test set and would be leaking data. This is information that you would not have if your model was in production and can therefore not use when fitting your model.

1

No, sklearn generally doesn't apply scaling inside of any of its models, instead relying on the user to do that. This seems like the right way to do it, since you might want to try different scaling techniques depending on your data. From the User Guide: Stochastic Gradient Descent is sensitive to feature scaling, so it is highly recommended to scale ...

Only top voted, non community-wiki answers of a minimum length are eligible