New answers tagged scikit-learn
0
CRF models are not DL models, they use text features in the traditional way as categorical data (equivalent to one-hot-encoding). However they are different from other supervised ML methods because they exploit the sequential information of the text.
Typically the text sequence is represented word by word, and each word can have several additional "...
1
Load the pickle file and your new data. Fit the loaded model on the data
model = pickle.load(open(pickle_file,'rb'))
model.fit(x_new, y_new)
2
permutation_importance is considering the top-level features. It is permuting each one sequentially and learning the importance.
So, the inner encoding i.e. OHE/tfid is not visible to it.
To get the importance of components of the top-level feature, you should encode it separately and then pass the encoded data to the permutation_importance
Get the pre-...
0
Since you are already using scikit-learn to fit the decision tree, you should scikit-learn to encode the features. For example, scikit-learn's OneHotEncoder has the get_feature_names attribute which return feature names for output features.
0
I think the comment is correct - there are not hyperparameters in the same sense as other ML classifiers.
You do want to make sure that you use the best version of Naive Bayes, based on your data (sklearn user guide: https://scikit-learn.org/stable/modules/naive_bayes.html#gaussian-naive-bayes)
I think one approach to using Naive Bayes in a robust manner ...
0
You can do this by the argument xgb_model. The splits are fixed but the leave values are updated by your new training data.
And you can increase the tree splitting using new data after the original ones.
Here is a minimal example as follows,
gbm_t = xgb.train(params_t, xgb_train_t, num_boost_round = num_round, evals= watchlist_t, xgb_model=gbm)
1
There are no combinations that work for all cases, hyperparameter tuning is still something that is mostly done by trial and error. Things like Gridsearch and Randomsearch exist though.
A good start is always the default setting. An idea if performance is an issue is to tune on a small percentage of the training set to later switch to the full set.
0
Unless you have a very good reason to use an ensemble inside the iterative imputer I would highly recommend to change the base estimator.
As mentioned on the previous answer, you can limit the tree's depth or change the max_features parameter to sqrt (both improve the execution time in ~20%) at the cost of prediction quality, but again the same question lies,...
1
From the User Guide:
By default, parameter search uses the score function of the estimator to evaluate a parameter setting. These are the sklearn.metrics.accuracy_score for classification and sklearn.metrics.r2_score for regression...
0
Question 1: Decision trees (and therefore random forests) allow weighted samples: you can provide different positive real-valued weights for each row in your training data, and the loss function cares more about the higher-weighted rows. When you use sample weights, you need to adjust all of your calculations, and so "number of samples" becomes &...
1
TL;DR - use the max_depth and max_samples arguments to ExtraTreesRegressor to reduce the maximum tree size. The sizes you pick might depend on the distribution of your data. As a starting point, you could start with max_depth=5 and max_samples=0.1*data.shape[0] (10%), and compare results to what you have already. Tweak as you see fit.
Apart from the fairly ...
0
Something for intuitive using sklearn's MultiLabelBinarizer, though slightly more verbose:
from sklearn.preprocessing import MultiLabelBinarizer
mlb = MultiLabelBinarizer()
mlb.fit(df['signals'])
new_col_names = mlb.classes_
# New DataFrame containing 0/1 values of the signals
signals_df = pd.DataFrame(mlb.transform(df['signals']), columns=new_col_names)
#...
1
None of them are the same.
linearSVC() uses one-vs-rest and SVC(kernel='linear) uses one-vs-one for classification.
To have the same results with the SVC poly kernel as with the SVC linear we have to set the gamma parameter to 1 otherwise the default is to use 1 / (n_features * X.var) weakening the value from the now linear kernel.
0
Yes - a Python script will have less overhead than a Jupyter Notebook
Pickle is the standard way to store a scikit-learn model, see model persistence documentation.
The two primary ways to scale Jupyter Notebooks is vertical (rent a bigger machine on a cloud service provider) or horizontal (spin-up a cluster).
0
Imagine you have a dataframe of four feature columns and a target. Two of the features are text columns that you want to perform tfidf on and the other two are standard columns you want to use as features in a RandomForest classifier.
I would use the following code:
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from ...
1
Models are trained on training data, and evaluated on test data based on assumption that unseen test data also comes from the same distribution with training data. So when you calculated statistics for training data, based on assumption that test data also comes from the same distribution you should apply same transformations to test data. You should fit ...
2
It might be because of the conflict of order between model.classes_ and series.unique()
For a binary labels,
model.classes_ = array([0, 1])
series.unique() = array([1, 0])
Try creating a constant value i.e. np.array[0,1] for labels and see.
1
There are probably many different strategies but it's a difficult problem when the imbalance is as severe as it is here.
Without any correction the model is likely to ignore the smallest classes, as you noticed. However forcing the class weight as if the data is balanced is certainly too strong a correction. A middle ground would be to resample the training ...
0
After some searching, I found this source code:
https://github.com/scikit-learn/scikit-learn/blob/7813f7efb5b2012412888b69e73d76f2df2b50b6/sklearn/tree/_splitter.pyx#L467
I think, but am not 100% sure as my code comprehension is imperfect, that:
Yes
x_0 = (a+b)/2, where a is the nearest neighbor to the left and b is the nearest neighbor to the right
0
vec1 = [1, 1, 0, 1, 1]
vec2 = [0, 1, 0, 1, 1]
print(cosine_similarity([vec1], [vec2]))
I passed the 2nd vec2 as Y and I got the output as a scalar.
0
weighted number of samples reaching node j
We have to compare the Parent to two children, then the children's impurity must be weighted.
Let's say the Parent has 100 samples and impurity=5
Children has the same data as [20, 2] and [80, 4]
Dip = 5 - (0.2 * 2 + 0.8 * 4)
=> 5 - 3.6
=> Dip = 1.4
"Weighted" means applying the appropriate ratio i....
2
This would not be possible since the two variables you are trying to predict are of a different type. You are first predicting the default label, which would be yes/no, so this is a classification problem. The second variable you are trying to predict is the prepayment percentage, which is a continuous variable, this is therefore a regression problem. You ...
1
After your initial validation of the model using train-test split, if you are satisfied with the performance, You can create a final model by training on the entire dataset. That way you put to use all available labeled for running inferences on brand new data.
You would simply perform a:
model = KNeighborsClassifier()
model.fit(X, y)
Where X, y represent ...
1
KNN is not fitted to "the k-nearest neighbor points closest to the test data points". You specify the fit option, like:
neigh = KNeighborsClassifier(n_neighbors=3)
neigh.fit(X, y)
Usually this will be xtrain, ytrain, while you test the model performance using "new" (unseen) data and compare the true targets to the prediction.
neigh....
1
I suppose that you want to fit a logistic function to your data. A general form of logistic function is :
$$y(x)=a+\frac{b}{1+c\: e^{-p\:x}}$$
So they are four parameters $a,c,b,p$ to optimize.
The usual method is a non-linear regression calculus. This is an iterative process which requires 'guessed' initial values for the parameters to start the iteration.
...
0
KNeighborsTransformer only gives you the indices of the nearest neighbors and the distances. You need to do more work to retrieve the points to fit your linear regression.
Here's a draft that appears to be working:
from sklearn.neighbors import NearestNeighbors
from sklearn.base import RegressorMixin, BaseEstimator, clone
from sklearn.linear_model import ...
0
It is possible, actually. The answer is not too different than the one given by @10xAI, but it is not trying to exploit the order of the random seeds implicitly, since it would break for parallel training. So the answer above could maybe only work for trees not trained in parallel. But not sure.
The actual working answer is simple, and it resides in using ...
0
Having two separate pipelines would be more efficient for larger datasets. Version the different cleaned/standardised/preprocessed datasets and run the lean learning and prediction model on a particular cleaned dataset.
That being said, if the cleaning process is cheap, then keep it in the same pipeline. You don't want to bloat your work if it is unnecessary....
0
One of the benefits of using scikit-learn's Pipeline is to appropriately apply the operations to both training and prediction datasets. The "whole dataset" you mention that is only the training data you currently have available. Scikit-learn's Pipeline can automatically handle additional data, such as production data.
Having two different pipelines ...
2
You are right, neg_mean_squared_error is simple -1 * mean_squared_error. This is because a convention in the Scikit-learn api that all the scorers follow.
According to scikit-learn documentation (some emphasis added):
For the most common use cases, you can designate a scorer object with the scoring parameter; the table below shows all possible values. All ...
0
Yes, it is perfectly legal to do so. Scaling such binary variables does not change the shape of its histogram and the shape is all that matters to most models.
1
Firstly, when you have an imbalanced dataset accuracy is not a good metric to be using (see https://en.wikipedia.org/wiki/Precision_and_recall#Imbalanced_data). You should consider what the ultimate use-case of this model is and what metric is properly capturing the performance of the model considering that use case. For example, when classifying the ...
0
No, coding-wise, it cannot use multi-dimensional arrays as single features. You can flatten the array though. Essentially, you want to reshape your input to be (189, 250).
0
There seems to be a confusion between multiclass and multilabel classification:
Multiclass is the regular case where the task consists in predicting among N possible classes. For example an image can be either a dog or a horse or a cat, but always exactly one among these three animals.
Multilabel is the when the task consists in predicting a set. For ...
1
You might benefit from random forests instead which aim to achieve the same objectives you are aiming for, i.e better generalization through pruning to remove overfitting.
scikit learn's random forest algorithm will let you specify how many or what proportion of variables you want to automatically drop across the many trees whose results will be averaged for ...
0
Clustering doesn't work like this: for example k-means assigns an instance to the closest centroid, and since there is always a closest centroid there is a always a cluster that an instance "belongs to".
So you need a different approach if you plan to have the possibility of in instance "not in any group":
redo the clustering on the full ...
1
The answer depends on the library you are using. If you are using scikit - you wouldn't need to one hot encode the targets. Scikit handles it automatically.
If you were using keras to build a neural network, you might want to use one hot encoded labels because the built in loss function in keras (e.g categorical crossentropy) expects labels to be one hot ...
3
I don't think it is possible to get it directly but we may utilize the random seed.
random_stateint, RandomState instance or None, default=None
Controls both the randomness of the bootstrapping of the samples used when building trees (if bootstrap=True)
This is from the RF Github code
def _generate_sample_indices(random_state, n_samples, ...
Top 50 recent answers are included
