Julio Jesus
• Member for 1 year, 8 months
• Last seen this week

29 views

Generally speaking the correct representation on td-idf encoding is a hyperparameter to be optimized. As suggested in the above's answers, you can go for the regularization parameter i.e min_df which ...

27 views

predict_proba method will return a numpy array of shape (n_samples,2) with the probability of Y == 1 and Y == 0 but you need to pass only the probability of Y == 1 for roc calculation so: from sklearn....

464 views

Specifically talking about Gaussian Naive Bayes, the decision boundary are ellipsoids characterized by the mean and standard deviation of the Gaussian distribution. Image: https://scikit-learn.org/...

36 views

As someone that is more used to use Python's structure, I highly recommend to use the package/class name before the method. So if you are using the method train, you want to specify that this method ...

29 views

This is a first approach, and even though this is not the best in terms of performance it makes the work: def urandom(frame): ls = list() for idx, row in frame.iterrows(): val = np....

959 views

Might be late but for anyone with the same question the answers (as almost everything with Scikit-learn) is the usage of Pipelines from sklearn.impute import SimpleImputer from sklearn.preprocessing ...

36 views

First things first: Check that you are joining by primary keys only i.e, yo do not have any duplicate value at any of the columns you are joining, otherwise you will end up with a huge and ...

191 views

You can think of this problem as basically trying to find dense areas inside a cloud with noise. This is not the only possible solution but you could use a clustering algorithm, and specifically one ...

25 views

Try adding theme to your plot layout So: library("reshape2") library("purrr") library("dplyr") library("dendextend") dendro <- as.dendrogram(aggl.clust.c) ...

36 views

You can try applying your preprocessor to your X_train and X_test: preprocessor = ColumnTransformer( transformers=[ ('num', numeric_transformer, numericas_all) ,('cat', categorical_transformer, ...

65 views

Even though virtually any supervised classification algorithm can be used when having categorical features by applying some encoding technique, my first thought is using Catboost, an algorithm ...

52 views

Additionally to the previous answer, I would go for POS tagging features (features that count the number of verbs, adverbs, nouns, etc contained in your review), since you are trying to distinguish ...

78 views

Once I assume you are using text data as your input matrix X. The first point is that you have to include your preprocessing step as you would do when not using a calibrated classifier, so as you ...

139 views

The correct way of calling the parameters inside Pipeline is using double underscore like named_step__parameter_name .So the first thing I noticed is in this line: parameters = {'vect__ngram_range': [(...

26 views

I'm assuming you already have made feature selection so all your ~200 features are the ones that describe your target So particularly for models that use SGD you can train your model in batches i.e ...

42 views

As far as I understand from your question, you are trying to compare sentences on word level, but it seems like you are interested in finding the number of words in sentence A that are contained in ...

46 views

A naive approach would be to use a supervised model to predict the target anomaly vs no anomaly that your IsolationForest model outputs, then if and only if this supervised binary classification model ...

49 views

As you well noticed there is no way to know the bin in wich an unseen data's target value will be. So what you can do is to train a model that splits/clusters your data and then run a model on each ...

46 views

In recent years I have read different approaches on what you mention, with the argument of applying an ensamble on feature selection, the same way it is applied to model prediction (stacking models) ...

134 views

Personally I find Victor Lavrenko's explanation of underfitting and overfitting the most intuitive and concise definition: This definition is very useful for at least these two points: This is not ...

179 views

Might be late but I found this question interesting: Try: import pandas as pd from sklearn.datasets import load_iris from sklearn.pipeline import Pipeline from sklearn.compose import ...

39 views

As you well mentioned, tree-based models are not sensitive to feature scaling, but on the contrary it might help with the convergency of finding the minimum in the optimization on boosted models I ...

26 views

First I'm going to use a simplier way (gives the same plot just without changing dots size according to its distance to core samples) of visualizing the cluster results: plt.scatter(X[:,0], X[:,1], c =...

3k views

An alternative to archived what you are looking for is: threshold = 12 drops = [l[0] for l in list(filter(lambda x: x[1] > threshold,[(col, (df.groupby((df[col] != 0).cumsum()).cumcount()).max()) ...

783 views

First of all a picture should not be taken to define if there are or not groups on your data, since no matter what projection you use (linear with PCA or manifold with tSNE) you are reducing a 64-...

2k views

Without having a complete knowledge of the features on that website I would say: Data visualization is only one part on data scientist (ds) pipeline from data understanding thought model validation ...

29 views

What you mentioned is true, for 99% of Scikit-learn's estimators, the data X must be numeric (I think only HistGradientBoosting works with no numerical categorical data) So when working with mixed ...

38 views

You can create n matrices where n is the number of different products {Alpha, Beta, ...,} then for each different product you group your daily value into weekly so you have something like Product1 : {...

43 views