6

Test accuracy better reflects generalization error, so you want the one with higher test accuracy. In your first setup, the higher train accuracy indicates overfitting, as it's significantly higher than train accuracy. This is also kind of why it generalizes less well than the second one.


4

Yes it would be possible that it happens. It means that this event happening has no importance for the target. Imagine a categorical feature with a lot of categories(high cardinality). Maybe only one of them does not have any influence in the target so if you do feature selection this feature might have a high chance of getting dropped. So this is normal ...


2

To summarize from the comment thread: there are two "weird" things going on here. 1. The zig-zag. As I addressed in the comments, and @BrianSpiering in an answer, this is probably a parity effect, arising from tied votes among the nearest neighbors when $k$ is even. 2. Training accuracy not decreasing (toward test accuracy) with increasing $k$. This was ...


2

Depending on your choice of accuracy metric, you'll find that different balancing ratios give the optimum value of the metric. To see why this is true, consider optimizing precision alone vs. optimizing recall alone. Precision is optimized (=1.0) when there are no false positives. Upweighting negative data reduces the positive rate, and therefore the false ...


2

The xyzSearchCV classes in sklearn require the estimator to be compatible with the sklearn API; in particular, the estimator('s clones)'s hyperparameters get set using set_params, which needs to be defined as a class method for the estimator. So a Keras Sequential model will not work. Keras does provide an sklearn wrapper: https://keras.io/scikit-learn-api/...


2

(To summarize the comment thread into an answer) Your original scores: Mean Absolute Error: 37216342513.01034 Root Mean Squared Error: 871869805169.7842 are based on the original-scale target variable and are between $10^{10}$ and $10^{12}$, at least significantly smaller than the mean of the features (and the target)? So these aren't automatically bad ...


2

First: I think you want the product functionality, not zip, since you are checking every df with every ref. In zip, you would check df_a with ref_1 and df_b with ref_2 only. Second: Your can look at the equation $(1+2+3+4)−(5+5+5+5)$ as $(1-5) + (2-5) + ...$ which is simply subtracting data frames and sum over columns. With these two consideration, ...


1

The minimum and maximum values are just known limits that are parts of the formula that reshapes the distribution of the data, so if a value is higher than the previously known value the resulting feature scaling (Normaliaztion) will be still appropriate. An alternative is z-scores if you don't feel like using minimum and maximum values. x'= (x-x̄) / σ ...


1

You’ve asked two questions: 1) Do you make decisions about model superiority based on training or testing performance? 2) Which model should you prefer? I’ll answer both. 1) First, come over to Cross Validated (the Stack Exchange site for statistics and similar topics, with some overlap to this site) and check out what Frank Harrell has to say about ...


1

You can use handle_unknown='ignore' in the OneHotEncoder; levels present in the test set but not the train set will be encoded as all-zeros rather than raising an error. But... that example you provide, I rather doubt it's worth it to keep all the levels. The coefficient learned for such a small level's dummy variable will be overly specialized (overfit). ...


1

It seems you are using a the standard scaler twice, once in your pipeline and once more in the TransformedTargetRegressor. Next to that, you are only fitting the scaler, never actually scaling the inputs (i.e. transforming the input).


1

The answer to this is simple. When we perform the cleaning of the dataset we'll need to do the whole cleaning process for training data first then we'll do the same data cleaning process for the test dataset too. So to avoid doing the same data cleaning process twice, we merge the training and testing data then we perform the data cleaning process and ...


1

Clean the data and check how each variable is varying with output. Drop the variables which has less variance among the output variable. sklearn.feature_selection contains multiple methods like SelectKBest, chi2, mutual_info_classif to select the best features. https://scikit-learn.org/stable/modules/feature_selection.html Use either PCA,forward stage ...


1

Good job looking at the tree and understanding what has happened. There is no problem splitting on the same feature multiple times. A continuous feature has many split points available. The tree continues to subset and refine. The split criteria shows what will be the "best" greedy split at this point. If a feature is income, perhaps the best split is \$100,...


1

The first variant is implemented: $$F1_{macro}= \ \sum_{classes} \frac{F1\text{ }of \text{ }class}{number\text{ }of\text{ }classes}$$ You can find an example calculation in this answer. Sometimes the scikit learn documentation does not include all the details. In these cases it is often helpful to look into the source code which is linked on all help-...


1

There's several ways that you can choose your k value for kNN - You can use the common formula k = sqrt(n) where n is the number of data points in your training set or you can try choosing k where there is a good balance between computation expense vs noise. Consider your what fits your problem: Do you care about runtime? The higher the k, the more ...


1

Your example shows that K-means (and clustering in general) is not a suitable tool to detect anomalies. Anomalies are, by definition, points (observations) deviating from normality, however that normality is defined. Clusters, on the other hand, are collections of points sharing some similarities. In your case, you use distance from cluster centroids as ...


1

I think there's no point in trying to catch all anomalies in one cluster. Anomalies are anomalies because they shouldn't belong to any cluster. In your case, it's better to cluster with n_clusters=1 and interpret outliers as anomalies. Also, k-means is probably not the best algorithm for your data because distance to centroid will still catch the outlier. ...


1

By multiplying the maximum by 10 in a loop, you are repeatedly multiplying by $\sim 10$, so that the final point is $\sim 10^{53}$ (hrm, the plot actually doesn't go quite that far?). Hence the last added point is so far away from the rest, it becomes too costly to include any other points in that cluster. (Doing so drags the mean of the cluster very far ...


1

If your target values are just 0 and 1, you should probably be treating it as classification, and use e.g. XGBClassifier instead of XGBRegressor. You brought up (originally in the comments on your SO post, now edited into your questions) a scenario when your true values might be limited to 0.5, 1.5, 2.5. That's unusual enough that I suspect the best answer ...


1

I agree with @CarlosMougan that the answer to your question "How should I deal with this?" may well be "There is no problem to deal with." This is essentially allowing your feature selection method to lump categories into the baseline. However, some folks seem to prefer categoricals' dummy variables to be kept together. I know of three ways to try to do ...


1

Probably it's problem with Pandas.Series, just change type to numpy array. X_train[:,8] = impC.fit_transform(np.array(X_train[:,8]).reshape(-1,1))


1

I don't think it's trying to impute across rows; rather, it's trying to impute in the dates column. You may want to use ColumnTransformer to select which columns to impute.


1

I would say the KMeans algo is doing exactly what it's supposed to do. I would be much more surprised if it DIDN'T do what you told it to do. I was also skeptical the first time I saw plots of my own KMeans calculations. Maybe plotting the data in a 3D chart would be more useful/practical. Here is some sample code that you should be able to adapt to your ...


1

I'm not sure what you are referring to in Q1, but for Q2, it looks like you are trying to dig down to the lower negative correlated items, right. I can't reproduce your exace example (I don't have the data), but I will give you a generic example, which you can easily adapt to your specific scenario. # get only numerics from your dataframe; correlations ...


1

No, SelectKBest and other *Select* transformers from sklearn.feature_selection do not change order of features, only drop not selected ones. Anyway, generally, machine learning models do not utilize relative order of a feature. If you need to check and reorder features, you can use scores_ and/or pvalues_ attributes of a fitted transformer (e.g. SelectKBest)...


1

This is still not implemented and not planned as it seems out of scope of sklearn, as per Github discussion #6773 and #13048. However, the documentation on linear models now mention that (P-value estimation note): It is theoretically possible to get p-values and confidence intervals for coefficients in cases of regression without penalization. The ...


Only top voted, non community-wiki answers of a minimum length are eligible