bradS
  • Member for 3 years, 11 months
  • Last seen this week
  • London, UK
How to make LightGBM to suppress output?
Accepted answer
7 votes

As @Peter has suggested, setting verbose_eval = -1 suppresses most of LightGBM output (link: here). However, LightGBM may still return other warnings - e.g. No further splits with positive gain. This ...

View answer
XGBoost and Random Forest: ntrees vs. number of boosting rounds vs. n_estimators
Accepted answer
5 votes

As I understand it, iterations is equivalent to boosting rounds. However, number of trees is not necessarily equivalent to the above, as xgboost has a parameter called num_parallel_tree which allows ...

View answer
XGBoost and Random Forest: ntrees vs. number of boosting rounds vs. n_estimators
4 votes

Following from your comment... Why would I want to create multiple trees per iteration? Multiple trees per iteration may mean that the model's prediction power improves much faster than using a ...

View answer
Is numpy.corrcoef() enough to find correlation?
Accepted answer
4 votes

On a side note, I don't think correlation is the correct measure of relation for you to be using, since Survived is technically a binary categorical variable. "Correlation" measures used should ...

View answer
How to handle "unknown" category in machine learning classification problems?
Accepted answer
4 votes

I think this is one of those topics with the most frustrating answer - it depends. To your questions: How can we handle these data which fall into "unknown" category? There are many ways of doing ...

View answer
How to measure the correlation between categorical variables and a continuous variable
Accepted answer
3 votes

The Chi-squared test measures the relationship between two categorical variables. To measure the relationship between a categorical feature and a continuous feature, you can use an ANOVA test. As an ...

View answer
Outliers handling
Accepted answer
3 votes

It's not always a good idea to remove data from your dataset. In some circumstances - and income is a good example - your data will be skewed / long-tailed and so will lie outside of the ...

View answer
How can I check if a bigger training data set would improve my accuracy of my scikit classifier?
3 votes

One idea: Split your data into train / hold out datasets. Train the model on a fraction of the training data (say 50%) and test on the holdout dataset. Train the model on a larger fraction of the ...

View answer
How to implement feature selection for categorical variables (especially with many categories)?
3 votes

Some algorithms will perform feature selection inherently - e.g. elastic net regression, random forest - so you will not necessarily need to do this prior to running the algorithm. You've identified ...

View answer
R langauge how to create xgb.DMatrix object from data frame (newbe)
Accepted answer
3 votes

Welcome to the site! Assume that y is your response, and x is your data set of predictors (where categorical variables have been appropriately converted to numeric). Your data does not necessarily ...

View answer
Boruta Feature Selection package
Accepted answer
3 votes

In R, Boruta relies on the ranger implementation of random forest. So: Converting input variables from categorical to numeric is not necessary. You will need to address NA values prior to running the ...

View answer
Xgboost interpretation: shouldn't cover, frequency, and gain be similar?
Accepted answer
3 votes

My layman's understanding of those metrics as follows: Gain = (some measure of) improvement in overall model accuracy by using the feature Frequency = how often the feature is used in the model. It'...

View answer
How to handle a large number of categories in one column effectively in machine learning?
2 votes

Target encoding calculated using an appropriate cross-validation strategy can also be powerful for high-cardinality categorical features. In some instances, frequency encoding can also be useful.

View answer
Which feature selection technique to pickup(Boruta vs RFE vs step selection)
2 votes

Some algorithms perform feature selection inherently - e.g. LASSO, random forests, and gradient-boosted models like XGBoost and LightGBM. If you are using those then there is no need for manual ...

View answer
How to deal with count data in random forest
Accepted answer
2 votes

Some thoughts: Your data is highly imbalanced. This is a critical issue which should be dealt with. Possible solutions include simple under-/over-sampling to more complicated synthetic approaches ...

View answer
Why decision tree needs categorical variable to be encoded?
2 votes

As I understand it, decision trees use the rules < threshold_value or >= threshold_value to group observations together, where threshold_value is the value of a variable which minimises the cost ...

View answer
Random forest vs. XGBoost vs. MLP Regressor for estimating claims costs
2 votes

Some ideas: Handling categorical features correctly: using one-hot encoding is one valid approach. Other approaches include target encoding (or mean encoding), and the hashing trick. There's no real ...

View answer
Trying to find the correlation between inputs and output
Accepted answer
2 votes

In a correlation framework above, the biggest driver of the output is the input which has the greatest absolute correlation value. Correlation lies in the range [-1,1], and: Negative correlation (...

View answer
ML Models: How to handle categorical feature with over 1000 unique values
2 votes

If necessary, there are other methods of encoding categorical features: Label encoding (might need some judgement regarding implied ordering) Target encoding Hashing trick A handy python package is ...

View answer
Converting ML Azure random forest to Python - what is "Number of random splits per node"?
2 votes

From the Notes section in your link above: "The features are always randomly permuted at each split." So using the max_features parameter should be analogous to "number of random splits per node".

View answer
Why are only 3-4 features important in my random forest?
Accepted answer
2 votes

RandomForestRegressor has a parameter called max_features, which is the number of features to consider when determining the optimal split. You haven't explicitly specified this, so Python will use the ...

View answer
Data cleaning and data transformation before EDA?
2 votes

Although not very helpful, the answer is probably "it depends". I like to do data cleaning and some EDA together since EDA can highlight appropriate treatments to clean the data - e.g. influencing ...

View answer
Decision trees for Rstudio v3.3
Accepted answer
2 votes

The "best" package depends on your goals and data really. A few tree / forest packages that I've come across: randomForest - an implementation of the original algorithm ranger - a flexible ...

View answer
Dropping columns or inputing numbers
1 votes

Columns 1 to 6: if the data is missing because it does not exist, does that tell you something about the variable/target/customer? If so, you want to preserve that information in your imputation. For ...

View answer
Dealing with a dataset with a mix of continuous and categorical variables
1 votes

It depends on which algorithm (and implementation) you are using. For instance, the linear regression implemented in sklearn requires all input variables to be numeric and so encoding will be ...

View answer
Adding extra variables to XGboost model is worsening the train and test accuracy
1 votes

In general, when you change the data being fed into a model you should also consider re-tuning the model parameters. It could be that the addition of the two new features in your data set means that ...

View answer
how can I replicate working of Multi Label Binarizer from sklearn package in R?
1 votes

One way to do this is to convert the source code from Python to R - see line 289 here. You could also check for functionality in existing R packages - for instance, mldr, mlr or caret packages.

View answer
Hyperparameter Tuning Time Series in Production
1 votes

Some ideas: Number of previous observations to use: depends on the process you are modelling. If the target is related in some way to many of the previous values, you may need to use more data in ...

View answer
VlookUp: how to loop it?
Accepted answer
1 votes

If you use a combination of INDEX and MATCH, you can let the desired look up array vary automatically as you drag the function across your columns, which is an alternative to manually fixing the look ...

View answer
How to handle missing date data?
Accepted answer
1 votes

I suspect you're working with the Ames house price dataset - one of Kaggle's introductory competitions. Replacing the missing values with the dataset mean / median is very general. I believe you ...

View answer