Skip to main content
famargar's user avatar
famargar's user avatar
famargar's user avatar
famargar
  • Member for 7 years, 11 months
  • Last seen more than a month ago
awarded
awarded
comment
handling constant columns in machine learning
If you have rare instances where the value in this feature is different, you should really understand why, and see if the different value correlates to the target. Say you want to predict a rare outcome (an individual will be convicted of pedophile) and the feature is “past pedophilia conviction”.
awarded
awarded
comment
How to convert longitude and latitude in time series data from daily to weekly?
Then your question is really "what do I have a good reason to think will predict the next location?". But that again, it depends on next location of what. In particular, for an inorganic object good predictors may be just position and speed along a known trajectory. For a living creature, predictors would likely depend on incentives and mean of transport (food location and fairly regular speed for animals, jobs/attractions and means-dependent speed for humans)
Loading…
comment
Which parameters are hyper parameters in a linear regression?
Wait, isn’t a hyperparameter something that is NOT learnt? I mean I get the first part of your comment, not the second one. But even for the first part - if regularisation parameter is an hyperparameter, then similarly the order of the polynomial could be seen as such?
comment
Which parameters are hyper parameters in a linear regression?
Nice answer. Wonder if in Linear Regression, the order of the polynomial is a hyperparameter?
awarded
accepted
revised
Is LASSO regression implemented in Statsmodels?
deleted 40 characters in body; edited body
Loading…
awarded
Loading…
awarded
Loading…
comment
Missing Values in Data
There are plenty of ways to impute missing data indeed. What I am commenting on is on what is simplest and more general. I can't answer the OP because what is most efficient really depends on the problem at hand. Linear interpolation is not the most general (you need to have the continuity assumption) and it's not the simplest - the simplest is dropping the rows, the next-to-simplest is mean (or median) imputation. The most sophisticated would be to train a classifier (for categorical variables) or a regression using the other values as dependent variables. Hope that clarifies.
comment
Missing Values in Data
Linear interpolation only makes sense under the assumptions I stated above. I elaborate further my example: customer A is 70 year old, customer C is 90, thus customer B is 80? If we take the known age distribution, and assume our services do not depend on age, then the chances for customer B to be 80y old are pretty close to zero. The simplest and most common way to impute values (unless the assumption of continuity of the value to be imputed along rows holds, which is rare) is mean imputation; that is to say, to replace the nulls with the mean of that quantity over the training set.
revised
Missing Values in Data
Some text cleanup
Loading…