New answers tagged regression
1
One way to go is to define functions that are non-zero only near your 'minimal' points, and add them. The goal is to avoid overlap, such that a function associated with a point won't modify the value of your global function around other points.
import numpy as np
def func0(x1):
lim = 0.01
dist = x1*x1
if dist<lim:
value = np.exp(-...
2
There is a loss called Root Mean Squared Log Error (RMSLE): $\sqrt[]{\frac{1}{n}\sum_{i=1}^n{(\log(y_i + 1) - \log(\hat{y_i} + 1))^2}}$ (do not forget the $+1$ as the $log$ is not defined at $0$)
You will find a brief explanation and discussion here. It has also been used in competitions as for example here.
-1
Since you have the exact specifications, you can directly hard-code the answers:
def f(x: float, y: float) -> float:
"Deterministically return the value of z."
if (x == 0.85) and (y == 0.5):
return 0.6 # Global minimum
elif (x == 0.2) and (y == 0.3):
return 0.7 # Local minimum
elif (x == 0.6) and (y == 0.8):
...
1
Nice question.
Short answer: the distinction between independent and dependent variables may not matter to your loss/error function, but it might matter for your modeling function.
Think about fitting a parabola with orthogonal regression. Here, it's really important which variable is dependent and independent, because the function is not one-to-one. So ...
0
Linear regression is easy. Or you could build some self training AI NN and do it the hard way.
Engine vs fuel efficient trade off would not be linear and there are so many other factors that go into a final trade off.
You must trade off ALL factors to optimise the system.
1
Since CARTs (Classification And Regression Tree) are a non-parametric algorithm, they should be able to find interactions between variables and non-linear behaviors.
Nevertheless, building polynomials can help them have a better performance.
-3
Add as additional constraint that first derivative is 0, and that its bigger/smaller than all the values on the interval implying maximum or minimum.
0
It really seems like a problem of formulating your probkem. On your 2D exemple you can split your $X_i$ along your axes and juxtapose them to build a data set. Each $X_i$ has two components : along axis 1 ($X_{i1}$) and 2 ($X_{i2}$).
Basically you would just build your data set by juxtaposing the 18 columns :
$X_{11}$ , $X_{12}$ , ..., $X_{i1}$ , $X_{i2}$ ,...
0
My guess: I agree with your colleagues. I see no reason to do anything other than a single neural network with multiple outputs. If necessary, increase the capacity of that single neural network until you see no further improvement.
An stacking ensemble where you have a few neural nets whose inputs are fed as input into another neural network is itself ...
0
It should even be easier than classification: you do not need the final layer.
Your input layer should have 18 nodes $(x_i,y_i,t_i)|i=1..6$
Hidden layers as you see fit (experiment with it, depending on data and results).
I would expect the best results from a fully connected layer and a 'relu' layer.
The output layer has 3 nodes $(x_7, y_7, t_7)$
Train ...
0
I think the answer is, you have to either create a very large number of categories, or do it completely differently than with the code I provided. The paper ... https://arxiv.org/pdf/1811.11296.pdf as a good solution to the problem.
0
It looks like you need a solution that captures sequential dependence.
A method would be Time Series. For a good explanation, you can check:
https://www.itl.nist.gov/div898/handbook/pmc/section4/pmc4.htm. The course presents different methods for time series problem, like: moving average, ARIMA.
0
What's a BU? Business Unit?
Can you tell anything about the variable and target variable? Can you include the scatter plot?
It is still not clear if the problem is not in the domain knowledge/assumptions.
A business example:
Profit depends on sales, which is generally assumed to have a positive correlation. But after a threshold marginal sales cost ...
1
Is 0.9113458623386644 my ridge regression accuracy(R squred) ? if it is, then what is meaning of 0.909695864130532 value.
These are both R^2 values:
https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Ridge.html#sklearn.linear_model.Ridge.score
The first score is the cross-validation score on the training set, and the second is your test ...
0
Decision trees usually base feature importance on the impurity reduction achieved by splitting on the features. In classification a usual choice is gini impurity, while regression trees typically use the mean squared error or node sample variance. This is also the case in scikit learn.
For a given (binary) node $m$ with left and right child nodes the ...
2
Weighting MSE is a way to give more importance to some prediction errors than to others in the overall score. This is useful if you are using MSE as a performance metric for your model, especially during the model training (loss function) or validation (hyper-parameter setting).
In the case you cite as example, more importance is given to cases with more ...
0
Alternatively, just implement a simple Grid Search algorithm yourself. The book "Introduction to Machine Learning with Python" by Mueller and Guido includes an example using an SVC:
# naive grid search implementation
from sklearn.svm import SVC
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, random_state=0)
print("Size of ...
1
If speed is the only only issue then i have few suggestions that will definitely improve the algorithm run time by 5-10times(which i experienced), without compromising on any other input:
1) Increase the number of jobs submitted in parallel, use (n_jobs = -1) in the algorithm parameters. This will run the algo in parallel instead of series(and will cut down ...
2
By passing a callable for parameter scoring, that uses the model's oob score directly and completely ignores the passed data, you should be able to make the GridSearchCV act the way you want it to. Just pass a single split for the cv parameter, as @jncranton suggests; you can even go further and make that single split use all the data for the training ...
1
There are a few ways of making this faster:
Decrease the CV value, as mentioned by @jncraton
Decrease the search space for the hyperparameters (test only a few parameters or decrease the ranges for parameters)
Additionally, you might consider using a more efficient way of hyperparameter searching by using hyperopt or nevergrad.
6
GridSearchCV is built around cross validation, but if speed is your main concern, you may be able to get better performance using a smaller number of folds.
From the docs:
class sklearn.model_selection.GridSearchCV(estimator, param_grid, scoring=None, n_jobs=None, iid='deprecated', refit=True, cv=None, verbose=0, pre_dispatch='2*n_jobs', error_score=nan, ...
0
Solutions to over fitting:
Simplify model by reducing the number of attributes in the training data or by constraining the model(regularization).
Gather more training data
Reduce the noise in the data (Fix data errors, remove outliers)
Solutions to under fitting:
Feature engineering
Reducing the constraints in the model.
To make good predictions your ...
0
You can build an optimization problem.
Your features will be the parameters, then you create a loss function and try to minimize it with gradient descent or brute force (depends on the search space of your problem)
0
There are several assumptions that are needed to be satisfied in order for logistic regression to work. From personal experience the two most important assumptions:
Features need to be not (or a little) correlated with each other, because if they are correlated then the change of one feature indicates also the change of the other that's a problem when you ...
1
Not knowing your data in detail, my intuition is that you could go for dummy (one hot) encoding. You could split each day in (say) 10 min. intervals $x$ (144 columns) and attach labels (up/down) $y$. Each time interval $x$ would be one dummy encoded column (true/false).
The model would be a binary classification (logistic) like:
$$ y = \beta_0 + \beta_1 ...
5
If you want to move from theory to application then I suggest to do exactly that: get your handy "dirty"!
UCI Machine Learning Repository has some easier datasets to get started. Kaggle is great too but before going for any competition look for an easier dataset from their repository.
If you prefer something with more guidance the book "Introduction to
...
4
Doing Kaggle problems is a good way to test your skills, and it is a good way to improve your skills. There are problems that don't require advanced techniques. For example, Titanic is an introductory problem. Also, solutions for many problems are available. You can do a problem yourself and then check how other people did it.
0
You could apply a square or a square root transformation to the y and check for which transformation y is not skewed.
1
Lambda is a tuning parameter („how much regularisation“, I think called alpha in sklearn) and you would choose lambda so that you optimise fit (e.g. by MSE). You can do this by running cross validation.
This page (for the GLMnet package in R) explains how to apply Lasso in a very instructive way (alpha is the elastic-net mixing parameter here, Lambda is ...
0
There is nothing that guarantee the six features will be a subset of the 55. Intuitively, if you parametrize you problem with number of features instead of lambda, it is easier to see how two variables could better explain the output than a single different one.
Another problem that can arise from these regularisation methods come from correlated variables....
1
I agree with @peter's detailed answer on points #1 through #5 and would like to supplement that with some more details:
Perform a PCA or MFA of the correlated variables and check how many predictors from this step explain all the correlation. For example, highly correlated variables might cause the first component of PCA to explain 95% of the variances in ...
2
A few hints/ideas regarding your task:
I would not kick out other explanatory variables $x$ too early. When I look at your correlation chart, I suspect that all variables (but serial no) have some impact on $y$. If you kick out these variables, you may throw away important information.
In case some $x$ are highly correlated, you may face the problem of ...
1
I just want to stress an important point: ConvLSTM() layers have been excluded from the new TensorFlow 2.0, which is largely based on Keras in models' specification.
It is substituted by ConvLSTM2D() layers, that take different arguments as input. (see docs here).
(An alternative is to manually create a combination of Conv2D() and LSTM() layers.)
That is to ...
1
Just a couple thoughts:
It looks like these "regimes" could be represented as a latent variable: you could probably design a bayesian model in which the OLS model depends on the value of this latent variable. This means that the model would still be trained only with the observed features, but would internally predict the value of the regime and this value ...
1
I'll go through your question one by one.
1) Can someone explain why we have to transform dependent variable using log-transformation (Normalization) when appear positive skewed y variable in regression model?
Not necessarily log transformations, any kind of transformation (square, square-root, log, Z-scores, you name it) necessary to make the ...
0
You may need to spend more time identifying the problem than looking for tricks that have worked in literature. So start with the simplest architecture and see how and on what values your network is converging. It's hard to know all the details but here are a few suggestions that might help.
You may try different initializations and also random re-...
0
In principle, you can do a regression with only factors as explanatory variables. Consider the example (in R):
df = data.frame(c(100,200,500,100,300), c(1,0,1,0,1), c("True", "False", "False", "False", "True"), c("A", "B", "B", "A", "A"))
colnames(df) = c("sales", "v1", "v2", "v3")
head(df)
reg = lm(sales~as.factor(v1)+as.factor(v2)+as.factor(v3), data=df)
...
1
A good place to start is with Analysis of Variance (ANOVA) models. The simplest case is where the response/outcome variable is continuous and you have 1 categorical predictor. This is called one-way ANOVA. With 2 categorical predictors you have a 2-way ANOVA and so on. With more than one predictor, interactions between the predictors are also typically ...
0
Try deeper network. Something like this.
self.model = models.Sequential()
self.model.add(layers.Dense(self.num_layers, activation="relu", input_shape=(self.x_train.shape[1],)))
self.model.add(layers.Dense(self.num_layers//2, activation="relu"))
self.model.add(layers.Dense(self.num_layers//4, activation="linear"))
self.model.add(layers.Dense(1))
self.model....
2
If you up to a predictive model, you look for a model which performs well on the test set and the metric of interest is the mean squared error which indicates by how much you fail to predict $y$ on average. So don't use $R^2$. Just compare all models based on MSE.
2
Question 1. Both. If you think in opposite to multivariate terms, than in univariate regression both input and output variables should be 1-d
Question 2. Multivariate regression where more than one independent variable (predictors) and more than one dependent variable (responses), are linearly related. So input needs to be more than 2 also.
5
Problem in this competition is VERY similiar. Instead of me copy-pasting some interesting ideas just check out winner write-ups in the discussions.
Some takeways:
Add count of values of features as new features.
This is information that LightGBM can’t see
Check unique value counts for all features of train and test set.
Model stacking almost always works ...
Top 50 recent answers are included
