David Waterworth
  • Member for 2 years, 11 months
  • Last seen this week
difference between model-based boosting and gradient boosting
2 votes

Gradient Boosting is fitting a base learner $f_{i}(X)$ to the gradient of the loss function of an existing model $F_{i-1}(X)$ i.e. find base learner $f_i$ which minimises $L(-g_i, f_t(x_i))$ where $...

View answer
Why do we use gradients instead of residuals in Gradient Boosting?
2 votes

I know this is an old question, but it took me a while to get my head around it The objective at each iteration of the gradient boosting algorithm is to find a base learner which gives the largest ...

View answer
Predict the average temperature for next 30 years
1 votes

At a high level you need to find historical data matching what you need to forecast (average temperature) and any factors which influence it. So you need to be a domain expert, have access to one, or ...

View answer
How exactly do Gaussian Processes (square dist kernel) enforce smoothness? (Aka how are they computed to do so?)
1 votes

It's by definition, when you fit a guassian process you specify the mean function m(x) and the covariance function (or kernel) k(x,x'). Often the mean function is 0 and the covariance is the radial ...

View answer
Significant drop from validation accuracy to test accuracy
1 votes

I don't think this is ususual, I experiance it quite frequently with regression problems. Generally I think it means the model is underspecified so instead of learning the actual relationship between ...

View answer
Splitting training and test set with financial data
Accepted answer
1 votes

I think what you're doing is correct, in fact it would be even more correct to introduce a gap between your test and train set, i.e. train <-head(stock_indicators,round(0.65*nrow(stock_indicators))...

View answer
Does neuron have weight?
1 votes

If you look at the 2nd equation under "Propagation function" $p_{j}(t)=\sum _{i=1}^No_{i}(t)w_{ij}+w_{0j}$ The $w_{0j}$ is the constant (bias), so it's reasonable to write that the $w_{ij}$ weights ...

View answer
Parallel hyperparameter optimization techniques?
1 votes

The python hyperopt library will evaluate multiple trials in parallel, it's open source and there's a paper. Also I'm fairly sure AWS Sagemaker has a distributed Baysian algorithm, it doens't meet ...

View answer
Analysis of Time Series data
1 votes

Speaking as someone from a finance background, the `usual' model for a stock price process is $\frac{dS}{S}=r dt + \sigma dW_t$ i.e. we assume the returns (not the absolute price changes, i.e. dS/S ...

View answer
Unsupervised Hierarchical Agglomerative Clustering
0 votes

I think I've figured out how to implement the algorithm described in the paper I'm studying. I suspect they used scipy.cluster.hierarchy. Anyway, my process is: Generate a distance matrix y from my ...

View answer
splitting into train test by train_test_split of float values?
0 votes

The issue is your data violates the requirements of StratifiedShuffleSplit, specifically it's not possible to do a 70:30 split of the data and maintain the same number of distinct y values in the test ...

View answer
Does the bias of an artificial neuron adjust or remain constant during training?
0 votes

The $\omega$ in that section is a vector of weights not a single weight, and when they write $\omega_i$ in this context they mean the weights for every connection and all the bias's at iteration $i$ (...

View answer
Large no of categorical variables with large no of categories
0 votes

If you're using python and sklearn I'd suggest you take a look at http://contrib.scikit-learn.org/categorical-encoding/ There's a large number of different encoding schemes you can experiment with. ...

View answer