Questions tagged [machine-learning]
Machine Learning is a subfield of computer science that draws on elements from algorithmic analysis, computational statistics, mathematics, optimization, etc. It is mainly concerned with the use of data to construct models that have high predictive/forecasting ability. Topics include modeling building, applications, theory, etc.
3,362
questions with no upvoted or accepted answers
9
votes
0
answers
2k
views
AdaBoost implementation and tuning for high dimensional feature space in R
I am trying to implement the AdaBoost.M1 algorithm (trees as base-learners) to a data set with a large feature space (~ 20.000 features) and ~ 100 samples in R. ...
- 91
8
votes
0
answers
132
views
Training value neural network AlphaGo style
I have been trying to replicate the results obtained by AlphaGo following their supervise learning protocol. The papers specify that they use a network that has two heads: a value head that predicts ...
- 101
8
votes
1
answer
252
views
How to predict advantage value in deep reinforcement learning
I'm currently working on a collection of reinforcement algorithms: https://github.com/lhk/rl_gym
For deep q-learning, you need to calculate the q-values that should be predicted by your network. There ...
- 181
7
votes
3
answers
4k
views
Why is 10000 used as the denominator in Positional Encodings in the Transformer Model?
I was working through the All you need is Attention paper, and while the motivation of positional encodings makes sense and the other stackexchange answers filled me in on the motivations of the ...
7
votes
0
answers
512
views
differences between LSQR and FTRL when working with very sparse data
I have a 2M instances dataset with millions of very very sparse dummy variables created using the hashing trick = ...
- 1,357
6
votes
2
answers
237
views
What does big O mean in KNN optimal weights?
Wiki gives this definition of KNN
In pattern recognition, the k-nearest neighbors algorithm (k-NN) is a
non-parametric method used for classification and regression. In both
cases, the input consists ...
- 623
6
votes
2
answers
325
views
Training stateful LSTM with different number of sequences
I'm using a stateful LSTM for stock market analysis, and I have varying amounts of data for each stock, ranging from 20 years to just a few weeks (i.e. for newly listed stocks).
I use 3 years of data ...
- 740
6
votes
3
answers
1k
views
how does XGBoost's exact greedy split finding algorithm determine candidate split values for different feature types?
Based on the paper by Chen & Guestrin (2016)
"XGBoost: A Scalable Tree Boosting System", XGBoost's "exact split finding algorithm enumerates over all the possible splits on all the features to ...
- 71
6
votes
1
answer
178
views
Predicting change of shapes/coordinates
I'm trying to find a way to predict/calculate how a shape (e.g. outline of a glacier) will change in the future—based on its history (previous shape) and additional factors (e.g. Δtemperature).
In my ...
- 161
6
votes
0
answers
298
views
Confidence value in AdaBoost?
I read this introduction about AdaBoost (http://www.cs.man.ac.uk/~nikolaon/~nikolaon_files/Introduction_to_AdaBoost.pdf), and am curious why confidence for each model is defined as
$$\alpha_j=\frac{...
- 143
6
votes
0
answers
95
views
Classify driver based on time-series sensor data
I want to build a model that can detect which driver is driving now the car based on a dataset that contains 20 driver records for 3600s each driver ( the dataset contains all the car sensors values ...
- 61
6
votes
0
answers
289
views
how to propagate error from convolutional layer to previous layer？
I've been trying to implement a simple convolutional neural network. But I've been stuck at this problem for over a week.
To be specific, assume there are 3 layers in a convolutional pass, marked as ...
- 161
5
votes
1
answer
636
views
Linear Regression bad results after log transformation
I have a dataset that has the following columns:
The variable I'm trying to predict is "rent".
My dataset looks a lot similar to what happens in this notebook. I tried to normalize the rent ...
- 157
5
votes
1
answer
98
views
What ML architecture fits fixed length signal regression?
My problem is of regression type -
How to estimate a fish weight using a fixed-length signal (80 data points) of the change in resistance when the fish swim through a gate with electrodes (basically 4 ...
- 51
5
votes
0
answers
2k
views
Dealing with categorical variables in Isolation Forest
Isolation Forest is widely used when dealing with outlier/anomaly detection when we have no labels. The theory behind is that making random split at random points and counting how many splits you do ...
- 6,082
5
votes
1
answer
342
views
Can I turn any binary classification algorithms into multiclass algorithms using softmax and cross-entropy loss?
Softmax + cross-entropy loss for multiclass classification is used in ML algorithms such as softmax regression and (last layer of) neural networks. I wonder if this method could turn any binary ...
5
votes
2
answers
418
views
Evaluation of regression models with different evaluations (MSE, variance, VAF etc.)
When comparing several regression models in terms of quality, it seems like most have agreed on the MSE.
There are also papers comparing "variance" and "variance accounted for (VAF)&...
- 183
5
votes
1
answer
496
views
Illustrating the dimensionality reduction done by a classification or regression model
Tl;DR: You can predict something, but how do you explain the prediction?
EDIT: I have built a website that tries to answer this question with means of embedding / visually clustering data according ...
- 151
4
votes
1
answer
1k
views
Difference Between Attention and Fully Connected Layers in Deep Learning
There have been several papers in the last few years on the so-called "Attention" mechanism in deep learning (e.g. 1 2). The concept seems to be that we want the neural network to focus on ...
- 836
4
votes
2
answers
528
views
Analysis of probability distribution of each features and Machine Learning
While I know that probability distributions are for hypothesis testing, confidence level constructions, etc. They definitely have many roles in statistical analysis.
However, it is not obvious to me ...
- 411
4
votes
2
answers
605
views
How do I know that model performance improvement is significant?
Say I am running a Machine Learning model that produces a certain result (say accuracy of 80%). I now change a minor detail in my model (say, in a Deep Learning model, increase the kernel size in one ...
- 141
4
votes
3
answers
104
views
Understanding Weighted learning in Ensemble Classifiers
I'm currently studying Boosting techniques in Machine Learning and I happened to understand that in Algorithms like Adaboost, each of the training samples is given a weight depending on whether it was ...
- 225
4
votes
1
answer
2k
views
Self-attention mechanism did not improve the LSTM classification model
I am doing an 8-class classification using time series data.
It appears that the implementation of the self-attention mechanism has no effect on the model so I think my implementations have some ...
- 125
4
votes
1
answer
4k
views
Weighted loss functions vs weighted sampling?
For image classification tasks, is there a practical difference between using weighted loss functions vs. using weighted sampling? (I would appreciate theoretical arguments, experience or published ...
4
votes
2
answers
1k
views
How to incorporate new features in an existing machine learning model?
Suppose we have trained a regression model $M$ on a fixed set of $n$ features, $F_1,F_2,…,F_n$ on a particular dataset $G$. Now assume that after model training, additional features ($F_{n+1},…$) ...
- 141
4
votes
1
answer
87
views
Training Machine Learning Model - Neural Network - Islands Problem
I was working on the following leetcode problem:
Given a 2d grid map of '1's (land) and '0's (water), count the number
of islands. An island is surrounded by water and is formed by
connecting ...
- 41
4
votes
4
answers
403
views
Backpropagation: Relevance of the error signal of a neuron
During my quest to understand back propagation in a more rigorous approach I have come across with the definition of error signal of a neuron which is defined as follows for the $j^{\text{th}}$ neuron ...
- 41
4
votes
1
answer
1k
views
SelectKBest and Correlation returns me excatly same feature selection. How?
Im working on selecting most effective features from a dataset with over that 2000 features. Im using different algorithms for that (selectKBest with chi-square, Extra Trees, Correlation etc.) But ...
4
votes
2
answers
6k
views
Saving and loading keras.callbacks.History object with np.save and np.load
I have been saving my training history in keras as follows:
...
- 51
4
votes
4
answers
658
views
Where does the "deep learning needs big data" rule come from
When reading about deep learning I often come across the rule that deep learning is only effective when you have large amounts of data at your disposal. These statements are generally accompanied by a ...
- 41
4
votes
1
answer
1k
views
Calculating saliency maps for text classification
I'm following the text classification with movie reviews TensorFlow tutorial, and wanted to extend the project by looking, for a certain input, which words influenced the classification the most.
I ...
- 41
4
votes
2
answers
154
views
What is the most appropriate machine learning approach for this scenario?
The scenario is pretty simple, and I'm sure it's been done a million times. The problem is I don't know the terminology to find the correct resources on the web.
Scenario: I have an environment that ...
- 141
4
votes
1
answer
789
views
Product classification in hierarchical categories based on multiple parameters and non-standard descriptions
I want to start a machine learning project in my company and a really big pain for spend analysts is to classify the products that buyers order for maintenance, tooling, raw material and such, as the ...
- 41
4
votes
2
answers
1k
views
How can I use two different datasets as a training model for svm
I know that you're supposed to scale your test data using the parameters (mean and stdev) from your training data. This is relatively simple; but what if the number of samples is limited in one ...
- 51
4
votes
1
answer
409
views
Time horizon T in policy gradients (actor-critic)
I am currently going through the Berkeley lectures on Reinforcement Learning. Specifically, I am at slide 5 of this lecture.
At the bottom of that slide, the gradient of the expected sum of rewards ...
4
votes
1
answer
129
views
EM-ELM Cross validation
I know that cross validation is used to find the best hyperparameters that minimize the average error. For example, the number of neurons that minimize the average error of cross-validation is ...
- 41
4
votes
1
answer
67
views
Hochreiter LSTM (p. 4): Maximal values of logistic sigmoid derivative times weight
My questions follow the below page 4 excerpt from Hochreiter's LSTM paper:
If $f_{l_{m}}$ is the logistic sigmoid function, then the maximal
value of $f^\prime_{l_{m}}$ is 0.25. If $y^{l_{m-1}}$ ...
4
votes
1
answer
450
views
Multiple activation functions with TensorFlow estimator DNNClassifier
I just want to know if is it possible to use tf.estimator.DNNClassifier with multiple different activation functions. I mean, could I use a DNNClassifier estimator which use different activation ...
- 41
4
votes
2
answers
267
views
Support Vector Regression trained with data sets
I am now searching for a long time on the internet and on papers for an answers of simple questions. Am I able to train a Support Vector Regression algorithm with different data sets? If yes, how is ...
- 41
4
votes
2
answers
4k
views
Fix first two levels of decision tree?
I am trying to build a regression tree with 70 attributes where the business team wants to fix the first two levels namely country and product type. To achieve this, I have two proposals:
Build a ...
- 41
4
votes
1
answer
259
views
How to automatically verify official documents?
I am new to machine learning and data science. I apologise if the question seems very basic. I have a requirement where I need to verify information submitted via a form with the corresponding ...
3
votes
0
answers
63
views
Where can I find implementation of the various improvements of K-nearest neighbors (KNN)?
I have been facing some challenges where traditional KNN algorithm perform well. I'd like to explore more advanced knn solutions. While researching possible solutions, I came across a paper titled
<...
- 2,567
3
votes
1
answer
4k
views
What are the "EMA" weights included with certain pre-trained models?
I came across this machine learning model called "Stable Diffusion v1." When downloading the pre-trained weights for this model, I noticed that there's a couple of different files:
...
- 131
3
votes
1
answer
321
views
Zero-shot learning for tabular data?
Can anyone point me to methods for zero-shot learning on tabular data? There is some very cool work being done for zero-shot learning on images and text, but I'm struggling to find work being done to ...
- 63
3
votes
0
answers
103
views
Intuitively, why do Non-monotonic Activations Work?
The swish/SiLU activation is very popular, and many would argue it has dethroned ReLU. However, it is non-monotonic, which seems to go against popular intuition (at least on this site: example 1, ...
- 53
3
votes
1
answer
57
views
String together a set of tokens into a sequence
I have this problem scenario - Given a set of tokens, string them or a subset of the tokens together using stop words into a sequence. I am clear that I can have potentially infinite pre-training data ...
- 141
3
votes
1
answer
43
views
Request: Confirmation on my understanding of overfitting and regularization concepts
Overfitted models tend to have largely different (some very high, some comparatively low) coefficients/weights for different feature values. So, this means the model (when drawn as graph) will have ...
- 151
3
votes
3
answers
137
views
Example for Boosting
Can someone exactly tell me how does boosting as implemented by LightGBM or XGBoost work in real case scenerio. Like I know it splits tree leaf wise instead of level wise, which will contribute to ...
- 193
3
votes
0
answers
57
views
Is it Scientifically Rigorous to create a multimodal ML Model with data from multiple sources
I'm attempting to create a multimodal machine learning model for disease diagnosis. However, I'm having quite a bit of difficulty finding public data sets with all the data I need.
For example, I need ...
3
votes
3
answers
123
views
Classification when the classification of the previous itens matter
I have a classification problem to solve, that seems to be common but I am struggling to find the name of this task and the best way to model this problem.
Suppose I have a series of events that are ...
- 31