Questions tagged [machine-learning]
Machine Learning is a subfield of computer science that draws on elements from algorithmic analysis, computational statistics, mathematics, optimization, etc. It is mainly concerned with the use of data to construct models that have high predictive/forecasting ability. Topics include modeling building, applications, theory, etc.
3,180
questions with no upvoted or accepted answers
9
votes
0
answers
2k
views
AdaBoost implementation and tuning for high dimensional feature space in R
I am trying to implement the AdaBoost.M1 algorithm (trees as base-learners) to a data set with a large feature space (~ 20.000 features) and ~ 100 samples in R. ...
- 91
8
votes
0
answers
143
views
Training value neural network AlphaGo style
I have been trying to replicate the results obtained by AlphaGo following their supervise learning protocol. The papers specify that they use a network that has two heads: a value head that predicts ...
- 101
8
votes
1
answer
335
views
How to predict advantage value in deep reinforcement learning
I'm currently working on a collection of reinforcement algorithms: https://github.com/lhk/rl_gym
For deep q-learning, you need to calculate the q-values that should be predicted by your network. There ...
- 181
7
votes
0
answers
517
views
differences between LSQR and FTRL when working with very sparse data
I have a 2M instances dataset with millions of very very sparse dummy variables created using the hashing trick = ...
- 1,357
6
votes
1
answer
299
views
What does big O mean in KNN optimal weights?
Wiki gives this definition of KNN
In pattern recognition, the k-nearest neighbors algorithm (k-NN) is a
non-parametric method used for classification and regression. In both
cases, the input consists ...
- 623
6
votes
1
answer
189
views
Predicting change of shapes/coordinates
I'm trying to find a way to predict/calculate how a shape (e.g. outline of a glacier) will change in the future—based on its history (previous shape) and additional factors (e.g. Δtemperature).
In my ...
- 161
6
votes
0
answers
312
views
Confidence value in AdaBoost?
I read this introduction about AdaBoost (http://www.cs.man.ac.uk/~nikolaon/~nikolaon_files/Introduction_to_AdaBoost.pdf), and am curious why confidence for each model is defined as
$$\alpha_j=\frac{...
- 143
6
votes
0
answers
100
views
Classify driver based on time-series sensor data
I want to build a model that can detect which driver is driving now the car based on a dataset that contains 20 driver records for 3600s each driver ( the dataset contains all the car sensors values ...
- 61
6
votes
0
answers
292
views
how to propagate error from convolutional layer to previous layer？
I've been trying to implement a simple convolutional neural network. But I've been stuck at this problem for over a week.
To be specific, assume there are 3 layers in a convolutional pass, marked as ...
- 161
5
votes
1
answer
2k
views
Difference Between Attention and Fully Connected Layers in Deep Learning
There have been several papers in the last few years on the so-called "Attention" mechanism in deep learning (e.g. 1 2). The concept seems to be that we want the neural network to focus on ...
- 916
5
votes
0
answers
176
views
Intuitively, why do Non-monotonic Activations Work?
The swish/SiLU activation is very popular, and many would argue it has dethroned ReLU. However, it is non-monotonic, which seems to go against popular intuition (at least on this site: example 1, ...
- 73
5
votes
1
answer
103
views
What ML architecture fits fixed length signal regression?
My problem is of regression type -
How to estimate a fish weight using a fixed-length signal (80 data points) of the change in resistance when the fish swim through a gate with electrodes (basically 4 ...
- 51
5
votes
0
answers
2k
views
Dealing with categorical variables in Isolation Forest
Isolation Forest is widely used when dealing with outlier/anomaly detection when we have no labels. The theory behind is that making random split at random points and counting how many splits you do ...
- 6,332
5
votes
1
answer
453
views
Time horizon T in policy gradients (actor-critic)
I am currently going through the Berkeley lectures on Reinforcement Learning. Specifically, I am at slide 5 of this lecture.
At the bottom of that slide, the gradient of the expected sum of rewards ...
5
votes
2
answers
519
views
Evaluation of regression models with different evaluations (MSE, variance, VAF etc.)
When comparing several regression models in terms of quality, it seems like most have agreed on the MSE.
There are also papers comparing "variance" and "variance accounted for (VAF)&...
- 183
5
votes
2
answers
4k
views
Fix first two levels of decision tree?
I am trying to build a regression tree with 70 attributes where the business team wants to fix the first two levels namely country and product type. To achieve this, I have two proposals:
Build a ...
- 51
5
votes
1
answer
507
views
Illustrating the dimensionality reduction done by a classification or regression model
Tl;DR: You can predict something, but how do you explain the prediction?
EDIT: I have built a website that tries to answer this question with means of embedding / visually clustering data according ...
- 151
4
votes
2
answers
758
views
Analysis of probability distribution of each features and Machine Learning
While I know that probability distributions are for hypothesis testing, confidence level constructions, etc. They definitely have many roles in statistical analysis.
However, it is not obvious to me ...
- 411
4
votes
4
answers
327
views
Decision boundary in a classification task
I have 1000 data points from the bivariate normal distribution $\mathcal{N}$ with mean $(0,0)$ and variance $\sigma_1^2=\sigma_2^2=10$ with the covariances being $0$. Also there are 20 more points ...
- 141
4
votes
1
answer
5k
views
Weighted loss functions vs weighted sampling?
For image classification tasks, is there a practical difference between using weighted loss functions vs. using weighted sampling? (I would appreciate theoretical arguments, experience or published ...
4
votes
2
answers
2k
views
How to incorporate new features in an existing machine learning model?
Suppose we have trained a regression model $M$ on a fixed set of $n$ features, $F_1,F_2,…,F_n$ on a particular dataset $G$. Now assume that after model training, additional features ($F_{n+1},…$) ...
- 141
4
votes
1
answer
97
views
Training Machine Learning Model - Neural Network - Islands Problem
I was working on the following leetcode problem:
Given a 2d grid map of '1's (land) and '0's (water), count the number
of islands. An island is surrounded by water and is formed by
connecting ...
- 41
4
votes
4
answers
486
views
Backpropagation: Relevance of the error signal of a neuron
During my quest to understand back propagation in a more rigorous approach I have come across with the definition of error signal of a neuron which is defined as follows for the $j^{\text{th}}$ neuron ...
- 41
4
votes
1
answer
1k
views
SelectKBest and Correlation returns me excatly same feature selection. How?
Im working on selecting most effective features from a dataset with over that 2000 features. Im using different algorithms for that (selectKBest with chi-square, Extra Trees, Correlation etc.) But ...
4
votes
0
answers
522
views
How to apply oversampling when doing Leave-One-Group-Out cross validation?
I am working on an imbalanced data for classification and I tried to use SMOTE previously to oversampling the training data. However, this time I think I need to use a leave-on group out (LOGO) cross-...
- 141
4
votes
2
answers
7k
views
Saving and loading keras.callbacks.History object with np.save and np.load
I have been saving my training history in keras as follows:
...
- 51
4
votes
0
answers
243
views
Training deep CNN with noisy dataset
I am training a Mask RCNN model with a train dataset that has been generated from some simple computer vision operations (color thresholding) and some morphological filtering.
The train set captures ...
- 171
4
votes
4
answers
743
views
Where does the "deep learning needs big data" rule come from
When reading about deep learning I often come across the rule that deep learning is only effective when you have large amounts of data at your disposal. These statements are generally accompanied by a ...
- 41
4
votes
1
answer
851
views
Product classification in hierarchical categories based on multiple parameters and non-standard descriptions
I want to start a machine learning project in my company and a really big pain for spend analysts is to classify the products that buyers order for maintenance, tooling, raw material and such, as the ...
- 41
4
votes
3
answers
118
views
Running an LSTM with Music Data
I'm working on a project for a class where I'm trying to create an algorithm that learns music and creates its own music.
I'm having trouble on how to set up the data for it to be inputted into the ...
- 41
4
votes
1
answer
135
views
EM-ELM Cross validation
I know that cross validation is used to find the best hyperparameters that minimize the average error. For example, the number of neurons that minimize the average error of cross-validation is ...
- 41
4
votes
1
answer
75
views
Hochreiter LSTM (p. 4): Maximal values of logistic sigmoid derivative times weight
My questions follow the below page 4 excerpt from Hochreiter's LSTM paper:
If $f_{l_{m}}$ is the logistic sigmoid function, then the maximal
value of $f^\prime_{l_{m}}$ is 0.25. If $y^{l_{m-1}}$ ...
4
votes
1
answer
469
views
Multiple activation functions with TensorFlow estimator DNNClassifier
I just want to know if is it possible to use tf.estimator.DNNClassifier with multiple different activation functions. I mean, could I use a DNNClassifier estimator which use different activation ...
- 41
4
votes
2
answers
273
views
Support Vector Regression trained with data sets
I am now searching for a long time on the internet and on papers for an answers of simple questions. Am I able to train a Support Vector Regression algorithm with different data sets? If yes, how is ...
- 41
4
votes
1
answer
274
views
How to automatically verify official documents?
I am new to machine learning and data science. I apologise if the question seems very basic. I have a requirement where I need to verify information submitted via a form with the corresponding ...
3
votes
0
answers
75
views
Where can I find implementation of the various improvements of K-nearest neighbors (KNN)?
I have been facing some challenges where traditional KNN algorithm perform well. I'd like to explore more advanced knn solutions. While researching possible solutions, I came across a paper titled
<...
- 2,378
3
votes
1
answer
63
views
String together a set of tokens into a sequence
I have this problem scenario - Given a set of tokens, string them or a subset of the tokens together using stop words into a sequence. I am clear that I can have potentially infinite pre-training data ...
- 141
3
votes
0
answers
229
views
Cluster tabular data with text in some columns
Let's say I have a following features in the my dataframe:
user_id
user_age
is_student
is_graduate
salary
resume
integer
integer
binary
binary
integer
text (up to 1000 symbols)
And also a few more ...
- 31
3
votes
0
answers
62
views
Is it Scientifically Rigorous to create a multimodal ML Model with data from multiple sources
I'm attempting to create a multimodal machine learning model for disease diagnosis. However, I'm having quite a bit of difficulty finding public data sets with all the data I need.
For example, I need ...
3
votes
3
answers
136
views
Classification when the classification of the previous itens matter
I have a classification problem to solve, that seems to be common but I am struggling to find the name of this task and the best way to model this problem.
Suppose I have a series of events that are ...
- 31
3
votes
0
answers
2k
views
Forecast Model to Estimate Customer Service Call Volume and Appropriate Staff
I am working on a project to predict the proper staffing needed for a customer service team using historical data.
I am new to machine learning, and I am not sure if my approach to this problem is the ...
- 31
3
votes
1
answer
1k
views
Understanding Conv1D Output Shape
I am a little confused with the output shape that Conv1D produces. Consider the code I have used as the following (a lot has been omitted for clarity):
...
3
votes
2
answers
2k
views
How to handle undefined or null data in a neural network
Let me preface this post with I am incredibly new to machine learning/neural networks. I am currently working on a classification neural network using TensorFlow whose input is multiple features of ...
- 31
3
votes
0
answers
254
views
Struggling to understand/implement Transformer Decoder
I'm struggling to understand the decoder in a Transformer model, specifically with regards to some aspects of its architecture as well as how it actually handles the data during training.
What I have ...
- 131
3
votes
1
answer
2k
views
Hopfield Network python implementation, Network doesn't converge to one of the learned patterns
I'm trying to implement a Hopfield Network in python using the NumPy library. The network has 2500 nodes (50 height x 50 width). The network learns 10 patterns from images of size 50x50 stored in &...
- 31
3
votes
0
answers
293
views
Unbiasedness of random forests
Suppose that I am trying to build a random forest by subsampling the data and choosing a single feature per tree randomly. For example, suppose there is some dataset,
$D = \{(x_{1},y_{1}), ......(x_{N}...
- 131
3
votes
0
answers
47
views
Finding the position of an arbitrary object in a static image?
One common object detection scenario involves finding trained models in an arbitrary scene. For example, we can train a model to understand what a "bicycle" looks like, by providing various ...
- 131
3
votes
0
answers
151
views
Clustering large set of images
I've got some big datasets of images (a few million each), and I would like to cluster them according to images' visual similarities. I've extracted a feature vector for each image; the space of ...
- 31
3
votes
0
answers
798
views
What exactly negative/positive value of Captum's Integrated Gradient mean?
I use Captum's Integrated Gradient to interprete my PyTorch's neural network. I know that from github and original paper mentioned that ...
Positive attribution score means that the input in that ...
- 31
3
votes
0
answers
126
views
Detecting pushups based on pose data
I've been playing with Google's MLKit, and decided to detect push ups.
As a quick test, I took the position of the left shoulder, and plotted the Y Axis. Here's how a variety of trials look:
Five ...
