Questions tagged [machine-learning]

Machine Learning is a subfield of computer science that draws on elements from algorithmic analysis, computational statistics, mathematics, optimization, etc. It is mainly concerned with the use of data to construct models that have high predictive/forecasting ability. Topics include modeling building, applications, theory, etc.

3,292 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
9 votes
0 answers
2k views

AdaBoost implementation and tuning for high dimensional feature space in R

I am trying to implement the AdaBoost.M1 algorithm (trees as base-learners) to a data set with a large feature space (~ 20.000 features) and ~ 100 samples in R. ...
AfBM's user avatar
  • 91
8 votes
0 answers
138 views

Training value neural network AlphaGo style

I have been trying to replicate the results obtained by AlphaGo following their supervise learning protocol. The papers specify that they use a network that has two heads: a value head that predicts ...
Phaune's user avatar
  • 101
8 votes
1 answer
286 views

How to predict advantage value in deep reinforcement learning

I'm currently working on a collection of reinforcement algorithms: https://github.com/lhk/rl_gym For deep q-learning, you need to calculate the q-values that should be predicted by your network. There ...
lhk's user avatar
  • 181
7 votes
0 answers
516 views

differences between LSQR and FTRL when working with very sparse data

I have a 2M instances dataset with millions of very very sparse dummy variables created using the hashing trick = ...
ihadanny's user avatar
  • 1,357
6 votes
1 answer
268 views

What does big O mean in KNN optimal weights?

Wiki gives this definition of KNN In pattern recognition, the k-nearest neighbors algorithm (k-NN) is a non-parametric method used for classification and regression. In both cases, the input consists ...
JJJohn's user avatar
  • 623
6 votes
2 answers
348 views

Training stateful LSTM with different number of sequences

I'm using a stateful LSTM for stock market analysis, and I have varying amounts of data for each stock, ranging from 20 years to just a few weeks (i.e. for newly listed stocks). I use 3 years of data ...
BigBadMe's user avatar
  • 750
6 votes
1 answer
187 views

Predicting change of shapes/coordinates

I'm trying to find a way to predict/calculate how a shape (e.g. outline of a glacier) will change in the future—based on its history (previous shape) and additional factors (e.g. Δtemperature). In my ...
Fabian Schultz's user avatar
6 votes
0 answers
306 views

Confidence value in AdaBoost?

I read this introduction about AdaBoost (http://www.cs.man.ac.uk/~nikolaon/~nikolaon_files/Introduction_to_AdaBoost.pdf), and am curious why confidence for each model is defined as $$\alpha_j=\frac{...
user3326682's user avatar
6 votes
0 answers
99 views

Classify driver based on time-series sensor data

I want to build a model that can detect which driver is driving now the car based on a dataset that contains 20 driver records for 3600s each driver ( the dataset contains all the car sensors values ...
John Karimov's user avatar
6 votes
0 answers
291 views

how to propagate error from convolutional layer to previous layer?

I've been trying to implement a simple convolutional neural network. But I've been stuck at this problem for over a week. To be specific, assume there are 3 layers in a convolutional pass, marked as ...
Shindou's user avatar
  • 161
5 votes
1 answer
1k views

Difference Between Attention and Fully Connected Layers in Deep Learning

There have been several papers in the last few years on the so-called "Attention" mechanism in deep learning (e.g. 1 2). The concept seems to be that we want the neural network to focus on ...
Adam's user avatar
  • 896
5 votes
1 answer
749 views

Linear Regression bad results after log transformation

I have a dataset that has the following columns: The variable I'm trying to predict is "rent". My dataset looks a lot similar to what happens in this notebook. I tried to normalize the rent ...
Caldass_'s user avatar
  • 167
5 votes
1 answer
102 views

What ML architecture fits fixed length signal regression?

My problem is of regression type - How to estimate a fish weight using a fixed-length signal (80 data points) of the change in resistance when the fish swim through a gate with electrodes (basically 4 ...
Shay's user avatar
  • 51
5 votes
0 answers
2k views

Dealing with categorical variables in Isolation Forest

Isolation Forest is widely used when dealing with outlier/anomaly detection when we have no labels. The theory behind is that making random split at random points and counting how many splits you do ...
Carlos Mougan's user avatar
5 votes
1 answer
431 views

Can I turn any binary classification algorithms into multiclass algorithms using softmax and cross-entropy loss?

Softmax + cross-entropy loss for multiclass classification is used in ML algorithms such as softmax regression and (last layer of) neural networks. I wonder if this method could turn any binary ...
CouldntLoginToMyPreviousAcc's user avatar
5 votes
2 answers
478 views

Evaluation of regression models with different evaluations (MSE, variance, VAF etc.)

When comparing several regression models in terms of quality, it seems like most have agreed on the MSE. There are also papers comparing "variance" and "variance accounted for (VAF)&...
MerklT's user avatar
  • 183
5 votes
1 answer
503 views

Illustrating the dimensionality reduction done by a classification or regression model

Tl;DR: You can predict something, but how do you explain the prediction? EDIT: I have built a website that tries to answer this question with means of embedding / visually clustering data according ...
BenoitParis's user avatar
4 votes
2 answers
655 views

Analysis of probability distribution of each features and Machine Learning

While I know that probability distributions are for hypothesis testing, confidence level constructions, etc. They definitely have many roles in statistical analysis. However, it is not obvious to me ...
Student's user avatar
  • 411
4 votes
3 answers
133 views

Understanding Weighted learning in Ensemble Classifiers

I'm currently studying Boosting techniques in Machine Learning and I happened to understand that in Algorithms like Adaboost, each of the training samples is given a weight depending on whether it was ...
AnonymousMe's user avatar
4 votes
1 answer
2k views

Self-attention mechanism did not improve the LSTM classification model

I am doing an 8-class classification using time series data. It appears that the implementation of the self-attention mechanism has no effect on the model so I think my implementations have some ...
Leo's user avatar
  • 125
4 votes
4 answers
251 views

Decision boundary in a classification task

I have 1000 data points from the bivariate normal distribution $\mathcal{N}$ with mean $(0,0)$ and variance $\sigma_1^2=\sigma_2^2=10$ with the covariances being $0$. Also there are 20 more points ...
thenac's user avatar
  • 141
4 votes
1 answer
4k views

Weighted loss functions vs weighted sampling?

For image classification tasks, is there a practical difference between using weighted loss functions vs. using weighted sampling? (I would appreciate theoretical arguments, experience or published ...
Jean-Pierre Coffe's user avatar
4 votes
2 answers
1k views

How to incorporate new features in an existing machine learning model?

Suppose we have trained a regression model $M$ on a fixed set of $n$ features, $F_1,F_2,…,F_n$ on a particular dataset $G$. Now assume that after model training, additional features ($F_{n+1},…$) ...
Ruben's user avatar
  • 141
4 votes
1 answer
95 views

Training Machine Learning Model - Neural Network - Islands Problem

I was working on the following leetcode problem: Given a 2d grid map of '1's (land) and '0's (water), count the number of islands. An island is surrounded by water and is formed by connecting ...
user87771's user avatar
4 votes
4 answers
446 views

Backpropagation: Relevance of the error signal of a neuron

During my quest to understand back propagation in a more rigorous approach I have come across with the definition of error signal of a neuron which is defined as follows for the $j^{\text{th}}$ neuron ...
utxeee's user avatar
  • 41
4 votes
1 answer
1k views

SelectKBest and Correlation returns me excatly same feature selection. How?

Im working on selecting most effective features from a dataset with over that 2000 features. Im using different algorithms for that (selectKBest with chi-square, Extra Trees, Correlation etc.) But ...
justRandomLearner's user avatar
4 votes
0 answers
510 views

How to apply oversampling when doing Leave-One-Group-Out cross validation?

I am working on an imbalanced data for classification and I tried to use SMOTE previously to oversampling the training data. However, this time I think I need to use a leave-on group out (LOGO) cross-...
npm's user avatar
  • 141
4 votes
2 answers
7k views

Saving and loading keras.callbacks.History object with np.save and np.load

I have been saving my training history in keras as follows: ...
Ben Groene's user avatar
4 votes
4 answers
701 views

Where does the "deep learning needs big data" rule come from

When reading about deep learning I often come across the rule that deep learning is only effective when you have large amounts of data at your disposal. These statements are generally accompanied by a ...
Aran's user avatar
  • 41
4 votes
1 answer
1k views

Calculating saliency maps for text classification

I'm following the text classification with movie reviews TensorFlow tutorial, and wanted to extend the project by looking, for a certain input, which words influenced the classification the most. I ...
Marc Jones's user avatar
4 votes
1 answer
810 views

Product classification in hierarchical categories based on multiple parameters and non-standard descriptions

I want to start a machine learning project in my company and a really big pain for spend analysts is to classify the products that buyers order for maintenance, tooling, raw material and such, as the ...
mad_dash2's user avatar
4 votes
1 answer
430 views

Time horizon T in policy gradients (actor-critic)

I am currently going through the Berkeley lectures on Reinforcement Learning. Specifically, I am at slide 5 of this lecture. At the bottom of that slide, the gradient of the expected sum of rewards ...
Dummie Variable's user avatar
4 votes
1 answer
134 views

EM-ELM Cross validation

I know that cross validation is used to find the best hyperparameters that minimize the average error. For example, the number of neurons that minimize the average error of cross-validation is ...
Pablo's user avatar
  • 41
4 votes
1 answer
73 views

Hochreiter LSTM (p. 4): Maximal values of logistic sigmoid derivative times weight

My questions follow the below page 4 excerpt from Hochreiter's LSTM paper: If $f_{l_{m}}$ is the logistic sigmoid function, then the maximal value of $f^\prime_{l_{m}}$ is 0.25. If $y^{l_{m-1}}$ ...
The AI Architect's user avatar
4 votes
1 answer
464 views

Multiple activation functions with TensorFlow estimator DNNClassifier

I just want to know if is it possible to use tf.estimator.DNNClassifier with multiple different activation functions. I mean, could I use a DNNClassifier estimator which use different activation ...
David C.'s user avatar
4 votes
2 answers
270 views

Support Vector Regression trained with data sets

I am now searching for a long time on the internet and on papers for an answers of simple questions. Am I able to train a Support Vector Regression algorithm with different data sets? If yes, how is ...
i.k.'s user avatar
  • 41
4 votes
2 answers
4k views

Fix first two levels of decision tree?

I am trying to build a regression tree with 70 attributes where the business team wants to fix the first two levels namely country and product type. To achieve this, I have two proposals: Build a ...
Aravind's user avatar
  • 41
4 votes
1 answer
267 views

How to automatically verify official documents?

I am new to machine learning and data science. I apologise if the question seems very basic. I have a requirement where I need to verify information submitted via a form with the corresponding ...
Prakhar Sinha's user avatar
3 votes
0 answers
73 views

Where can I find implementation of the various improvements of K-nearest neighbors (KNN)?

I have been facing some challenges where traditional KNN algorithm perform well. I'd like to explore more advanced knn solutions. While researching possible solutions, I came across a paper titled <...
Lucas Morin's user avatar
  • 2,196
3 votes
0 answers
132 views

Intuitively, why do Non-monotonic Activations Work?

The swish/SiLU activation is very popular, and many would argue it has dethroned ReLU. However, it is non-monotonic, which seems to go against popular intuition (at least on this site: example 1, ...
Jason's user avatar
  • 53
3 votes
1 answer
60 views

String together a set of tokens into a sequence

I have this problem scenario - Given a set of tokens, string them or a subset of the tokens together using stop words into a sequence. I am clear that I can have potentially infinite pre-training data ...
Deepak Saini's user avatar
3 votes
1 answer
45 views

Request: Confirmation on my understanding of overfitting and regularization concepts

Overfitted models tend to have largely different (some very high, some comparatively low) coefficients/weights for different feature values. So, this means the model (when drawn as graph) will have ...
Curious's user avatar
  • 151
3 votes
0 answers
59 views

Is it Scientifically Rigorous to create a multimodal ML Model with data from multiple sources

I'm attempting to create a multimodal machine learning model for disease diagnosis. However, I'm having quite a bit of difficulty finding public data sets with all the data I need. For example, I need ...
Pulkith Paruchuri's user avatar
3 votes
3 answers
129 views

Classification when the classification of the previous itens matter

I have a classification problem to solve, that seems to be common but I am struggling to find the name of this task and the best way to model this problem. Suppose I have a series of events that are ...
bratao's user avatar
  • 31
3 votes
0 answers
1k views

Forecast Model to Estimate Customer Service Call Volume and Appropriate Staff

I am working on a project to predict the proper staffing needed for a customer service team using historical data. I am new to machine learning, and I am not sure if my approach to this problem is the ...
Tony's user avatar
  • 31
3 votes
1 answer
1k views

Understanding Conv1D Output Shape

I am a little confused with the output shape that Conv1D produces. Consider the code I have used as the following (a lot has been omitted for clarity): ...
Wong Wai Kwun's user avatar
3 votes
2 answers
888 views

What can we learn from PCA on non linear data?

Suppose we have dataset with 10 features which are not linear: ...
Boom's user avatar
  • 297
3 votes
2 answers
1k views

How to handle undefined or null data in a neural network

Let me preface this post with I am incredibly new to machine learning/neural networks. I am currently working on a classification neural network using TensorFlow whose input is multiple features of ...
Tis's user avatar
  • 31
3 votes
0 answers
238 views

Struggling to understand/implement Transformer Decoder

I'm struggling to understand the decoder in a Transformer model, specifically with regards to some aspects of its architecture as well as how it actually handles the data during training. What I have ...
cuuupid's user avatar
  • 131
3 votes
0 answers
2k views

Hopfield Network python implementation, Network doesn't converge to one of the learned patterns

I'm trying to implement a Hopfield Network in python using the NumPy library. The network has 2500 nodes (50 height x 50 width). The network learns 10 patterns from images of size 50x50 stored in &...
Emoticon's user avatar

1
2 3 4 5
66