Questions tagged [machine-learning]

Machine Learning is a subfield of computer science that draws on elements from algorithmic analysis, computational statistics, mathematics, optimization, etc. It is mainly concerned with the use of data to construct models that have high predictive/forecasting ability. Topics include modeling building, applications, theory, etc.

3,325 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
8 votes
0 answers
2k views

AdaBoost implementation and tuning for high dimensional feature space in R

I am trying to implement the AdaBoost.M1 algorithm (trees as base-learners) to a data set with a large feature space (~ 20.000 features) and ~ 100 samples in R. ...
user avatar
  • 81
7 votes
0 answers
95 views

Training value neural network AlphaGo style

I have been trying to replicate the results obtained by AlphaGo following their supervise learning protocol. The papers specify that they use a network that has two heads: a value head that predicts ...
user avatar
  • 91
7 votes
0 answers
165 views

How to predict advantage value in deep reinforcement learning

I'm currently working on a collection of reinforcement algorithms: https://github.com/lhk/rl_gym For deep q-learning, you need to calculate the q-values that should be predicted by your network. There ...
user avatar
  • 171
6 votes
1 answer
274 views

Training stateful LSTM with different number of sequences

I'm using a stateful LSTM for stock market analysis, and I have varying amounts of data for each stock, ranging from 20 years to just a few weeks (i.e. for newly listed stocks). I use 3 years of data ...
user avatar
  • 730
6 votes
0 answers
501 views

differences between LSQR and FTRL when working with very sparse data

I have a 2M instances dataset with millions of very very sparse dummy variables created using the hashing trick = ...
user avatar
  • 1,287
5 votes
3 answers
792 views

Why is 10000 used as the denominator in Positional Encodings in the Transformer Model?

I was working through the All you need is Attention paper, and while the motivation of positional encodings makes sense and the other stackexchange answers filled me in on the motivations of the ...
user avatar
5 votes
1 answer
717 views

How to use multiple text features for NLP classifier?

I am trying to build text classifier, Usually, we have one text column and ground truth. But I am working on a problem where dataset contains many text features. I am exploring different ways how to ...
user avatar
5 votes
0 answers
158 views

What does big O mean in KNN optimal weights?

Wiki gives this definition of KNN In pattern recognition, the k-nearest neighbors algorithm (k-NN) is a non-parametric method used for classification and regression. In both cases, the input consists ...
user avatar
  • 554
5 votes
2 answers
125 views

What are the reasons for drawing initial neural network weights from the Gaussian distribution?

Are there theoretical or empirical reasons for drawing initial weights of a multilayer perceptron from a Gaussian rather than from, say, a Cauchy distribution?
user avatar
5 votes
2 answers
215 views

Evaluation of regression models with different evaluations (MSE, variance, VAF etc.)

When comparing several regression models in terms of quality, it seems like most have agreed on the MSE. There are also papers comparing "variance" and "variance accounted for (VAF)&...
user avatar
  • 183
5 votes
1 answer
141 views

Predicting change of shapes/coordinates

I'm trying to find a way to predict/calculate how a shape (e.g. outline of a glacier) will change in the future—based on its history (previous shape) and additional factors (e.g. Δtemperature). In my ...
user avatar
5 votes
0 answers
243 views

Confidence value in AdaBoost?

I read this introduction about AdaBoost (http://www.cs.man.ac.uk/~nikolaon/~nikolaon_files/Introduction_to_AdaBoost.pdf), and am curious why confidence for each model is defined as $$\alpha_j=\frac{...
user avatar
5 votes
0 answers
81 views

Classify driver based on time-series sensor data

I want to build a model that can detect which driver is driving now the car based on a dataset that contains 20 driver records for 3600s each driver ( the dataset contains all the car sensors values ...
user avatar
5 votes
0 answers
280 views

how to propagate error from convolutional layer to previous layer?

I've been trying to implement a simple convolutional neural network. But I've been stuck at this problem for over a week. To be specific, assume there are 3 layers in a convolutional pass, marked as ...
user avatar
  • 151
4 votes
1 answer
333 views

Linear Regression bad results after log transformation

I have a dataset that has the following columns: The variable I'm trying to predict is "rent". My dataset looks a lot similar to what happens in this notebook. I tried to normalize the rent ...
user avatar
  • 127
4 votes
1 answer
843 views

Self-attention mechanism did not improve the LSTM classification model

I am doing an 8-class classification using time series data. It appears that the implementation of the self-attention mechanism has no effect on the model so I think my implementations have some ...
user avatar
  • 115
4 votes
1 answer
80 views

What ML architecture fits fixed length signal regression?

My problem is of regression type - How to estimate a fish weight using a fixed-length signal (80 data points) of the change in resistance when the fish swim through a gate with electrodes (basically 4 ...
user avatar
  • 41
4 votes
3 answers
2k views

Bert-Transformer : Why Bert transformer uses [CLS] token for classification instead of average over all tokens?

I am doing experiments on bert architecture and found out that most of the fine-tuning task takes the final hidden layer as text representation and later they pass it to other models for the further ...
user avatar
4 votes
1 answer
2k views

Weighted loss functions vs weighted sampling?

For image classification tasks, is there a practical difference between using weighted loss functions vs. using weighted sampling? (I would appreciate theoretical arguments, experience or published ...
user avatar
4 votes
0 answers
1k views

Dealing with categorical variables in Isolation Forest

Isolation Forest is widely used when dealing with outlier/anomaly detection when we have no labels. The theory behind is that making random split at random points and counting how many splits you do ...
user avatar
4 votes
1 answer
276 views

How to incorporate new features in an existing machine learning model?

Suppose we have trained a regression model $M$ on a fixed set of $n$ features, $F_1,F_2,…,F_n$ on a particular dataset $G$. Now assume that after model training, additional features ($F_{n+1},…$) ...
user avatar
  • 141
4 votes
1 answer
77 views

Training Machine Learning Model - Neural Network - Islands Problem

I was working on the following leetcode problem: Given a 2d grid map of '1's (land) and '0's (water), count the number of islands. An island is surrounded by water and is formed by connecting ...
user avatar
4 votes
3 answers
552 views

Is it a best practice to exclude retweets from the data set?

I am going to build machine learning algorithm to identify fake tweets. The data set has huge retweets which I think might be an issue. Do you think given that the focus is the original tweet, it is ...
user avatar
4 votes
4 answers
270 views

Backpropagation: Relevance of the error signal of a neuron

During my quest to understand back propagation in a more rigorous approach I have come across with the definition of error signal of a neuron which is defined as follows for the $j^{\text{th}}$ neuron ...
user avatar
  • 41
4 votes
2 answers
6k views

Saving and loading keras.callbacks.History object with np.save and np.load

I have been saving my training history in keras as follows: ...
user avatar
4 votes
3 answers
533 views

Where does the "deep learning needs big data" rule come from

When reading about deep learning I often come across the rule that deep learning is only effective when you have large amounts of data at your disposal. These statements are generally accompanied by a ...
user avatar
  • 41
4 votes
1 answer
961 views

Calculating saliency maps for text classification

I'm following the text classification with movie reviews TensorFlow tutorial, and wanted to extend the project by looking, for a certain input, which words influenced the classification the most. I ...
user avatar
4 votes
2 answers
125 views

What is the most appropriate machine learning approach for this scenario?

The scenario is pretty simple, and I'm sure it's been done a million times. The problem is I don't know the terminology to find the correct resources on the web. Scenario: I have an environment that ...
user avatar
4 votes
2 answers
1k views

How can I use two different datasets as a training model for svm

I know that you're supposed to scale your test data using the parameters (mean and stdev) from your training data. This is relatively simple; but what if the number of samples is limited in one ...
user avatar
  • 51
4 votes
2 answers
1k views

how does XGBoost's exact greedy split finding algorithm determine candidate split values for different feature types?

Based on the paper by Chen & Guestrin (2016) "XGBoost: A Scalable Tree Boosting System", XGBoost's "exact split finding algorithm enumerates over all the possible splits on all the features to ...
user avatar
  • 51
4 votes
1 answer
100 views

EM-ELM Cross validation

I know that cross validation is used to find the best hyperparameters that minimize the average error. For example, the number of neurons that minimize the average error of cross-validation is ...
user avatar
  • 41
4 votes
1 answer
58 views

Hochreiter LSTM (p. 4): Maximal values of logistic sigmoid derivative times weight

My questions follow the below page 4 excerpt from Hochreiter's LSTM paper: If $f_{l_{m}}$ is the logistic sigmoid function, then the maximal value of $f^\prime_{l_{m}}$ is 0.25. If $y^{l_{m-1}}$ ...
user avatar
4 votes
1 answer
359 views

Multiple activation functions with TensorFlow estimator DNNClassifier

I just want to know if is it possible to use tf.estimator.DNNClassifier with multiple different activation functions. I mean, could I use a DNNClassifier estimator which use different activation ...
user avatar
4 votes
2 answers
245 views

Support Vector Regression trained with data sets

I am now searching for a long time on the internet and on papers for an answers of simple questions. Am I able to train a Support Vector Regression algorithm with different data sets? If yes, how is ...
user avatar
  • 41
4 votes
2 answers
3k views

Fix first two levels of decision tree?

I am trying to build a regression tree with 70 attributes where the business team wants to fix the first two levels namely country and product type. To achieve this, I have two proposals: Build a ...
user avatar
  • 41
4 votes
1 answer
460 views

Illustrating the dimensionality reduction done by a classification or regression model

Tl;DR: You can predict something, but how do you explain the prediction? Your usual classification/regression setup Lets say the data is a classic regression/classification problem: several numerical ...
user avatar
4 votes
1 answer
240 views

How to automatically verify official documents?

I am new to machine learning and data science. I apologise if the question seems very basic. I have a requirement where I need to verify information submitted via a form with the corresponding ...
user avatar
3 votes
1 answer
38 views

String together a set of tokens into a sequence

I have this problem scenario - Given a set of tokens, string them or a subset of the tokens together using stop words into a sequence. I am clear that I can have potentially infinite pre-training data ...
user avatar
3 votes
1 answer
39 views

Request: Confirmation on my understanding of overfitting and regularization concepts

Overfitted models tend to have largely different (some very high, some comparatively low) coefficients/weights for different feature values. So, this means the model (when drawn as graph) will have ...
user avatar
3 votes
3 answers
62 views

Example for Boosting

Can someone exactly tell me how does boosting as implemented by LightGBM or XGBoost work in real case scenerio. Like I know it splits tree leaf wise instead of level wise, which will contribute to ...
user avatar
  • 183
3 votes
2 answers
126 views

Analysis of probability distribution of each features and Machine Learning

While I know that probability distributions are for hypothesis testing, confidence level constructions, etc. They definitely have many roles in statistical analysis. However, it is not obvious to me ...
user avatar
  • 359
3 votes
0 answers
38 views

Is it Scientifically Rigorous to create a multimodal ML Model with data from multiple sources

I'm attempting to create a multimodal machine learning model for disease diagnosis. However, I'm having quite a bit of difficulty finding public data sets with all the data I need. For example, I need ...
user avatar
3 votes
3 answers
111 views

Classification when the classification of the previous itens matter

I have a classification problem to solve, that seems to be common but I am struggling to find the name of this task and the best way to model this problem. Suppose I have a series of events that are ...
user avatar
  • 31
3 votes
0 answers
418 views

Forecast Model to Estimate Customer Service Call Volume and Appropriate Staff

I am working on a project to predict the proper staffing needed for a customer service team using historical data. I am new to machine learning, and I am not sure if my approach to this problem is the ...
user avatar
  • 31
3 votes
1 answer
458 views

Understanding Conv1D Output Shape

I am a little confused with the output shape that Conv1D produces. Consider the code I have used as the following (a lot has been omitted for clarity): ...
user avatar
3 votes
0 answers
119 views

Struggling to understand/implement Transformer Decoder

I'm struggling to understand the decoder in a Transformer model, specifically with regards to some aspects of its architecture as well as how it actually handles the data during training. What I have ...
user avatar
  • 131
3 votes
1 answer
114 views

How do I know that model performance improvement is significant?

Say I am running a Machine Learning model that produces a certain result (say accuracy of 80%). I now change a minor detail in my model (say, in a Deep Learning model, increase the kernel size in one ...
user avatar
  • 131
3 votes
1 answer
144 views

Using softmax for multilabel classification (as per Facebook paper)

I came across this paper by some Facebook researchers where they found that using a softmax and CE loss function during training led to improved results over sigmoid + BCE. They do this by changing ...
user avatar
3 votes
0 answers
147 views

Unbiasedness of random forests

Suppose that I am trying to build a random forest by subsampling the data and choosing a single feature per tree randomly. For example, suppose there is some dataset, $D = \{(x_{1},y_{1}), ......(x_{N}...
user avatar
  • 131
3 votes
3 answers
70 views

Understanding Weighted learning in Ensemble Classifiers

I'm currently studying Boosting techniques in Machine Learning and I happened to understand that in Algorithms like Adaboost, each of the training samples is given a weight depending on whether it was ...
user avatar

1
2 3 4 5
67