Questions tagged [machine-learning]
Methods and principles of building "computer systems that automatically improve with experience."
6,692
questions
1
vote
1answer
10 views
Thoughts on Feature Engineering of a duration_in_program Variable
So I am trying to predict which customers would leave a loyalty program sponsored by X firm, using an ML classification model.
I further believe that the duration for which a customer has been in the ...
1
vote
0answers
14 views
Curve fitting through cloud of daily values using machine learning
I want to plot laboratory values (SCr_v) over time and find the best fitting regression curves (see plot 1 and 2). I don't want to restrict myself to a specific model if possible. Is there a function ...
0
votes
0answers
11 views
problem using .fit() for an image generator
I am trying to use use the metod .fit() in order to fit a generator as it is done in the following link : https://keras.io/preprocessing/image/ , but if I try to do this for my code :
...
2
votes
0answers
7 views
is it a good idea to take the derivative or integral of some features and add them as new features in machine learning?
I'm learning how to do feature Engineering and come across some ideas in my head that's why I want to ask if I had some dataset with some features let's say 2 features and I have a timestamp column ...
1
vote
0answers
9 views
Maximum Entropy Policy Gradient Derivation
I am reading through the paper on Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review by Sergey Levine. The link to the paper is https://arxiv.org/pdf/1805.00909. I am ...
2
votes
2answers
17 views
Are stationarity and low autocorrelation the prerequisite of regression model?
As said in the title, are stationarity and low autocorrelation the prerequisite of general / linear regression model ? That is, if a time series is non-stationary or has large autocorrelation, would ...
0
votes
0answers
5 views
Tensorflow: how to interpret Tensorboard profile? How to increase efficiency according to it?
I use Tesla K80 for training a DNN model built with Tensorflow 2.0. I want to speed up the training. I have already used tf.data.Dataset input pipeline. I'm not using TFRecords files though. From my ...
0
votes
0answers
8 views
How to find identify important features that influence outcome?
I have a dataset of 5K records where I try to predict Yes or No class.
Currently I have around 70 variables.
I know we have random forest ...
0
votes
2answers
18 views
k-fold cross validation with RNNs
is it a good idea to use k-fold cross-validation in the recurrent neural network (RNN) to alleviate overfitting?
A potential solution could be ...
0
votes
0answers
8 views
When clipping gradient is useful? [closed]
When clipping gradient is useful? It is useful for exploding gradient! But, when
it is useful? for instance the weight=5 is it useful?When we are able to use clipping gradient?
1
vote
0answers
9 views
Using Keras/TS for Multivariate Time Series Prediction w/ Univariate output?
I've been reading through a few tutorials for using Keras/TensorFlow for multivariate time-series prediction (primarily using LSTM models).
One example uses air pollution as an example. In this ...
1
vote
0answers
16 views
Hierarchical Clustering on transaction data
Problem Statement: Let's say I have buyer transactional data for every product, features are categorical and numeric. I want to cluster purchases that have similar attributes in terms of who's ...
1
vote
2answers
19 views
Machine Learning Analysis for Redaction Purposes of Personally Identifying Information from Open Text Fields
Let's say that I wanted to use machine learning to find and redact personally identifying information (PII) from millions of records with open text fields.
Let's also say the PII could include a ...
0
votes
1answer
21 views
How do I turn images dataset into a numpy array?
I have a directory for a dataset of images, I I want to transorm it to a numpy array in order to be able to fit an image generator to it. What I have tried to do is the following:
...
0
votes
0answers
16 views
Why does a class weight fraction improve precision compared to under-sampling approach where precision drops?
I have an imbalanced data where the ratio between positive to negative samples is 1:3 (positive samples are 3 times higher than negative). For my case it is is important to have a higher precision (...
0
votes
0answers
23 views
Help required for medical imaging research | Deep Learning Project
I am a cs student currently working on Brain tumour segmentation using cascading of two U-Net research project. I have researched over the internet about the cascading of CNN but I found nothing about ...
0
votes
0answers
8 views
modified mean squared error with sign being taken into consideration
Suppose we have a regression model y_pred=f(X) with corresponding label y. Typical MSE is ...
0
votes
0answers
16 views
problem with distribution of dataset
I am doing image classification with CNN and I have a training set and a test set with different distributions. To try to overcome this problem I am thinking about doing a standardization using ...
0
votes
0answers
9 views
Exponentiated Gradient
I am currently trying to understand exponentiated gradient from this paper.
Here is an implementation of the Algorithm in Python.
So my question using exponentiated-gradient algorithm we can update ...
0
votes
1answer
17 views
performances evaluation of image classification with different distribution for train and test set
I am implementing a CNN to do image classification of 4 classes representing different weathers : Haze, Sunny, Rainy, Snowy. I have as training set 3200 images, and as test set 3038 images.
The ...
0
votes
0answers
13 views
Tensorflow API: What does the metric `tf.keras.metrics.TopKCategoricalAccuracy` do?
According to the API doc, this metric
"Computes how often targets are in the top K predictions."
But how come the following codes prouce the result 1?
0.95>0.9>0.8>0.1>0.05, both 0.95 and 0.8 lead ...
0
votes
1answer
28 views
How to treat time based ticket prices for train/test split
I have a dataset of airfare price tickets that were scraped throughout a 6 month period where each observation represents a particular price for a specific flight on a specific date that it was ...
0
votes
0answers
11 views
my cifar10 kaggle result around 10% is there something wrong with my pytorch code/?
I'm quite new to pytorch so I want check is there something wrong
I got final submission code score around 10%
here is my code
...
1
vote
1answer
18 views
How to use weights from our data set properly? Should we use them at all?
The census data set I'm using: https://archive.ics.uci.edu/ml/datasets/Adult
So, I'm currently using this census data to make observations and predict whether someone is married or not. However, when ...
1
vote
0answers
6 views
Derivation of Bayes classifier in Murphy's book
I am reading Kevin Murphy's Machine Learning book (MLAPP, 1st printing) and want to know how he got the expression for the Bayes classifier using minimization of the posterior expected loss.
He wrote ...
0
votes
1answer
29 views
Deep learning model gives random results
First I am new to machine learning if it is an obvious question, I am sorry.
...
0
votes
0answers
15 views
0
votes
0answers
32 views
An interesting task on machine learning [migrated]
There are 5 programs. Each program is a binary classifier, which classifies letters - "Spam" and "Not spam." All classifiers are independent and have an accuracy of 0.6.
Suppose that there is a new ...
0
votes
0answers
11 views
Structural Risk Minimization: Inequality in “Foundations of Machine Learning”
I am reading the chapter about structural risk minimization in Mohri's book.
However, I have trouble understanding the derivation of inequality 4.5 on page 83. More precisely, why is
$$\sum_k \...
1
vote
1answer
23 views
Where are WEKA installed packages stored
I would like to use the WEKA library in a Java program but I can't seem to find the methods I installed using WEKA's package manager.
Does anyone know where the installed methods are stored?
For ...
1
vote
3answers
32 views
group the similar words
array(['Ruby on Rails', 'Ruby', 'AWS DynamoDB', 'Python', 'MySQL',
'Swift', 'Android', 'iOS', 'JavaScript', 'React Native', 'ReactJS',
'TypeScript', 'Vue.js', 'Webpack', 'Amazon Web ...
2
votes
2answers
21 views
What is my training score the mean_train_score or mean_test_score?
I am using sklearn to train some models (random forest, decision tree). For the training I am using RandomsearchCV with Stratified k-fold as cross-validation. Then I make a predictions on the test set ...
1
vote
0answers
18 views
Where can I find datasets suitable for an undergraduate research project? [closed]
I have tried Kaggle but couldn’t find anything suitable. Basically I want to do a machine learning or deep learning project. Ideally it would have something to do with computer forensics or security. ...
1
vote
1answer
20 views
Problem with overfitting for a CNN
I am doing image classification with a CNN and I am having trouble building a network that does not do overfitting. I have in my training set 2000 images of 4 classes, while in my test set I have 3038 ...
1
vote
1answer
10 views
Error while using predict [closed]
After splitting into test and train the glm function is used on train set. For example
m1 = glm(target ~ ., data = train, family="binomial")
Then
...
1
vote
1answer
27 views
Creating a loss function with two minima?
I'm using keras to build a model. The model has one output, and I want a loss similar to mse loss, but there are two values to predict, and I'm fine if the model predicts either one of them, so I want ...
1
vote
0answers
15 views
How to Manually Classify using SVM?
Consider points x1 = (1,1), x2=(1,0), x3=(1,-1) from class C1 and points x4 = (-1,1), x5=(-1-1) from class C2. Classify
the given data with SVM
How do we manually classify data by finding the ...
0
votes
0answers
15 views
Chess position evaluation with Machine Learning
I have been working on a neural network which can be used as an evaluation function of a chess board (returning how good this position is for the white player). The input is the chess board and the ...
1
vote
0answers
25 views
Warning during training a CNN
I was training a CNN when the following appeared:
after this the training continue, but I don't understand why it happens. Should I do something or leave it?
[EDIT] The problem is that it does this ...
0
votes
1answer
7 views
Understanding AC_errorRate loss function
I'm reading an article about Rolling Window Regression: a Simple Approach for Time Series Next value Predictions.
He explains about 5 different loss functions. I managed to understand the first four, ...
1
vote
1answer
21 views
Can a recommendation system be used as a binary classifier?
I have a computer-generated music project, and I'd like to classify short passages of music as "good" or "bad" via machine learning.
I won't have a large training set. I'll start by generating 500 ...
1
vote
0answers
13 views
What is the shape of Decision Boundary for binary classifiers?
There are different types of Binary Classifiers each will produce a different type of decision boundary some produce decision boundary as straight-line others as circle or square, some of the ...
0
votes
1answer
15 views
How to interpret Variance Inflation Factor (VIF) results?
From various books and blog posts, I understood that the Variance Inflation Factor (VIF) is used to calculate collinearity. They say that VIF till 10 is good. But I have a question.
As we can see in ...
0
votes
1answer
32 views
Where do I start? I have a massive dataset (from web scraping) and want to predict y from 20 variables
I'm trying to do my best to recreate an algorithm that a company is using by sucking in all of its outputs and adding in some relevant variables (I won't be able to perfectly recreate it as I'm ...
0
votes
0answers
7 views
Plotting Graph by Implementing Python [closed]
I have a question about The Polynominal Regression and python. I was wondering how to plot this data given below into graph of Polynominal Regression by using the python.
How to do that ? I am facing ...
0
votes
1answer
15 views
How do I plot the predictions made by a LinearRegression model?
I have made a linear regression using sklearn. I am wondering if there is a convenient way where I can plot the prediction versus a specified variable?
Alternatively, is there any other good way of ...
1
vote
0answers
16 views
How to create an roc plot and calculate AUC for an svm (that does not return probabilities)?
I have some SVM classifier outputting final classifications for every sample in the test set, something like
1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1
and so on.
The "...
0
votes
1answer
11 views
How to reshape ndarray to calculate the mean squared error?
y_train.shape = (7654,1)
y_test.shape = (1914,1)
mae = mean_absolute_error(y_train, y_test) gives:
"ValueError: Found input variables with inconsistent numbers ...
0
votes
2answers
22 views
How to fit integer output in neural network
I have a data set that contains 104 input features and 96 output values to be predicted, the input features are floating values normalized to [-1,1], and each out put value would be one integer ...
0
votes
0answers
7 views
How to use machine learning to database design [closed]
How can we use machine learning to design database?
What are steps to proceed?
