Questions tagged [machine-learning]

Methods and principles of building "computer systems that automatically improve with experience."

Filter by
Sorted by
Tagged with
1
vote
1answer
10 views

Thoughts on Feature Engineering of a duration_in_program Variable

So I am trying to predict which customers would leave a loyalty program sponsored by X firm, using an ML classification model. I further believe that the duration for which a customer has been in the ...
0
votes
1answer
195 views

Process mining with ML

I have a little more general question. My dataset consists of N sequences of events. Example of one sequence could be [A,B,C,D,X,Y] and another [A,B,Z], where letters represent different events. The ...
1
vote
0answers
14 views

Curve fitting through cloud of daily values using machine learning

I want to plot laboratory values (SCr_v) over time and find the best fitting regression curves (see plot 1 and 2). I don't want to restrict myself to a specific model if possible. Is there a function ...
0
votes
2answers
73 views

ML, Statistics and Mathematics

I have just started getting my hands wet in ML and every time I try delving deeper into the concepts/code, I face the challenges of the mathematics and its cryptic notations. Coming from a Computer ...
0
votes
0answers
11 views

problem using .fit() for an image generator

I am trying to use use the metod .fit() in order to fit a generator as it is done in the following link : https://keras.io/preprocessing/image/ , but if I try to do this for my code : ...
2
votes
1answer
98 views

Is color information only extracted in the first input layer of a convolutional neural network?

In a convolutional neural network (CNN), since the RGB values get multiplied in the first convolutional layer, does this mean that color is essentially only extracted in the very first layer? ...
1
vote
2answers
2k views

How to build an image dataset for CNN?

I don't understand how images are actually fed into a CNN? If I have a directory containing a few thousand images, what steps do I need to take in order to feed them to a neural network (for instance ...
1
vote
1answer
383 views

How to create a dummy model in Tensorflow

I am a newbie in Machine learning. I found this example using tflearn somewhere. It is the part of the program where we initialize a dummy model before training ...
1
vote
4answers
68 views
+50

scaling images for image classification

I am trying to do image classificaition with a dataset that contains images of different sizes. The images are in a folder called Train, which contains 4 subfolders callsed HAZE,RAINY,SNOWY and SUNNY. ...
0
votes
2answers
404 views

Should the cost function be zero using TensorFlow's sigmoid_cross_entropy_with_logits?

I'm building a CNN to make a binary classification (1 or zero). For this, I'm using the cost function sigmoid_cross_entropy_with_logits. But for some reason, the ...
1
vote
0answers
15 views

How to Manually Classify using SVM?

Consider points x1 = (1,1), x2=(1,0), x3=(1,-1) from class C1 and points x4 = (-1,1), x5=(-1-1) from class C2. Classify the given data with SVM How do we manually classify data by finding the ...
0
votes
2answers
232 views

Including identifier in machine learning model as feature vs separate model for every identifier

I am new to machine learning and i am building a model to predict number of customers for the model branch at specific hour/season/other feature. I know it will be bad idea to pit id(...
1
vote
2answers
19 views

Machine Learning Analysis for Redaction Purposes of Personally Identifying Information from Open Text Fields

Let's say that I wanted to use machine learning to find and redact personally identifying information (PII) from millions of records with open text fields. Let's also say the PII could include a ...
2
votes
0answers
7 views

is it a good idea to take the derivative or integral of some features and add them as new features in machine learning?

I'm learning how to do feature Engineering and come across some ideas in my head that's why I want to ask if I had some dataset with some features let's say 2 features and I have a timestamp column ...
1
vote
0answers
9 views

Maximum Entropy Policy Gradient Derivation

I am reading through the paper on Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review by Sergey Levine. The link to the paper is https://arxiv.org/pdf/1805.00909. I am ...
5
votes
0answers
54 views
+50

Time-series prediction: Model & data assumptions in AI/ML models vs conventional models

I was wondering if there was a good paper out there that informs about model and data assumptions in AI/ML approaches. For example, if you look at Time Series Modelling (Estimation or Prediction) ...
2
votes
2answers
17 views

Are stationarity and low autocorrelation the prerequisite of regression model?

As said in the title, are stationarity and low autocorrelation the prerequisite of general / linear regression model ? That is, if a time series is non-stationary or has large autocorrelation, would ...
10
votes
6answers
8k views

tool to label images for classification

Can anyone recommend a tool to quickly label several hundred images as an input for classification? I have ~500 microscopy images of cells. I want to assign categories such as 'healthy', 'dead', 'sick'...
0
votes
0answers
5 views

Tensorflow: how to interpret Tensorboard profile? How to increase efficiency according to it?

I use Tesla K80 for training a DNN model built with Tensorflow 2.0. I want to speed up the training. I have already used tf.data.Dataset input pipeline. I'm not using TFRecords files though. From my ...
0
votes
2answers
39 views

How do I correctly build model on given data to predict target parameter?

I have some dataset which contains different paramteres and data.head() looks like this Applied some preprocessing and performed Feature ranking - ...
1
vote
1answer
18 views

How to use weights from our data set properly? Should we use them at all?

The census data set I'm using: https://archive.ics.uci.edu/ml/datasets/Adult So, I'm currently using this census data to make observations and predict whether someone is married or not. However, when ...
0
votes
0answers
8 views

How to find identify important features that influence outcome?

I have a dataset of 5K records where I try to predict Yes or No class. Currently I have around 70 variables. I know we have random forest ...
0
votes
2answers
18 views

k-fold cross validation with RNNs

is it a good idea to use k-fold cross-validation in the recurrent neural network (RNN) to alleviate overfitting? A potential solution could be ...
0
votes
0answers
9 views

how to use user KPI score data for making recommendations based on improving the performance

I have a dataset with these data points: user_id login_points meeting_complete points meeting_missed_points call_points lead_created_points and some features which tells the user activity and ...
2
votes
1answer
203 views

Improve results using user input

I've developed a tool that retrieve the closest expressions from a database based on what the user typed. (using word embedding - a comparison is made between each expression from the database and the ...
0
votes
0answers
8 views

When clipping gradient is useful? [closed]

When clipping gradient is useful? It is useful for exploding gradient! But, when it is useful? for instance the weight=5 is it useful?When we are able to use clipping gradient?
1
vote
0answers
9 views

Using Keras/TS for Multivariate Time Series Prediction w/ Univariate output?

I've been reading through a few tutorials for using Keras/TensorFlow for multivariate time-series prediction (primarily using LSTM models). One example uses air pollution as an example. In this ...
1
vote
0answers
16 views

Hierarchical Clustering on transaction data

Problem Statement: Let's say I have buyer transactional data for every product, features are categorical and numeric. I want to cluster purchases that have similar attributes in terms of who's ...
5
votes
1answer
1k views

Netflow anomaly detection python packages

Is anyone aware of any open source / python packages for Netflow Anomaly detection ? I found some on github but anyone who has more experience with it. please advise.
0
votes
0answers
16 views

Why does a class weight fraction improve precision compared to under-sampling approach where precision drops?

I have an imbalanced data where the ratio between positive to negative samples is 1:3 (positive samples are 3 times higher than negative). For my case it is is important to have a higher precision (...
0
votes
0answers
23 views

Help required for medical imaging research | Deep Learning Project

I am a cs student currently working on Brain tumour segmentation using cascading of two U-Net research project. I have researched over the internet about the cascading of CNN but I found nothing about ...
3
votes
2answers
42 views
+50

Splitting a pdf containing batch of scanned documents

My question is primarily: is there any ML research paper about splitting a pdf containing a batch of scanned documents (eg bank statements) into individual documents? I have searched for this but I ...
0
votes
1answer
28 views

How to treat time based ticket prices for train/test split

I have a dataset of airfare price tickets that were scraped throughout a 6 month period where each observation represents a particular price for a specific flight on a specific date that it was ...
0
votes
1answer
21 views

How do I turn images dataset into a numpy array?

I have a directory for a dataset of images, I I want to transorm it to a numpy array in order to be able to fit an image generator to it. What I have tried to do is the following: ...
0
votes
0answers
9 views

Exponentiated Gradient

I am currently trying to understand exponentiated gradient from this paper. Here is an implementation of the Algorithm in Python. So my question using exponentiated-gradient algorithm we can update ...
1
vote
1answer
164 views

How do I perform Leave One Out Cross Validation For Top n Recommendation Sytems?

I am new in making recommendation systems . I am using the surpriselib library to evaluate my recommendations. All the Accuracy Metrics are well supported in this library. But I also want to compute ...
0
votes
0answers
16 views

problem with distribution of dataset

I am doing image classification with CNN and I have a training set and a test set with different distributions. To try to overcome this problem I am thinking about doing a standardization using ...
2
votes
1answer
42 views

Item-based recommender using K-NN

I'm trying to build an item-based recommender using k-nn. I have a list of items, all of which have some properties (features) in common. ...
4
votes
3answers
246 views

What is the best question generation state of art with nlp?

I was trying out various projects available for question generation on GitHub namely NQG,question-generation and a lot of others but I don't see good results form them either they have very bad ...
0
votes
1answer
17 views

performances evaluation of image classification with different distribution for train and test set

I am implementing a CNN to do image classification of 4 classes representing different weathers : Haze, Sunny, Rainy, Snowy. I have as training set 3200 images, and as test set 3038 images. The ...
0
votes
0answers
8 views

modified mean squared error with sign being taken into consideration

Suppose we have a regression model y_pred=f(X) with corresponding label y. Typical MSE is ...
1
vote
1answer
82 views

does KNN have a hypothesis space? if yes, Is there a way to clearly define the hypothesis space with the instances?

I am learning this post, "A few useful things to know about machine learning" The author says A classifier must be represented in some formal language that the computer can handle.Conversely, ...
1
vote
2answers
32 views

Simple question about prediction classes of item in question vs not item in question

Let's say I wanted to use transfer learning to train a model to detect object A vs everything else. In this case, do I provide 2 types of input, images of object A and images of everything else, and ...
2
votes
1answer
126 views

Dynamic clustering

I am performing anomaly detection on different datasets and thought to first cluster the dataset and submit each of the clusters to different AD models. I am using HDBSCAN, and in my test dataset I ...
7
votes
5answers
407 views

What are some of the best practices for sharing data and models with colleagues?

As a data scientist who recently joined a new team, I wanted to ask the community how they share data and models among their colleagues. Currently I have to resort to storing data in some central ...
0
votes
1answer
40 views

Non-Real Time Data Augmentation for CNN Classification. What are the drawbacks?

When people talk about and use data augmentation, are they mostly referring to real-time data augmentation? In the case of image classification, that would involve augmenting the data right before ...
0
votes
0answers
13 views

Tensorflow API: What does the metric `tf.keras.metrics.TopKCategoricalAccuracy` do?

According to the API doc, this metric "Computes how often targets are in the top K predictions." But how come the following codes prouce the result 1? 0.95>0.9>0.8>0.1>0.05, both 0.95 and 0.8 lead ...
1
vote
0answers
25 views

Warning during training a CNN

I was training a CNN when the following appeared: after this the training continue, but I don't understand why it happens. Should I do something or leave it? [EDIT] The problem is that it does this ...
0
votes
0answers
11 views

my cifar10 kaggle result around 10% is there something wrong with my pytorch code/?

I'm quite new to pytorch so I want check is there something wrong I got final submission code score around 10% here is my code ...
0
votes
1answer
29 views

Deep learning model gives random results

First I am new to machine learning if it is an obvious question, I am sorry. ...