Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [machine-learning]

Methods and principles of building "computer systems that automatically improve with experience."

1
vote
0answers
15 views

Grouping probabilities of full model vs fitting model on subset of variables

Suppose I want to predict some event probability with a set of features. Some of the features may be gender or some other categorical variable. Assume the probabilities are well calibrated. Now, say,...
1
vote
0answers
8 views

Model Performance using Precision as evaluation metric

I am dealing with an imbalanced class with the following distribution : (Total dataset size : 10763 X 20) 0 : 91% 1 : 9% To build model on this dataset having class imbalance, I have compared ...
0
votes
1answer
14 views

What kind of storage media should I use for large data storage [on hold]

In my hands there is a data of about 60k × 1k. This data is stored in the txt file. Because the data is large, I would like to use the hard disk capacity efficiently for what kind of data file or ...
0
votes
1answer
16 views

Supervised multiclass classification : is ANN a good idea ? or use other classifiers?

I have a problem deciding what to use since i'm just beginning to creating predictive models. Let's say I have a training dataset with 5 or 6 features and a testing dataset. (With around 50k rows in ...
3
votes
2answers
1k views

Can a neural network compute $y = x^2$?

In spirit of the famous Tensorflow Fizz Buzz joke and XOr problem I started to think, if it's possible to design a neural network that implements $y = x^2$ function? Given some representation of a ...
0
votes
0answers
11 views

Example of sharing of parameters among features and classes

This question ask for the meaning of 'sharing of parameters between features and classes'. The only answer to the question gives a nice explanation of how logistic regression does not share parameters ...
1
vote
0answers
19 views

Kappa Goes up as Accuracy Goes Down

I have recently been trying to train a randomForest model on a binary outcome with a very uneven class split. 282 control ~82% 63 case ~18% There are a total of 147 predictors that I'm testing for ...
2
votes
2answers
26 views

What activation function should I use for a specific regression problem?

Which is better for regression problems create a neural net with tanh/sigmoid and exp(like) activations or ReLU and linear? Standard is to use ReLU but it's brute force solution that requires certain ...
0
votes
0answers
10 views

Spatio-temporal Dataset - Machine Learning [on hold]

Which ML algorithm to use when dealing with spatio-temporal dataset. Also, can anyone direct me to a tutorial to learn how to perform such ML process? Regards.
1
vote
1answer
26 views

Unsupervised learning for anomaly detection

I've started working on an anomaly detection in Python. My dataset is a time series one. The data is being collected by some sensors which record and collect data on semiconductor making machines. ...
0
votes
0answers
12 views

Perceptron algorithm depends on the learning rate?

I am working on the problem above. I know that by rescaling I can make η irrelavant. But I do not know how exactly. Is it making new vector, namely w' = w/η ? Moreover, if I do this, would the final ...
3
votes
3answers
40 views

Scaling neural networks

While using Neural Networks (TensorFlow: Deep Neural Regressor), when increasing your training data size from a sample to the whole data (say a 10x larger dataset), what changes should you make to the ...
0
votes
0answers
21 views

NLP - How to detect the presence of a phrase and it's derivatives

I have a dataset with a free form text field as one of the variables. Essentially I want to determine if a record has the phrase "The cat is not present". However, this phrase could be written as "cat ...
0
votes
1answer
20 views

How to get probability of classification

I have binary classification, I tried several model KNN, SVM, decision tree and random forest. I have 50 000 samples, X_train has 50 000 rows and 2300 columns. ...
0
votes
0answers
7 views

GoogleNet Inception Module Output Dimensions

In google Net inception module, we are shown that the output spatial dimension does not change after application of 3x3, 5x5 convolution operations. Is there implicit zero padding on input that is ...
0
votes
0answers
77 views

Algorithm for SVD based recommendation engine

I am trying to build an SVD based recommendation engine for MovieLens database. After going through multiple online tutorials and resources I have understood how SVD works if a user-rating matrix is ...
0
votes
0answers
14 views

Rank the Documents from the cluster of documents [on hold]

We have around 100K documents and we'd like to rank the document on a scale from 1 to 10 (Preferably using continuous values like 1.9, 2.6, 3.4...). We don't have ...
3
votes
1answer
26 views

Newton method and Vanishing Gradient

I read the article on Vanishing Gradient problem, which states that the problem can be rectified by using ReLu based activation function. Now I am not able to understand that if using ReLu based ...
0
votes
0answers
15 views

How to re-train a model with new data in Marketing context

If I develop and then deploy a model for Marketing like customer churn and then take actions based on the results of this model, how can I retrain the model some months later? I think that the ...
0
votes
1answer
12 views

How to Build Mobile Application for Image Recognition?

I want to write an application on (Android) phone for image recognition. The (Keras) model itself is written and trained on a desktop machine and works satisfactorily with standard images. However, I ...
1
vote
0answers
24 views

Are there any public datasets about mental health of patients containing physiological and psychological symptoms?

I would like to segment mental illnesses with clustering using machine learning. To do so I need training dataset which contains physiological and psychological symptoms of an subject. The closest ...
0
votes
2answers
11 views

Can feedback loops occur in machine learning that cause the model to become less precise?

In discussions about ML algorithms, in for instance crime prediction, it is often claimed by non-experts that there are problems with feedback loops causing the model to become biased and give the ...
1
vote
1answer
30 views

Meaning of this notion in 0-1 loss?

I am reading a paper and encountered this notion: $$1_{\{Y=1\}}$$ To me it seems to be the expression as below, but I am not entirely sure and I don't think the author explictly explained it: <...
0
votes
1answer
18 views

Algorithm suggestion for anomaly detection in multivariate time series data

I have time series data containing user actions at certain time intervals eg ...
0
votes
1answer
15 views

Weighted samples in Tensorflow for convolutional neural networks

For my binary classification problem (A vs B), each image in either class has its individual weight. This means, for example, if I have 10000 images for A, not all of the images are equally important....
0
votes
1answer
24 views

How exactly do I extract the important features from strings for machine learning?

Forgive me for my ignorance. Linked below is an image of my dataset with 1000 tuples. https://i.stack.imgur.com/WHIlx.png I have the following questions (1) How exactly do I go about extracting ...
2
votes
1answer
42 views

In which cases shouldn't we drop the first level of categorical variables?

Beginner in machine learning, I'm looking into the one-hot encoding concept. Unlike in statistics when you always want to drop the first level to have k-1 dummies (...
1
vote
0answers
12 views

classification of groups of images

I have been looking through research and cannot find too much on this topic. I believe that I might be searching for the wrong terms. What I am trying to find is any literature on classifying a ...
1
vote
1answer
24 views

How feed a numpy array in batches in Keras

I have the data in the following format: 1: DATA NUMPY ARRAY (trainX) A numpy array of a set of numpy array of 3d np arrays. To be more articulate the format is: [[3d data], [3d data], [3d data], [...
0
votes
1answer
12 views

Can video descriptions be learned from videos + human descriptions? [on hold]

So I have a dataset of short videos with simple content and humans describing them, for example ("ball is pushed"; "cube is grabbed"; "book is grabbed"; etc.). What strategies are there in order to ...
1
vote
2answers
25 views

How exactly do I go about extracting features from timestamps for machine learning? [on hold]

My dataset has a timestamp column with the following format: 06/24/18 0:56 How exactly do I convert this information into features that can be used for classification algorithms like logistic ...
3
votes
2answers
66 views

Manual feature engineering based on the output

So, I'm working on a ML model that would have as potential predictors : age , a code for his city , his social status ( married / single and so on ) , number of his children and the output signed ...
0
votes
1answer
27 views

Machine Learning methods suited for CPU

I have a large number of x86_64 cores available to me, but no GPUs or TPUs. Which Machine Learning techniques are suited for execution on a CPU? I would imagine more "statistical learning" techniques ...
0
votes
0answers
17 views

Python Library for Neural Networks (no Tensors)

I’m having an absolute nightmare with Keras and TF and I think by now it’s time to attempt a different approach with a different library (plan C is to build the network from scratch) My neural ...
1
vote
0answers
35 views

Problem about tuning hyper-parametres

I have tried GridSearchCV and BayesSearchCV for tuning my LightGBM algorithm (for binary classification). I have used 10 iterations and I have indicated scoring ="roc_auc" In the first iteration, I ...
0
votes
0answers
12 views

Using a discriminator to distinguish ground truth and predicted boxes for FRCNN

We have implemented an object detection framework in Keras based on the Faster R-CNN model. Currently, we would like to find a way to automatically classify images on which the model is performing ...
0
votes
0answers
14 views

Best approach to hierarchical classification? [on hold]

What would be the best approach in doing a hierarchical classification? I thought about doing a flat classification with just leaf nodes and not considering the hierarchy but that doesn't sound like a ...
2
votes
1answer
43 views

Newton's method optimization for Deep Learning

I'm reading this paper "Deep learning via Hessian-free optimization" by J. Martens, I am having difficulty figure out the following statement: In the standard Newton's method, $q_{\theta}(p)$ is ...
0
votes
0answers
7 views

Extracting disaggregated time series from an aggregate time series

I've got $N = 5000$ individual time series representing hourly electricity demand from $N$ households. I also know whether each house has electric heating or not. Assume there are $N^1$ houses with ...
1
vote
1answer
24 views

How to deal with memory insufficient read by pandas in python

I use pandas.read_csv to read a huge file for machine learning, but I got memory error. Someone recommend me to set arg ...
0
votes
0answers
13 views

Identify credit card shape using machine learning mobile

I have asked a previous question related to identifying and Isolating credit card from background here I was able to get a good model to work on Desktop/Laptop systems using semantic segmentation. ...
0
votes
1answer
15 views

How to categorize machine learning problems, techniques, solutions and algorithms in one super flowchart, consisting of all possible categories? [on hold]

I am trying to create a flowchart that categorizes everything under the sun that exists in machine learning. From the different techniques to the evaluation metrics to miscellaneous techniques used to ...
0
votes
0answers
7 views

Model and Type for Medication Impact

I have following 3 datasets from various patients: Diagnosis Dataset: Patient_Key, Diagnosis Code,Diagnosis Date Medication Dataset: Patient_Key, Medication Name, Dose, Fill Date Lab Result Dataset:...
0
votes
0answers
10 views

Learning with missing features (MNAR)

I want to learn from features that may have some missing informations. The value of the variable that's missing is related to the reason it's missing (MNAR) To better understand my case, here is an ...
2
votes
2answers
60 views

How important is the input data for a ML model?

Last 4-6 weeks, I have been learning and working for the first time on ML. Reading blogs, articles, documentations, etc. and practising. Have asked lot of questions here on Stack Overflow as well. ...
2
votes
1answer
32 views

Detect sensitive data from unstructured text documents

I know this question is broad, but I need an advice to know if it's possible to achieve what I want to do. The problem is that I have around 2500 documents with sensitive data being replaced by four ...
2
votes
2answers
45 views

Scikit learn train test split without mixing participants in trails

I have dataset of trails. There are 90 participants that appear in 90 trails each. So 450 trails in total. I'm looking to split my dataset so participants doesn't get mixed over training and test set....
0
votes
0answers
10 views

Preparing data for training, where to find suitable text

I've been tasked with producing training data for a supervised ML system using Tensorflow. However, I'm having a hard time generating the training data. The system's ultimate goal, is to be able to ...
0
votes
1answer
18 views

Collinearity and Multicollinearity in the features?

What are some advanced or basic methods most used by data scientists/ML Engineers to detect collinearity (or) multicollinearity between features?