Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [machine-learning]

Methods and principles of building "computer systems that automatically improve with experience."

5
votes
4answers
360 views

Couple PCA plot and clusters to labels

I am trying my first 'project' concerning machine learning and I am a bit stuck. However, I am not sure if it's even possible but here goes my question. What I want to achieve is clustering user ...
2
votes
1answer
179 views

Generic strategy for object detection

I have a huge collection of objects from which only a tiny fraction are in a class of interest. The collection is initially unlabelled, but labels can be added using an expensive operation (for ...
4
votes
2answers
88 views

Dataset to give same eigenvectors?

Any suggestion on what kind of dataset lets say $n \times d$ ($n$ rows, $d$ columns) would give me same eigenvectors?. I believe it should be the one with same absolute values in each cell. Like ...
6
votes
1answer
230 views

How many possible hypotheses

The training set contains $p$ papers. Each paper is annotated as research or non-research. To develop the research paper filter, we consider the $W$ most frequent phrases in a paper. The research ...
5
votes
1answer
529 views

How to detect overfitting of a stock screener

The project I am working on allows users to create Stock Screeners based on both technical and fundamental criteria. Stock Screeners are then "backtested" by simulating the results of applying in ...
3
votes
3answers
1k views

Predicting Soccer: guessing which matches a model will predict correcly

I took on a project to predict the outcome of soccer matches but it turned out to be a very challenging task. I tried out different models but I only got 50-54% accuracy on my test dataset. Some of ...
1
vote
2answers
405 views

Does reinforcement learning always work on grid world?

Does reinforcement learning always need a grid world problem to be applied to? Can anyone give me any other example of how reinforcement learning can be applied to something which does not have a ...
1
vote
1answer
434 views

Combining data sets without using row.name

I start with a data.frame (or a data_frame) containing my dependent Y variable for analysis, my independent X variables, and some "Z" variables -- extra columns that I don't need for my modeling ...
2
votes
3answers
849 views

training one SVM model for predicting more than one response variable

Please I want to know if there is any SVM R package that can handle more than one response variable (y) at a time. that is to train one model for predicting more than one response variable. it could ...
0
votes
1answer
442 views

How to test/validate unlabeled data in association rules in R?

I produced association rules by using the arules package (apriori). I'm left with +/- 250 rules. I would like to test/validate the rules that I have, like answering the question: How do I know that ...
47
votes
6answers
64k views

strings as features in decision tree/random forest

I am new to machine learning! Right now I am doing some problems on an application of decision tree/random forest. I am trying to fit a problem which has numbers as well as strings (such as country ...
1
vote
1answer
82 views

Splitting binary classification into smaller susbsets

As an example. If you are tying to classify humans from dogs. Is it possible to approach this problem by classifying different kinds of animals (birds, fish, reptiles, mammals, ...) or even smaller ...
1
vote
1answer
259 views

Matrix factorization for like/dislike/unknown data

Most literature focus on either explicit rating data or implicit (like/unknown) data. Are there any good publications to handle like/dislike/unknown data? That is, in the data matrix there are three ...
6
votes
2answers
12k views

Python or R for implementing machine learning algorithms for fraud detection [closed]

I was wondering which language can I use: R or Python, for my internship in fraud detection in an online banking system: I have to build machine learning algorithms (NN, etc.) that predict transaction ...
1
vote
0answers
81 views

perceptron implementation, preocedure for post-feature vecor: bag-of-words model

I have code to generate feature vectors in the style of the bag-of-words model, you can see it on my github page. It renders output of the form: ...
1
vote
2answers
3k views

How to appropriately set weights for weighted KNN

I'm currently trying to predict a continuous variable using KNN. Instead of treating each neighbor equally I would like to use the weights to create a weighted average. The weights by themselves are ...
1
vote
1answer
531 views

training neural net with multiple sets of time-series data

I have the following data ($x^1_i$, $y^1_i$) for $i=1,2,...N_1$ ($x^2_i$, $y^2_i$) for $i=1,2,...N_2$ ... ($x^m_i$, $y^m_i$) for $i=1,2,...N_m$ Is it possible to train a neural net to produce ...
2
votes
1answer
390 views

Definition of likelihood function

I am confused by the definition of the likelihood function in different contexts. In the case of linear and logistic regression, it is defined as y given x In the case naive bayes and LDA, it is ...
1
vote
2answers
490 views

How to start the process of coming up with the predicted math score?

I am working on a problem(non competition) from hacker rank https://www.hackerrank.com/challenges/predict-missing-grade Basically you're given test data of a bunch of students of their scores in ...
1
vote
1answer
1k views

Kaggle Titanic Survival Table an example of Naive Bayes?

is the survival table classification method on the Kaggle Titanic dataset an example of an implementation of Naive Bayes ? I am asking because I am reading up on Naive Bayes and the basic idea is as ...
1
vote
0answers
512 views

How do I use Hierarchical Dirichlet Process (HDP) implementations (hdpfaster by C.Wang or hca by W.Buntine at mloss.org) to discover number of topics?

Context of question: I want to find semantically similar documents in corpora. For that, I'm first trying Latent Dirichlet Allocation (LDA) with divergences (Hellinger, Kullback-Leibler, Jensen-...
1
vote
1answer
1k views

Dividing percentage

A book I'm now reading, "Apache Mahout Cookbook" by Pierro Giacomelli, states that To avoid [this], you need to divide the vector files into two sets called the 80-20 split <...> A good ...
4
votes
2answers
3k views

Machine Learning on financial big data [closed]

Disclaimer: although I know some things about big data and am currently learning some other things about machine learning, the specific area that I wish to study is vague, or at least appears vague to ...
2
votes
1answer
147 views

Reasons and prevention of trivial (and less trivial) misclassification errors?

I was not sure about posting this question with mentioning the name of the company, which I quite respect and admire. However, I've figured that a wider exposure might help the team to fix this and ...
0
votes
2answers
65 views

Should I use epochs > 1 when training data is unlimited?

If I have virtually endless training data (it's synthesized) is there still purpose in having epochs? I.e. training on the same samples multiple times?
5
votes
1answer
2k views

Question and Answer Chatbot for Customer Support

I want to build a chatbot that serves as a first line customer support on a retail website. I have a large log of chat sessions between customers and support professionals that I can use. I am ...
4
votes
3answers
578 views

machine learning on athlete performances to predict the time in a future race

Example Data I have a dataset (in R as a data frame) of race results for athletes. ...
1
vote
2answers
584 views

How to determine whether a bad performance is caused by data quality?

I'm using a set of features, says $X_1, X_2, ..., X_m $, to predict a target value $Y$, which is a continuous value from zero to one. At first, I try to use a linear regression model to do the ...
3
votes
1answer
296 views

How to compute F1 score?

Recently I read about path ranking algorithm in a paper (source: Knowledge Vault: A Web-Scale Approach to Probabilistic Knowledge Fusion). In this paper was a table (Table 3) with facts and I tried ...
0
votes
1answer
2k views

Machine learning for Point Clouds Lidar data

Our main use case is object detection in 3d lidar point clouds i.e. data is not in RGB-D format. We are planning to use CNN for this purpose using theano. Hardware limitations are CPU: 32 GB RAM Intel ...
4
votes
2answers
2k views

Proper way of fighting negative outputs of a regression algorithms where output must be positive all the way

Maybe it is a bit general question. I am trying to solve various regression tasks and I try various algorithms for them. For example, multivariate linear regression or an SVR. I know that the output ...
9
votes
2answers
220 views

Amplifying a Locality Sensitive Hash

I'm trying to build a cosine locality sensitive hash so I can find candidate similar pairs of items without having to compare every possible pair. I have it basically working, but most of the pairs in ...
3
votes
1answer
871 views

Pre-processing (center, scale, impute) among training sets (different forms) and the test set - what is a good approach?

I am currently working on a multi-class classification problem with a large training set. However, it has some specific characteristics, which induced me to experiment with it, resulting in few ...
11
votes
3answers
5k views

Help regarding NER in NLTK

I have been working in NLTK for a while using Python. The problem I am facing is that their is no help available on training NER in NLTK with my custom data. They have used MaxEnt and trained it on ...
0
votes
1answer
364 views

Can 2 dimensional input be applied to SVM? [closed]

When considering Support Vector Machine, in an take in multiple inputs. Can each of these inputs be a vector?? What i am trying to say is, can the input be a 2 dimensional vector??
5
votes
7answers
15k views

Machine learning toolkit for Excel

Do you know of any machine learning add-ins that I could use within Excel? For example I would like to be able to select a range of data and use that for training purposes and then use another sheet ...
6
votes
1answer
1k views

How to extract features and classify alert emails coming from monitoring tools into proper category?

My company provides managed services to a lot of its clients. Our customers typically uses following monitoring tools to monitor their servers/webapps: OpsView Nagios Pingdom Custom shell scripts ...
3
votes
4answers
1k views

How to learn a classifier from a dataset with high imbalance

What are the most useful techniques for learning a binary classifier from a dataset with a high degree of imbalance (i.e., a dataset with the "target" class being much rarer than the "background" ...
12
votes
2answers
7k views

High-dimensional data: What are useful techniques to know?

Due to various curses of dimensionality, the accuracy and speed of many of the common predictive techniques degrade on high dimensional data. What are some of the most useful techniques/tricks/...
8
votes
1answer
2k views

What happens when we train a linear SVM on non-linearly separable data?

What happens when we train a basic support vector machine (linear kernel and no soft-margin) on non-linearly separable data? The optimisation problem is not feasible, so what does the minimisation ...
2
votes
2answers
148 views

Could someone please offer me some guidance on some kind of particular, SPECIFIC project that I could attemp, to “get my feet wet, so to speak” [closed]

I am COMPLETELY new to the field of Data Science, mainly because every employer I have worked for, simply COULDN'T sell any customers anything that would use techniques learned in this field. Of ...
34
votes
3answers
2k views

When to use what - Machine Learning [closed]

Recently in a Machine Learning class from professor Oriol Pujol at UPC/Barcelona he described the most common algorithms, principles and concepts to use for a wide range of machine learning related ...
9
votes
1answer
11k views

What is the difference between feature generation and feature extraction?

Can anybody tell me what is the purpose of feature generation? and why feature space enrichment is needed before classifying an image? Is it a necessary step? Is there any method to enrich feature ...
0
votes
1answer
187 views

What are the current killing machine learning methods? [closed]

I was wondering whether we could list machine learning winning methods to apply in many fields of interest: NLP, image, vision, medical, deep package inspection, etc. I mean, if someone will get ...
1
vote
2answers
834 views

Probability distribution in input-output pairs

This question might sound silly. But I have been wondering why do we assume that there is a hidden probability distribution between input-output pairs in machine learning setup ? For example, if we ...
2
votes
2answers
382 views

Generate tags for live chat transcripts

I'm wondering if there's a way to automatically generate a list of tags for live chat transcripts without domain knowledge. I've tried applying NLP chunking to the chat transcripts and keep only the ...
4
votes
4answers
515 views

Determine highly correlated segments

Given a dataset that has a binary (0/1) dependent variable and a large collection of continuous and categorical independent variables, is there a process and ideally a R package that can find ...
3
votes
2answers
3k views

Aspect based sentiment analysis using machine learning approach

I am very new in machine learning. I have annotated data with category, aspect, opinion word and sentiment. for example, for the bellow text "The apple was really tasty" I have category->food, ...
0
votes
1answer
368 views

Learning cost function for linear regression

(Me: Never learned calculus or advanced math and I started Stanford openclasses for machine learning. I know basic matrix calculations.) One chapter of my course is about cost function. I have been ...
1
vote
1answer
146 views

Correlation threshold for Neural Network features selection

I'm trying to do a correlation analysis between inputs and outputs inspecting the data in order to understand which input variables to include. What could be a threshold in the correlation value to ...