Questions tagged [machine-learning]

Methods and principles of building "computer systems that automatically improve with experience."

Filter by
Sorted by
Tagged with
62
votes
8answers
77k views

Clustering geo location coordinates (lat,long pairs)

What is the right approach and clustering algorithm for geolocation clustering? I'm using the following code to cluster geolocation coordinates: ...
19
votes
2answers
10k views

How to increase accuracy of classifiers?

I am using OpenCV letter_recog.cpp example to experiment on random trees and other classifiers. This example has implementations of six classifiers - random trees, boosting, MLP, kNN, naive Bayes and ...
31
votes
4answers
14k views

Meaning of latent features?

I am learning about matrix factorization for recommender systems and I am seeing the term latent features occurring too frequently but I am unable to understand ...
3
votes
2answers
2k views

ARMA/ARIMA on energy forecasts timeseries: strange prediction

I'm trying to use ARMA/ARIMA with the statsmodel Python package, in order to predict the gas consumption. I tried with a dataset of this format: Using only the gas column. ...
17
votes
2answers
6k views

How to choose the features for a neural network?

I know that there is no a clear answer for this question, but let's suppose that I have a huge neural network, with a lot of data and I want to add a new feature in input. The "best" way ...
7
votes
2answers
2k views

Efficient dynamic clustering

I have a set of datapoints from the unit interval (i.e. 1-dimensional dataset with numerical values). I receive some additional datapoints online, and moreover the value of some datapoints might ...
147
votes
17answers
120k views

Best python library for neural networks

I'm using Neural Networks to solve different Machine learning problems. I'm using Python and pybrain but this library is almost discontinued. Are there other good alternatives in Python?
5
votes
3answers
4k views

Stochastic gradient descent in logistic regression

I am very new to machine learning and in my first project have stumbled across a lot of issues which I really want to get through. I'm using logistic regression with R's ...
8
votes
2answers
190 views

What are some standard ways of computing the distance between individual search queries?

I made a similar question asking about distance between "documents" (Wikipedia articles, news stories, etc.). I made this a separate question because search queries are considerably smaller than ...
37
votes
5answers
10k views

What are some standard ways of computing the distance between documents?

When I say "document", I have in mind web pages like Wikipedia articles and news stories. I prefer answers giving either vanilla lexical distance metrics or state-of-the-art semantic distance metrics,...
8
votes
2answers
169 views

Linearly increasing data with manual reset

I have a linearly increasing time series dataset of a sensor, with value ranges between 50 and 150. I've implemented a Simple Linear Regression algorithm to fit a regression line on such data, and I'm ...
11
votes
1answer
584 views

Fisher Scoring v/s Coordinate Descent for MLE in R

R base function glm() uses Fishers Scoring for MLE, while the glmnet appears to use the coordinate descent method to solve the ...
29
votes
4answers
28k views

What algorithms should I use to perform job classification based on resume data?

Note that I am doing everything in R. The problem goes as follow: Basically, I have a list of resumes (CVs). Some candidates will have work experience before and some don't. The goal here is to: ...
7
votes
1answer
1k views

Linear Regression in R Mapreduce(RHadoop)

I m new to RHadoop and also to RMR... I had an requirement to write a Mapreduce Job in R Mapreduce. I have Tried writing but While executing this it gives an Error. Tring to read the file from hdfs ...
10
votes
1answer
170 views

Prediction with non-atomic features

I would like to use non-atomic data, as a feature for a prediction. Suppose I have a Table with these features: ...
3
votes
1answer
1k views

Data preparation and machine learning algorithm for click prediction

I am new to machine learning. I have a task at hand of predicting click probability given user information like city, state, OS version, OS family, device, browser family, browser version, etc. I have ...
10
votes
3answers
140 views

Handling a regularly increasing feature set

I'm working on a fraud detection system. In this field, new frauds appear regularly, so that new features have to be added to the model on ongoing basis. I wonder what is the best way to handle it (...
2
votes
1answer
850 views

Difference Between Hadoop Mapreduce(Java) and RHadoop mapreduce

I understand Hadoop MapReduce and its features but I am confused about R MapReduce. One difference I have read is that R utilizes maximum RAM. So do perform parallel processing integrated R with ...
6
votes
1answer
2k views

How to normalize results of Singular Value Decomposition (SVD) between 0 and 1?

I'm building a recommender system and using SVD as one of the preprocessing techniques. However, I want to normalize all my preprocessed data between 0 and 1 because all of my similarity measures (...
2
votes
3answers
2k views

How to use neural networks with large and variable number of inputs?

I'm new to machine learning, but I have an interesting problem. I have a large sample of people and visited sites. Some people have indicated gender, age, and other parameters. Now I want to restore ...
5
votes
4answers
563 views

How does Google categorize results from its image search?

While doing a Google image search, the page displays some figured out categories for the images of the topic being searched for. I'm interested in learning how this works, and how it chooses and ...
8
votes
3answers
11k views

Why does Gradient Boosting regression predict negative values when there are no negative y-values in my training set?

As I increase the number of trees in scikit learn's GradientBoostingRegressor, I get more negative predictions, even though there are no negative values in my ...
17
votes
5answers
3k views

Detecting cats visually by means of anomaly detection

I have a hobby project which I am contemplating committing to as a way of increasing my so far limited experience of machine learning. I have taken and completed the Coursera MOOC on the topic. My ...
11
votes
4answers
4k views

Feature Extraction Technique - Summarizing a Sequence of Data

I often am building a model (classification or regression) where I have some predictor variables that are sequences and I have been trying to find technique recommendations for summarizing them in the ...
4
votes
2answers
331 views

What are your favorite sources for news about Machine Learning and Data Science? [closed]

Data Science and Machine Learning include a lot of different topics and it´s hard to stay up-to-date about all the news about papers, researches or new tutorials and tools. What sources do you use to ...
17
votes
2answers
6k views

Binary classification model for unbalanced data

I have a dataset with the following specifications: Training dataset with 193,176 samples with 2,821 positives Test Dataset with 82,887 samples with 673 positives There are 10 features. I want to ...
5
votes
2answers
1k views

How to select algorithms for ensemble methods?

There is a general recommendation that algorithms in ensemble learning combinations should be different in nature. Is there a classification table, a scale or some rules that allow to evaluate how far ...
-6
votes
1answer
211 views

Online k-means explanation [closed]

Please, could someone recommend a paper or blog post that describes the online k-means algorithm.
22
votes
4answers
600 views

What statistical model should I use to analyze the likelihood that a single event influenced longitudinal data

I am trying to find a formula, method, or model to use to analyze the likelihood that a specific event influenced some longitudinal data. I am having difficultly figuring out what to search for on ...
26
votes
4answers
12k views

Word2Vec for Named Entity Recognition

I'm looking to use google's word2vec implementation to build a named entity recognition system. I've heard that recursive neural nets with back propagation through structure are well suited for named ...
11
votes
4answers
12k views

Is GLM a statistical or machine learning model?

I thought that generalized linear model (GLM) would be considered a statistical model, but a friend told me that some papers classify it as a machine learning technique. Which one is true (or more ...
9
votes
4answers
3k views

Suggest text classifier training datasets

Which freely available datasets can I use to train a text classifier? We are trying to enhance our users engagement by recommending the most related content for him, so we thought If we classified ...
15
votes
4answers
2k views

What are the implications for training a Tree Ensemble with highly biased datasets?

I have a highly biased binary dataset - I have 1000x more examples of the negative class than the positive class. I would like to train a Tree Ensemble (like Extra Random Trees or a Random Forest) on ...
9
votes
1answer
204 views

Learning signal encoding

I have a large number of samples which represent Manchester encoded bit streams as audio signals. The frequency at which they are encoded is the primary frequency component when it is high, and there ...
14
votes
4answers
2k views

Looking for example infrastructure stacks/workflows/pipelines

I'm trying to understand how all the "big data" components play together in a real world use case, e.g. hadoop, monogodb/nosql, storm, kafka, ... I know that this is quite a wide range of tools used ...
8
votes
4answers
1k views

Online machine learning tutorial

Does anyone know some good tutorials on online machine learning technics? I.e. how it can be used in real-time environments, what are key differences compared to normal machine learning methods etc. ...
14
votes
3answers
9k views

Best way to classify datasets with mixed types of attributes

I would like to know what is the best way to classify a data set composed of mixed types of attributes, for example, textual and numerical. I know I can convert textual to boolean, but the vocabulary ...
102
votes
10answers
102k views

Choosing a learning rate

I'm currently working on implementing Stochastic Gradient Descent, SGD, for neural nets using back-propagation, and while I understand its purpose I have some ...
19
votes
7answers
11k views

How can I predict traffic based on previous time series data?

If I have a retail store and have a way to measure how many people enter my store every minute, and timestamp that data, how can I predict future foot traffic? I have looked into machine learning ...
9
votes
2answers
3k views

Difference between using RMSE and nDCG to evaluate Recommender Systems

What kind of error measures do RMSE and nDCG give while evaluating a recommender system, and how do I know when to use one over the other? If you could give an example of when to use each, that would ...
119
votes
15answers
115k views

Python vs R for machine learning

I'm just starting to develop a machine learning application for academic purposes. I'm currently using R and training myself in it. However, in a lot of places, I have seen people using Python. What ...
5
votes
1answer
7k views

How can we calculate AUC for a simple decision tree?

The setup is simple: binary classification using a simple decision tree, each node of the tree has a single threshold applied on a single feature. In general, building a ROC curve requires moving a ...
10
votes
2answers
2k views

Debugging Neural Networks

I've built an artificial neural network in python using the scipy.optimize.minimize (Conjugate gradient) optimization function. I've implemented gradient checking, double checked everything etc and I'...
17
votes
3answers
3k views

One-Class discriminatory classification with imbalanced, heterogenous Negative background?

I'm working on improving an existing supervised classifier, for classifying {protein} sequences as belonging to a specific class (Neuropeptide hormone precursors), or not. There are about 1,150 known ...
12
votes
9answers
3k views

What are some easy to learn machine-learning applications? [closed]

Being new to machine-learning in general, I'd like to start playing around and see what the possibilities are. I'm curious as to what applications you might recommend that would offer the fastest ...
42
votes
10answers
44k views

Can machine learning algorithms predict sports scores or plays?

I have a variety of NFL datasets that I think might make a good side-project, but I haven't done anything with them just yet. Coming to this site made me think of machine learning algorithms and I ...
16
votes
2answers
976 views

Where in the workflow should we deal with missing data?

I'm building a workflow for creating machine learning models (in my case, using Python's pandas and sklearn packages) from data ...
9
votes
3answers
4k views

Human activity recognition using smartphone data set problem

I'm new to this community and hopefully my question will well fit in here. As part of my undergraduate data analytics course I have choose to do the project on human activity recognition using ...
13
votes
4answers
7k views

Algorithm for generating classification rules

So we have potential for a machine learning application that fits fairly neatly into the traditional problem domain solved by classifiers, i.e., we have a set of attributes describing an item and a "...
15
votes
4answers
5k views

How to specify important attributes?

Assume a set of loosely structured data (e.g. Web tables/Linked Open Data), composed of many data sources. There is no common schema followed by the data and each source can use synonym attributes to ...