All Questions
35,554
questions
5
votes
1
answer
194
views
Data scheduling for recommender
I do at the moment some data experiments with the Graphlab toolkit. I have at the first next SFrame, with the three columns:
Users Items Rating
The pair in the ...
- 313
3
votes
2
answers
1k
views
Recommendation engine with mahout
I have a list user data: user name, age, sex, address, location etc., and
a set of product data: Product name, Cost, description etc.
Now I would like to build a recommendation engine that will be ...
- 1,890
3
votes
2
answers
1k
views
API for historical housing prices
I'm looking for an (ideally free) API that would have time series avg/median housing prices by zip code or city/state.
Quandl almost fits the bill, but it returns inconsistent results across ...
- 131
3
votes
2
answers
748
views
How to run R programs on multicore using doParallel package?
I am running SVM algorithm in R.It is taking long time to run the algorithm.I have system with 32GB RAM.How can I use that whole RAM memory to speed my process.
- 33
3
votes
1
answer
460
views
What is the best practice to classify category of named entity in sentence
I have 1-4 gram text data from wikipedia for 14 categories, which I am using for NE classification.
I feed named entity from sentence to lucene indexer which searches named entity from these 14 ...
- 131
0
votes
0
answers
261
views
Analyze paragraphs using Neuroph
Currently we are regularly analyzing sets of paragraphs every month. I would like to automate this and split each paragraphs into chunks of data.
To do this I would like to employ a neural network. ...
- 109
1
vote
1
answer
14k
views
Change aliases of filter items in Tableau
Data sample contains a single feature: random integer number from 1 to 4.
Is it possble to change 1,2,3,4 representation on the filter card to some custom names, ...
- 5,414
4
votes
2
answers
135
views
Reduction of multiple answers to single variable
The questionnaire for the data is here
The first question takes multiple entry for the same question, I want to reduce this to a single variable. How do I do it?
The clean data is available here.
NB:...
- 41
1
vote
1
answer
91
views
Ontology database
this is my first ever stack exchange question.
I'm trying to build a tool right now and one of the features of the tool is the ability to break down a product or service into it's associated ...
3
votes
3
answers
6k
views
Visualizing Support Vector Machines (SVM) with Multiple Explanatory Variables
I was wondering if anyone was aware of any methods for visualizing an SVM model where there are more than three continuous explanatory variables. In my particular situation, my response variable is ...
- 158
2
votes
2
answers
286
views
Visualization using D3
I am new to D3 programming (any programming, for that matter). I have protein-protein interaction data in JSON format and csv format. I would like to use that data for network visualization.
Data ...
- 795
2
votes
2
answers
140
views
Parsing data from a string
I think this is something that experienced programmers do all the time. But, given my limited programming experience, please bear with me.
I have an excel file which has particular cell entries that ...
- 105
2
votes
3
answers
1k
views
Choosing the right data mining method to find the effect of each parameter over the target
I am dealing with a lot of categorical data right now and I would like to use an appropriate data mining method in any tool [preferably R] to find the effect of each parameter [categorical parameters] ...
- 1,045
2
votes
0
answers
188
views
Non-linear transformations input dataset for support vector machines
I have two classes (A,B) that I would like to classify using a SVM. Say that I have a class C and a function f. Can I do this:
...
- 121
2
votes
1
answer
339
views
3
votes
1
answer
126
views
Dimension reduction for logical arrays
I have measurements of 4 devices at two different points of time. A measurement basically consists of an array of ones and zeros corresponding to a bit value at the corresponding location:
...
- 131
9
votes
3
answers
14k
views
Clustering of documents using the topics derived from Latent Dirichlet Allocation
I want to use Latent Dirichlet Allocation for a project and I am using Python with the gensim library. After finding the topics I would like to cluster the documents using an algorithm such as k-means(...
- 211
2
votes
2
answers
512
views
Geospatial Social Network Analysis Visualization
I have a data set that keeps track of who referred someone to a program, and includes the geo coordinates of both parties for each record.
What will be the best way to visualize this kind of data set?...
6
votes
2
answers
1k
views
Processing data stored in Redshift
We're currently using Redshift as our data warehouse, which we're very happy with. However, we now have a requirement to do machine learning against the data in our warehouse. Given the volume of data ...
- 63
3
votes
1
answer
209
views
Modeling when the response variable has too many 0's and few continuous values?
For problems where the data represents online fraud or insurance (where each row represents a transaction), it is typical for the response variable to denote the value of fraud committed in dollars. ...
- 1,605
4
votes
0
answers
39
views
Method to create master product database to validate entries, and enrich data set [closed]
We have a ruby-on-rails platform (w/ postgreSQL db) for people to upload various products to trade. Of course, many of these products listed are the same, while they are described differently by the ...
3
votes
1
answer
144
views
How can I create a custom tag in JPMML?
I'm trying to create a logistic regression model in jpmml, then write the PMML to a file. The problem I'm having, is that I can't find any way to create a custom tag, such as "shortForm" and "longForm"...
- 33
3
votes
0
answers
429
views
Opening biosemi bdf data using MNE and biosig using python [closed]
I have a biosemi bdf of EEG data which contains 32 channel.
I've opened it using biosig, everything works great, a first list is channel and inside each list there are eeg data.
But if I open it ...
- 251
3
votes
1
answer
101
views
Analyzing mobile usage. What kind of approach should I apply?
I need to analyse a dataset about mobile phone usage (#calls, #sms, #internetConnections) per each cell and hour in the different days.
...
- 1,667
5
votes
1
answer
172
views
Classification of DNA Sequences
I have a database of 3190 instances of DNA consisting of 60 sequential DNA nucleotide positions classified according to 3 types: EI, IE, Other.
I want to formulate a supervised classifier.
My ...
- 723
3
votes
0
answers
302
views
How does the supposed "Unified Architecture for NLP" from Collobert and Weston 2008 really works?
In this paper (here) they suppose a "unified architecture for NLP" with deep neural networks with multitask learning
My problem is to understand the layered architecture in figure 1, see ...
- 449
4
votes
3
answers
10k
views
Large categorical dataset for regression
I need to collect several large datasets (thousands of samples, dozens of features) for regression with only categorical inputs. I already look for such datasets in the UCI repository, but I did not ...
- 1,717
2
votes
0
answers
128
views
FUZZY ARTMAP for continuous data [closed]
I was going through an IEEE research paper which has used Fuzzy ARTMAP for predicting the price of electricity given some highly correlated data.
As per my basic understanding about Fuzzy ARTMAP, it ...
- 129
5
votes
3
answers
523
views
What technologies are fastest at performing joins on large datasets?
By "large", I mean in the range of 100m to 10b rows.
I'm currently using both Hadoop MapReduce and Amazon RedShift. MapReduce has been a little disappointing here. Redshift works very well if the ...
- 51
7
votes
2
answers
297
views
The meaning of multi-class classification rules
The meaning of multi-class classification rules
...
- 153
3
votes
1
answer
86
views
Validity of data
I have a data set that is pivoted in to the following format:
[key] [id] [0] [1] [5] [10] [15] [60] [120] [180],.. [365]
So key could be
[Products] [1000] [15,000] [4000]... etc
Where products is ...
- 31
3
votes
1
answer
2k
views
Clarification about Octave data size limit
I'm just starting to work on a relatively large dataset after ML course in Coursera.
Trying to work on https://archive.ics.uci.edu/ml/datasets/YearPredictionMSD.
Got an accuracy of 5.2 in training and ...
- 69
6
votes
3
answers
755
views
Similarity measure for ordered binary vectors
I would like to ask your opinion on how to choose a similarity measure. I have a set of vectors of length N, each element of which can contain either 0 or 1. The vectors are actually ordered sequences,...
- 121
2
votes
1
answer
233
views
Learning resources for Data Science for applications in road traffic data? [closed]
My Background
I am a graduate student in Civil Engineering. For the analyses of road traffic data (vehicle trajectories as time series) I work with big data sets mostly about a million data points or ...
- 344
20
votes
2
answers
11k
views
Data science without knowledge of a specific topic, is it worth pursuing as a career? [closed]
I had a conversation with someone recently and mentioned my interest in data analysis and who I intended to learn the necessary skills and tools. They suggested to me that while it is great to learn ...
- 323
3
votes
1
answer
428
views
Python Machine Learning Experts [closed]
I'd like to apply some of the more complex supervised machine learning techniques in python - deep learning, generalized addative models, proper implementation of regularization, other cool stuff I ...
- 33
15
votes
3
answers
5k
views
Modelling Unevenly Spaced Time Series
I have a continuous variable, sampled over a period of a year at irregular intervals. Some days have more than one observation per hour, while other periods have nothing for days. This makes it ...
- 420
0
votes
1
answer
186
views
Best format for recording time stamp and GPS
In most data acquisition settings it is useful to tag your data with time and location. If I write the data to csv file, what are the best formats that I can use for this two variables if I want to ...
- 13
2
votes
1
answer
130
views
scikit-learn OMP mem error
I tried to use OMP algorithm available in scikit-learn. My net datasize which includes both target signal and dictionary ~ 1G. However when I ran the code, it exited with mem-error.
The machine has ...
- 21
4
votes
2
answers
915
views
Anomaly detection in multiple parameters
I am a newbie to data science with a typical problem. I have a data set with metric1, metric2 and metric3. All these metrics are interdependent on each other. I want to detect anomalies in metric3. ...
- 141
4
votes
3
answers
2k
views
Text-Classification-Problem, what is the right approach?
I'm planing to write a classification program that is able to classify unknown text in around 10 different categories and if none of them fits it would be nice to know that. It is also possible that ...
- 163
3
votes
0
answers
109
views
How to find a model for a short discrete time-series?
I'm examining the activity of customers over the years which have about one event per year. This results is many short time-series for which I found the distributions (hit/miss over 4 years sorted by ...
- 355
8
votes
2
answers
970
views
Matrix properties and machine learning/data mining
I'm doing some data analysis in a Statistical Pattern Recognition course using PRML. We analyzed a lot of matrix properties, like eigenvalues, column independence, positive semi-definite matrix, etc. ...
- 81
4
votes
2
answers
309
views
Detecting Spam using Machine Learning
The most online tutorials like to use a simple example to introduce to machine learning by classify unknown text in spam or not spam. They say that this is a binary-class problem. But why is this a ...
- 163
63
votes
10
answers
66k
views
Machine learning - features engineering from date/time data
What are the common/best practices to handle time data for machine learning application?
For example, if in data set there is a column with timestamp of event, such as "2014-05-05", how you can ...
- 1,071
5
votes
1
answer
3k
views
Neural Networks getting stuck at local optima
I'm training a NN with 8 features and 8000 training examples with a single output (0, 1) using the scipy.optimise CG algorithm and the results are somewhat inconsistent. The goal is to get the NN to ...
- 101
3
votes
1
answer
96
views
Ranked tag recommendation for university courses
Our system allows an admin to manage a database of university courses. These courses have multiple fields, like the department, a title, and a description.
I am adding the ability to add learning ...
- 131
3
votes
1
answer
3k
views
How do I show plots when using Graphlab?
Sorry, if this topic is not connected directly to Data Science.
I want to understand how the Graphlab tool works. Firstly I want to execute the toy examples from the Gallery site. When I try to ...
- 313
4
votes
2
answers
378
views
MATLAB Perceptron
I cant seem to figure out why I have a high percentage error.
I'm trying to get a perceptron between X1 and X2 which are Gaussian distributed data sets with distinct means and identical co-variances.
...
- 41
2
votes
1
answer
45
views
making logical inference from a simuation generated data
I have data collected from a computer simulation of football games which seem to have recurring patterns of the following form.
if madrid plays arsernal and the match ends under 3 goal, then on their ...
- 61