Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
4
votes
2answers
95 views

Dataset to give same eigenvectors?

Any suggestion on what kind of dataset lets say $n \times d$ ($n$ rows, $d$ columns) would give me same eigenvectors?. I believe it should be the one with same absolute values in each cell. Like ...
78
votes
6answers
112k views

strings as features in decision tree/random forest

I am doing some problems on an application of decision tree/random forest. I am trying to fit a problem which has numbers as well as strings (such as country name) as features. Now the library, scikit-...
1
vote
1answer
967 views

What is a good hardware setup for using Python across multiple users [closed]

This question is likely somewhat naive. I know I (and my colleagues) can install and use Python on local machines. But is that really a best practice? I have no idea. Is there value in setting up a ...
6
votes
2answers
13k views

Python or R for implementing machine learning algorithms for fraud detection [closed]

I was wondering which language can I use: R or Python, for my internship in fraud detection in an online banking system: I have to build machine learning algorithms (NN, etc.) that predict transaction ...
5
votes
1answer
2k views

Best Python library for statistical inference

I'm curious if anyone has Python library suggestions for inferential statistics. I'm currently reading An Introduction to Statistical Learning, which uses R for the example code, but ideally I'd like ...
0
votes
1answer
244 views

Python for data analytics [closed]

What are some data analytic package & feature in python which helps do data analytic?
0
votes
2answers
554 views

minimization with a negative cost function: works in MATLAB, not in Python

I'm trying to use a particular cost function (based on doubling rate of wealth) for a classification problem, and the solution works well in MATLAB. See https://github.com/acmyers/compareCostFXs When ...
2
votes
1answer
584 views

Normalize weekly data - Python

I have a weekly dataset and I have to normalize this data. Data is something like this : ...
12
votes
3answers
6k views

Help regarding NER in NLTK

I have been working in NLTK for a while using Python. The problem I am facing is that their is no help available on training NER in NLTK with my custom data. They have used MaxEnt and trained it on ...
26
votes
5answers
3k views

VM image for data science projects

As there are numerous tools available for data science tasks, and it's cumbersome to install everything and build up a perfect system. Is there a Linux/Mac OS image with Python, R and other open-...
3
votes
1answer
13k views

Querying DBpedia from Python [closed]

How can I get information about an entity from DBpedia using Python? Eg: I need to get all DBpedia information about USA. So I need to write the query from python (SPARQL) and need to get all ...
2
votes
3answers
367 views

Creating a Graph from Address Data

What's the best / easiest way to create a graph from address data? For example, if I have 100 houses from all across a city is there any easy way to determine the shortest distance between two houses ...
4
votes
2answers
4k views

Scikit Learn: KMeans Clustering 3D data over a time period (dimentionality reduction?)

I have a dataset of xyz coordinates with a date component in a pandas dataframe ex: date1: $[x_1,y_1,z_1]$, date2: $[x_2,y_2,z_2]$, date3: $[x_3,y_3,z_3]$, .. I would like to classify a sample of ...
6
votes
2answers
3k views

Python interface to Titan Database

How can I connect to Titan database from Python ? What I understand is that Titan (Graph database) provides an interface (Blueprint) to Cassandra (Column Store) and bulb is a python interface to ...
2
votes
0answers
200 views

How does one feed graph optimization problems into Python's anneal function in SciPy?

I am interested in graph problems like 2-color, max-clique, stable sets, etc but the documentation for scipy.optimize.anneal seems to be for ordinary functions. How would one apply this library ...
2
votes
2answers
276 views

Graphlab vs Mahout [closed]

I have some question regarding to the choice of the better implementation. I would know the differences and advantages of Mahout Apache (Java implementation) versus Graphlab (Python implementation) in ...
2
votes
3answers
2k views

Sentiment analysis using python

I have some text files containing movie reviews I need to find out whether the review is good or bad. I tried the following code but it's not working: ...
11
votes
3answers
1k views

What regression to use to calculate the result of election in a multiparty system?

I want to make a prediction for the result of the parliamentary elections. My output will be the % each party receives. There is more than 2 parties so logistic regression is not a viable option. I ...
4
votes
2answers
1k views

How to ensemble classifier incorporating all features in python?

I am doing a text classification task(5000 essays evenly distributed by 10 labels). I explored LinearSVC and got an accuracy of 80%. Now I guess whether accuracy ...
9
votes
3answers
13k views

Clustering of documents using the topics derived from Latent Dirichlet Allocation

I want to use Latent Dirichlet Allocation for a project and I am using Python with the gensim library. After finding the topics I would like to cluster the documents using an algorithm such as k-means(...
3
votes
0answers
364 views

Opening biosemi bdf data using MNE and biosig using python [closed]

I have a biosemi bdf of EEG data which contains 32 channel. I've opened it using biosig, everything works great, a first list is channel and inside each list there are eeg data. But if I open it ...
3
votes
1answer
428 views

Python Machine Learning Experts [closed]

I'd like to apply some of the more complex supervised machine learning techniques in python - deep learning, generalized addative models, proper implementation of regularization, other cool stuff I ...
2
votes
0answers
117 views

scikit-learn OMP mem error

I tried to use OMP algorithm available in scikit-learn. My net datasize which includes both target signal and dictionary ~ 1G. However when I ran the code, it exited with mem-error. The machine has ...
3
votes
1answer
3k views

How do I show plots when using Graphlab?

Sorry, if this topic is not connected directly to Data Science. I want to understand how the Graphlab tool works. Firstly I want to execute the toy examples from the Gallery site. When I try to ...
4
votes
9answers
3k views

Python and R good tutorials? [closed]

I would like to learn both Python and R for usage in data science projects. I am currently unemployed, fresh out of university, scouting around for jobs and thought it would be good if I get some ...
10
votes
2answers
8k views

Stochastic gradient descent based on vector operations?

let's assume that I want to train a stochastic gradient descent regression algorithm using a dataset that has N samples. Since the size of the dataset is fixed, I will reuse the data T times. At each ...
2
votes
1answer
395 views

data processing, correlation calculation

I have product purchase count data which looks likes this: user item1 item2 a 2 4 b 1 3 c 5 6 ... ... ... These data are ...
11
votes
2answers
299 views

How much time do scikit classifiers take to classify?

I am planning to use scikit linear support vector machine (SVM) classifier for text classification on a corpus consisting of 1 million labeled documents. What I am planning to do is, when a user ...
-6
votes
2answers
287 views

which programming language has a large library that can do machine learning algorithm, R, matlab or python [closed]

As what I described in the title, we are especially interested in those for dealing with big data----ts efficiency and stability, and used in industry not in experiment or university. Thanks!
7
votes
1answer
1k views

How to numerically estimate MLE estimators in python when gradients are very small far from the optimal solution?

I am exploring how to model a data set using normal distributions with both mean and variance defined as linear functions of independent variables. Something like N ~ (f(x), g(x)). I generate a ...
-3
votes
2answers
1k views

Which one will be the dominating programming language for next 5 years for analytics , machine learning . R or python or SAS [closed]

Which one will be the dominating programming language for next 5 years for analytics , machine learning . R verses python verses SAS. Advantage and disadvantage.
103
votes
12answers
110k views

SVM using scikit learn runs endlessly and never completes execution

I am trying to run SVR using scikit-learn (python) on a training dataset that has 595605 rows and 5 columns (features) while the test dataset has 397070 rows. The data has been pre-processed and ...
1
vote
1answer
2k views

How to build parse tree with BNF [closed]

I need to build parse tree for some source code (on Python or any program language that describe by CFG). So, I have source code on some programming language and BNF this language. Can anybody give ...
17
votes
2answers
8k views

Recommending movies with additional features using collaborative filtering

I am trying to build a recommendation system using collaborative filtering. I have the usual [user, movie, rating] information. I would like to incorporate an ...
14
votes
8answers
16k views

Is Python suitable for big data

I read in this post Is the R language suitable for Big Data that big data constitutes 5TB, and while it does a good job of providing information about the ...
2
votes
1answer
134 views

Getting GitHub repository information by different criteria

New to the Data Science forum, and first poster here! This may be kind of a specific question (hopefully not too much so), but one I'd imagine others might be interested in. I'm looking for a way to ...
11
votes
1answer
5k views

t-SNE Python implementation: Kullback-Leibler divergence

t-SNE, as in [1], works by progressively reducing the Kullback-Leibler (KL) divergence, until a certain condition is met. The creators of t-SNE suggests to use KL divergence as a performance criterion ...
62
votes
8answers
81k views

Clustering geo location coordinates (lat,long pairs)

What is the right approach and clustering algorithm for geolocation clustering? I'm using the following code to cluster geolocation coordinates: ...
61
votes
9answers
9k views

Tools and protocol for reproducible data science using Python

I am working on a data science project using Python. The project has several stages. Each stage comprises of taking a data set, using Python scripts, auxiliary data, configuration and parameters, and ...
3
votes
2answers
2k views

ARMA/ARIMA on energy forecasts timeseries: strange prediction

I'm trying to use ARMA/ARIMA with the statsmodel Python package, in order to predict the gas consumption. I tried with a dataset of this format: Using only the gas column. ...
147
votes
17answers
122k views

Best python library for neural networks

I'm using Neural Networks to solve different Machine learning problems. I'm using Python and pybrain but this library is almost discontinued. Are there other good alternatives in Python?
7
votes
2answers
5k views

Struggling to integrate sklearn and pandas in simple Kaggle task

I'm trying to use the sklearn_pandas module to extend the work I do in pandas and dip a toe into machine learning but I'm struggling with an error I don't really understand how to fix. I was working ...
10
votes
3answers
12k views

Why does Gradient Boosting regression predict negative values when there are no negative y-values in my training set?

As I increase the number of trees in scikit learn's GradientBoostingRegressor, I get more negative predictions, even though there are no negative values in my ...
17
votes
2answers
6k views

Binary classification model for unbalanced data

I have a dataset with the following specifications: Training dataset with 193,176 samples with 2,821 positives Test Dataset with 82,887 samples with 673 positives There are 10 features. I want to ...
26
votes
4answers
13k views

Word2Vec for Named Entity Recognition

I'm looking to use google's word2vec implementation to build a named entity recognition system. I've heard that recursive neural nets with back propagation through structure are well suited for named ...
8
votes
3answers
975 views

What to consider before learning a new language for data analysis

I'm currently in the very early stages of preparing a new research-project (still at the funding-application stage), and expect that data-analysis and especially visualisation tools will play a role ...
120
votes
15answers
116k views

Python vs R for machine learning

I'm just starting to develop a machine learning application for academic purposes. I'm currently using R and training myself in it. However, in a lot of places, I have seen people using Python. What ...
10
votes
2answers
2k views

Debugging Neural Networks

I've built an artificial neural network in python using the scipy.optimize.minimize (Conjugate gradient) optimization function. I've implemented gradient checking, double checked everything etc and I'...
17
votes
3answers
3k views

One-Class discriminatory classification with imbalanced, heterogenous Negative background?

I'm working on improving an existing supervised classifier, for classifying {protein} sequences as belonging to a specific class (Neuropeptide hormone precursors), or not. There are about 1,150 known ...
2
votes
5answers
1k views

Are there any python based data visualization toolkits?

Data visualization is an important sub-field of data science and python programmers need to have available toolkits for them. Is there a Python API to Tableau? Are there any Python based data ...