Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
0
votes
1answer
827 views

Anomaly detection on a time-series data in a CSV format using python

I have a time versus current data for a days work which is as follows: ...
0
votes
0answers
32 views

How to use a predfined separate prune set in Weka (alternatives in R or Python also acceptable)

I recently ran into a problem that seems to have considerate local effects, in space and time in fact: Two objects that are close are likely yo be in a similar state (if they are in the same year). I'...
5
votes
1answer
970 views

Implementation of Tree Kernels in Python

I'm looking to use a tree kernel (e.g. Zelenko et al., 2003, Moschitti 2006, etc.) for text classification in Python. However, I'm still a little unclear on the implementation. Is there an existing ...
3
votes
3answers
514 views

Antonym search for expanding search terms

I some code that identifies ngrams of interest in a small bit of text, then searches for the ngram in a larger bit of text to provide a snippet of the term used in context. One ngram might be ...
0
votes
2answers
1k views

CSR scipy matrix does not update after updating its values

I have the following code in python: ...
1
vote
1answer
1k views

How to run XGBoostregressor using reg: tweedie as objective?

I installed XGBoost for anaconda on windows 10 based on the instructions provided here. It seems that xgboost 0.6 is already installed. It performs well using "...
3
votes
4answers
10k views

Check similarity between time series

I have time series of parameters A, B, C and D. All of ...
3
votes
0answers
297 views

TensorFlow doesn't learn when input=output (or probably I am missing something)

I have used TensorFlow before for training a model to recognize images (something similar to this example: https://github.com/tensorflow/models/tree/master/inception). I was trying to do something ...
0
votes
2answers
4k views

Plotting k-means output - python

Can anyone help with plotting k-means results from GraphLab k-means tool. ...
1
vote
1answer
2k views

Computing confidence interval of average output from random forest in scikit learn

I am computing confidence intervals for random forests using the package available here: http://contrib.scikit-learn.org/forest-confidence-interval/auto_examples/plot_mpg.html#sphx-glr-auto-examples-...
1
vote
0answers
642 views

Does the time to train a model using keras increase linear with epoches?

Training a model in keras can go very fast. I trained the following model. To train the model with 1 epoch finishes in a few second. However, using 20 epochs it lasts half an hour or so? An epoch ...
8
votes
4answers
8k views

How to run a pyspark application in windows 8 command prompt

I have a python script written with Spark Context and I want to run it. I tried to integrate IPython with Spark, but I could not do that. So, I tried to set the spark path [ Installation folder/bin ] ...
2
votes
1answer
1k views

categorizing articles using deep learning

Is it possible to write a program using deep learning that categorizes an article (news, entertainment,..., etc. ) into a positive category or negative category based on a specific theme?
1
vote
2answers
290 views

How to interpret the AUC score in this case?

I just run a random forest model on a imbalance dataset. I got the set of AUC and the confusion matrix. The AUC seemed not bad but actually the model predict every instance as positive. So how it ...
1
vote
1answer
377 views

sklearn RandomForestRegressor oob_score_ looks wrong?

I am a newbie to datascience. I installed the Jupyter notebook and was trying to create model for the kaggle titanic dataset. Below is the code I wrote - ...
1
vote
5answers
2k views

Predictive modeling on big data set that can't fit into memory

I am trying to build a Decision-Tree model on top of a dataset that is about 10G in size on my local computer. However, I only have 8G memory. What I am doing now is just random sampling certain ...
1
vote
1answer
937 views

How to find the filename associated with a prediction in Keras?

My question is really simple, how to find the filename associated with a prediction in Keras? That is, if I have a set of 100 test samples named and I get a numpy array which contains the estimated ...
4
votes
1answer
5k views

How to customise cost function in Scikit learn's model?

For example, when I have a problem that false negative should be penalised more, how can I incorporate that requirement in the algorithm such as SVM?
1
vote
1answer
669 views

Example of Logistic Regression using a proportion as the dependent variable?

I am trying to run a logistic regression on a data set where my dependent variable is a proportion of a binary variable, rather than the binary variable itself. I have seen a ton of documentation ...
-3
votes
2answers
336 views

What is most likely the bare minimum knowledge one has to have to become data scientist? [closed]

I am a Python developer but I want to become a data scientist. My Question: At its core what is the bare minimum I need to have to make this transition? I know it cannot simply be that I need to ...
3
votes
1answer
3k views

How do I show plots when using Graphlab?

Sorry, if this topic is not connected directly to Data Science. I want to understand how the Graphlab tool works. Firstly I want to execute the toy examples from the Gallery site. When I try to ...
9
votes
1answer
12k views

XGBoost Linear Regression output incorrect

I am a newbie to XGBoost so pardon my ignorance. Here is the python code : ...
0
votes
1answer
381 views

Using Scorer Object for Classifier Score Method for scikit-learn

I have written my custom scorer object which is necessary for my problem and which I've called "p_value_scoring_object". For the function sklearn.cross_validation.cross_val_score one of the ...
4
votes
4answers
13k views

Library to count number of objects in image?

I'm reading a paper on counting cells, humans, etc. Is there an off-the-shelf library for Python/Theano or TensorFlow I could use?
0
votes
2answers
323 views

Features from images using opencv in Python

I want to start with image processing (image classification). I installed opencv and plan to use it with Python. Being starter at this, I am looking for references regarding trivial ...
0
votes
1answer
576 views
0
votes
3answers
1k views

Multi-label Text Classification

I am trying to build a multi-label classifier for suggesting tags on blog posts. The textual data is full of noise. The approach I have been following until now was a BOW approach with Tf-idf ...
2
votes
1answer
2k views

emission probability using hmmlearn package in python

I am learning hmm and try to implement it in Python hmmlearn package(http://hmmlearn.github.io/hmmlearn/hmm.html#building-hmm-and-generating-samples). However I am not quite understand what the ...
0
votes
1answer
412 views

How do I use Machine Learning to analyze process statistics of automated test results?

(I am newbieeee in machine learning) At the end of every automated test (Python), I collect a lot of information from our product. The information includes CPU, Memory Usage, Disk Activity, Postfix ...
0
votes
1answer
963 views

Dynamic learning rates in XGBoost cross-validation

XGBoost's xgb.train() method takes a learning_rates parameter, which can take a custom function to apply a dynamic learning rate,...
0
votes
1answer
327 views

Kmeans: Between class intertia

I'm using KMeans of scikit-learn for clustering of forums. The attribute inertia_ of model gives the within class inertia but how can i get the between class inertia? In other word how can I verify ...
1
vote
0answers
736 views

Image as input and output in keras

I am trying to make a model of this image. Here is the relevant code: ...
5
votes
1answer
5k views

Plotting different values in pandas histogram with different colors

I am working on a dataset. The dataset consists of 16 different features each feature having values belonging to the set (0,1,2). In order to check the distribution of values in each column, I used <...
1
vote
1answer
2k views

tensorflow mnist tutorial

i have a question about the tutorial of tensorflow to train the mnist database how do i create my own batch without using next_batch() , the idea is to train with a batch of 50 ,then 100 and so but it ...
1
vote
0answers
1k views

Markov chain modelling?

I am working on a personal loan dataset. For each loan, we recorded its credit status monthly after it was drawn by the borrower. Let's say there were 6 status coded by A-F. My project is to use ...
6
votes
3answers
2k views

Convolutional Neural Network not learning EEG data

I have trained a simple CNN (using Python + Lasagne) for a 2-class EEG classification problem, however, the network doesn't seem to learn. loss does not drop over epochs and classification accuracy ...
0
votes
1answer
591 views

What are recommended methods for multi-task prediction?

Currently, we are working on a school project which is trying to predict the number of crimes in some area/neighbourhood. There are 8 different categories for crimes and we've tried to find the ...
6
votes
1answer
13k views

How do i pass data into keras?

I am currently struggling to understand how i should train my regression network using keras. I am not sure how I should pass my input data to the network. Both the input data and the output data is ...
8
votes
2answers
16k views

Counting indexes in pandas

I feel like this is a rudimentary question but I'm very new to this and just haven't been able to crack it / find the answer. Ultimately what I'm trying to do here is to count unique values on a ...
2
votes
3answers
5k views
2
votes
1answer
2k views

Pros and Cons of Python and R for Data Science [closed]

So, let's start out by saying I am NOT asking which is better. I like both of these languages for data science and I think this is a matter of and rather than or since there is no need to choose one ...
2
votes
1answer
345 views

Plotting relationship between 2 data points where one data point is a boolean

I am working with the titanic survivors data set. I have the data as a DataFrame and I can create 1D visualizations such as histograms, and also see the correlations by calling data.corr(). I would ...
0
votes
2answers
368 views

Contrasting logistic regression vs decision tree performance in specific example

I have a set of 10,000 integers, and another set of 100. The integers in the first set are mapped to integers in the second set according to some rules (not mathematical rules, think of these values ...
2
votes
1answer
577 views

Collapse a list to most common spellings

I have a user-generated list of proper names. For the sake of conversation, imagine it's pet names. This list will have many variations of the same name: misspellings, alternative spellings, and ...
1
vote
1answer
3k views

Output data from scikit learn logistic regression

I have run a logistic regression using scikit learn in python. I know want to output the results to put into a csv and then load into Tableau. To do that I need to combine the y_test, y_actual, and ...
3
votes
1answer
636 views

Python Matplotlib Radar not correctly centered

I'm trying to create radar plots for players's profiling. I got this solution from this previous question, but the graphic is not correctly centered and in fact there is a label that ends outside the ...
0
votes
2answers
627 views

Visualizing a large number of variables in python

I have a large number of variables (2000) and want to visualize them using Python. Also, I would like to see the summary statistics of these variables. It seems that drawing box plots for all of these ...
1
vote
1answer
11k views
0
votes
1answer
632 views

Which approach for user classification on chat text (classifier, representation, features)?

I'm trying to train a classifier to classify text from a chat between 2 users so later on I can predict who of the two users is more likely to say X sentence/word. To get there I mined the text from ...
1
vote
1answer
1k views

Implicit Training Models in Spark MLlib?

I have something of a ground floor question I’d like to ask. I’m looking at various options for recommendation engines using Spark. I feel that I have a decent grasp of the basics of collaborative ...

1 75 76 77 78 79 83