Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
1
vote
0answers
243 views

MLP batch iteration in python

I'm using the MLPRegressor in sklearn to train a network with approx 1000 inputs and a continuous output variable. Essentially, the issue is one of image classification (1000 pixels) with the ...
1
vote
0answers
512 views

Why is this trained model misclassifying new cases?

Below is text / code from a recent SO post of mine that I hope you could comment on: Using Keras/Theano, I successfully trained a model using the Keras sample code for CIFAR10. It ran for about 2 ...
0
votes
1answer
425 views

DecisionTree split quality using chi-squared statistic

In most of the popular packages (rpart etc.), the decision tree split is based on information gain, entropy or Gini index. Is there a popular decision tree package ( either 'R' or python) where the ...
0
votes
1answer
100 views

Choosing the correct learning algorithm

I am kind of new to the data mining subject but i need help to choose a learning algorithm for my application: The problem: identifying that a certain curve or data set belongs to a certain fault in a ...
-5
votes
1answer
2k views

i want to build a linear regression

i am getting error when i try to run my script. I have stored my data in txt file formar ...
3
votes
3answers
184 views

What type of a problem is this?

The problem: I have a huge dataset which is more than 200GBs in size. The dataset contains around 200 columns (predictors). The task at hand is to predict which product to promote to the customer so ...
2
votes
3answers
258 views

Obtain a model where each feature vector is past few samples and labels are future few samples?

So, I have this data set where each instance is made of past 20 samples of 2 variables. Labels are next five samples. So every instance looks like this: ...
0
votes
1answer
72 views

Data Mining Gear/Goods Websites for Specific Prices

This is a bit of a stretch topic, but I find myself often browsing gear websites (like, Gear Patrol and Uncrate) for birthday and gift ideas. I find it cumbersome to click through each and every ...
1
vote
3answers
2k views

Reduce dimension, then apply SVM

Just out of curiousity, is it generally a good idea to reduce the dimension of training set before using it to train SVM classifier? I have a collection of documents, each of them is represented by a ...
10
votes
1answer
20k views

Reshaping of data for deep learning using Keras

I am a beginner to Keras and I have started with the MNIST example to understand how the library actually works. The code snippet of MNIST problem in Keras example folder is given as : ...
1
vote
0answers
718 views

Multi Model data set Visualization Python

I have a data-set with two variable frequency values as given below ...
11
votes
3answers
5k views

Issue with IPython/Jupyter on Spark (Unrecognized alias)

I am working on setting up a set of VMs to experiment with Spark before I spend go out and spend money on building up a cluster with some hardware. Quick note: I am an academic with a background in ...
2
votes
2answers
7k views

Use spark_csv inside Jupyter and using Python

My ultimate goal is to use Jupyter together with Python for data analysis using Spark. The current hurdle I face is loading the external spark_csv library. I am ...
10
votes
2answers
17k views

How to convert categorical data to numerical data in Pyspark

I am using Ipython notebook to work with pyspark applications. I have a CSV file with lots of categorical columns to determine whether the income falls under or over the 50k range. I would like to ...
2
votes
0answers
539 views

Is there any implementation of Recursive Auto Encoders in Tensorflow?

I am looking for the implementation of Recursive Auto Encoders (RAE) in tensor flow python. I want to model English sentence representations from a sequence to sequence neural network model. RAE is ...
-1
votes
1answer
569 views

OOP for data science [closed]

I'm an R user trying to learn python. I normally create functions and do loops but I 've never created an object. I wonder if there's something that I'm really missing. Can you give a useful example ...
4
votes
2answers
569 views

How to reduce dimensionality of audio data that comes in form of matrices and vectors?

I'm working on a project involved with identifying different types of sounds (such as screams, singing, and bangs) from each other. We've got our data a reasonable number of different transformations ...
1
vote
2answers
662 views

Predict buying behavior under the condition that a customer is advertised or not

I'm trying to predict response of customers on a marketing campaign. As of now, I have data from one Marketing campaign and rfm data of my customers. Some proportion of customers say 60%, received ...
-2
votes
1answer
355 views

Numpy .eye() in Python 3

I'm using numpy.eye(N,M, dtype = int) to print a matrix whose diagnol elements are 1 and rest of the elements are 0. Code: ...
1
vote
0answers
564 views

Is there any ability to use two ore more inputs for Elman recurrent neural network?

I have a problem with using neurolab python library: I'm trying to predict some time-series with help of Elman recurrent neural network: ...
1
vote
0answers
368 views

Memory error - Hierarchical Dirichlet Process, HDP gensim

I am running Hierarchical Dirichlet Process, HDP using gensim in Python but as my corpus is too large it is throwing me following error: ...
2
votes
1answer
880 views

How do I obtain the weight and variance of a k-means cluster?

I am trying to reproduce the results of this paper, but using python and the HMMlearn library instead of matlab. The paper describes a procedure for using HMM (Hidden Markov Model) in order to predict ...
25
votes
6answers
3k views

Machine learning techniques for estimating users' age based on Facebook sites they like

I have a database from my Facebook application and I am trying to use machine learning to estimate users' age based on what Facebook sites they like. There are three crucial characteristics of my ...
0
votes
1answer
81 views

How to implement accurate counts or sums when you have different numbers of days of the week?

I have a dataset of transaction data for a retail outlet. I am using pandas and want to analyse revenue by day of the week, but there are unequal days in the dataset (i.e. an extra weekend). I have ...
9
votes
2answers
66k views

How to group identical values and count their frequency in Python?

Newbie to analytics with Python so please be gentle :-) I couldn't find the answer to this question - apologies if it is already answered elsewhere in a different format. I have a dataset of ...
1
vote
0answers
315 views

Python: How to make model predict in a generalized manner using ML Algorithm

I have a dataset for which I am trying to predict target variables. ...
0
votes
2answers
2k views

Use of cross validation for Polynomial Regression

I've two text files which contains my data. One text file on X axis another text file on Y axis Then using scatter function from python I did the data visualization After that, I used polyfit function ...
5
votes
1answer
3k views

Determinate K in K-Means Clustering

I have salary data of several user (Python list). Now I am using KMeans to cluster them. Given this data, Is there a way to figure out the best value for 'K' automatically through program? I tried ...
0
votes
2answers
584 views

Hierarchical Clustering customized Linkage function

In my clustering project, I need to customize the linkage function, so that after each cluster merging I can update the inter-cluster distance in my own way. Currently I'm using scikit-learn ...
1
vote
0answers
171 views

Scoring consistency within dataset

Suppose I am given a set of structured data. The data is known to be problematic, and I need to somehow "score" them on consistency. For example, I have the data as shown below: ...
2
votes
0answers
1k views

How to use Python's FastFM library (factorization machines) for recommendation tasks?

I have a dataset of <user, item> pairs where each entry records which user bought which item. e.g. ...
1
vote
1answer
413 views

Sklearn and PCA. Why is max n_row == max n_components?

I posted my question on stack overflow, but there someone suggested that I should try it here. What I'm doing now :) OK, first to my data. I have a word-bi-gram frequency matrix (1100 x 100658, ...
1
vote
2answers
3k views

Technical question: how to build exe from python code?

I have a predictive model in IPython notebook. How can I create dll or exe from it? And is it possible to save in executable file already trained model? I want a program with very simple syntax(X - ...
1
vote
1answer
70 views

Need some tips regarding starting out with the field's specific programming languages, with a heavy focus on data visualization

I've got rather expansive development experience, but I'm new to data science. I have been trying to familiarize myself with the main concepts and deciding between R and Python to put my time into. ...
1
vote
1answer
544 views

How to create visualisation on medical data?

I want to create a data visualization on medical data i.e. patient's medical history, allergies one is having, any chronic patient's name must be highlighted, etc. And, there can be a separate ...
4
votes
1answer
1k views

How to explain the outcome of k-means clustering?

I am currently conducting some analysis using NTSB aviation accident database. There are cause statements for most of the aviation incidents in this dataset that describe the factors lead to such ...
1
vote
1answer
2k views

What is a visual bag of words and how is it implemented?

I'm currently working on implementing a bag of visual words in Python. I get the general gist of how it works but I can't seem to find any sources that explain it in more detail to a level where I can ...
4
votes
1answer
2k views

Testing bimodality of data

I am trying to see if my data is multimodal (in fact, I am more interested in bimodality of the data). I performed dip test and it does evidence against unmodal data. However, I want to see, in ...
3
votes
1answer
270 views

Document Categorization Problem

I'm very new to data science in general, and have been tasked with a big challenge. My organization has a lot of documents that are all sorted on document type (not binary format, but a subjectively ...
1
vote
1answer
2k views

Understanding churn prediction model [closed]

guys! I have a dataset with a bunch of costumer-behavior features and the output being "Churned"/"Not churned". I applied a simple Random Forest Classifier and got a nice performance. With this, I ...
3
votes
1answer
2k views

Python plot for confusion matrix similar to confusion wheel?

I have a confusion matrix with 7 classes and would like to represent the matrix in a graph. Something similar like a confusion wheel. Mainly I need to show, the correct observations in each class and ...
2
votes
1answer
2k views

Image Segmentation with a challenging background

I'm working on an animal classification problem, with the data extracted from a video feed. The recording was made in a pen, so the problem is quite challenging with a dark background and many shadows:...
1
vote
1answer
146 views

Histogram of some values only

I have a pandas dataframe df, and I want to show the histogram. df.hist(bins=100, label="myhist") Now, for some reason I have lots of zeros in this df, so I only ...
1
vote
3answers
1k views

K Means giving poor results

I have several user names and their salaries. Now I need to cluster user based on their salaries. I am using KMeans clustering and following is my code ...
2
votes
1answer
309 views

Is there an equivalent of the Bandit package in r (Used in implementation of Multi arm bandit algorithm) in python?

I want to implement MAB(Multi arm bandit algorithm ) in python. There is a package called bandit in R Check here, is there anything similar to this in python
4
votes
1answer
12k views

How to plot/visualize clusters in scikit-learn (sklearn)?

I have done some clustering and I would like to visualize the results. Here is the function I have written to plot my clusters: ...
3
votes
0answers
856 views

Fuzzy Rules with more than two variable in python

I am trying to build a fuzzy inference system in python using skfuzzy library. I have 4 variables depending on which output class is decided. ...
6
votes
1answer
130 views

How are Hyperplane Heatmaps created and how should they be interpreted?

For nonlinear data, when we are using Support Vector Machines, we can use kernels such as Gaussian RBF, Polynomial, etc to achieve linearity in a different (potentially unknown to us) feature space ...
2
votes
2answers
174 views

Is there an appropriate sequence of parameters to be considered when building a pipeline in Scikit learn?

DISCLAIMER: I understand this could be considered a subjective type of question, but I'm hoping it is specific enough to not be flagged. Please give it a chance. I am doing a GridSearch in scikit-...
2
votes
1answer
289 views

Hyperparameters in Gaussian Process

My academical background is in physics and analysis (PDE's), but now I'am reading about data science. I'm reading about Gaussian Process implementation in Sci-Kit Learn I could not find a simple ...