Questions tagged [python]
Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).
3,944
questions
0
votes
0answers
8 views
Reasons that LIME and SHAP might not agree with intuition
I'm leveraging the Python packages lime and shap to explain single (test-set) predictions that a basic, trained model is making ...
1
vote
1answer
53 views
How to evaluate model with imbalanced data binary classification?
I have a binary classification problem. I am using Area under precision recall curve as the evaluation metric. The dimensions of my data are (211, 1361). The data is imbalanced so I have used various ...
1
vote
0answers
8 views
Some of the p-values are NaN - logistic regression
I am trying to do logisitc regression, but have this issue - some of the p values are NaN
...
0
votes
1answer
30 views
In a Time Series Problem, is it possible to forecast quantities by learning the patterns of other items? What are my options?
Suppose I own a store that sells a variety of apples and I have the following stats each month.
Report Date
Type of Apple (TA)
Quantity Available(QA)
Quantity Sold in the Past 30 days(QS30)
...
-1
votes
0answers
16 views
Renaming Columns [closed]
I want to create a new dataframe based upon an original dataframe. I am not really sure how to approach this and which is why the consultation. I want to take the name of one element in the string ...
0
votes
0answers
18 views
Learning Process in Machine Learning
I want to use Unsupervised Leaning to detect Anomalies within a huge Csv file (consisting of headers that are named and thousands of rows belonging to them)
Here on this link I have read about the ...
5
votes
3answers
78 views
+100
How can we extract fields from images?
I am making an document parser which extracts data fields from the documents and store them in a structured way. Each field in my dataset is horizontal which is easy to extract.
But the model fails ...
0
votes
0answers
17 views
0
votes
1answer
14 views
Estimating 3D function - f(x,y) according to its minimums
Problem Statement:
I was given 3 minimums of Y = f(X1, X2) such that:
Local Minimum1: X1 = 0.20; X2 = 0.30; Ymin1 = 0.70
Local Minimum2: X1 = 0.60; X2 = 0.80; Ymin2 = 0.80
Global Minimum:...
1
vote
1answer
255 views
Custom Named Entity Recognition using deep learning
I have a dataset with two columns. First column has some text (news article) and the second column contains names of people (not exactly English names) present in those news articles (first column). I'...
32
votes
8answers
67k views
How to clone Python working environment on another machine?
I developed a machine learning model with Python (Anaconda + Flask) on my workstation and all goes well. Later, I tried to ship this program onto another machine where of course I tried to set up the ...
1
vote
1answer
27 views
Using TSNE to Visualize Clusters in Python
I'm using TSNE to visualize my clusters but the output seems a bit strange. There are supposed to be 3 clusters but instead, there are 4 lines. Is there something wrong with how I'm visualizing them ...
1
vote
2answers
29 views
KNN with mixed data (feature set)
I have a dataset where the feature set consists of hour of the day (between 0 to 23), day of the week (Monday to Sunday), number of shops (a positive integer) and road category (0 to 8 on an ordinal ...
1
vote
1answer
25 views
Comparing excel data sets in Pandas
Pretty new to Python, but as an SEO I'm looking at the benefits of using notebooks in my workflow.
I've got two excel files which I've cleaned and imported into a new notebook using pandas.
I'm ...
1
vote
0answers
8 views
Scipy.optimize getting back x0 for optimization of input to recurrent neural network
I need help with this optimization problem which is either not getting solved at all or is taking a copious amount of time.
I am trying to find optimized input to an RNN (GRU) model of a process ...
0
votes
1answer
24 views
How to load pre-processed .txt data and use it for training convolutional networks?
I have a lot of '.txt' (pre-processed) files related to C3D and I need to use this to train and test while using CNN model.
How does one go about this?
Most of data in each .txt files are formatted ...
2
votes
1answer
53 views
Is it possible to make F1_Score differentiable and use it directly as a Loss function?
One of the metrics that is widely used in binary classification is the F1 score:
$F_1 = 2\cdot \frac{recall \cdot precision}{recall+precision}$
The problem of the F1-score is that it is not ...
1
vote
1answer
55 views
NLP Feature creation from phrase matching
I'm building a model to classify email content, to decide whether the email should lead to a JIRA ticket being "Raised" or "Not Raised". The problem I am having is the data is highly imbalanced with ...
0
votes
1answer
79 views
CVNN and Tensorflow
How the algorithm use Complex Value Neural Network ( all complex value, like: input, weight,bias and output) in python
0
votes
0answers
11 views
How to create POS Embeddings for input?
I am trying to append POS embeddings to an input word embedding (which is from Glove). In order to do that, I need represent POS tags as embeddings.
I saw the link in this channel already; however I ...
2
votes
0answers
8 views
Using an unmerged branch from StatsModels for Anaconda
I am interested in using a statistical test from StatsModels, one that is not available in the default Anaconda StatsModels folder on my drive.
See this blog, which as some point mentions, almost ...
2
votes
1answer
31 views
How to validate regex based Resume parser efficiently
I am using rule based logic to extract features from resume. Basically I am trying to find if the candidate switched the company in less than 1 year. So I have the code in place to find it using ...
1
vote
0answers
19 views
Filter and plot values using matplotlib
In order to understand data a bit more, I want to plot one column from my csv file using matplotlib as a dotted graph
This is how my my csv file looks like.
I want to filter the csv with ...
0
votes
1answer
39 views
Getting Rid of duplicate data
I extract data from a website, on which I have no control. The data is multiplied.
The original report should look like this.
...
0
votes
1answer
443 views
Group small values in a pie chart
When plotting a Pie chart from a dictionary using Python's matplotlib, I get some classes that are too mashed together because of their small size. Is there a way to group the smallest values together ...
1
vote
3answers
589 views
convert dataframe without header to dictionary with a row of number
I am trying to put a dataframe into dictionary ,with the first column as the key ,the numbers in a row would be the value . how should I do it ? I tried to_dict but did not work , my df has no header ...
4
votes
1answer
399 views
How to implement patternet in python as it is in matlab?
NOTE: This question was first posted in cross-validated website but I was instructed to move it off that website as it was not a good fit.
I am new in implementation of machine learning, neural ...
0
votes
0answers
13 views
0
votes
2answers
22 views
Binary classification for variable outputs
I am creating a binary classifier in Keras and here's the code
...
-1
votes
0answers
13 views
Keras - Preparing the Embedding layer with Word2Vec
I have a word2vec model. But I can't implement word2vec with keras embedding layer. In this tutorial, Preparing the Embedding layer section, they used GloVe. I want to use word2vec instead. How can I ...
1
vote
1answer
32 views
very important features, but rarely input
The features are of capital gain and capital loss, but very small amounts of people have one or the other. As far as I can see it may not signify any great difference, though I'm not sure how I would ...
0
votes
2answers
53 views
Dummy encoding the categorical variables using the changed version of OneHotEncoder
This is my code, I was trying to dummy encode the first column of X using OneHotEncoder but it was showing error and the documentation page of OneHotEncoder says that it has been changed and I wasn't ...
0
votes
0answers
24 views
How to fill missing value based on other column in Pandas dataframe? [closed]
For the below sample dataframe, I would like to map the respective state with zone and fill na's with the respective zone names. How can we code for the below instead manually filling missing values?
...
-1
votes
0answers
20 views
How to update package in my Anaconda Environment? [closed]
I am trying to update numpy from 1.15 to 1.18. Using the command
conda install -c anaconda numpy
did not help as the only thing that was updated are the 'ca-...
2
votes
1answer
24 views
Unable to learn weights of a Word2Vec model
I was going to implement a word embedding model - namely Word2Vec - by following this TensorFlow tutorial and adapting the code a little bit. Unfortunately, though, my model won't learn anything. I've ...
-2
votes
1answer
32 views
Which is the correct method for outlier analysis on a dataset for modelling?
I'm trying to build a regression model but my data-set have many outliers points which I need to analyze and then remove them.
There are two ways to do it,
1) First do all the analysis on every ...
1
vote
1answer
41 views
Representing geospatial information
I am trying to train a model to predict the location of a storm at a given time. The dataset includes the longitude and latitude of the storm at the given "timestamps" but I am not sure if that is the ...
0
votes
3answers
101 views
pandas apply function with multiple condition?
if i want to apply lambda with multiple condition how should I do it?
df.train.age.apply(lambda x:0 (if x>=0 and x<500))
or is there much better methods?
2
votes
1answer
247 views
Randomstate and kmeans issues
I try to cluster a dataframe of 227 rows in 5 clusters using kmeans algorithm. Each time I run my code I got different labels and different clusters which make my analysis afterwards a bit tricky.
...
2
votes
1answer
23 views
Trying to return more than just the top result from sklearn NearestNeighbors
I'm trying to compare a list of names (duplicated into a clean file and a messy file). I then compare the files against each other. My problem is that it returns only the top 1 result for each, ...
1
vote
1answer
19 views
Forecasting - 3 independent variables
I have 3 independent input variables: time, temperature and pressure, and one output - usefulness.
Example:
Number of the sample; Time; Temperature; Pressure; Usefulness
1;1;30;50;9%
2;3;50;50;80% ...
1
vote
1answer
237 views
String handling by OneHotEncoder
I am reading everywhere on new questions and blogs that since version 0.20, OneHotEncoder is able to handle string features.
Moreover, the documentation is what looks more ambiguous. Here are the ...
1
vote
1answer
369 views
K-means clustering for one class classification
I want to know if I can use the k-means clustering algorithm for a one class classification (as in the case of one class SVM), which means I have data for 2 classes, and I labelled only the one class ...
5
votes
2answers
924 views
Question about sklearn's StratifiedShuffleSplit
I'm reading through the book Hands-On Machine Learning with Scikit-Learn and Tensorflow by Aurélien Géron. In a regression project on California Housing Prices, he goes over the concept of stratified ...
2
votes
1answer
40 views
Why do probabilities sum to one and how can I set optimal threshold level?
I am working on a text classification use case.
The training data has two classes, so the XBBoostClassifier and onevsrest model is classifying the test data into either of the two classes.
But my ...
1
vote
0answers
31 views
Can anyone direct me on how to got to [closed]
I am a newbie to python.
I have uploaded about 2000 images into my google drive, assigned a variable to all content in the folder and tried to split the folder's content into training and testing, ...
1
vote
1answer
33 views
Feature extraction from Resume using Python without rule based logic
I am working on Resume parser project. Currently I am using rule based regex to extract features like University, Experience, Large Companies etc.
So Basically I have a set of Universities name in a ...
-1
votes
3answers
77 views
Can tanh be used as an output for a binary classifier?
I am creating a binary classifier in Keras and here;s the code
...
1
vote
0answers
26 views
Need help regarding my thesis project related to Data Analysis [closed]
I am working on a project with a company that manufactures Injection molding machines. I am to perform data analysis on some selected parameters as a part of my thesis to tell the company if from ...
0
votes
4answers
4k views
Unable to generate error bars with seaborn
Can you please help me add error bars to my graph?
This is the csv:
...
