Questions tagged [python]
Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).
4,853
questions
0
votes
1answer
149 views
XGBOOST CV() producing error
I am getting the following error while using xgboost.cv() (scikit-learn interface). I am working on a regression problem. Below is the code and trace. No idea why ...
0
votes
0answers
16 views
+50
Estimate eps value in DBSCAN using KNN algorithm
I would like to estimate the best eps value for the DBSCAN algorithm on this dataset by following this set of rules:
Set a minPts: 10
Compute the reachability distance of the 10-th nearest neighbour ...
0
votes
0answers
5 views
Heatmap of large 2D array using datashader and plotly
I’m trying to show a heatmap of a large 2D array (160x250000 entries). This should go into a dash app so I'm using plotly to deal with graphics and my idea was to use datashader for performance but I’...
-1
votes
0answers
6 views
how to convert a large xml file containing multiple xml pages into python pandas?
Every xml page is separated by a \n in the large file. So i have a large xml file sometimes about 0.5gb and in it there are multiple xml pages separated by \n.
imagine that i need to take this file ...
0
votes
0answers
10 views
In python how to extract the unique values base particular column getting the relevant rows data
I have a folder with multiple csv files stores. i am trying to fetch the common companies and parameters based on the year Column, and based on the Risk Return column.
The condition for Risk return &...
1
vote
2answers
37 views
Determine how each feature contribute to XGBoost Classification
so for a summary of what I have done:
My dataset has 5 classes and 10 parameters. I used XGBclassifer from sklearn to investigate if I could use those 10 parameters to predict the class of each data ...
0
votes
1answer
11 views
improving performance for a limited dataset with noisy images, pattern recognition
I am trying to recognize doodles in noisy images like in this one below. My dataset consists of only 10 000 images and 30 categories I've implemented a CNN but it is giving me a 6% accuracy. I am ...
1
vote
1answer
44 views
White circles extraction from a little darker background
I'm trying to extract three white circles on the top left corner from the image below:
I've tried to use:
...
12
votes
2answers
92k views
How to plot two columns of single DataFrame on Y axis
i have two DataFrames(Action,Comedy). Action contains two columns(year,rating) ratings columns contains average rating with respect to year.
Comedy Dataframe contains same two columns with different ...
1
vote
2answers
5k views
How to save Tensorflow predictions to data frame?
I am new to Tensorflow. I have trained Tensorflow model, but I need to take model predictions and add them to my original test set as a column. How can I do that?
...
0
votes
0answers
8 views
Error in python_config after calling 'to_categorical' in R [closed]
I am currently in the process of cleaning/augmenting a dataset in R before I can begin training my model.
I am currently using R.
I am using Keras, Caret, and Tidyverse to augment, train and ...
1
vote
1answer
45 views
Clustering sequences of sentence embeddings
I have a sequence of events, right now I am not worried about their actual times, just the order. This is a sequence of web page views.
I have modelled my data as the following, where each element ...
1
vote
4answers
2k views
Model Validation accuracy stuck at 0.65671 Keras
I am using conv1d to classify EEG signals, but my val_accuracy stuck at 0.65671. No matter what changes i do, it never go beyond 0.65671.
Here is the architecture
...
3
votes
1answer
969 views
Advice on imputing temperature data with StatsModels MICE
This may be a dumb question but I can't figure out how to actually get the values imputed using StatsModels MICE back into my data. I have a dataframe (dfLocal) with hourly temperature records for ...
0
votes
1answer
79 views
How to feed training set labels into Keras LSTM
I’m implementing an LSTM with Keras and I know that I have to reshape the training dataset in a 3D object. Basically I have a ...
-1
votes
0answers
12 views
i keep getting " 'list' object has no attribute 'lower' error when ever i try to use countvectorizer /tf-idf functions [closed]
I have a column of data that has tokenized rows in each column that looks likes this:
0 [albert, betty, dave, wonder, jobe]
0 [working, way, chilling, classics]
...
2
votes
2answers
77 views
Differentiate between positive and negative clusters
I have applied k-means clustering on my dataset of Amazon Alexa reviews.
...
-1
votes
0answers
17 views
Normal distribution histogram [closed]
How I can create histogram for normal distribution without using built-in function in python?
The user should enter the value of mean and std first....
2
votes
2answers
28 views
In feature selection, I came across a situation where NaN were filled by median of the column values
Why the median value is used for NaN? Why not something else like mean? What is the logic behind using the median value?
1
vote
1answer
48 views
Recommendations based on other products seen
I am trying to develop a basic book recommender system to get in touch with the field and start learning methods and how to prepare the data.
The Dataframe I am using is pretty plain, it has the ...
0
votes
1answer
415 views
Neural Network Sensordata as Input
I have a dataset consisting of sensor recordings about human movement. There are 22 classes of different movement like sitting or walking and 19 sensor values.
Each recording of a movement has about ...
-1
votes
1answer
18 views
ROC Curve and AUC value of SVM model
I am new to ML. I have a question so I am evaluating my SVM model.
Example:
...
0
votes
1answer
206 views
Streamlit widgets doesn't display in Jupyter Notebook
I'm trying to create streamlit widgets in Jupyter Notebook but when executing the following code nothing happens.
...
0
votes
2answers
30 views
Standardization with positive and negatives
I have a data set that has a few columns such as:
Total cost: mean = 3,000,000
Percent complete: mean = 50
final profit %: mean = 14
I know with such different orders of magnitude before I fit a ...
0
votes
2answers
5k views
How to build a webscraper for indeed.com
How can I scrape a URL such as: https://www.indeed.com/jobs?q=data+scientist+%2420%2C000&l=New+York&start=10
where I want to get the following parameters: job title, company name, job ...
1
vote
1answer
161 views
Tensorflow MirroredStrategy() looks like it may only be working on one GPU?
I finally got a computer with 2 gpus, and tested out https://pytorch.org/tutorials/beginner/blitz/data_parallel_tutorial.html and https://github.com/tensorflow/models/tree/master/tutorials/image/...
0
votes
1answer
27 views
Model always predicting the same cluster k-means [closed]
I have been working with amazon alexa reviews and want to classify positive and negative reviews into two clusters using k-means and GloVe for word-embeddings.
The clustering on the data given to the ...
0
votes
1answer
67 views
extract document topic vectors from lda model
how can I extract document-topic matrix from LDA model and use it as input features an svm classifier? I am using gensim for implementation
0
votes
2answers
36 views
Python: advice on which machine learning algorithm to use for a problem which involves lots of randomness
I'm new to machine learning so I'll summarize my problem with two examples without getting technical (because I can't).
The dog vs. cat classification example is solvable, in the sense that a human ...
0
votes
3answers
27 views
Problem of having imbalanced classes in the test set while using oversampling [closed]
I have an imbalanced dataset. My classes are 0 and 1. The number of 0 class instances is about 20 times more than the number 1 class instances. I know that I should apply oversampling after train test ...
0
votes
1answer
6k views
ValueError: pos_label=1 is not a valid label: array(['N', 'Y'], dtype='<U1')
X = train_encoded_df.iloc[:, 1: ]
y = train_encoded_df["Loan_Status"]
print("Precision:",metrics.precision_score(y_test, y_pred))
My training data contains the ...
0
votes
2answers
24 views
Clustering based on missing values
I have a dataframe of the form:
...
0
votes
1answer
13 views
How to interpret this Plot of Model Loss from a BiLSTM model?
Hi everyone,
the above graph is produced by a BiLSTM model i just trained and tested. I can't seem to interpret it while it is very different from the references that i acquired by googling online.
...
0
votes
3answers
37 views
How to find the driver features towards a particular result in Classification problems
In a classifier model, we can predict the outcome class, but here I need to find out the features that drive towards a particular result in a classification problem, that are a strong indicator of a ...
1
vote
1answer
522 views
Detect blur image using ssdmobilenet and tensorflowlite
I have clear images of cards vs blurry images of card. My task is to capture photo when the image is not blurry, as you can see from the description I need this code to run in real time on android ...
0
votes
1answer
399 views
Error with pandas dataframe (needs to be 1-dimensional)
I am trying to determine the conformal predictions for my model with my data. But it gives me following error that occurs at icp.calibrate(X_cal, y_cal) :
...
1
vote
1answer
81 views
Naive Bayes always predicting the same label
I have been trying to write a naive bayes classifier from scratch that is supposed to predict the class label of the nominal car.arff dataset. However the classifier always predicts the most common ...
0
votes
1answer
16 views
Should I normalize/standardize the whole dataset during preprocessing or it is sufficient to do It in some columns? [closed]
I am a beginner and my doubts concern the normalization/standardization: during the pre-processing should I normalize/standardize the whole dataset or it's sufficient for some columns? And can I apply ...
0
votes
0answers
10 views
Unable to parse the rest of the xml tag? [closed]
This is a sample from the xml file:
"tag1 start" Zalog is a formerly independent settlement in the eastern part of the capital "tag2 start" Ljubljana "tag2 finish" in ...
0
votes
1answer
118 views
Generating normally distributed data frame with 3 columns
I am trying to generate a normally distributed data frame in Python with 3 columns. Each column requires a separate input of mean, standard deviation, lower and upper values. I am trying to generate ...
-1
votes
0answers
14 views
What is the wrong in my histogram code?
The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all()
...
2
votes
1answer
57 views
Understanding and find the best eps value for DBSCAN
I'm trying to run the DBSCAN algorithm on this .csv. In the first part of my program I load it and plot the data inside it to check its distribution. This is the first part of the code:
...
2
votes
2answers
21 views
How to collect info about unseen bugs given user's comments/feedbacks? [closed]
I have a dataframe which looks like:
...
0
votes
1answer
44 views
Association Rules with Python (coded dataset)
I have this dataset which I really need to use association rules techniques on. The dataset has like 90 variables, many of which are ordinal. Thing is, the data is already coded using numbers instead ...
-2
votes
0answers
6 views
Why am I getting an error for saving a .pkl file (FileNotFound Error, No such file or directory) [closed]
[][2
The code for the model to be trained is fine, even for writing the .pkl file i think. But there is an error in finding the directory to write it into. Im not sure why im getting this error, is ...
0
votes
1answer
19 views
Adding and multiplying higher values based on different columns of a dataframe
I am trying to calculate scores based on different values in the dataframe. Since these scores are based on different conditions, I am having problems where in the final calculation, which I need in a ...
0
votes
1answer
14 views
Difference between Word Embedding and Text Embedding
I am working on a dataset of amazon alexa reviews and wish to cluster them in positive and negative clusters. I am using Word2Vec for vectorization so wanted to know the difference between Text ...
2
votes
1answer
53 views
Negative loss, 100% accuracy
I have a very small dataset with 567 images. I've used pretrained model resnet50 in transfer learning. But whenever i fit my model it's giving me 100% accuracy , 100% validation accuracy and in per ...
0
votes
0answers
15 views
I have collected a large set of user comments. What cool things can I do with it? [closed]
I have collected a dataset of more than 10.000 user comments of a popular news site. For each comment I have
Unique User Identifier
Unique Comment Identifier
The comment text itself
All responses of ...
1
vote
1answer
27 views
why is an optimizer used in something that is not a neural network?
why is an optimizer used in something that is not a neural network?
in the neural style transfer architecture the image is updated instead of the neural network weights.
in stylegan encoder (find ...
