Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (which should be asked on Stack Overflow).

2,056 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
9 votes
2 answers
389 views

Are there any graph embedding algorithms like this already?

I wrote an algorithm for generating node embeddings based on the graph's topology. Most of the explanation is done in the readme file and the examples. The question is: Am I reinventing the wheel? ...
monomonedula's user avatar
9 votes
0 answers
2k views

Why is my Keras model not learning image segmentation?

Edit: as is turns out, not even the model's initial creator could successfully fine-tune it. This is most likely a problem of implementation, or possibly related to the non-intuitive way in which the ...
Matt's user avatar
  • 199
9 votes
0 answers
3k views

Python : Feature Matching + Homography to find Multiple Objects

I'm trying to use OpenCV via Python to find multiple objects in a train image and match it with the key points detected from a query image. For my case, I'm trying to detect the tennis courts in the ...
Reward's user avatar
  • 91
8 votes
4 answers
5k views

How to perform feature selection on dataset with categorical and numerical features?

I am working on a dataset with 30 columns (29 numerical, 1 non-ordinal categorical). I hot-encoded the categorical feature and reached at 35 columns. To improve training efficiency, I want to perform ...
Songyu Yan's user avatar
8 votes
1 answer
1k views

Gensim LDA model: return keywords based on relevance (λ - lambda) value

I am using the gensim library for topic modeling, more specifically LDA. I created my corpus, my dictionary, and my LDA model. With the help of the pyLDAvis library I visualized the results. When I ...
Tasos Lytos's user avatar
7 votes
0 answers
2k views

Using the Python Keras multi_gpu_model with LSTM / GRU to predict Timeseries data

I'm having an issue with python keras LSTM / GRU layers with multi_gpu_model for machine learning. When I use a single GPU, the predictions work correctly ...
Palisadoes's user avatar
7 votes
0 answers
2k views

Fine tuning accuracy lower than Raw Transfer Learning Accuracy

I've used transfer learning on Inception V3 with ImageNet weights on Keras with Tensorflow backend on python 2.7 to create an image classifier. I first extracted and saved the bottleneck features from ...
Varun's user avatar
  • 71
6 votes
3 answers
186 views

What Clustering Method Should I Use?

My data is a group of 10 thousand points (each having an node location (x,y)) that are spread across a plane. They are also chromatically-colored based on their weight. I need to finalize a bayesian ...
ChessGrandMaster's user avatar
6 votes
2 answers
4k views

Triangle Pattern Recognition on Financial Market with Python

I'm working on a personal project to find Triangles on any stock in Python. I detect the max and min points (shift(-5,+5) because if I consider only shift(-1+1) I have a lot of lines) and write lines ...
Martin Bouhier's user avatar
6 votes
0 answers
150 views

Unable to transform (greatly performing) Autoencoder into Variational Autoencoder

Following the procedure described in this SO question, I am trying to transform my (greatly performing) convolutional Autoencoder into a Variational version of the same Autoencoder. As explained in ...
user87590's user avatar
6 votes
1 answer
8k views

Keras - Implementation of custom loss function with multiple outputs

I am trying to replicate (a way smaller version) the AlphaGo Zero system. However, in the network model, I am having a problem. The loss function I am supposed to implement is the following: $$l = (z -...
ihavenoidea's user avatar
6 votes
2 answers
3k views

How to deal with missing data for Bernoulli Naive Bayes?

I am dealing with a dataset of categorical data that looks like this: ...
Chuck's user avatar
  • 161
6 votes
2 answers
206 views

Gridsearch XGBoost for ensemble. Do I include first-level prediction matrix of base learners in train set?

I'm not quite sure how I should go about tuning xgboost before I use it as a meta-learner in ensemble learning. Should I include the prediction matrix (ie. df containing columns of prediction results ...
doyz's user avatar
  • 161
6 votes
0 answers
12k views

Tuning Gradient Boosted Classifier's hyperparametrs and balancing it

I am not sure if it is a correct stack. Maybe I should have put my question into crossvalidated. Nevertheless, I perform following steps to tune the hyperparameters for a gradient boosting model: ...
user1877600's user avatar
5 votes
2 answers
3k views

Multidimensional scaling producing different results for different seeds

I took the data from here and wanted to play around with multidimensional scaling with this data. The data looks like this: In particular, I want to plot the cities in a 2D space, and see how much it ...
Kristada673's user avatar
4 votes
0 answers
71 views

Does ROC AUC different between crossval and test set indicate overfitting or other problem?

I am training a composite model (XGBoost, Linear Regression, and RandomForest) to predict injured people probability. Well, the results of cross-validation with 5 folds. Well, I can see any problem ...
GregOliveira's user avatar
4 votes
1 answer
2k views

Why does classifier chain ask for at least 2 classes, when I have it

I'm using Classifier Chain with logistic regression and when i try to use fit, i get This solver needs samples of at least 2 classes in the data, but the data contains only one class: 1 but I'm ...
ronald reagan's user avatar
4 votes
1 answer
1k views

SelectKBest and Correlation returns me excatly same feature selection. How?

Im working on selecting most effective features from a dataset with over that 2000 features. Im using different algorithms for that (selectKBest with chi-square, Extra Trees, Correlation etc.) But ...
justRandomLearner's user avatar
4 votes
0 answers
513 views

How to apply oversampling when doing Leave-One-Group-Out cross validation?

I am working on an imbalanced data for classification and I tried to use SMOTE previously to oversampling the training data. However, this time I think I need to use a leave-on group out (LOGO) cross-...
npm's user avatar
  • 141
4 votes
2 answers
7k views

Saving and loading keras.callbacks.History object with np.save and np.load

I have been saving my training history in keras as follows: ...
Ben Groene's user avatar
4 votes
1 answer
1k views

Advice on dealing with very large datasets - HDF5, Python

Recently, I've started working on an application for the visualization of really big datasets. While reading online it became apparent that most people use HDF5 for storing big, multi-dimensional ...
munieq11's user avatar
4 votes
1 answer
1k views

Calculating saliency maps for text classification

I'm following the text classification with movie reviews TensorFlow tutorial, and wanted to extend the project by looking, for a certain input, which words influenced the classification the most. I ...
Marc Jones's user avatar
4 votes
1 answer
817 views

Product classification in hierarchical categories based on multiple parameters and non-standard descriptions

I want to start a machine learning project in my company and a really big pain for spend analysts is to classify the products that buyers order for maintenance, tooling, raw material and such, as the ...
mad_dash2's user avatar
4 votes
1 answer
874 views

Scikit-learn average_precision_score() vs. auc score of precision_recall_curve()

I've been searching around for an explanation to this, and haven't come across one yet- in scikit-learn, when I compute the auc() of the ...
sarahwie's user avatar
4 votes
2 answers
238 views

Benefits of using Deep Learning-specific hyperparameter optimization tools vs. sklearn?

There are quite a few library for hyperparameter optimization that are specific to Keras or other Deep Learning libraries, like Hyperas or Talos. My question is, what's the main benefit of using ...
Edgar Derby's user avatar
4 votes
1 answer
466 views

Multiple activation functions with TensorFlow estimator DNNClassifier

I just want to know if is it possible to use tf.estimator.DNNClassifier with multiple different activation functions. I mean, could I use a DNNClassifier estimator which use different activation ...
David C.'s user avatar
4 votes
1 answer
365 views

Why is my U-matrix visually not separating the classes?

I am visualizing the U-matrix generated using a Self Organizing Map codebook to (visually) identify regions of similarity in the data. Although I would like to use SOM to identify clusters in ...
Rohit Gavval's user avatar
4 votes
1 answer
961 views

What preprocessing steps to be followed before image comparison?

1 down vote favorite For example I am trying to find the similarity between two images using skimage - SSIM. The code block will be as follows ...
Vivek Srinivasan's user avatar
3 votes
0 answers
74 views

Where can I find implementation of the various improvements of K-nearest neighbors (KNN)?

I have been facing some challenges where traditional KNN algorithm perform well. I'd like to explore more advanced knn solutions. While researching possible solutions, I came across a paper titled <...
Lucas Morin's user avatar
  • 2,244
3 votes
1 answer
126 views

Reinforcement Learning applied to Optimisation Problem

Problem Statement: We are given an optimisation problem; with production centres, source airport, destination airports, transfer points and finally delivered to the customers. This is better explained ...
Alpha's user avatar
  • 31
3 votes
0 answers
2k views

Identify MCAR, MNAR and MAR in the data

If I have missing values in a dataset, I can't just blindly impute them with mean/median/mode or any other technique. I have to identify what kind of missing values they are, namely: MCAR (missing ...
spectre's user avatar
  • 2,105
3 votes
1 answer
1k views

How to identify/recognize that a sentence about talks about future?

Brief Introduction: I have a report/paragraph in which there are sentences with reference to future plans/outlooks/expectations for a particular entity. I want to extract all such sentences for now. ...
Krs's user avatar
  • 31
3 votes
0 answers
244 views

Struggling to understand/implement Transformer Decoder

I'm struggling to understand the decoder in a Transformer model, specifically with regards to some aspects of its architecture as well as how it actually handles the data during training. What I have ...
cuuupid's user avatar
  • 131
3 votes
0 answers
2k views

Hopfield Network python implementation, Network doesn't converge to one of the learned patterns

I'm trying to implement a Hopfield Network in python using the NumPy library. The network has 2500 nodes (50 height x 50 width). The network learns 10 patterns from images of size 50x50 stored in &...
Emoticon's user avatar
3 votes
0 answers
378 views

Is it possible to increase the number of images of one class using data augmentation, which is not applied on the other class, in the same dataset?

I have 2 classes for my image classification problem, say class A and class B, and I am using tensorflow and keras for the same. One of them have around 5K images while the other have just around 2K ...
Hrushikar Teja K's user avatar
3 votes
2 answers
1k views

Why does my random forest classifier predicts one class more often?

I have a random forest classifier that predicts 0 class about twice as often as class 1. It also predicts class 0 with higher probabilities than class 1. It is not a imbalanced dataset. I tried ...
Ondřej Vitík's user avatar
3 votes
0 answers
753 views

What exactly negative/positive value of Captum's Integrated Gradient mean?

I use Captum's Integrated Gradient to interprete my PyTorch's neural network. I know that from github and original paper mentioned that ... Positive attribution score means that the input in that ...
3ORZ's user avatar
  • 31
3 votes
0 answers
913 views

PyTorch: Train without dataloader (loop trough dataframe instead)

I was wondering if it is bad practice to instead of using built in tools such as dataloader just loop trough each row in a pandas df. Lets say I am doing text classification and my training loop looks ...
Isbister's user avatar
  • 193
3 votes
1 answer
161 views

Issues with self-implemented logistic regression

I am trying to self-implement a logistic regression algorithm to do some self-learning but I am having a bit of trouble with achieving similar accuracy to the logistic regression of sklearn. Here is ...
Steve Ahlswede's user avatar
3 votes
1 answer
510 views

Semantic network using word2vec

I have thousands of headlines and I would like to build a semantic network using word2vec, specifically google news files. My sentences look like ...
Math's user avatar
  • 161
3 votes
0 answers
696 views

Explain FastText model using SHAP values

I have trained fastText model and some fully connected network build on its embeddings. I figured out how to use Lime on it: complete example can be found in Natural Language Processing Is Fun Part 3: ...
Mikhail_Sam's user avatar
3 votes
1 answer
241 views

Is it possible to solve Rubik's cube using DQN?

I'm trying to solve Rubik's cube using deep learning and I came across with DQN, so I decided to give it a try. I developed all the code and started training but I got this results: Loss goes up and ...
Javier Jiménez de la Jara's user avatar
3 votes
1 answer
979 views

Error when trying Transfer Learning

I'm trying to train a model which is an extension of Google's Inception-V3 for the purpose of recognizing and classifying whether there is any pneumonia using x-ray images. I've used Tensorflow-Hub ...
MetaInformation's user avatar
3 votes
1 answer
911 views

Character-level embeddings in python

I'm working on an NLP task that requires the use of character level embeddings, and I've been trying to use Spacy. However, it seems that spacy uses word-level embeddings for the word vectors, and I ...
rmaguiar's user avatar
  • 163
3 votes
0 answers
40 views

Serializing a trained classification model into a set of actionable insights

I'm looking for ways to convert a trained classification model into a list of insights based on the resulting parameters of the model. To make an example, let's assume we trained a decision tree to ...
ozz1k's user avatar
  • 83
3 votes
0 answers
813 views

Balancing the dataset using imblearn undersampling, oversampling and combine?

I have the imbalanced dataset: data['Class'].value_counts() Out[22]: 0 137757 1 4905 Name: Class, dtype: int64 ...
hanzgs's user avatar
  • 163
3 votes
1 answer
261 views

Transfer Learning Question: Extending the Functionality of a Multipose-Estimation Machine Learning Model?

I have experimented with a number of different machine learning models used for pose estimation. Most of them output a heatmap and offsets for the detected person(s) in the image. I really like the ...
Josh Sharkey's user avatar
3 votes
0 answers
51 views

Information Extraction from image / text - approach?

I need assistance with a ML project I am currently trying to create. I receive a lot of invoices from a lot of different suppliers - all in their own unique layout. I need to extract 3 key elements ...
oliverbj's user avatar
  • 185
3 votes
1 answer
233 views

tensorflow pseudo inverse doesn't work for complex matrices!

The Tensorflow documentation here says that: tf.linalg.pinv is ''analogous to numpy.linalg.pinv. It differs only in default value of rcond''. However, tf.linalg.pinv requires the matrix to ...
eMichel's user avatar
  • 33
3 votes
0 answers
2k views

How to apply a groupby rolling function to create multiple columns in the dataframe

I am setting up a volume profile series over a stock data. I have implemented the market profile code from this github repo and the link to the data is here and the example here. Some Sample of data ...
Scrappy Coco's user avatar

1
2 3 4 5
42