Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (which should be asked on Stack Overflow).

2,141 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
9 votes
2 answers
259 views

Are there any graph embedding algorithms like this already?

I wrote an algorithm for generating node embeddings based on the graph's topology. Most of the explanation is done in the readme file and the examples. The question is: Am I reinventing the wheel? ...
user avatar
8 votes
0 answers
2k views

Why is my Keras model not learning image segmentation?

Edit: as is turns out, not even the model's initial creator could successfully fine-tune it. This is most likely a problem of implementation, or possibly related to the non-intuitive way in which the ...
user avatar
  • 189
7 votes
0 answers
1k views

Find missing object(s) in image with a priori knowledge about the missing object(s) (w.r.t base image)

Problem Statement: I am working on developing a method, or borrow/modify/combine existing ones, where given an golden image (reference or base with all expected objects to be present), it is able to ...
user avatar
  • 4,036
7 votes
0 answers
776 views

Gensim LDA model: return keywords based on relevance (λ - lambda) value

I am using the gensim library for topic modeling, more specifically LDA. I created my corpus, my dictionary, and my LDA model. With the help of the pyLDAvis library I visualized the results. When I ...
user avatar
7 votes
0 answers
2k views

Python : Feature Matching + Homography to find Multiple Objects

I'm trying to use OpenCV via Python to find multiple objects in a train image and match it with the key points detected from a query image. For my case, I'm trying to detect the tennis courts in the ...
user avatar
  • 71
6 votes
2 answers
1k views

How to perform feature selection on dataset with categorical and numerical features?

I am working on a dataset with 30 columns (29 numerical, 1 non-ordinal categorical). I hot-encoded the categorical feature and reached at 35 columns. To improve training efficiency, I want to perform ...
user avatar
6 votes
1 answer
7k views

Keras - Implementation of custom loss function with multiple outputs

I am trying to replicate (a way smaller version) the AlphaGo Zero system. However, in the network model, I am having a problem. The loss function I am supposed to implement is the following: $$l = (z -...
user avatar
6 votes
2 answers
144 views

Gridsearch XGBoost for ensemble. Do I include first-level prediction matrix of base learners in train set?

I'm not quite sure how I should go about tuning xgboost before I use it as a meta-learner in ensemble learning. Should I include the prediction matrix (ie. df containing columns of prediction results ...
user avatar
  • 161
6 votes
0 answers
1k views

Fine tuning accuracy lower than Raw Transfer Learning Accuracy

I've used transfer learning on Inception V3 with ImageNet weights on Keras with Tensorflow backend on python 2.7 to create an image classifier. I first extracted and saved the bottleneck features from ...
user avatar
  • 61
6 votes
1 answer
2k views

In XGBoost, how to change eval function and keeping same objective?

I would like to keep the objective as "reg:linear" and eval_metric as customized RMSE as follows: ...
user avatar
5 votes
3 answers
142 views

What Clustering Method Should I Use?

My data is a group of 10 thousand points (each having an node location (x,y)) that are spread across a plane. They are also chromatically-colored based on their weight. I need to finalize a bayesian ...
user avatar
5 votes
2 answers
2k views

Triangle Pattern Recognition on Financial Market with Python

I'm working on a personal project to find Triangles on any stock in Python. I detect the max and min points (shift(-5,+5) because if I consider only shift(-1+1) I have a lot of lines) and write lines ...
user avatar
5 votes
0 answers
115 views

Unable to transform (greatly performing) Autoencoder into Variational Autoencoder

Following the procedure described in this SO question, I am trying to transform my (greatly performing) convolutional Autoencoder into a Variational version of the same Autoencoder. As explained in ...
user avatar
5 votes
2 answers
2k views

Multidimensional scaling producing different results for different seeds

I took the data from here and wanted to play around with multidimensional scaling with this data. The data looks like this: In particular, I want to plot the cities in a 2D space, and see how much it ...
user avatar
5 votes
1 answer
495 views

How can I detect partially obscured objects using Python?

I'm building a computer vision application using Python (OpenCV, keras-retinanet, tensorflow) which requires detecting an object and then counting how many objects are behind that front object. So, ...
user avatar
5 votes
0 answers
1k views

Using the Python Keras multi_gpu_model with LSTM / GRU to predict Timeseries data

I'm having an issue with python keras LSTM / GRU layers with multi_gpu_model for machine learning. When I use a single GPU, the predictions work correctly ...
user avatar
5 votes
2 answers
2k views

How to deal with missing data for Bernoulli Naive Bayes?

I am dealing with a dataset of categorical data that looks like this: ...
user avatar
  • 151
5 votes
0 answers
11k views

Tuning Gradient Boosted Classifier's hyperparametrs and balancing it

I am not sure if it is a correct stack. Maybe I should have put my question into crossvalidated. Nevertheless, I perform following steps to tune the hyperparameters for a gradient boosting model: ...
user avatar
4 votes
3 answers
553 views

Is it a best practice to exclude retweets from the data set?

I am going to build machine learning algorithm to identify fake tweets. The data set has huge retweets which I think might be an issue. Do you think given that the focus is the original tweet, it is ...
user avatar
4 votes
2 answers
2k views

Pytorch - Loss is decreasing but Accuracy not improving

It seems loss is decreasing and the algorithm works fine. But accuracy doesn't improve and stuck. ...
user avatar
4 votes
2 answers
6k views

Saving and loading keras.callbacks.History object with np.save and np.load

I have been saving my training history in keras as follows: ...
user avatar
4 votes
1 answer
290 views

Feature Engineering Lists\Vectors as values in dataframe

Let's say I have a dataframe where some of the columns have lists of strings as values. I would like to use ML Algorithms on this dataframe. In this case, I can: I could add many columns of 1's and ...
user avatar
4 votes
1 answer
963 views

Calculating saliency maps for text classification

I'm following the text classification with movie reviews TensorFlow tutorial, and wanted to extend the project by looking, for a certain input, which words influenced the classification the most. I ...
user avatar
4 votes
1 answer
811 views

Scikit-learn average_precision_score() vs. auc score of precision_recall_curve()

I've been searching around for an explanation to this, and haven't come across one yet- in scikit-learn, when I compute the auc() of the ...
user avatar
4 votes
2 answers
218 views

Benefits of using Deep Learning-specific hyperparameter optimization tools vs. sklearn?

There are quite a few library for hyperparameter optimization that are specific to Keras or other Deep Learning libraries, like Hyperas or Talos. My question is, what's the main benefit of using ...
user avatar
4 votes
1 answer
359 views

Multiple activation functions with TensorFlow estimator DNNClassifier

I just want to know if is it possible to use tf.estimator.DNNClassifier with multiple different activation functions. I mean, could I use a DNNClassifier estimator which use different activation ...
user avatar
4 votes
1 answer
329 views

Why is my U-matrix visually not separating the classes?

I am visualizing the U-matrix generated using a Self Organizing Map codebook to (visually) identify regions of similarity in the data. Although I would like to use SOM to identify clusters in ...
user avatar
4 votes
1 answer
924 views

What preprocessing steps to be followed before image comparison?

1 down vote favorite For example I am trying to find the similarity between two images using skimage - SSIM. The code block will be as follows ...
user avatar
3 votes
1 answer
43 views

Reinforcement Learning applied to Optimisation Problem

Problem Statement: We are given an optimisation problem; with production centres, source airport, destination airports, transfer points and finally delivered to the customers. This is better explained ...
user avatar
  • 31
3 votes
0 answers
120 views

Struggling to understand/implement Transformer Decoder

I'm struggling to understand the decoder in a Transformer model, specifically with regards to some aspects of its architecture as well as how it actually handles the data during training. What I have ...
user avatar
  • 131
3 votes
0 answers
191 views

Is it possible to increase the number of images of one class using data augmentation, which is not applied on the other class, in the same dataset?

I have 2 classes for my image classification problem, say class A and class B, and I am using tensorflow and keras for the same. One of them have around 5K images while the other have just around 2K ...
user avatar
3 votes
2 answers
378 views

Why does my random forest classifier predicts one class more often?

I have a random forest classifier that predicts 0 class about twice as often as class 1. It also predicts class 0 with higher probabilities than class 1. It is not a imbalanced dataset. I tried ...
user avatar
3 votes
0 answers
315 views

What exactly negative/positive value of Captum's Integrated Gradient mean?

I use Captum's Integrated Gradient to interprete my PyTorch's neural network. I know that from github and original paper mentioned that ... Positive attribution score means that the input in that ...
user avatar
  • 31
3 votes
1 answer
94 views

What parameters to use when normalising training, validation, and testing data?

I know a similar post was made here, but I wanted to ask some follow up questions. I am conducting a cross-validation search to find values of a set of hyper-parameters and need to normalise the data. ...
user avatar
3 votes
1 answer
76 views

Issues with self-implemented logistic regression

I am trying to self-implement a logistic regression algorithm to do some self-learning but I am having a bit of trouble with achieving similar accuracy to the logistic regression of sklearn. Here is ...
user avatar
3 votes
1 answer
114 views

Is it possible to solve Rubik's cube using DQN?

I'm trying to solve Rubik's cube using deep learning and I came across with DQN, so I decided to give it a try. I developed all the code and started training but I got this results: Loss goes up and ...
user avatar
3 votes
1 answer
802 views

Error when trying Transfer Learning

I'm trying to train a model which is an extension of Google's Inception-V3 for the purpose of recognizing and classifying whether there is any pneumonia using x-ray images. I've used Tensorflow-Hub ...
user avatar
3 votes
1 answer
370 views

Character-level embeddings in python

I'm working on an NLP task that requires the use of character level embeddings, and I've been trying to use Spacy. However, it seems that spacy uses word-level embeddings for the word vectors, and I ...
user avatar
  • 163
3 votes
0 answers
31 views

Serializing a trained classification model into a set of actionable insights

I'm looking for ways to convert a trained classification model into a list of insights based on the resulting parameters of the model. To make an example, let's assume we trained a decision tree to ...
user avatar
  • 63
3 votes
0 answers
640 views

Balancing the dataset using imblearn undersampling, oversampling and combine?

I have the imbalanced dataset: data['Class'].value_counts() Out[22]: 0 137757 1 4905 Name: Class, dtype: int64 ...
user avatar
  • 163
3 votes
1 answer
206 views

Transfer Learning Question: Extending the Functionality of a Multipose-Estimation Machine Learning Model?

I have experimented with a number of different machine learning models used for pose estimation. Most of them output a heatmap and offsets for the detected person(s) in the image. I really like the ...
user avatar
3 votes
0 answers
27 views

Information Extraction from image / text - approach?

I need assistance with a ML project I am currently trying to create. I receive a lot of invoices from a lot of different suppliers - all in their own unique layout. I need to extract 3 key elements ...
user avatar
  • 175
3 votes
1 answer
179 views

tensorflow pseudo inverse doesn't work for complex matrices!

The Tensorflow documentation here says that: tf.linalg.pinv is ''analogous to numpy.linalg.pinv. It differs only in default value of rcond''. However, tf.linalg.pinv requires the matrix to ...
user avatar
  • 33
3 votes
0 answers
2k views

How to apply a groupby rolling function to create multiple columns in the dataframe

I am setting up a volume profile series over a stock data. I have implemented the market profile code from this github repo and the link to the data is here and the example here. Some Sample of data ...
user avatar
3 votes
1 answer
378 views

Error while plotting Logistic Regression Classification

I was trying to plot by using the following code ...
user avatar
3 votes
2 answers
141 views

Machine learning model with simultaneous function optimization

Consider the following scenario. I am a sculpturer and customers ask me for what price I am willing to provide them with some statues. Their request for sculptures can vary in difficulty, quantity, ...
user avatar
  • 31
3 votes
0 answers
39 views

Improving a simple trig model

I have some data which I know is well approximated as a trig function, and I can fit it with scipy.optimize.curve_fit as follows: ...
user avatar
3 votes
1 answer
975 views

find bigrams in pandas

I have a DataFrame with 4 columns: 'Headline', 'Body_ID', 'Stance', 'articleBody', with 'Headline' and 'articleBody containing cleaned and tokenized words. I want to find bi-grams using nltk and have ...
user avatar
  • 71
3 votes
2 answers
107 views

How can we create an label, value detector?

I am trying to implement an text detector using MaskRCNN such that the model detects the label and value as shown in the image below. Detecting the same is easier for fields like page date and order ...
user avatar
  • 81
3 votes
1 answer
1k views

Why does classifier chain ask for at least 2 classes, when I have it

I'm using Classifier Chain with logistic regression and when i try to use fit, i get This solver needs samples of at least 2 classes in the data, but the data contains only one class: 1 but I'm ...
user avatar

1
2 3 4 5
43