Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

1,504 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6
votes
1answer
34 views

How to add data points to a trained KNN

I have a trained KNN, I created with https://github.com/kevinzakka/blog-code/blob/master/knn/knn.py I want to add more data points to the KNN but I am on a raspberry pi so limited by RAM and ...
5
votes
0answers
77 views

Are there any graph embedding algorithms like this already?

I wrote an algorithm for generating node embeddings based on the graph's topology. Most of the explanation is done in the readme file and the examples. The question is: Am I reinventing the wheel? ...
5
votes
0answers
1k views

Why is my Keras model not learning image segmentation?

Edit: as is turns out, not even the model's initial creator could successfully fine-tune it. This is most likely a problem of implementation, or possibly related to the non-intuitive way in which the ...
5
votes
1answer
350 views

Find matching text from a text column

This is my first time to use Data Analytics tool to figure out a solution to a problem. I have a table with following columns ...
5
votes
1answer
235 views

Binary (Unary) Recommendation System with Biased Views

I would like to create a content recommendation system based on binary click data that also takes views into account. What content a user has been exposed to, and therefore has the chance to click ...
5
votes
1answer
2k views

Recurrent neural network multiple types of input Keras

For a project I want to use recurrent neural networks, however my knowledge on this subject is still somewhat limited. I do have some experience with convolutional nets and traditional neural networks....
4
votes
1answer
39 views

weightage attributes

If I have a dataset with binary classification and has many attributes with value of (0 or 1) means the occurrence of attribute is represented by 1 and absence is represented by 0, can I add weight ...
4
votes
2answers
229 views

Trained Tensorflow model performs poorly on inference

I trained an image classification model using Keras with Tensorflow backend. The model got good accuracy on validation dataset as well as on the testing data, I save the entire model to ...
4
votes
1answer
97 views

How to predict value in every 120 minutes using LSTM in python

I want to predict value in every 120 minutes continuous using LSTM model. Here I wrote the code for predction. But I'm not getting proper prediction values . Here from start time I need to predict ...
4
votes
2answers
174 views

Text topic classification in tensorflow

I want to create a CNN in tensorflow that does the following: Classify a recipe headline and find out the topic. For instance <...
4
votes
1answer
728 views

Implementing spatio-temporal convolutions in pytorch

I am trying to implement a layer to perform the (2+1)D convolutions described in this paper: https://arxiv.org/pdf/1711.11248.pdf The basic idea is as follows: Let's say I have a 3D convolutional ...
4
votes
1answer
3k views

Memory error on using data generator in keras

I am using the following augmentations on dataset of size 9 GB: ...
4
votes
0answers
71 views

Gridsearch XGBoost for ensemble. Do I include first-level prediction matrix of base learners in train set?

I'm not quite sure how I should go about tuning xgboost before I use it as a meta-learner in ensemble learning. Should I include the prediction matrix (ie. df containing columns of prediction results ...
4
votes
2answers
11k views

Multivariate Time-Series Clustering

I have a streaming data along with timestamp dataset that looks like this: 1.png Timestamp can be inclusive of "seconds" too, but the data may or may not change every second. it depends on the ...
4
votes
0answers
1k views

In XGBoost, how to change eval function and keeping same objective?

I want to keep objective as "reg:linear" and eval_metric as customised rmse as follows. ...
4
votes
1answer
423 views

How to implement patternet in python as it is in matlab?

NOTE: This question was first posted in cross-validated website but I was instructed to move it off that website as it was not a good fit. I am new in implementation of machine learning, neural ...
4
votes
0answers
10k views

Tuning Gradient Boosted Classifier's hyperparametrs and balancing it

I am not sure if it is a correct stack. Maybe I should have put my question into crossvalidated. Nevertheless, I perform following steps to tune the hyperparameters for a gradient boosting model: ...
4
votes
0answers
1k views

How does Seaborn calculate error bars when using estimators other than the arithmetic mean?

If I create a barplot using Seaborn and specify the geometric mean or the median as the estimator, does Seaborn know to use the appropriate standard error formula to create error bars?
4
votes
2answers
44 views

Cluster elements that appear in the same lists

Suppose I have a multitude of sets with (unordered) combinations of elements and I want to determine which elements tend to appear together. For example Given the following sets: ...
3
votes
0answers
17 views

Serializing a trained classification model into a set of actionable insights

I'm looking for ways to convert a trained classification model into a list of insights based on the resulting parameters of the model. To make an example, let's assume we trained a decision tree to ...
3
votes
0answers
24 views

sum of occurrences in bins

I'm looking at popularity of food stalls in a pop-up market: ...
3
votes
1answer
74 views

tensorflow pseudo inverse doesn't work for complex matrices!

The Tensorflow documentation here says that: tf.linalg.pinv is ''analogous to numpy.linalg.pinv. It differs only in default value of rcond''. However, tf.linalg.pinv requires the matrix to ...
3
votes
1answer
33 views

Data prediction using scikit-learn and a list

I have a group of lists detailing temperatures over differing amounts of time. My goal is to use machine learning to identify periods in which a machine is turned on and off, where turning on the ...
3
votes
0answers
22 views

Improving a simple trig model

I have some data which I know is well approximated as a trig function, and I can fit it with scipy.optimize.curve_fit as follows: ...
3
votes
1answer
52 views

Is there a real C4.5 implementation in Python ? (handling missing value)

To my understanding, C4.5 comes with 4 improvements compared to ID3: Handling missing values in both training data and "test" data, Handling continuous data Handling costs on attributes. The pruning ...
3
votes
0answers
65 views

Unable to transform (greatly performing) Autoencoder into Variational Autoencoder

Following the procedure described in this SO question, I am trying to transform my (greatly performing) convolutional Autoencoder into a Variational version of the same Autoencoder. As explained in ...
3
votes
0answers
1k views

Keras - Implementation of custom loss function with multiple outputs

I am trying to replicate (a way smaller version) of the AlphaGo Zero system. However, in the network model, I am having a problem. The loss function I am supposed to implement is the following: $$l = ...
3
votes
1answer
55 views

The output of Model from Decision Tree and Random Forests are different?

I have been made a model using both Decision Tree and Random Forest. But, when I tried to test the model on the same DataFrame the output is different. How is this possible? The data file from my ...
3
votes
0answers
108 views

Gensim LDA model: return keywords based on relevance (λ - lambda) value

I am using gensim library for topic modeling, more specifically LDA. I have created my corpus, my dictionary and my lda model, and with the help of pyLDAvis library I visualize the results. When I ...
3
votes
1answer
169 views

Fast Python implementation of the gradient descent

I'm looking for fast Python implémentations of gradient descent optimization algorithm. I have a convex problem , with no constraint, so for now I'm using the BFGS algorithm implemented in scikit-...
3
votes
1answer
224 views

SelectKBest and Correlation returns me excatly same feature selection. How?

Im working on selecting most effective features from a dataset with over that 2000 features. Im using different algorithms for that (selectKBest with chi-square, Extra Trees, Correlation etc.) But ...
3
votes
1answer
1k views

How do I create a Keras custom loss function for a one-hot-encoded binary classifier?

I have a CNN binary classifier with one-hot-encoded labels that I've written using Keras and it's just not training to the metric I want to encourage. My data is very imbalanced (91% class 0, 9% class ...
3
votes
3answers
3k views

Input contains NaN, infinity or a value too large for dtype('float32'). Without ruining Dataset

I have a datasheet that is all in Binary but sometimes there are missing cases. In one of the inputs, there is a blank, I don't want to completely remove the whole column because some of the rows have ...
3
votes
0answers
209 views

Pyspark: Filter dataframe based on separate specific conditions

How can I select only certain entries that match my condition and from those entries, filter again using regex? For instance, I have this dataframe (df) ...
3
votes
1answer
535 views

Advice on imputing temperature data with StatsModels MICE

This may be a dumb question but I can't figure out how to actually get the values imputed using StatsModels MICE back into my data. I have a dataframe (dfLocal) with hourly temperature records for ...
3
votes
0answers
77 views

Feature scaling worsens performance?

I noticed that feature scaling can destroy completely neural networks performance in some cases. Below are my results that you can reproduce easily. I use a neural network to approximate function $g$ ...
3
votes
0answers
258 views

Adjust class weights due to class imbalance and class importance Multi class classification XGBoost

With respect to this question and the answer given by @Esmailian, Would anyone be able to let me know if Class B has a higher importance or the positive class ( i.e. it needs to have a higher ...
3
votes
0answers
54 views

Is there an algorithm that imputes missing values based on n nearest columns? (KNN hybrid)

I have a dataset of 70 columns that have missing values. Each column has a few columns (3-5) that it is significantly more correlated than the others but each column's correlated columns are very ...
3
votes
2answers
76 views

Is ARIMA appropriate for time series prediction involving a mix of explanatory and independent variables?

I have a table with the following columns: Date(Month,Year), Sold_Past_Month, Quantity_Available, Quantity_Shipping_In, Missed_Sales, Quantity_Needed Quantity_Needed is the dependent variable that ...
3
votes
1answer
182 views

How can I detect partially obscured objects using Python?

I'm building a computer vision application using Python (OpenCV, keras-retinanet, tensorflow) which requires detecting an object and then counting how many objects are behind that front object. So, ...
3
votes
0answers
757 views

What are the differences between Glueviz and Orange that come installed with Anaconda and how do they compare on performance and workflows?

I've been using Anaconda for quite a while now and these two libraries glue and Orange have sparked interest lately. I've ...
3
votes
2answers
144 views

Error plotting with datetime and value using matplotlib in python

Here I have a dataset with three values. I plot these three values in one graph using python. While I plot the graph first plot line is not coming properly. Because start point and end point combined ...
3
votes
1answer
117 views

Pandas DataFrame Rollup Error

I am trying to compute Percentage Contributions on a pandas dataframe. I have a data frame which has three columns (Buyer, ...
3
votes
1answer
190 views

ML regression poor performance

I am experimenting with 3 years time series electrical demand data (kW) for a building and attempting to create regression supervised ML models from sci kit learn regressor algorithms but I have very ...
3
votes
2answers
744 views

How to get probabilities values with keras?

tensorflow version = '1.12.0' keras version = '2.1.6-tf' I'm using keras with tensorflow backend. I want to get the probabilities values of the prediction. I want the probabilities to sum up to 1. ...
3
votes
0answers
964 views

Using the Python Keras multi_gpu_model with LSTM / GRU to predict Timeseries data

I'm having an issue with python keras LSTM / GRU layers with multi_gpu_model for machine learning. When I use a single GPU, the predictions work correctly ...
3
votes
0answers
508 views

FREAK feature extraction OpenCV

I want to know how to use FREAK feature extraction in python, I read the documentation but I need some examples. I wrote this code : ...
3
votes
1answer
66 views

How to get the survival duration prediction for each individual in the data by using the Kaplan-Meier method?

I am trying to learn how to use the Kaplan-Meier survival estimator model in the lifelines package. The documentation says that the KaplanMeierFitter.fit function ...
3
votes
0answers
133 views

PCA and FastICA in scikit-learn giving near identical results

So after importing my data, transforming it, and splitting into training and test sets I tried running this script for PCA: ...
3
votes
0answers
2k views

ValueError: Numpy arrays that you are passing to your model is not the size the model expected

I am trying to perform concatenation on the Bidirectinal LSTM layer. I have my model defined like this: ...

1
2 3 4 5
31