Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

Keras model's embedding weight get NaN value

I am working on 3 categorical and 19 numerical features in which I plan to use trained embedding weights (from categorical features). After training, and get weights from embedding layers, I got NaN ...
0
votes
0answers
6 views

How can I use Ensemble learning of two models with different features as an input?

I have a fake news detection problem and it predicts the binary labels "1"&"0" by vectorizing the 'tweet' column, I use three different models for detection but I want to use ...
1
vote
0answers
12 views

How to make the values in different columns in the correct order based on another data frame (mapping) in Python Pandas

I am pretty new to Python and Pandas and I struggle with combining a messy dataframe from excel with a mapping. I have tried to find some solutions on the Internet, however with no success. My first ...
1
vote
0answers
10 views

Trouble with anomaly/novelty detection (on microscale) - need easy practical guide with Keras

I am relatively new to the field of machine learning. However, I already have solved simple image classification tasks with Keras (for example building CNNs and classifying MNIST...). The rough deep ...
3
votes
2answers
94 views

How to perform text classification on a dataset with many imbalanced classes

I am completely new to NLP and I have been tasked with performing text classification on a dataset containing 193k records. The number of classes is 107. The class with the highest number of records ...
0
votes
0answers
10 views

Aggregate grouped data conditionally over many columns doing different operations in Python/Pandas

I'm not sure this is a general coding question or not but I hope this is the correct forum. Consider the following reduced example data frame df: ...
1
vote
1answer
27 views

Scikit-learn's implementation of AdaBoost

I am trying to implement the AdaBoost algorithm in pure Python (or using NumPy if necessary)....
1
vote
2answers
33 views

How to get best data split from cross validation

I have trained a Random forest regressor which is giving me a rmse score of 70.72. But when I tried the same model in cross_val_score with a cv of 10, it gave me an array that looks something like ...
1
vote
0answers
15 views

Types of averages when analyzing sentences

I have a list of words and their frequencies in a text corpus. So there are words like "a", "what", "some" that have really high frequencies, and other like "...
0
votes
0answers
13 views

Tutorial fruits and vegetables segmentation and classification with python

There's a great number of tutorials for fruits and vegetables classification with python. But the neural network is trained to classify a single fruit. What if there is several fruits on the same ...
1
vote
1answer
25 views

How to fix my CSV files? (ValueError: Found array with 0 sample(s) (shape=(0, 1)) while a minimum of 1 is required)

I have tried to import two csv files into df1 and df2. Concatenated them to make df3. I ...
2
votes
0answers
23 views

Fuzzy Address Matching using Rapid Fuzz

I am using RapidFuzz for matching US Addresses from two separate datasets. I was able to get the results that I was hoping for using the below code: ...
-1
votes
0answers
26 views

Polynomial with degree 5 will have low bias and high variance

i wanna know that if Polynomial with degree 5 will have low bias and high variance will the model be under fitting or over fitting.
0
votes
1answer
14 views

does Gradient descent algorith always gives optimal solution? [closed]

Which gradient descent algorithm always gives optimal solution? Stochastic Gradient Descent Mini-Batch Gradient Descent Batch Gradient Descent All of the above
0
votes
0answers
6 views

Invalid value encountered in double_scalars and UndefinedMetricWarning: Precision is ill-defined

This question I asked in stack overflow but didn't receive any comments yet Question Link
2
votes
1answer
22 views

Reading the classification report of evaluation metric?

I am using the "classification_report" from: from sklearn.metrics import classification_report in order to evaluate a classification model. How can I ...
0
votes
0answers
11 views

How do I represent audio file in a format that can be saved in dataset and then can be used in modelling?

i used the following code to read my audio file from scipy.io import wavfile samplerate, data = wavfile.read("the path of my audio") but when i tried to ...
0
votes
0answers
20 views

How to put feature names from a 4D NumPy array into a pandas dataframe with a one to one relationship [closed]

I have a 4D NumPy array containing the features: importer, product, demand sector, and ...
0
votes
0answers
11 views

TypeError: import_optional_dependency() got an unexpected keyword argument 'errors' [closed]

I am trying to work with Featuretools to develop an automated feature engineering workflow for the customer churn dataset. The end outcome is a function that takes in a dataset and label times for ...
0
votes
0answers
6 views

cost function diverging in batch gradient descent

I am trying to implement the gradient descent method in python. I would like the calculation to stop when abs(J-J_new) reaches a certain tolerance level (i.e. it ...
0
votes
0answers
8 views

Calculating all near-tangential lines for inflection points of a time series

I have been thinking about this problem for a while and I'm curious if anyone knows of a good paper on this, has any ideas for algorithms or improvements to the framework. The task is to store the ...
1
vote
0answers
7 views

Building crop recommender system with past cultivation data not with ratings

I am planning to create a Crop recommendation system for farmers using the past ten years' crop cultivation data. it is a mobile application. whenever a farmer selects his location, the system will ...
0
votes
0answers
10 views

Model variance increases during training

I trained a regression model with lightgbm and the learning curve doesn't look good: The model variance increases during training, which shows a kind of overfitting. Now, I tried many ways to fix ...
0
votes
0answers
12 views

How to create custom stochastic layer in tensorflow 2.0?

Recently, I have been looking into Stochastic neural networks and would like to try creating one; however, I am not sure where to start. I have experience in Python and have been learning TensorFlow ...
1
vote
0answers
12 views

Adding additional classes in stanford NLP NER or Spacy

For stanford NER 3 class model, Location, Person, Organization recognizers are available. Is it possible to add additional classes to this model. For example : Sports as one class to tag sports names. ...
1
vote
0answers
27 views

CDF plot overlay histogram in python

I have a dataframe called df['ProgressStep'] I would like to get overlaid CDF plot in histogram. Have tried 2 methods, neither one meet my target perfectly. please help to fine tune the code, either ...
1
vote
0answers
14 views

Hidden integers being attached to my data in Azure ML

I've been trying to solve an issue with a piece of time data for a while now. I cannot convert it to DateTime using the Edit Metadata module, or turn it into a numeric value, or bin it. However, every ...
3
votes
1answer
43 views

Using output of ML algorithms as an input to different ML algorithm (Ensemble Learning)

I want to assign weights to multiple models and make an single ensemble model. I want to use my outputs as the input to a new machine learning algorithm and the algorithm will learn the correct ...
0
votes
0answers
7 views

How to use AraVec word embeddings to build text classification model?

I am working with ArSarcasm dataset from Hugging Face. I have cleaned the tweets from all the noise with a few preprocessing steps, tokenized the tweets and lemmatized the tokens (snippet of code can ...
2
votes
1answer
23 views

Learning a board game using a genetic neural network

I've never really done any practical machine learning, this is just a hobby for me. I'm trying to create a process using a neural network to learn the board game "7 Wonders." Here's how I ...
1
vote
1answer
25 views

Lung segmentation by Kmeans contains white border

I'm new to image processing, I'm trying to segment lung CT images by Kmeans by the following: ...
1
vote
1answer
24 views

Normalization with learning/test dataset in [0,1]

Say you split your data into two sets: training and test sets. You know that the inputs of your data are in [lower_bounds, upper_bounds]. Now, assume that you would like to do a min-max normalization ...
0
votes
0answers
6 views

Geospatial clustering plot with zoom in Python?

I need to construct an interactive clustering plot. Ideally as the user zooms in the clusters would split-up into smaller clusters at certain zoom levels. I am planning to have several discrete levels ...
1
vote
0answers
11 views

Classification for Ordinal labels - what tree-based methds can i use?

I have a label that has a natural ordering e.g. 0,1,2,3 where 0 is the worst activity measure and 3 is the best. For each label given by the model i need to also give the probability that it belongs ...
0
votes
0answers
16 views

Correlation between continuous variables and multi class categorical variables in python

I was trying to figure out a way of finding a correlation between continuous variables and a non-binary target categorical label. The only thing I though of is by fitting the labels into Multinomial ...
0
votes
0answers
9 views

Stemming/lemmatization for German words

I have a huge dataset of German words and their frequency in a text corpus (so words like "der", "die", "das" have a very high frequency, whereas terminology-like words ...
3
votes
1answer
45 views

Cosine similarity between sentence embeddings is always positive

I have a list of documents and I am looking for a) duplicates; b) documents that are very similar. To do so, I proceed as follows: Embed the documents using paraphrase-xlm-r-multilingual-v1. ...
3
votes
1answer
80 views

How to visualize a hierarchical clustering as a tree of labelled nodes in Python?

The chapter "Normalized Information Distance", visualizes a hierarchical clustering as a tree of nodes with labels: Unfortunately I cannot find out how to replicate this visualization, ...
1
vote
0answers
9 views

Node embeddings and densely connected graphs

I have a dataset of users and their interests (represented by categories) and I’m trying to embed the graph which results from connecting users if they have a common interest, so I’ll add an edge for ...
1
vote
1answer
19 views

Sampling Technique for mixed data type

I am looking for a very specific sampling technique which pertains to a very large dataset with mixed data type i.e, I have categorical as well as continous variables and want to have a sample that ...
0
votes
0answers
29 views

ValueError: `validation_split` is only supported for Tensors or NumPy arrays, found following input: RaggedTensor

I have following inputs to be trained on a CNN: x = np.array(Images) y = [ [[0]], [[76., 5., 9., 1., 0., 0.], [54., 4., 10., 51.]] ] Since the ...
1
vote
1answer
26 views

How are the values for the `sex` feature in sklearn Diabetes dataset obtained?

I'm just starting out with using sklearn for my own Machine Learning project and I'm using sklearn's built-in "Diabetes&...
1
vote
1answer
13 views

How to set a breakpoint inside a custom metric function in keras

I am trying to write my own custom metric functions in keras and I wanted to start with a test function so I implemented a f1_score function using sklearn, next I will need to customize the ...
0
votes
0answers
21 views

How can I compute the AUC by using Gaussian Mixture Model?

By using this code, can I compute the AUC: ...
0
votes
0answers
5 views

Read 'n' no of sheets from excel and create multiple dcc.Tabs on dashboard using 'for loop' in python [closed]

What I am trying to do? I want to build a dashboard with following: Read tables from excel sheets (It has n number of sheets). Ex: Sheet1, Sheet2. Each sheet has 1 table. Ex: Sheet1 has Table1, ...
0
votes
0answers
9 views

Memory problem when running Tensorflow with Flask and Docker

I've trained a Tensorflow object detection model and want to deploy it with Flask. My problem is that with every new request the memory used by docker container rises by ~100mb, which is not freed ...
0
votes
0answers
9 views

Keras LSTM Time series output not match to the actual data

I use LSTM model in Keras to predict time series data. I use MinMaxScaler to normalize data and create model like this code. ...
0
votes
1answer
18 views

Taking the squared Euclidean distance for kNN classification of images

A problem I'm working on states: Computes the squared Euclidean distance between each element of the training set and each element of the test set. Images should be flattened and treated as vectors. ...
0
votes
1answer
22 views

Concatenate two tensors of different shape

I have two tensors: ...

1
2 3 4 5
113