Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
1 vote
0 answers
5 views

Is there anyway to classify the category on give amazon reviews using python

I am trying to find a model or way to classify text which falls into a category and its a positive or negative feedback. For ex. we have three columns Review : Camera's not good battery backup is not ...
user avatar
0 votes
0 answers
7 views

How to train ML model for time series data

I am trying to build a machine learning model in python. I used pytorch and sklearn to make the model. My model is a bit ...
user avatar
  • 101
0 votes
1 answer
12 views

Using Sci-Kit Learn Clustering and/or Random-Forest Classification on String Data with Multiple Sub-Classifications

I have a set of data with some numerical features and some string data. The string data is essentially a set of classes that are not inherently related. For example: ...
user avatar
-2 votes
1 answer
11 views

Why Do a Set of 3 Clusters Across 1 Dimension and a Set of 3 Clusters Across 2 Dimensions Form 9 Apparent Clusters in 3 Dimensions?

I am sorry if this is a well-known phenomenon but I can't quite wrap my head around this. I have a related question: How To Develop Cluster Models Where the Clusters Occur Along Subsets of Dimensions ...
user avatar
0 votes
0 answers
9 views

Accuracy drops when adding a fully connected layer for dimensionality reduction to a ResNet50

I'm training a ResNet50 for image classification and I'm interested in decreasing the dimensionality of the embedded layer, in order to apply some clustering techniques. The suggested dimension is ...
user avatar
  • 101
0 votes
1 answer
13 views

Can anyone tell me how can I get the following output?

Here is my code; ...
user avatar
1 vote
1 answer
13 views

Random search grid not displaying scoring metric

I want to do a grid search of some few hyperparameters through a XGBClassifier of a binary class, but whenever i run it the score value (roc_auc) is not being ...
user avatar
0 votes
0 answers
5 views

In Python, how can I transfer/remove duplicate columns from one dataset, such that the rows and columns of all datasets would be equal?

So I've been trying to improve my Random Decision Tree model for the Titanic Challenge on Kaggle by introducing a Validation Dataset, and now I encounter this roadblock, as shown by the images below: ...
user avatar
0 votes
0 answers
10 views

How to solve this ValueError: Dimensions must be equal

I'm trying to train an autoencoder model with colored image samples but I got this error ...
user avatar
  • 1
0 votes
0 answers
5 views

ML methods for vector correlation

I am dealing with a timeseries consisting of input flow sampled every 5 minutes over 441 days. My aim is to find any possible correlation from data coming from: The same day of the week The same ...
user avatar
0 votes
1 answer
17 views

Need an example of a custom class whose instance is fed to sklearn Pipeline / make_pipeline to use with GridSearchCV

According to sklearn.pipeline.Pipeline documentation, the class whose instance is a pipeline element should implement fit() and transform(). I managed to create a custom class that has these methods ...
user avatar
  • 131
0 votes
0 answers
5 views

OpenCV warpAffine error during image augmentation using Albumentations

I have been trying to do image augmentation using a library called Albumentations. But I got some error from OpenCV while transforming the images. I ran the code below on Kaggle's notebook. The ...
user avatar
  • 1
0 votes
0 answers
12 views

Neural network model in Google Colab significantly slower with increased data points

I've made a neural network in google colab. The original data contained 12 000 rows (csv 350KB). 4 features and 2 labels. Each epoch would run within a few ms. I tried to increase the amount of data ...
user avatar
  • 101
0 votes
1 answer
13 views

How to solve MemoryError problem

I've created and normalized my colored image dataset of 3716 sample and size 493*491 as x_train, its type is list I'm tring to convert it into numpy array as follows ...
user avatar
  • 1
0 votes
0 answers
10 views

How to match the input image with the ground truth image (the order)

I'm a beginner learning deep learning and trying to do semantic segmentation problems on histologic image using python and TensorFlow. There is 2 main file : Images.npy and Labels.npy (Ground truth). ...
user avatar
  • 1
0 votes
0 answers
13 views

How to optimize Pandas DataFrame reorganizing?

I have a DataFrame which looks like this: date person value 2022-05-01 A 5 2022-05-01 B 4 2022-05-02 A 5 2022-05-02 B 9 I want to convert it to that form: ...
user avatar
0 votes
0 answers
8 views

semantic segmentation using kmeans or mean shift

i know what semantic segmentation is and i know how to do semantic segmentation using deep learning but my question here can i do semantic segmentation with a traditional way like kmeans or mean shift ...
user avatar
  • 1
0 votes
0 answers
13 views

How to optimize hyperparameters in Bert?

I am using the BERT model in order to classify stereotypes in sentences. I wanted to know if there is a way to automate the optimization of hyperparameters such as 'epochs', 'batchs' or 'learning rate'...
user avatar
  • 320
0 votes
0 answers
12 views

Python: How to construct a joyplot with values taken from a column in pandas dataframe as y axis

I have a dataframe df in which the column extracted_day consists of dates ranging between 2022-05-08 to 2022-05-12. I have another column named gas_price, which consists of the price of the gas. I ...
user avatar
1 vote
1 answer
22 views

High Performance Classification or Similarity Algorithim for Mixed Data Types?

I have a database holding 10-ish features that describe different breeds of dogs. They are mostly categorical features, but some provide ranges for values. Here's a demo representation of the database,...
user avatar
0 votes
0 answers
4 views

After a certain number of moves my tictactoe program raise an exception

I'm trying to build an AI to play tictactoe(cs50ai pset0). i have built 7 essential functions for this purpose. 1.player function that takes as an argument a board and returns whose turn is it. ...
user avatar
  • 1
0 votes
1 answer
10 views

expanding 1x1xn global average pooling output to the size of the input map HxWxn

I am trying to recreate the model in this paper : BiSeNetV2 There is one module called Context Embedding Block, in which Global Average Pooling is used to embed global information into the feature ...
user avatar
0 votes
0 answers
16 views

Dealing with near duplicates using NLP

I have a dataframe like as shown below ...
user avatar
  • 2,129
0 votes
0 answers
6 views

Manual garbage collection throwing KeyErrors using Python inside Azure Function

I have a python script that I push up to an Azure Function App (It's specifically a blob storage trigger). My function app is hitting memory caps every so often, and I thought it might be a good idea ...
user avatar
0 votes
1 answer
15 views

Keras Callback to save all the values of each batch per epoch

I would like to gather all the values during training for each epoch. When using the fit function of tensorflow I only receive the last value after run through all ...
user avatar
2 votes
0 answers
57 views

random_state on train_test_split() appears to have large effect in performance metrics?

To summarize the problem: I have a data set with ~1450 samples, 19 features and a binary outcome where classes are fairly balanced (0.51 to 0.49). I split the data into a train set and a test set ...
user avatar
  • 31
0 votes
0 answers
5 views

Constant trend for test error

I'm using keras functional API in order to model a multi-input/multi-output relation in the form (y1,y2) = f(x1,x2,x3). I have a really simple architecture composed by 2 hidden layer of 64 nodes and ...
user avatar
0 votes
0 answers
16 views

Song playlist recommendation system

I want to build a recommender system to suggest similar songs to continue a playlist (similar to what Spotify does by recommending similar songs at the end of a playlist). I want to build two models: ...
user avatar
  • 1
1 vote
3 answers
95 views
+50

How To Develop Cluster Models Where the Clusters Occur Along Subsets of Dimensions in Multidimensional Data?

I have been exploring clustering algorithms (K-Means, K-Medoids, Ward Agglomerative, Gaussian Mixture Modeling, BIRCH, DBSCAN, OPTICS, Common Nearest-Neighbour Clustering) with multidimensional data. ...
user avatar
1 vote
1 answer
57 views

How to use confidence labels?

I have 2 sets of training data in csv files. The training data have class labels, 1 for memorable, and 0 for not memorable. In addition, there is also a confidence label for each sample. The class ...
user avatar
1 vote
0 answers
25 views

Multi Linear Regression on String Values

I'm using datasets which involves mostly of string values. The main outcome of the project is that it should predict success. Now I can use OneHotEncoding to convert string values in numerical format ...
user avatar
0 votes
0 answers
4 views

Stale element reference exception when webscraping

I am trying to scrape a website and have written up a working script. The problem is that after some time running the script I get the stale element reference exception telling me the referenced ...
user avatar
  • 1
0 votes
1 answer
13 views

How to count words in a dataframe?

I would like to count how many Male and female who answer (ex. Biking / Cycling). Below is the sample data:
user avatar
0 votes
0 answers
15 views

pytorch resize and pad image

Does torch have the same functionality as tf.image.resize_with_pad? I need to resize a picture and pad it with zeros to a wanted size. E.g: ...
user avatar
3 votes
2 answers
38 views
+50

Approaches to fit a theoretical model on a physical device

Happy to join this community. Thank you in advance for your kind help! :) Intro I have a physical device characterized by its internal parameters, of which I know the nominal values. I also have the ...
user avatar
0 votes
0 answers
22 views

Python: calculate the average correlation coefficient

I am calculating the volatility (standard deviation) of returns of a portfolio of assets using the variance-covariance approach. Correlation coefficients and asset volatilities have been estimated ...
user avatar
  • 101
0 votes
0 answers
14 views

write a dataframe using a column that contains a cycle (number of the weekin every year) in python

I want to write a dataframe with a column of the number of week and make it as a row that contains 53 columns (I mean weeks). I really need your help. NOTE THAT: each year has 53 week, and the column ...
user avatar
  • 1
1 vote
0 answers
6 views

Evaluation of the preprocessing to make a dataset anonymous

I have a very huge dataset from the NLP area and I want to make it anonymous. Is there any way to check if my pre-processing is correct? Generaly, is there any way to evaluate how good is the pre-...
user avatar
0 votes
0 answers
10 views

NaNs in predictions with LSTM

I have an LSTM model that I have trained and tested it with a dataset. Now I want to test it to an other dataset and I use the following snippet: ...
user avatar
  • 101
0 votes
1 answer
18 views

Input shape error

I've this item: ['6', '1', '6', '843537', '3', '0', '5', '1006709', '3', '1', '4'] with shape: (11,) but when go to predict with: ...
user avatar
0 votes
0 answers
12 views

use pre-built model to predict audio file

i am using this code to build a model that recognizes emotions in speech, but i can't figure out how to use it after loading it in a new python file. this is what i have but the results are always the ...
user avatar
-1 votes
2 answers
16 views

could not convert string to float: 'YELLOW'

...
user avatar
0 votes
0 answers
12 views

Time series classification using multiples multivariate multi-length timeseries

0 I would like to develop a time series classification algorithm to classify use a of parachute. My data consist of multiple recording files (around 5min at 100hz, length of the recording vary) with a ...
user avatar
0 votes
0 answers
11 views

Federated Learning: some clients with 0% accuracy

Suppose that I am doing a Federated Learning experiment using MNIST. As you know MNIST has 10 classes. Now, Federated Learning is useful especially in cases like hospitals, for collaborations, because ...
user avatar
1 vote
1 answer
24 views

Tensor dot product with rank one tensor from vector

I'm trying to compute an inner product between tensors in numpy. I have a vector $x$ of shape (n,) and a tensor $y$ of shape ...
user avatar
  • 11
1 vote
1 answer
31 views

Imbalanced Binary Dataset in Keras. Finding the best threshold after fit s.t. sensitivity and specificity is maximized?

I have made an ANN in Keras that works on an imbalanced binary dataset. The data is used after fitting the model to predict the binary classes and I want to choose a threshold s.t. sensitivity and ...
user avatar
1 vote
0 answers
24 views

How to draw each ROC curve of an SVM model with cross validation

I would like to make a graph like the following in python: That is, one curve for each fold. I have the following code where I use an SVM model to classify some data ...
user avatar
  • 320
0 votes
0 answers
8 views

Is it possible to implement an rdd version of a for loop having map and reduce using pyspark?

I need to test an algorithm that computes a function on a dataframe where in each execution I drop a column and computes the function. This is a example in python pyspark but without using rdd: ...
user avatar
  • 1
0 votes
0 answers
10 views

What kind of Keras data classification mode should I pick for my kind of machine learning problem?

So I have to come up with a machine learning model for a specific problem that I have. As far as I know, Keras allows for four different types of classification: Binary, Categorical, Multiclass ...
user avatar
0 votes
0 answers
14 views

Which LSTM Training Strategy Performs better?

I would like to use LSTM for predicting multiple time series (Time series about sales per day in multiple countries. In parts, contradicting regional trends are present within the data. Sales is the '...
user avatar
  • 1

1
2 3 4 5
124