Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
4 votes
1 answer
924 views

What preprocessing steps to be followed before image comparison?

1 down vote favorite For example I am trying to find the similarity between two images using skimage - SSIM. The code block will be as follows ...
0 votes
0 answers
3 views

Tensorflow-directml is not making any processing time difference

I don't see any time difference when running tensorflow-directml on intel HD hardware. Can someone help me on why running on a GPU is not making any difference? Am I executing the code correctly? ...
3 votes
1 answer
456 views

How do I resolve this error: "expected string or bytes-like object" please?

I was doing Lemmatisation but I got the above error. I think it's because my data isn't in string form, is that correct? If so, which part of my data isn't in string form? This is my data source: ...
2 votes
4 answers
212 views

Are there ML Libs in Python robust to missing data?

So I was searching on how to handle missing data and came across this post from Machine Learning Mastery. This article states that some algorithms can be made robust to missing data, such as Naive ...
0 votes
0 answers
2 views

not able to show evaluation metrics for object detection on MSCOCO like dataset using FASTAI

I want to show evaluation metrics for coco object detection,I have saved my learner in .pth file and I have data in variable data(databunch type fastai) , I have the code from the notebook finding ...
0 votes
1 answer
58 views

Which classification_report metrics are appropriate to report/interpret for a binary label? Individual or macro average for both classes? scikit-learn

First, please forgive my ignorance; I am a newbie but dedicated to learning more. Example: I have a using a random forest classifier to predict a binary outcome. The binary outcome equals 1 if people ...
2 votes
1 answer
202 views

Combine RepeatedStratifiedKFold and crossval

Is it okay to combine RepeatedStratifiedKFold and cross_val_score? The result in the example below are 30 accuracy values (3x10 splits). How to calculate the final accuracy value for the 3 times 10 ...
0 votes
1 answer
11 views

How to fine tune n_components parameter in UMAP?

I am using UMAP for clustering. However I can't find any information about methods to fine tune n_components parameter (which is very important). As good as I understand I can't use explained variance ...
3 votes
1 answer
2k views

One hot encoding as input to recurrent neural networks

I'm trying to predict next label in a pattern based on previous labels using recurrent neural network. In total I have 100 labels Example of input pattern: ...
3 votes
1 answer
108 views

Is it possible to solve Rubik's cube using DQN?

I'm trying to solve Rubik's cube using deep learning and I came across with DQN, so I decided to give it a try. I developed all the code and started training but I got this results: Loss goes up and ...
1 vote
1 answer
627 views

Can we do multivariate time series analysis using holt-winter ( Exponential smoothing) method?

Just like we have a method like ARIMAX and SARIMAX where we can provide exog and endog variable for perfroming multivariate analysis. I was hoping is there a way, we can achieve same using ETS as well....
3 votes
1 answer
678 views

Keras functional API Layer name not captured with TimeDistributed wrapper

...
0 votes
0 answers
5 views

Can label encoder be used to deal with unordered category variables?

Many people say that one hot should be used to deal with unordered category variables, but I see that there is a high praise project on kaggle, which directly uses label encoder to deal with unordered ...
0 votes
2 answers
1k views

Could not identify NUMA node of platform GPU ID 0 on M1 MacBook

I am unable to identify this Warning Below. I am using M1 MacBook Air CODE ...
0 votes
2 answers
127 views

Can I do the below problem with ML?

My dataset has 3 fields ColumnX, ColumnA, Column B as below. Column X, Column_A , Column_B AA, 1 , KEYWORD1 KEYWORD2 KEYWORD3 AA, 2 , KEYWORD4 KEYWORD5 KEYWORD6 XX, 3 , ...
1 vote
2 answers
64 views

Update the saved model after training

Will saving a trained model this way give me a model trained on every chunk of data or just the last chunk? ...
1 vote
1 answer
957 views

Determine model hyper-parameter values for grid search

I built machine learning model for Ridge,lasso, elastic net and linear regression, for that I used gridsearch for the parameter tuning, i want to know how give value range for **params Ridge ** below ...
3 votes
1 answer
15k views

How to create a new dataframe using the another dataframe

I have created and worked on a DataFrame for a project. It looks like the following: Critics Items Ratings a...........1..........5 b...........2..........3 b...........3..........2 c...........
2 votes
1 answer
1k views

How to impute using simple imputer (custom function)

I am imputing my data using simple imputer from sklearn. i want to test many different ways of applying transformations to the data. i.e for logisitcic regression i would like to remove nans and ...
0 votes
1 answer
81 views

How to predict custom value after using linear regression?

I'm new to machine learning, and I'm currently practicing by playing around with datasets that I find on Kaggle. Currently I'm trying to predict the price of an Audi, based on the model, mileage and ...
0 votes
1 answer
902 views

I am getting this error ValueError: not enough values to unpack (expected 4, got 2)

I have written this code: ...
1 vote
1 answer
69 views

Multiple values for a single parameter in the mlflow run command

How to pass multiple values to each parameter in the mlflow run command? The objective is to pass a dictionary to GridSearchCV as a param_grid to perform cross validation. In my main code, I retrieve ...
1 vote
1 answer
16 views

How can I implement lambda-mart with lightgbm?

I have a learning to rank task at hand and I want to use the lightgbm implementation of LambdaMART. I'm also following this notebook. ...
0 votes
2 answers
3k views

Dimension Mismatch Error during dot product in Python

I have two matrices user_vecs and item_vecs I am trying to take the dot product of the two to build a recommendation engine: The ...
0 votes
1 answer
172 views

How can I load the custom data in the data loader (pytorch)

Note: I have extracted the frame for all videos and save it in the folder with the same name of video train_data, class, video ---> These are folders img --> these are jpg files, so each class ...
1 vote
1 answer
502 views

How to interpret my logistic regression result with statsmodels

so I'am doing a logistic regression with statsmodels and sklearn. My result confuses me a bit. I used a ...
1 vote
1 answer
102 views

Data quality improvement as a part of preprocessing: Imputation

I have a python pandas dataframe representing a superset. The data contains a lot of nulls which I want to overwrite with real values. the superset has: both numerical and categorical data some ...
0 votes
2 answers
105 views

CNN for image classification with two outputs

Is it possible to classify my images (cars parts) by the type of cars part(door, window ...) and also by the view of the image( front, back, right, left, top and bottom). My pictures are labelled like ...
3 votes
2 answers
134 views

Manipulating noise to get some data in right format and apply it to task using PPO

Warning: I understand that my question may seem strange, stupid, and impossible, but let's just think about this interesting problem. I would not ask a question like: how to create an AGI in google ...
0 votes
1 answer
9 views

Creating a grid type 3D data array from data points

I have a 3 data column $(X, Y, Z)$ ranges from $(min, max)$. For example, $X = (0, 5)$, $Y=(0, 3)$, $Z=(0, 2)$. By using them I need to create a numpy array in the form of $[(0, 0, 0), (0, 0, 1), (0, ...
1 vote
2 answers
4k views

How to find the size of a tensor in bytes?

Supposing I have a tensor named encoding which is of the shape (1,32) Now I would like to find out the size of the tensor in ...
1 vote
1 answer
50 views

Grouping numeric data into efficient group/pool

I have devices assigned with different data plan. But based on device behaviour amount of data used by device changes during the month. I need to put the device into appropriate data plan based on ...
0 votes
0 answers
43 views

Reinforcement learning using univalent and multivalent heterogeneous features

Problem introduction I have a game in which human players play levels (just like the famous casual game candy crush) where each level has its own properties and its own difficulty. In said game, the ...
0 votes
0 answers
5 views

Asynchronous vs synchronous polling when collecting time-series data from various sensors?

I'm leading a project in my small company where we are performing data collection on some water heating equipment we supply. I will be using a Raspberry Pi with Python scripts to aggregate data from a ...
2 votes
1 answer
420 views

Digitize graphs. Pull out data from points in a scatter plot

I have a project where I would like to extract data from a series of scatter plots that are image files (Jpeg or png). The plots are similar but the axes scales are not always exactly the same. I have ...
1 vote
1 answer
426 views

How to filter rows in Python pandas dataframe with duplicate values in the columns to be filtere

Overall context: I have a data frame that contains observations for every five minute starting at 5 AM in the morning and ending at 8 PM in the evening for several days. I need to filter all the ...
2 votes
1 answer
990 views

Linear regression : ValueError: operands could not be broadcast together with shapes (3,) (1338,)

I try to use linear regression for insurance data . But had error on the when try to call a function with features parameter. Here is my code: ...
0 votes
1 answer
168 views

How to define quadratic weighted kappa as eval_metric in catboost classifier

I am using catboost for a multiclass classification problem. I want to use quadratic weighted kappa as the evaluation metric. Catboost already has WKappa as an eval_metric but it is linearly weighted ...
1 vote
1 answer
97 views

Extracting component means and convariances from mixture model

I am currently trying to write a simple multivariate gaussian mixture model using tensorflow probability. Specifically, I have some 2-dimensional input and 2-dimensional output data and am looking to ...
3 votes
1 answer
424 views

Advice on dealing with very large datasets - HDF5, Python

I've recently started working on an application for visualization of really big datasets. While reading online it became apparent that most people use HDF5 for storing big, multi-dimensional datasets ...
3 votes
1 answer
939 views

find bigrams in pandas

I have a DataFrame with 4 columns: 'Headline', 'Body_ID', 'Stance', 'articleBody', with 'Headline' and 'articleBody containing cleaned and tokenized words. I want to find bi-grams using nltk and have ...
1 vote
1 answer
63 views

Features and LSTM

I have a problem while developing an LStm model. I have 4 feaures that I want to use to make a prediction. When I test my model with a single feaure I get average results but when I test with all 4 ...
1 vote
1 answer
263 views

Cross-validation average score

I am using Repeated K-folds (RepeatedKFold(n_splits=10, n_repeats=10, random_state=999) from sklearn) to provide reliable scores for a linear regression on my ...
0 votes
1 answer
24 views

How do I train a CNN using my own dataset?

I'm trying to run an image classification CNN on my own set of labeled images. Currently, I have the images of each label stored in folders, named by the labels. I can also store the images as numpy ...
0 votes
1 answer
217 views

Text classification analysis based on similarity

I have been reading a lot of literature regarding text classification and different approaches/models, especially using Python language, but probably I am still missing something on how to build the ...
2 votes
1 answer
508 views

Binary image classifier always predicting one class

I am trying to design a model for binary image classification, this is my first classifier and I am following an online tutorial but the model always predicts class 0 My dataset contains 3620 and ...
1 vote
0 answers
8 views

Is It Fundamentally Correct To The Text Classification Model To Train First Without Pre-Trained Word Vectors And Then With Pre-Trained Word Vectors?

Is this solution fundamentally correct to the text classification (sentiment analysis) model to train it by these three steps: train the model without pre-trained word vectors untill reaches the ...
1 vote
0 answers
11 views

Is This Solution Fundamentally Correct To The Text Classification Model With Pre-Trained Word Vectors?

Is it fundamentally correct to training text classification (sentiment analysis) model with pre-trained word vectors; first with the locked embedding layer, and then train again with locked additional ...
1 vote
2 answers
3k views

Accumulators in Spark (PySpark) without global variables?

BACKGROUND Consider the following textbook example which uses accumulators to add vectors. ...
0 votes
1 answer
9 views

Plotly graph does not show x-axis values correctly

Consider the following code : ...

1
2 3 4 5
125