Questions tagged [python]
Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).
6,231
questions
4
votes
1
answer
924
views
What preprocessing steps to be followed before image comparison?
1
down vote
favorite
For example I am trying to find the similarity between two images using skimage - SSIM. The code block will be as follows
...
CommunityBot
- 1
0
votes
0
answers
3
views
Tensorflow-directml is not making any processing time difference
I don't see any time difference when running tensorflow-directml on intel HD hardware. Can someone help me on why running on a GPU is not making any difference? Am I executing the code correctly?
...
- 5,844
3
votes
1
answer
456
views
How do I resolve this error: "expected string or bytes-like object" please?
I was doing Lemmatisation but I got the above error. I think it's because my data isn't in string form, is that correct? If so, which part of my data isn't in string form?
This is my data source: ...
CommunityBot
- 1
2
votes
4
answers
212
views
Are there ML Libs in Python robust to missing data?
So I was searching on how to handle missing data and came across this post from Machine Learning Mastery.
This article states that some algorithms can be made robust to missing data, such as Naive ...
CommunityBot
- 1
0
votes
0
answers
2
views
not able to show evaluation metrics for object detection on MSCOCO like dataset using FASTAI
I want to show evaluation metrics for coco object detection,I have saved my learner in .pth file and I have data in variable data(databunch type fastai) , I have the code from the notebook finding ...
0
votes
1
answer
58
views
Which classification_report metrics are appropriate to report/interpret for a binary label? Individual or macro average for both classes? scikit-learn
First, please forgive my ignorance; I am a newbie but dedicated to learning more.
Example: I have a using a random forest classifier to predict a binary outcome. The binary outcome equals 1 if people ...
CommunityBot
- 1
2
votes
1
answer
202
views
Combine RepeatedStratifiedKFold and crossval
Is it okay to combine RepeatedStratifiedKFold and cross_val_score?
The result in the example below are 30 accuracy values (3x10 splits).
How to calculate the final accuracy value for the 3 times 10 ...
CommunityBot
- 1
0
votes
1
answer
11
views
How to fine tune n_components parameter in UMAP?
I am using UMAP for clustering. However I can't find any information about methods to fine tune n_components parameter (which is very important). As good as I understand I can't use explained variance ...
- 2,168
3
votes
1
answer
2k
views
One hot encoding as input to recurrent neural networks
I'm trying to predict next label in a pattern based on previous labels using recurrent neural network. In total I have 100 labels
Example of input pattern:
...
CommunityBot
- 1
3
votes
1
answer
108
views
Is it possible to solve Rubik's cube using DQN?
I'm trying to solve Rubik's cube using deep learning and I came across with DQN, so I decided to give it a try. I developed all the code and started training but I got this results:
Loss goes up and ...
CommunityBot
- 1
1
vote
1
answer
627
views
Can we do multivariate time series analysis using holt-winter ( Exponential smoothing) method?
Just like we have a method like ARIMAX and SARIMAX where we can provide exog and endog variable for perfroming multivariate analysis. I was hoping is there a way, we can achieve same using ETS as well....
CommunityBot
- 1
3
votes
1
answer
678
views
0
votes
0
answers
5
views
Can label encoder be used to deal with unordered category variables?
Many people say that one hot should be used to deal with unordered category variables, but I see that there is a high praise project on kaggle, which directly uses label encoder to deal with unordered ...
0
votes
2
answers
1k
views
Could not identify NUMA node of platform GPU ID 0 on M1 MacBook
I am unable to identify this Warning Below.
I am using M1 MacBook Air
CODE
...
- 101
0
votes
2
answers
127
views
Can I do the below problem with ML?
My dataset has 3 fields ColumnX, ColumnA, Column B as below.
Column X, Column_A , Column_B
AA, 1 , KEYWORD1 KEYWORD2 KEYWORD3
AA, 2 , KEYWORD4 KEYWORD5 KEYWORD6
XX, 3 , ...
CommunityBot
- 1
1
vote
2
answers
64
views
Update the saved model after training
Will saving a trained model this way give me a model trained on every chunk of data or just the last chunk?
...
CommunityBot
- 1
1
vote
1
answer
957
views
Determine model hyper-parameter values for grid search
I built machine learning model for Ridge,lasso, elastic net and linear regression, for that I used gridsearch for the parameter tuning, i want to know how give value range for **params Ridge ** below ...
CommunityBot
- 1
3
votes
1
answer
15k
views
How to create a new dataframe using the another dataframe
I have created and worked on a DataFrame for a project. It looks like the following:
Critics Items Ratings
a...........1..........5
b...........2..........3
b...........3..........2
c...........
CommunityBot
- 1
2
votes
1
answer
1k
views
How to impute using simple imputer (custom function)
I am imputing my data using simple imputer from sklearn. i want to test many different ways of applying transformations to the data. i.e for logisitcic regression i would like to
remove nans and ...
CommunityBot
- 1
0
votes
1
answer
81
views
How to predict custom value after using linear regression?
I'm new to machine learning, and I'm currently practicing by playing around with datasets that I find on Kaggle. Currently I'm trying to predict the price of an Audi, based on the model, mileage and ...
CommunityBot
- 1
0
votes
1
answer
902
views
I am getting this error ValueError: not enough values to unpack (expected 4, got 2)
I have written this code:
...
CommunityBot
- 1
1
vote
1
answer
69
views
Multiple values for a single parameter in the mlflow run command
How to pass multiple values to each parameter in the mlflow run command?
The objective is to pass a dictionary to GridSearchCV as a param_grid to perform cross validation.
In my main code, I retrieve ...
CommunityBot
- 1
1
vote
1
answer
16
views
How can I implement lambda-mart with lightgbm?
I have a learning to rank task at hand and I want to use the lightgbm implementation of LambdaMART. I'm also following this notebook.
...
- 664
0
votes
2
answers
3k
views
Dimension Mismatch Error during dot product in Python
I have two matrices user_vecs and item_vecs
I am trying to take the dot product of the two to build a recommendation engine:
The ...
CommunityBot
- 1
0
votes
1
answer
172
views
How can I load the custom data in the data loader (pytorch)
Note: I have extracted the frame for all videos and save it in the folder with the same name of video
train_data, class, video ---> These are folders
img --> these are jpg files, so each class ...
CommunityBot
- 1
1
vote
1
answer
502
views
How to interpret my logistic regression result with statsmodels
so I'am doing a logistic regression with statsmodels and sklearn.
My result confuses me a bit. I used a ...
CommunityBot
- 1
1
vote
1
answer
102
views
Data quality improvement as a part of preprocessing: Imputation
I have a python pandas dataframe representing a superset. The data contains a lot of nulls which I want to overwrite with real values.
the superset has:
both numerical and categorical data
some ...
CommunityBot
- 1
0
votes
2
answers
105
views
CNN for image classification with two outputs
Is it possible to classify my images (cars parts) by the type of cars part(door, window ...) and also by the view of the image( front, back, right, left, top and bottom). My pictures are labelled like ...
CommunityBot
- 1
3
votes
2
answers
134
views
Manipulating noise to get some data in right format and apply it to task using PPO
Warning:
I understand that my question may seem strange, stupid, and impossible, but let's just think about this interesting problem. I would not ask a question like: how to create an AGI in google ...
- 16.5k
0
votes
1
answer
9
views
Creating a grid type 3D data array from data points
I have a 3 data column $(X, Y, Z)$ ranges from $(min, max)$. For example,
$X = (0, 5)$, $Y=(0, 3)$, $Z=(0, 2)$. By using them I need to create a numpy array in the form of
$[(0, 0, 0), (0, 0, 1), (0, ...
- 5,844
1
vote
2
answers
4k
views
How to find the size of a tensor in bytes?
Supposing I have a tensor named encoding which is of the shape (1,32) Now I would like to find out the size of the tensor in ...
1
vote
1
answer
50
views
Grouping numeric data into efficient group/pool
I have devices assigned with different data plan. But based on device behaviour amount of data used by device changes during the month. I need to put the device into appropriate data plan based on ...
CommunityBot
- 1
0
votes
0
answers
43
views
Reinforcement learning using univalent and multivalent heterogeneous features
Problem introduction
I have a game in which human players play levels (just like the famous casual game candy crush) where each level has its own properties and its own difficulty. In said game, the ...
- 1
0
votes
0
answers
5
views
Asynchronous vs synchronous polling when collecting time-series data from various sensors?
I'm leading a project in my small company where we are performing data collection on some water heating equipment we supply. I will be using a Raspberry Pi with Python scripts to aggregate data from a ...
2
votes
1
answer
420
views
Digitize graphs. Pull out data from points in a scatter plot
I have a project where I would like to extract data from a series of scatter plots that are image files (Jpeg or png). The plots are similar but the axes scales are not always exactly the same. I have ...
CommunityBot
- 1
1
vote
1
answer
426
views
How to filter rows in Python pandas dataframe with duplicate values in the columns to be filtere
Overall context:
I have a data frame that contains observations for every five minute starting at 5 AM in the morning and ending at 8 PM in the evening for several days. I need to filter all the ...
CommunityBot
- 1
2
votes
1
answer
990
views
Linear regression : ValueError: operands could not be broadcast together with shapes (3,) (1338,)
I try to use linear regression for insurance data . But had error on the when try to call a function with features parameter.
Here is my code:
...
CommunityBot
- 1
0
votes
1
answer
168
views
How to define quadratic weighted kappa as eval_metric in catboost classifier
I am using catboost for a multiclass classification problem. I want to use quadratic weighted kappa as the evaluation metric.
Catboost already has WKappa as an eval_metric but it is linearly weighted ...
CommunityBot
- 1
1
vote
1
answer
97
views
Extracting component means and convariances from mixture model
I am currently trying to write a simple multivariate gaussian mixture model using tensorflow probability. Specifically, I have some 2-dimensional input and 2-dimensional output data and am looking to ...
CommunityBot
- 1
3
votes
1
answer
424
views
Advice on dealing with very large datasets - HDF5, Python
I've recently started working on an application for visualization of really big datasets. While reading online it became apparent that most people use HDF5 for storing big, multi-dimensional datasets ...
CommunityBot
- 1
3
votes
1
answer
939
views
find bigrams in pandas
I have a DataFrame with 4 columns: 'Headline', 'Body_ID', 'Stance', 'articleBody', with 'Headline' and 'articleBody containing cleaned and tokenized words. I want to find bi-grams using nltk and have ...
CommunityBot
- 1
1
vote
1
answer
63
views
Features and LSTM
I have a problem while developing an LStm model. I have 4 feaures that I want to use to make a prediction. When I test my model with a single feaure I get average results but when I test with all 4 ...
- 11
1
vote
1
answer
263
views
Cross-validation average score
I am using Repeated K-folds (RepeatedKFold(n_splits=10, n_repeats=10, random_state=999) from sklearn) to provide reliable scores for a linear regression on my ...
CommunityBot
- 1
0
votes
1
answer
24
views
How do I train a CNN using my own dataset?
I'm trying to run an image classification CNN on my own set of labeled images. Currently, I have the images of each label stored in folders, named by the labels. I can also store the images as numpy ...
- 156
0
votes
1
answer
217
views
Text classification analysis based on similarity
I have been reading a lot of literature regarding text classification and different approaches/models, especially using Python language, but probably I am still missing something on how to build the ...
CommunityBot
- 1
2
votes
1
answer
508
views
Binary image classifier always predicting one class
I am trying to design a model for binary image classification, this is my first classifier and I am following an online tutorial but the model always predicts class 0
My dataset contains 3620 and ...
CommunityBot
- 1
1
vote
0
answers
8
views
Is It Fundamentally Correct To The Text Classification Model To Train First Without Pre-Trained Word Vectors And Then With Pre-Trained Word Vectors?
Is this solution fundamentally correct to the text classification (sentiment analysis) model to train it by these three steps:
train the model without pre-trained word vectors untill reaches the ...
1
vote
0
answers
11
views
Is This Solution Fundamentally Correct To The Text Classification Model With Pre-Trained Word Vectors?
Is it fundamentally correct to training text classification (sentiment analysis) model with pre-trained word vectors; first with the locked embedding layer, and then train again with locked additional ...
1
vote
2
answers
3k
views
Accumulators in Spark (PySpark) without global variables?
BACKGROUND
Consider the following textbook example which uses accumulators to add vectors.
...
CommunityBot
- 1
0
votes
1
answer
9
views