Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
0 votes
1 answer
54 views

Plot multiple time series from single dataframe

I have a dataframe with multiple time series and columns with labels. My goal is to plot all time series in a single plot, where the labels should be used in the legend of the plot. The important ...
user avatar
0 votes
0 answers
8 views

Multi-level timeseries forecasting? How to do it?

So, I just finished a 48 hr datathon, and I did terribly, to be honest. It was my first datathon. We were given a list of datasets: 5 months of taxi demand data (January to May) Weather dataset Zone ...
user avatar
0 votes
0 answers
7 views

Unable to debug where torch Adam optimiser is going wrong

I was implementing a training loop in vscode. I have created a Adam optimizer using XLM-Roberta model as follows: ...
user avatar
  • 137
0 votes
0 answers
28 views

under sample to get specific number of samples per class using tomek links of imblearn

I have a dataset with classes in my target column distributed like shown below. ...
user avatar
0 votes
1 answer
40 views

parallel work on KNN in python

I have a question, related to parallel work on python How I can use Processers =1,2,3... on k nearest neighbor algorithm when K=1, 2, 3,.. to find the change in time spent, speedup, and efficiency. ...
user avatar
0 votes
0 answers
15 views

find speedup for different number of processes

I am new to data science I need to create code to find speedup compared with the number of processes while using a k-nearest neighbor. which (k=1,2,3,4,5,6,7). this process should be after downloading ...
user avatar
0 votes
0 answers
16 views

Clustering Algorithm + Euclidean Distance to find similarities

Goal: Create a tool that recommends similar players based on their statistical profile Process: (1) Standardize data (2) UMAP to reduce dimensionality (c. 50 features) (3) First-Stage Clustering: GMM ...
user avatar
0 votes
0 answers
21 views

Could not load file or assembly 'Python.Runtime' or one of its dependencies. Attempt to load an incorrectly formatted program

I'm trying to use keras.net in visual studio 2022. I'm using a .Net application of 32bits so I had to install python 32 bits. I installed keras.net and tenwsorflow with python 32bits but I got this ...
user avatar
0 votes
0 answers
28 views

Orange3 and OCR Help

I would like to be able to utilize Orange to OCR pictures with pytesseract. I have been able to create simple code in the Python Script widget in order to read one image at a time, but I want to be ...
user avatar
  • 1
1 vote
0 answers
14 views

Multiple Timeseries Anomaly Detection - identifying which feature is anomalous?

I have a multivariate time series where I have features such as: temperature set point energy used relative humidity, etc. Currently, I'm creating univariate anomaly detection models in Python using ...
user avatar
1 vote
0 answers
34 views

Learning Python after R [closed]

I just started to learn Python after I worked about 5 years with R. I'm using Spyder as people are recommending this IDE as closest to R and more friendly to Data Science applications. As a ...
user avatar
0 votes
1 answer
24 views

Using large files in a OVHcloud AI notebook

I’m trying to train an artificial intelligence model on OVHCloud AI Notebooks with the Common Voice’s dataset from Mozilla. The problem is that this database has a 70 GB size. I have tried to download ...
user avatar
2 votes
0 answers
24 views

How to perform Timeseries Forecasting on dataset with repeating dates?

I have a dataset, I have to perform timeseries forecasting on that dataset. The dataset has date column, the dates in date column are duplicated. We have 5 classes, a date will have a sales record for ...
user avatar
0 votes
0 answers
7 views

Customized Auto-ML code (no Auto-ML package)

I have been coding for a few hours and I'm wondering If I'm not inventing the wheel again. It has been pretty hard to find similar questions on this also, so I'm wondering if this is completely wrong ...
user avatar
  • 101
0 votes
0 answers
42 views

LightGBM predict_proba in thousandths place

Can someone explain to me how my lightgbm classification model's predict_proba() is in thousandths place for the positive class: ...
user avatar
  • 101
0 votes
0 answers
11 views

my k-fold validation is giving a lot of 100% in the concatenated confusion matrix, is it because of overfitting?

The confusion matrix is a concatenated one from a 5-fold stratified cross-validation of my data set. I used rbf kernel for the svm classifier. Is it telling me the classifier is overfitting? Plus when ...
user avatar
  • 1
0 votes
0 answers
11 views

Adding L2 regularization on top of pretrained EfficientNet model causes fluctuating validation accuracy and loss

I've built a model on top of pretrained EfficientNetB2 which would achive ~50% validation accuracy after 3-4 epochs and then would start to overfit. In order to counter that I added l2 regularization ...
user avatar
0 votes
0 answers
6 views

What kind of hypothesis testing in Python can be used to validate that 4 job titles are significantly different based on their skillset?

I have 4 job titles, for each of which I scraped hundreds of job descriptions and classified them by if they contain words related to a predefined list of skills. For each job description, I now have ...
user avatar
0 votes
0 answers
18 views

Predicting using previous data

I am doing an experiment using ultrasonic radar which rotates 180 degrees clockwise and anti-clockwise. When the sensor encounters an obstacle in front, the algorithm should determine which direction (...
user avatar
  • 1
0 votes
0 answers
24 views

Is this a successful implementation of KL Divergence from scratch, and how can I graph all distributions?

I'm attempting to implement a KL Divergence on two imaginary dice from scratch and I'm not sure if it is correct. By 'how do I graph all distributions' I'm wondering if the KL Divergence just a number ...
user avatar
  • 157
0 votes
0 answers
55 views

AttributeError: 'MissingValues' object has no attribute 'to_list' while i am using LabelEncoder() from sklearn

the same process when done in python 3.9 with pandas and csv dataset it is working fine but how should i use label encoder on geopandas dataframe with python 3.6 and sklearn version 0.24.2.
user avatar
1 vote
0 answers
26 views

Training validation and accuracy are very high, but testing accuracy is low

The training accuracy and validation accuracy of my model is extremely high. ...
user avatar
  • 111
0 votes
0 answers
6 views

iteration and if condition -python

...
user avatar
0 votes
0 answers
13 views

Good values for GridSearchCV testing

Good morning, I'm solving a data problem in several and I'm testing the different models between them: Ridge, Lasso and ElasticNet. I wanted the best parameters for my L1 and L2, but how do I choose ...
user avatar
1 vote
1 answer
22 views

How to plot one graph of ROC curve for 4 separate ML model located in different python notebooks

if we have 4 different notebooks for different ML model results .. and we have to plot one ROC curve graph which shows the ROc of all 4 models. how can we do this this is my code in every notebook to ...
user avatar
  • 175
2 votes
2 answers
186 views

How to balance sensitivity(sn) and specificity(sp) of an Artificial Neural Network model?

I have been working on a binary classification problem of protein sequences. I have used a feed-forward neural network with two hidden layers. I have the training and validation accuracy/loss curves ...
user avatar
0 votes
1 answer
275 views

Using "Demon Adam" as optimizer in Tensorflow

I am working with a simple neural network in Google Colab using Python with Tensorflow where I've only tried to use the optimizers already available in keras such as Nadam, Adam, Adadelta, Adagrad etc....
user avatar
  • 103
0 votes
0 answers
6 views

Prophet Weight Cloumn On Time Series (Python)

Can we add a weight column for times on Prophet? So I want the values ​​of old dates to be less important while recent dates are more important. Is there a parameter or method for this? Or does the ...
user avatar
0 votes
0 answers
15 views

How to produce these tensors efficiently/fast?

I would like to produce the following tensor of size (N*N) where the ones (D) appear as follows: ...
user avatar
0 votes
0 answers
26 views

SHAP KernelExplainer AttributeError numpy.ndarray

I've developed a text classifier of the form of python function that can input a np.array of strings (each string is one observation). ...
user avatar
  • 101
1 vote
0 answers
50 views

Reduce MNIST dataset

I am working on the MNIST dataset. How I can reduce 50% of this data? (x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
user avatar
1 vote
1 answer
29 views

How to test likelihood hypothesis on dataset?

How to test the following hypothesis? The larger the fare the more likely the customer is to be travailing alone. Using the data below, how would one be able to test the hypothesis? ...
user avatar
0 votes
0 answers
12 views

How to print 2-ngrams in LimeTextExplainer

I try to explain the importance of a sentence using the following pipeline with LimeTextExplainer from LIME package. ...
user avatar
  • 101
1 vote
1 answer
32 views

XGBoost results changing when one row is removed

I have a training dataset of 2,600 rows and 26 columns. I trained an XGBoost (1.3.1) Classification model using the data and evaluated it using a test set of c. 800 rows. Whilst experimenting I found ...
user avatar
0 votes
0 answers
11 views

does R2 diverge because of a lack of input dimensions?

I try to improve my R2 score between theoretical and real output values. On the picture you can see two cases: the blue one is an artificial case I’m completely mastering with 7 dimensions as input ...
user avatar
0 votes
0 answers
12 views

Formatting df's multi index

I have a multi-index dataframe used in a block of code. It's index looks like this: ...
user avatar
  • 101
0 votes
0 answers
11 views

When i'm training the pre-trained model on kaggle dataset, it is showing 79% accuracy. when i test on different dataset it shows 42% accuracy

This is the code i used to train the pre-trained model 'inception v3'. And can i know it is showing accuracy as 0.8717 but the val_accuracy as 0.7781. why there is a that much difference. Can anyone ...
user avatar
0 votes
0 answers
14 views

Is RoI Pooling appropriate to retrieve fine details from objects of varying sizes?

I’m using a RoI Pooling after a CNN that extract features from images of varying sizes, containing defects I want to classify. The images and defects sizes range from a couple of pixels to ~100 pixels....
user avatar
1 vote
0 answers
35 views

Using F_regression to find the best significant features

We are trying to use SelectKBest F_Regression scoring function on a pool of 1000 numerical features, and solve a regression problem. Also, we wanted to paralellize the execution of SelectKBest and we ...
user avatar
0 votes
1 answer
75 views

How can I remove legend from the figure in seaborn?

Here is my diagrams, I want to remove the labels of the second bar that is C1,C2,C3,C4,C5, because it is repeating.
user avatar
0 votes
0 answers
11 views

How to properly setup jensen_shannon_divergence and infinity norm in tensorflow data validation for skew and drift checks

Tensorflow data validation offers the capability of checking data skew and drift and the documentation also mention that "Setting the correct distance is typically an iterative process requiring ...
user avatar
  • 193
0 votes
0 answers
41 views

regression.fit(x_train, y_train) is not working on python

I try to deal with my homework. The Job is to take the Achievement data and perform a multi-linear regression on it. The code is published here. I am quite new to programming in Python and in data ...
user avatar
0 votes
0 answers
13 views

what type of machine learning should i implement for this case

I'm still newbie in machine learning and i need a algorithm that can study linear functions it doesn't have to be a function as i have x and y coordinates and i can feed it that, what it should do is ...
user avatar
0 votes
1 answer
28 views

is my confusion matrix is showing differently or is it correctly showing?

I am totally new to Python and machine learning. I have only basic knowledge in both fields. I am trying to train pre-trained model with other data. Below is my code please look into it. I have ...
user avatar
1 vote
0 answers
44 views

What approach I should take to extract number entity from dataset

I have the training, validation, and test dataset. The first column has store data and the second column has store numbers. I need to develop an entity extractor model which can extract store numbers ...
user avatar
  • 11
0 votes
1 answer
62 views

How to handle this kind of timeseries

I am using Python and I have a sample dataset of this kind: ...
user avatar
1 vote
0 answers
150 views

K-Means time series clustering with multiple time series for each data point

I have been trying to cluster my data through K-Means. However, for each datapoint that I have, there is 4 different time series (In, Out for Weekend/Wekeday). I have been looking to do multivariate ...
user avatar
0 votes
1 answer
32 views

Multiple conda enviroments in (R) functions using reticulate

For an internally used R package I need to have certain functions load python environments to do part of the processing. The environment cannot be the same one for all functions, unfortunately, due to ...
user avatar
0 votes
0 answers
135 views

ValueError: Input contains NaN, infinity or a value too large for dtype('float64') - fitting a model

I am attempting to train a model on a pandas DataFrame and am running into ValueError issue. I have verified that there are no ...
user avatar
  • 302
1 vote
1 answer
26 views

How variable alpha changes SGDRegressor behavior for outlier?

I am using SGDRegressor with a constant learning rate and default loss function. I am curious to know how changing the alpha parameter in the function from 0.0001 to 100 will change regressor behavior....
user avatar
  • 111

1
3 4
5
6 7
125