Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (which should be asked on Stack Overflow).

Filter by
Sorted by
Tagged with
0 votes
0 answers
33 views

heavy underfitting of keras LSTM regression

I moved the question from stackoverflow to here. I used keras LSTM to do the standard regression project of ...
user6703592's user avatar
0 votes
1 answer
27 views

calculate the predicted value based on coefficient and constant in python

i have the coefficients and the constant (alpha). i want to multiply and add the values together like this example. (it has to be done for 300000 rows) Prediction = constant + (valOfRow1 * col1) + (-...
Mostafa Bouzari's user avatar
0 votes
0 answers
23 views

ROC curve for multiclassification - results sound not correct

I'm working on a multiclassification task using LSTM algorithm, i generated my roc curve plots but they give scores like 1 , 0.99, 0.97 however i have an accuracy of 0.97, Precision 0.65, Sensitivity/...
biihu's user avatar
  • 11
0 votes
0 answers
47 views

ValueError Found array with 0 sample shape= (0,2) while a minimum of 1 is required by KMeans

I am new to logistic regression and cam across this error. Please help. import pandas as pd from sklearn.cluster import KMeans import matplotlib.pyplot as plt column_names = ["GENDER","...
user22366511's user avatar
0 votes
0 answers
111 views

Failed to find data adapter that can handle input: (<class 'list'> containing values of types {"<class 'numpy.ndarray'>"})

...
abhi singh's user avatar
0 votes
0 answers
7 views

What are the paprements needed and how should be filled using LIME with NLP in Python?

it is my first time using LIME and I have never used any interpretation technique before. most likely I am doing something wrong but I cannot figure out what is it. I tried googling and going through ...
asmgx's user avatar
  • 539
0 votes
0 answers
25 views

How to get best result by using hyperparameter tuning and stacking regression?

I want to use regression ML models to get the best possible R2 score. So I decided to do both hyperparameter tuning and combining models with stacking regressor. I'm wondering if I should do ...
User's user avatar
  • 1
0 votes
0 answers
28 views

Memory / Segfault doing auto_arima

I am trying to do a day based SARIMA modelling using the pmdarima package like so: ...
silverdagger's user avatar
1 vote
1 answer
141 views

Given historical states and action/reward data, is reinforcement learning a reasonable approach?

To summarize my problem: I want to maximize my total reward over all timesteps I have 3 discrete actions at each time step. The state vector for each time step has 5 features. The features are ...
dcl's user avatar
  • 251
0 votes
1 answer
26 views

Is it possible to optimize ML models further and get better R2 score than the one suggested by lazypredict?

I'm using lazypredict in Jupyter Notebook. It automatically run different ML models and sort the R^2 score result of them in a form of table so that one can realize which models perform the best. I'm ...
User's user avatar
  • 1
0 votes
0 answers
75 views

Get graph execution error for BiLSTM and LSTM on keras

I want to use BiLSTM mode for text classification tasks. I use a data generator to get already batched and embedded files that have been split into 64 different files(for training) and 4 files each ...
Nekon's user avatar
  • 1
1 vote
1 answer
48 views

causal inference vs sensitivity analysis to create acurate ML predictions?

I recently discovered the concept of "causal inference" and I'm trying to understand if I've grasped its essence correctly. If I understand it correctly, it involves finding what influences ...
lelorrain7's user avatar
0 votes
1 answer
76 views

Is there a way to automatically split large clusters that are greater than some maximum number of points?

I ran HDBSCAN on these coordinates and got some clusters but some are too large. HDBSCAN has a minimum cluster size parameter, but no maximum size. All I want is to intuitively divide larger clusters ...
Ben Hendel's user avatar
0 votes
0 answers
10 views

Updating Predictive Model to Forecast from Today’s Date

I have a historical dataset with four numerical columns: c1, c2, c3, and c4. Column c4 is calculated and represents the number of days between the active date and the closed date. I have created a ...
Tahmeed's user avatar
2 votes
0 answers
44 views

LSTM output capped at a maximum

I am using a LSTM built using to forecast a single-value (solar irradiance) by using weather data as my input. When predicting my validation test, I get a weird results as it looks like all my ...
Adam Jaamour's user avatar
0 votes
0 answers
40 views

Data Cleanup for Regression

I have a simple dataset of 1 output and 1 input and want to fit a linear regression to the dataset. The data has a certain level of noise to it (potentially driven by another input, which I will ...
felix_the_cat's user avatar
0 votes
0 answers
17 views

Why does the first call to a TensorFlow function execute much slower than the second call?

I was doing an Image Classification problem using TensorFlow. I was generating the mean images for two image datasets having the same size. The dataset was generated using the tf.data API. Thereafter ...
Harsh Khare's user avatar
0 votes
0 answers
39 views

Multi GPU training using Pytorch fabric

I am launching the Pytorch fabric below : ...
Shrinidhi M's user avatar
0 votes
0 answers
28 views

How to remove augmented data from test data

I am working with this dataset https://data.mendeley.com/datasets/hxsnvwty3r/1 for object classification model like CNN. In the description of the dataset, I see in the description there are "...
usan's user avatar
  • 1
0 votes
0 answers
26 views

Multi input model tensorflow with fixed data as input

I am tring to implement the following architecture. alpha and beta are fixed matrices, they are matrices I want to input on every forward pass. Meaning for every batch they should be the same My ...
Ahmed Gado's user avatar
0 votes
0 answers
13 views

Is it possible okay to use regression MLP for ordinal classification problem when target variable is numerical?

I have a target variable of 1-10 that represent difficulty level. These are individual classes represented by integers with 1 being the easiest and 10 most difficult. I have decided to use regression ...
Yoseph Ismail's user avatar
0 votes
0 answers
21 views

Interpolation techniques predicting negative values in a column with positive values

I tried different interpolation techniques & for some numeric columns with only positive values the interpolation techniques: Polynomial, Cubic Spline & Radial Basis are predicting negative ...
Apoorva's user avatar
  • 275
0 votes
0 answers
11 views

clustering droplet data along trajectories

I have regular t,x,y,z-datapoints of droplets, that move continuously from top to bottom on different but similar paths. The droplets also have a volume that is approximately measured, they can appear ...
Jonas Hilti's user avatar
0 votes
1 answer
454 views

chainlit test failed

I am using windows OS. I created a folder named ABC under C. In command prompt, I typed cd C:\ABC and then typed the following commands: ...
Karl 17302's user avatar
0 votes
0 answers
18 views

Seeking guidance on data analysis techniques to estimate project LOE based on numeric counts of outputs produced

I’m looking for some basic guidance on where to focus my research in support of some data analysis I’m looking to perform. The problem space is identifying a methodology for estimating Level of Effort ...
Steven's user avatar
  • 109
0 votes
0 answers
18 views

How does TensorFlow method tf.data.Dataset.reduce() work?

I was trying to compute the mean of the images that I fetched from a GCS bucket using TensorFlow's tf.data input pipeline. For this, I came across two methods: ...
Harsh Khare's user avatar
0 votes
0 answers
25 views

how to find a repeated event in time series?

I need to find the repeated events in time series. if one event always has the similar value, similar description, and happens at similar date, then it is a repeated pattern. Let me make an example ...
Feng Chen's user avatar
  • 207
0 votes
1 answer
23 views

I cannot run MNIST MWE (hello world for DL)

I have installed Anaconda and want tor run MWE for MNIST but I'me getting this error: D:\STAZENE_last\Anaconda2\Lib\site-packages\torch\cuda\__init__.py:107: ...
user2925716's user avatar
0 votes
1 answer
364 views

Training embeddings on own dataset

In my project I follow the retrieval augmented generation (RAG) approach. I want to create embeddings for my own dataset and use it in combination with llama-2. In the dataset are german annual ...
Christian01's user avatar
0 votes
0 answers
29 views

How to train Keras Sequential model in batches?

newbie here. I have a large dataset in many local CSV files. It's impossible to fit it in memory within budget. How can I train a Sequential model in batches? I've found ...
Putnik's user avatar
  • 105
0 votes
0 answers
18 views

How to detect a kick signal

Background: I am measuring the reflection signal from the pipe. First strong kick is called firing kick. The later small bumps ...
ii2's user avatar
  • 111
0 votes
1 answer
44 views

How is the backward propagation is done in pytroch? When to use torch.no_grad, also when and where is the gradinte calcuated?

I have this training loop in pytorch. the loss_fn = nn.CrossEntropyLoss() and optim = torch.optim.Adam(net.parameters(), lr=lr) <...
Ahmed Gado's user avatar
0 votes
1 answer
24 views

A model for recognizing suspicious user activity on a forum

I have a log of user actions on my website. Some users were blocked for spamming manually. This is also shown in the log. Here is an example of the dataset: ...
Quaw's user avatar
  • 13
1 vote
0 answers
14 views

Clustering Similar Articles Using Mixed Data: Seeking Advice and Validation

Question: I'm working on a project where I need to cluster a dataset of articles based on various features, including text, numeric values, and categorical data. I've implemented a clustering approach ...
sara sara's user avatar
0 votes
0 answers
13 views

Macro Averaging vs. Samples Averaging in multilabel classification problems

I am currently working on a multilabel classification problem and I have developed some models to solve it using the SciKit-Learn framework. I wish now to evaluated the models by producing scores for ...
user2566415's user avatar
1 vote
0 answers
98 views

How to split graph data into train and test sets for link prediction problem using node emebbdings and cosine similarity

I would like to predict new links using node embeddings and cosine similarity, but I am unsure how to split the data set into training and testing, and how to evaluate new links. This is my code ...
Ghada Kahlaoui's user avatar
0 votes
0 answers
20 views

Need help in Webscraping T20 World Cup 2022 data

I have a list of links of match scorecards and i have stored the respective html content of each link in a list. I need to get the csv files I am attaching as the output. Any help would be ...
karan shah's user avatar
0 votes
1 answer
24 views

Why most keras examples use shape=(None,) instead of shape=(None)?

I'm reading several keras tutorials and found that many examples are written like this: keras.Input(shape=(None,), dtype="int64", name="english")...
skan's user avatar
  • 185
0 votes
1 answer
57 views

Likelihood function of a Beta-Normal distribution

I'm going over a paper regarding the calculation of credit scoring By Kalkbrener and Onwunta. In the article, they derive the following Likelihood function, in order to find the MLE for $R^2$: (...
Alex Teush's user avatar
0 votes
0 answers
34 views

Tensorflow model wont train when using validation data

Im trying to train a tensorflow neural network. Since the training data is too big for my computer RAM I have divided it into sub-datsets and sequentially use them for training. I have a problem ...
Sergio Pardina's user avatar
0 votes
0 answers
17 views

Detection of noise and signal components with deep learning

There are thousands of datasets from the signal in the bellow image. With these datasets and machine learning, I want to first detect the jumping points (blue in the image) which are noises, and after ...
alexjan's user avatar
0 votes
1 answer
65 views

Combine two separate models created via Transfer Learning?

Suppose have two 'image classification' models created by transfer learning on the same base model[1], each producing a different set of labels/classes. Trained at different times, with different ...
barryhunter's user avatar
1 vote
1 answer
44 views

How to know the confidence of a classification on unlabeled data generated after model training?

I have created (in python) the code for a Random Forest classification model for a labeled dataset using sklearn. The model works very well. ...
Daniel Vieira's user avatar
0 votes
0 answers
31 views

What is numpy.dot(a, b)[i,j,k,m] = sum(a[i,j,:] * b[k,:,m]) in numpy.dot() documentation?

I was trying to understand difference between np.dot and np.matmul and in the docs of np.dot() there is this code dot(a, b)[i,j,k,m] = sum(a[i,j,:] * b[k,:,m]). ...
Tinted's user avatar
  • 1
0 votes
0 answers
10 views

How to solve MemoryError problem?

I have audio signals which I need to convert them to melspectrogram. I am using Deepfilternet to remove background noise. when I use the output of Deepfilternet for next phase like padding, it showed ...
Zara Nz's user avatar
0 votes
0 answers
21 views

Python Darts API

I'm using the darts API for some time series prediction in python, while looking at the documentation for the RNN model I saw this parameters called training length. The description is not very clear ...
Guilherme Takata's user avatar
1 vote
0 answers
7 views

Huge variance for RandomForestRegressor models

The experiment is the following: train a RFR with a 15k train rows get predictions on 8k test rows, save predictions as y_hat0 remove 1 random row from the training set and retrain the RFR save ...
Oleg's user avatar
  • 11
0 votes
0 answers
7 views

YOLOV4 model training

I can't start my model tutorial. I ran it correctly, I tried it before, but now it doesn't even start. why could it be? Did the update come?
Elif Deniz's user avatar
0 votes
0 answers
22 views

Wavelet choice for wav file analysis

Context I am working on the Maestro v3.0 dataset https://magenta.tensorflow.org/datasets/maestro#v300 to familiarize myself with signal processing. Years ago, I mainly worked on image processing with ...
zar3bski's user avatar
  • 101
1 vote
1 answer
51 views

Time series prediction on test set produces multiple cycles instead of following trend

I have a non-stationary time series where I am trying to build a model for forecasting. So far on test set it produces multiple cycles no matter which technique I use. There's just one feature ...
Thermal_insulator's user avatar