Skip to main content

Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (which should be asked on Stack Overflow).

Filter by
Sorted by
Tagged with
0 votes
0 answers
4 views

ValueError: If using all scalar values, you must pass an index when converting API JSON response to DataFrame

I'm encountering a ValueError when attempting to convert an API JSON response to a Pandas DataFrame. The error message states: "If using all scalar values, you must pass an index." Below is ...
0 votes
1 answer
280 views

Best python library for training using Hidden Marov model with Gaussian Mixture

I would like to train my data using HMM- GMM (Baum Welch approach with gaussian Mixture) to find the best parameters suited for my data. Note : My data is ...
2 votes
2 answers
639 views

Is there are way to impute missing values by clustering, regression and stochastic regression

I'd like to know if there are any libraries that allow imputation by clustering, regression and stochastic regression. So far, I've done imputation by mean, median and KNN. I'm trying to evaluate the ...
2 votes
1 answer
49 views

How to analyze time series data and create time series model in Python?

I am trying to understand time-series data and model. In youtube tutorial and others, mostly univariate examples are shown. And they are applicable or suitable for those conditions. What if our ...
2 votes
1 answer
4k views

Variational AutoEncoder giving negative loss

I'm learning about variational autoencoders and I've implemented a simple example in keras, model summary below. I've copied the loss function from one of Francois Chollet's blog posts and I'm getting ...
0 votes
1 answer
369 views

Facebook Prophet add Regressor

I've been searching for a long time to answer my question, but I haven't found anything. So I hope you can help. I'm searching for an opportunity to add a regressor to my prophet model in python. I ...
0 votes
0 answers
14 views

Seeking Data Science Internship Opportunities and Networking Advice [closed]

I'm an engineering student specializing in data science from Tunisia, and I will be graduating next year. Additionally, I am close to completing the IBM Data Science Professional Certificate. I'm ...
1 vote
1 answer
650 views

Using if statement as a model in orange

I am seeking to evaluate logistic regression against a rule-based (nested if-else statements) classification in Orange. The steps I've taken to investigate this are as follows: I could not identify ...
1 vote
1 answer
410 views

CASE WHEN SQL Type Statement in pandas [closed]

I'm trying to create a new column called Class which is a calculated column. Basically when the Math class is NA then the class will be the CS class, otherwise it will be vice versa. How can I do ...
1 vote
2 answers
1k views

What would be the best way to map similar ngrams

I'm trying to map similar ngrams using Wordnet and synsets. For example: elder brother and older sibling should map to the same ...
0 votes
1 answer
526 views

What is the best\correct data split approach over time-series data to compare performance of forecasting future data among ML and DL regressors?

Let's say I have dataset contains a timestamp (non-standard timestamp column without datetime format) as a single feature and count as Label/target to predict ...
0 votes
3 answers
327 views

How to apply a different Loss function to one specific Label?

I got a recurrent neural network in Keras, which classifies on 14 labels. The first label is the most important one and should be predicted with the highest accuracy. The other labels don't have to be ...
0 votes
1 answer
147 views

Mixed effect random forest model for Python Windows

Does anybody know if there is a Mixed effect random forest model for Python Windows? The merf package https://anaconda.org/search?q=merf+ seems to only be available on a linux environment? thanks!
0 votes
1 answer
216 views

Cross Validation with One Class Classification in Python

I'm trying to do cross-validation with One Class Classification - I'm using the PyOD lib - but I don't know if I'm doing it right. The precision is too low and I'm also not able to bring up the mean ...
1 vote
1 answer
51 views

How to Balance Dataset extracted using image_dataset_from_directory

I'm new to tensorflow, so I've been trying to find the best way to do class balancing over a dataset where I used image_dataset_from_directory to load. But I haven't find the way to do it. I saw from ...
1 vote
1 answer
1k views

Found input variables with inconsistent numbers of samples: [11232, 5616]

I don't know what is the reason for the error please guide me and help me out. I am at a learning stage.
1 vote
0 answers
14 views

Stuck with constant loss and network not learning

I am trying to predict certain function coefficients (output: a, b) based on its curve (input: frequency_response) with the help of https://github.com/Blealtan/efficient-kan (Kolmogorov-Arnold Network)...
0 votes
1 answer
75 views

Transitioning from a python script for data transformation to BigQuery

So I have a dataset spread over multiple and ever-growing excel files all of which looks like: email order_ID order_date [email protected] 1234 23-Mar-2021 [email protected] 1235 23-Mar-2021 user2@...
1 vote
1 answer
1k views

Clustering for Sparse Data Matrix of high dimension

I currently have a dataset of 1000 entries with 512 features that are sparse. I want to cluster them. I have attempted using kmeans, but found that the clustering wasn't very good, and have been ...
2 votes
2 answers
575 views

Estimating the uncertainty of regression models

Given a regression model, with n features, how can I measure the uncertainty or confidence of the model for each prediction? Suppose for a specific prediction the accuracy is amazing, but for another ...
2 votes
1 answer
216 views

How can I reduce the number of dimensions using a Clustering algorithm in a mixed dataset?

I am working with a mixed data set, corresponding to TV consumption data, with the aim of reducing the number of features to only those relevant to detect TV consumption patterns (or consumption ...
0 votes
0 answers
21 views

What is the most accurate way of computing the evaluation time of a neural network model?

I am training some neural networks in pytorch to use as an embedded surrogate model. Since I am testing various architectures, I want to compare the accuracy of each one, but I am also interested in ...
0 votes
2 answers
774 views

Handling unknown words when making NER Models

I'm working on my custom Named Entity Recognition model that I'm making in Python's Keras lib. I have read that I should enumerate all words that are appearing, so that I get vectorized sequences. I ...
1 vote
1 answer
324 views

LSTM with input of actual time step

I'm working on an implementation of LSTM neural network to forecast energy consumption. I have a dataset with load, series of weather parameters and indicator of it's bank holiday or not. I first ...
2 votes
1 answer
84 views

Which is the best algorithm for entity extraction for unstructured document

I have unstructured documents from which I have to extract the information like let buyer name, seller name, expiry date, buying date etc. I had planned to use spacy(Custom entity recolonization(...
0 votes
1 answer
51 views

Using a SVM to classify audio data

If I had 1000 audio files where three people are independently saying an animal at the same time, there can be 9 independent labels of animals. What features should I select from the audio file, and ...
1 vote
1 answer
1k views

How to match a word from column and compare with other column in pandas dataframe

I have the below dataframe ...
6 votes
3 answers
15k views

When to use Random Forest

I understand Random Forest models can be used both for classification and regression situations. Is there a more specific criteria to determine where a random forest model would perform better than ...
0 votes
1 answer
18 views

Why is my Transformer model outputting the same class for every token and not improving despite decreasing loss?

I'm currently training a Transformer model for a sequence classification task using CrossEntropyLoss. My input tensor has the shape (batch_size, classes, seq_len) and my target tensor has the shape (...
1 vote
1 answer
240 views

activation function for binarized neural networks

I am trying to implement a binarized neural network using keras and tensorflow. Here is my current code: ...
6 votes
2 answers
9k views

High RMSE and MAE and low MAPE

I have used a few regression models on the same dataset and obtained error metrics for them as shown below, The RMSE(Root Mean Squared Error) and MAE(Mean Absolute Error) for model A is lower than ...
3 votes
2 answers
4k views

R NbClust for Python?

Does anyone know of a Python equivalent to the R package NbClust? I am searching for a way to determine the optimal number of clusters for many 'similar datasets' and NbClust sounds good but it is for ...
1 vote
1 answer
421 views

How to find average lag time with variance & confidence of two time series

I have two variables as time series, one a consequent of the other, I would like to find the average time delay it takes the dependent variable to act on the independent variable. Additionally, I ...
0 votes
1 answer
142 views

Summarize 3 variables into one and calculate a „skill“value + ranking

I have a data set that looks like this: Name Top Speed Number Sprints Cumulative Sprint distance Xyz 55 300 33.3 Xyz123 45 350 32.0 Top Speed is in km/h. Cumulated Sprint distance is in km. Number ...
1 vote
1 answer
613 views

Distribution of Regression Residuals: Is this a normal distribution?

I've created a histogram as well as a QQPlot from the residuals of my Regression Model: Mean: 0.35 Standard Deviation: 18.14 Judging from these plots, is it okay to say that my residuals are normally ...
1 vote
1 answer
220 views

How to find the distribution of a single variable based on population distribution

I am trying to find the age distribution of the subset of married people of a population. However, I realise that just visualising the married count by age (Figure 1) is highly dependent on the ...
0 votes
1 answer
189 views

How can I calculate total days past due between billing events?

I am dealing with a dataframe with subscription events partitioned by username, subscription status, and relative timestamps. For each of the dates, there are changes in time when the subscription ...
1 vote
1 answer
209 views

SVM radial basis generate equation for hyperplane

I need to generate an equation for hyperplane, I have two independent variables and one binary dependent variable. Regarding this following equation for svm , $f(x)=sgn( sum_i alpha_i K(sv_i,x) + b )$...
0 votes
3 answers
891 views

Random Forest Target/prediction maximum minimum

I have a regression random forest that is targeting a continuous variable (sales amount). Is there any configuration for the classifier that allows me to set a maximum and a minimum for the target? ...
0 votes
0 answers
30 views

Fuzzy Name Matching with Machine Learning. Input data encoding

I have a huge dataset: Last name, first name, date of birth of Indian residents and I need to match them for similarity. The matching is fuzzy, the data looks like this (names are fictitious for the ...
0 votes
1 answer
128 views

Can lag features be applied into test data without label?

can lag features be applied into test data without label? I've been wondering. I tried to build random forest model using dataset: training data (with label Y) and testing data (without label Y). The ...
2 votes
1 answer
83 views

How do I interpret probability results in conjunction with my known precision/accuracy/recall scores?

I have a Random Forest Classifier (trained with sklearn) modeling a binary data set. Here's what the configuration looks like (I've tuned it for precision intentionally): ...
1 vote
1 answer
1k views

Naive Bayes always predicting the same label

I have been trying to write a naive bayes classifier from scratch that is supposed to predict the class label of the nominal car.arff dataset. However the classifier always predicts the most common ...
0 votes
2 answers
436 views

Time Series - Anomaly Detection

I have time-series data with alerts (every minute) that I need to find anomalies in. I am looking for a library which can do unsupervised learning of this data and detect anomalies in the data. Which ...
2 votes
1 answer
4k views

Getting confusion matrix with Keras flow_from_directory

For a homework I have to analyse a set of images. For this I plan to use convolutional neural network. The images are split onto specific folders : A test set with 624 photos dataset/test/normal (...
0 votes
1 answer
104 views

Azure Cloud SQL - Querying large number of rows with Python

I have a Python Flask application that connects to an Azure Cloud SQL Database, and uses the Pandas read_sql method with SQLAlchemy to perform a select operation on a table and load it into a ...
1 vote
1 answer
17 views

Tracking people problem

I am trying to implement a system for counting people in a building using Python and OpenCV, but I have one problem. The system for tracking people doesn't work very well; it can't track a person all ...
0 votes
1 answer
188 views

How to Calculate semantic similarity between video captions?

I intend to calculate the accuracy of a caption generated by comparing it to a number of reference sentences. For example, the captions for one video are as follows: These captions are for the same ...
1 vote
2 answers
1k views

How can I do a train test split for an unbalanced panel data set in Python?

I have an unbalanced, panel pandas data frame. I would like to split this data into a training set and a testing set. Python's train_test_split method will not work ...
0 votes
2 answers
369 views

Does autoencoder with few images not work?

I am trying to load 2 images to an autoencoder but for some reason, it does not rebuild the input image. An autoencoder is supposed to compress and decompress an image. However, when passing two ...

1
2 3 4 5
135