Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (which should be asked on Stack Overflow).

Filter by
Sorted by
Tagged with
0 votes
0 answers
12 views

How to solve MemoryError problem?

I have audio signals which I need to convert them to melspectrogram. I am using Deepfilternet to remove background noise. when I use the output of Deepfilternet for next phase like padding, it showed ...
Zara Nz's user avatar
0 votes
0 answers
23 views

Python Darts API

I'm using the darts API for some time series prediction in python, while looking at the documentation for the RNN model I saw this parameters called training length. The description is not very clear ...
Guilherme Takata's user avatar
1 vote
0 answers
7 views

Huge variance for RandomForestRegressor models

The experiment is the following: train a RFR with a 15k train rows get predictions on 8k test rows, save predictions as y_hat0 remove 1 random row from the training set and retrain the RFR save ...
Oleg's user avatar
  • 11
0 votes
0 answers
7 views

YOLOV4 model training

I can't start my model tutorial. I ran it correctly, I tried it before, but now it doesn't even start. why could it be? Did the update come?
Elif Deniz's user avatar
0 votes
0 answers
22 views

Wavelet choice for wav file analysis

Context I am working on the Maestro v3.0 dataset https://magenta.tensorflow.org/datasets/maestro#v300 to familiarize myself with signal processing. Years ago, I mainly worked on image processing with ...
zar3bski's user avatar
  • 101
1 vote
1 answer
51 views

Time series prediction on test set produces multiple cycles instead of following trend

I have a non-stationary time series where I am trying to build a model for forecasting. So far on test set it produces multiple cycles no matter which technique I use. There's just one feature ...
Thermal_insulator's user avatar
0 votes
1 answer
30 views

How can I solve this kind of problem about predicting the sequence of once in life events?

So let's imagine I have a dataset of children. For each of them a have a bunch of characteristics (generation, gender, race, class, urban/rural, religion, bmi, number of siblings etc..) and plus the ...
Floralys's user avatar
0 votes
3 answers
161 views

Clusterise a group of answers taking the question as relevant information

I am solving a problem where I group answers to a given question into clusters using k-means algorithm. The steps I follow are: For every answer I get the corresponding vector. Reduce the vector ...
jesantana's user avatar
0 votes
0 answers
38 views

Perform k-means clustering over multiple columns using python but without library sklearn.kmeans

I'm on a project with the topic "clustering" using the KMeans method. so I have a data set with 10 columns and 1000 rows divided into 5 clusters (in this case there are 5 robot movements). I'...
Higashida Cyza's user avatar
0 votes
1 answer
75 views

Workflow for reinforcement learning in Python

I'm new to reinforcement learning but (I believe) I have certain experience with machine learning. I know if you work on, say, a classification problem, your workflow will be (in a very high level) ...
ElonMuskofBadIdeas's user avatar
0 votes
0 answers
29 views

Imputation of multi-peaked dataset

I am a beginner to model fitting, and I have been working on generating a model for a CO2 emissions data set. The distribution of the data points in a number of these columns are very markedly dual-...
Phindolin's user avatar
0 votes
0 answers
423 views

Aggregate a DataFrame using multiple conditions in groupby

I have a dataframe with many duplicates. I have determined rules for duplicate identification, for example : If street_name AND ...
Rusoiba's user avatar
  • 829
0 votes
1 answer
39 views

9-dimensional data classification

I have a dataset with 9 columns and 20 million rows. I have now split the dataset into two parts using IQR. One part is my inliers, the other part is my outliers. I now want to build a categorizer ...
Sponge_Data_45's user avatar
0 votes
1 answer
46 views

What data am I supposed to store into the "input_data" variable?

I'm supposed to pass preprocessed input to a TFLite model to make inferences. I have an excel file containing data and a tensorflowlite model file. I received instructions on how to do this; here is ...
Feynman's user avatar
0 votes
0 answers
46 views

Matrix Factorization with SVD/Gradient Descent - Dealing with new users

I'm working on implementing matrix factorization for a recommender system. I implemented matrix factorization, once with stochastic gradient descent and once with scipy's svds() function. The thing ...
0diraf's user avatar
  • 1
0 votes
0 answers
24 views

Use ML.NET trained model in Python

How would i use a ML.NET trained (with VS ML Model Builder) model in Python? It's a .zip file btw. ZIP file structure: ...
Marcelsky's user avatar
0 votes
1 answer
89 views

`ValueError: There must be exactly two hue levels to use `split`.', while two levels?

The following plotting issue occurs in preparation for a model build: ...
Tim's user avatar
  • 5
0 votes
1 answer
53 views

Are there any R packages that support Deep RNNs?

I recently found an interesting paper on what it really means for a recurrent neural network (RNN) to be deep here. Depth can be added in several different ways (state to state, input to state, etc.) ...
noNameTed's user avatar
0 votes
0 answers
15 views

How do I reconstruct irregularly spaced lat-lon data (in the form of a matrix) using spherical harmonic fitting?

Introductory Links to spherical harmonics give a mildly rigorous introduction to the concept; but very few links describe how to actually use them. I have latitude-longitude data arranged the ...
requiemman's user avatar
0 votes
2 answers
44 views

Data Science Project Data Workflow Structure

I'm in the middle of a project of marketing regarding the sales prediction with promotions. The client has very complex business processes and so the data needs a lot of preprocessing (joins, filters, ...
ru.mp's user avatar
  • 63
-1 votes
1 answer
88 views

Falcon 7B LLM Evaluation using TruLens

The problem I am facing is after defining the prompt template, creating a chain using Langchain, defining the huggingface evaluation module from trulens_eval to check the toxicity of the response and ...
RAUNAK GHOSH's user avatar
0 votes
0 answers
49 views

Correlation Matrix for Very Large Data Set

I am tasked to create a correlation matrix on a large portion of an MRI imaging data set (all numeric entries). The dimensions of the data is around 28,000x1,800. I have already set a correlation ...
user123's user avatar
  • 101
0 votes
0 answers
27 views

Multivariate Time Series Forecasting of only 1 of the variables

I have a dataset that has time series data in the following format - 1 column for the date, 5 dependent variables, 1 independent variable (need to predict) I want to see if the dependent variables ...
aa aa's user avatar
  • 1
0 votes
0 answers
22 views

Can somebody explain to me, how the Hyperparameters in this expamle work? I only uderstood the meaning for gamma

I am quite new to this topic, but I want to understand how Reinforcement Learning (RL) works in this example (https://gymnasium.farama.org/tutorials/training_agents/reinforce_invpend_gym_v26/). I have ...
Peter's user avatar
  • 1
0 votes
1 answer
38 views

Stick to R or switch to Python as a newby in Reinforcement Learning?

Background I would consider myself to have a decent level in R and virtually 0 skills in any other programming language - something quite common in my field. Note also that my background is not in ...
ionatura's user avatar
0 votes
0 answers
34 views

How to solve error with 2D labels in partial fit regression problems?

I would like to use continual learning (incremental learning) in my code. The dataset is composed of cartesian coordinates in which X and y is the same. I've solved the fact of using a 2D in the y ...
Raúl Parada Medina's user avatar
0 votes
0 answers
26 views

Why do I get such a low accuracy despite having good metrics?

I'm working on an image segmentation problem. I'm fine-tuning the network, and when displaying the metrics I obtain, I have some doubts. I am going to provide some details about the network: Solver: ...
PicaR's user avatar
  • 304
0 votes
2 answers
69 views

How to apply supervised machine learning when the target variable depends on multiple rows (with varying number of rows)?

Here is a example dataset that resembles the one I am using. Let's call one row = one experiment. I'm trying to predict Y based on features X1 through X7 The problem I am facing is very similar to ...
Izem's user avatar
  • 1
1 vote
1 answer
206 views

What is the most straightforward way to run python code in a chrome extension?

I am currently trying to develop a chrome extension for a project. My knowledge of Javascript is quite limited and this is my first using it, so I am trying to reduce as much as possible its use. The ...
Manuel's user avatar
  • 11
0 votes
0 answers
47 views

Build a model with cross-validation on entire dataset to learn insights?

Goal : Use XGBoost regression to learn insights from data. Prediction or forecasting not needed. Hypothesis : If the model fits the entire dataset well, it can maybe capture its "physics" in ...
cwanderroycbooks's user avatar
1 vote
1 answer
30 views

Patterns binary classification - model doesn't overfit

I am working on a very basic binary classification problem. For each set of four float numbers $(x,y,z,w)$, I want to check if they fall or not into one category. I have written a model in Keras with ...
apt45's user avatar
  • 121
0 votes
0 answers
55 views

Implicit feedback based recommendation - User - User recommendation

My dataset involves customer_id, item_id and Count of purchases. My datset size is small...we have 15 users and 317 items.. Currently, am trying to build a recommendation system based on user based ...
The Great's user avatar
  • 2,555
0 votes
1 answer
27 views

How should I approach generating recommendations for fish stock management?

I've been asked to "Model and study the size and age composition of fish populations as dynamical systems for a given area and generate recommendations for seasonal fish stock management based on ...
user18937561's user avatar
0 votes
0 answers
43 views

Mean Average Precision with 11 points interpolation method Python libs

I want to calculate mAP with 11 points interpolation method for object detection, as described here: https://learnopencv.com/mean-average-precision-map-object-detection-model-evaluation-metric/ What ...
Ars ML's user avatar
  • 61
1 vote
0 answers
25 views

What approaches can I take to reconstruct missing climate data?

I have climate datasets consisting of tables where each row has four elements: latitude, longitude, rainfall, and timestamp. This dataset covers the entirety of the planet. However this dataset is ...
requiemman's user avatar
0 votes
0 answers
19 views

How to feed patient data to tflite file model?

So I have the patient data I need in an excel file, and I need to feed that data to a tflite file. Could someone point me in the right direction to do this using python, please (preferably a code ...
Feynman's user avatar
0 votes
2 answers
93 views

Should I call inverse_transform?

I built a K-Nearest Neighbors Regressor GridSearchCV which calls a sklearn pipeline as the estimator. Training appears to be working fine. My confusion is after calling predict and checking the RMSE ...
J. S.'s user avatar
  • 3
0 votes
0 answers
25 views

How to check for correlations using ANOVA between the selected feature columns and the target feature column

Suppose I have a set of feature columns that I select to check for correlations with the target feature 'is_Star'; which is of binary values and the former consists of numeric values Why do i end up ...
meKafka's user avatar
0 votes
1 answer
27 views

Do prediction intervals for Random Forest predictions are average of prediction intervals of trees being Random Forest estimators?

I am working on adding prediction intervals for each prediction value of new input samples. Do prediction intervals for Random Forest predictions are average of prediction intervals of trees being ...
Paulina's user avatar
  • 15
0 votes
3 answers
88 views

Heatmap not respecting the color bounds

For the following dataframe of a chi2 correlation study, i started to plot a heatmap: ...
Guilherme Raibolt's user avatar
0 votes
1 answer
96 views

Splitting CSV data with missing target variable using train_test_split in Python

I have a dataset in CSV format that consists of a training dataset with around 300 instances and a test dataset with around 100 instances. The issue is that the target variable (the column we want to ...
Yasmine Bzgn's user avatar
0 votes
0 answers
16 views

Error in seq2seq translation when passing predicted output to rnn due to input shape not always being the same

I'm working on a language translator and I'm getting an error I'm unsure about. During the decoding process when using argmax on the predicted output I am sometimes getting an RuntimeError ...
bailey.bailey's user avatar
0 votes
0 answers
24 views

How to build a NN for classifying custom timeseries data?

I'm trying to learn a little bit of machine learning and it occurred to me that my own household produces a lot of interesting data. Specifically, the ModBus-enabled digital power meter installed a ...
Sixtyfive's user avatar
  • 125
0 votes
0 answers
11 views

Model outputting repeated result in Flask back-end

I am trying to implement this working audio classification model into a Flask app for use on a website. However, anytime I upload an audio file, even one that was used on the test/training of the ...
DaddyMuffin's user avatar
0 votes
1 answer
28 views

How can I find what kind of args and kargs a pandas function takes?

I was wondering how do I find what are valid kargs and args for a given function? For example in pandas there is this function cumin DataFrame.cummin(axis=None, skipna=True, *args, **kwargs); how do I ...
Sam's user avatar
  • 111
0 votes
1 answer
49 views

Training and validation loss sometimes not decreasing in Keras dense layer with the same data and random seed

I want to use a Multi-Layer-Perceptron in Keras (Dense layer) to map 6 inputs to 1 output. For doing this I use the following code: ...
PeterBe's user avatar
  • 81
0 votes
1 answer
96 views

How can I compare the accuracy of imputation models if there is already missing dataset in the file?

Let's say I have a dataset of 50,000 where about 2% were already missing from the beginning. From what I have learned, we need to use indicators to compare the imputation model with the ground truth ...
Amisha Dhimal's user avatar
1 vote
0 answers
49 views

Highlight specific paragraphs from documents

I have a bunch of documents in which I want to highlight certain paragraphs/keyphrases. I have a list of the most frequently appearing sentences and I want to search for these paragraphs/keyphrases in ...
spectre's user avatar
  • 1,896
0 votes
0 answers
22 views

Should I remove noise by fitting a curve in my case?

I have an object that is moving towards target point and I want to determine when it will reach or pass through it. I receive points in real time, which represent the current location of the object. ...
theateist's user avatar
  • 141
0 votes
0 answers
24 views

s3 object as corpus_file for gensim Doc2Vec

Is it possible to use a txt or jsonl file in an s3 bucket as the corpus_file input for a gensim Doc2Vec model? I am looking for something of the form: ...
Mutasim Mim's user avatar