Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
0
votes
0answers
15 views

How to use YAML files as input for creating custom labels in Pandas?

I have a data frame that has a column called Price. This column contains continuous numeric values. The custom labels are created using the following code: ...
0
votes
0answers
15 views

Using Raspberry PI as a remote classifier

I am currently looking for similar projects to use a Raspberry PI as a classifier. I want to run a service on the microcontroller permanently, so that the model is loaded only once and then waits for ...
1
vote
0answers
9 views

How Can I Estimate the Full Path Route of an Aircraft From a Given Halft Path?

I have a lot of real data of aircrafts paths which starts from the departure airport and ends on the arrival airport. Each aircrafts data is a time series of points like below: Point(lat, lon, ...
0
votes
0answers
41 views

is there a way to compare two k-prototype models performance?

i try to understand which of my two k-prototype models to use both models contains the same categorical features one of the models contains a bit more numerical features in other words is there a ...
0
votes
0answers
10 views

Anomaly Detection in Discrete Sequences

So, I have a set of discrete sequences, let's say, composed by letters, representing a certain action: A -> B -> C -> D B -> E -> A -> C ... My ...
0
votes
1answer
15 views

how to do feature engineering on Atari Pong game in code?

In RL context, I know that features are explanatory variables that represent or describe the states. If I want to do feature engineering on atari games and use it to solve RL task, how should I ...
0
votes
0answers
9 views

NLP: Checking that answers to a question are correct

Question answering is a common topic within NLP, but my problem is a little different: rather than answering a question, I have a question, an (open-ended) answer, and what I want to check is if that ...
0
votes
1answer
36 views

How to gps data anomaly detection in python

I have gps format dataset lat, lon. I want to detection anomaly using python. I tested knn, smv, cof, iforest using pycaret. But i did not. These colors anomlay because the angle change is too much ...
0
votes
0answers
5 views

Tracking methods for pose estimation task

I'm working on the task of tracking the output of a pose estimation model. This model returns for each frame 17 joints for each person in the scene and I want to track each pose in order to assign a ...
0
votes
1answer
34 views

Interpreting 'values' of a Decision Tree

I am trying to interpret my decision tree here which was resulted as a part of pre-pruning- I am trying to understand why the values in my nodes are in decimal places. Ideally, they should represent ...
0
votes
0answers
12 views

Options for creating a server based runtime environment for our users

We are a medium sized organization looking to provide some sort of server based computing environment for our users. Ideally we are looking for a self service solution where users can schedule and ...
0
votes
0answers
15 views

Does every column need to be np.log?

I have questions regarding data cleaning for machine learning. Let's say my dataset has three columns with different skewness For example: label column skewness = 1.500, feature column 1 ...
0
votes
0answers
12 views

Keras CNN low training and val acc

I don't what is wrong with my current network architecture maybe some of you can help. I have a dataset that is highly imbalanced so i have implemented a datagenerator which balances the image data so ...
0
votes
0answers
10 views

How can you predict the amount of each expression of the dependent variable for given independent variables?

The starting position is the following: There are categories 1 and 2, as well as features A, B and C. A representation would look like this: What is a way to not only predict the occuring categories (...
0
votes
2answers
18 views

How to convert String data to something more meaningful for regression model training

I have a bunch of data of employees and their salaries. I would like to build a regression model that predicts The columns in question are countries, employment_status, job_title, education All of ...
0
votes
0answers
16 views

Separate discrete and continuous variables

I know how to separate numerical and categorical data as follows: ...
0
votes
3answers
41 views

How to compare two columns of two dataframes and indicate the changes?

I have two dataframes, one is current week's information, one is of last week. I want to create a new dataset that lists all the changes during the week. Please see the following example: if there is ...
0
votes
0answers
12 views

How can I improve calibration curves?

I am training a binary xgboost classifer with an imbalance of : 85% = 0 class and 14 % = class 1. This was achieved after i took a random sample fromaround 11m to 1M. When i calibrate i get the ...
0
votes
0answers
11 views

Forecasting for multiple (Irregular spaced) Timeseires with interdependence

I have already asked a simmilar question, but i thoguth that this was not phrased well and hence i am trying a new post were i ask a better question. Let me know if this is ok. Judging by some of the ...
0
votes
1answer
15 views

Predicting Antennae Coverage using SPLAT

I have been trying to predict the network coverage based on antennae characteristics (such as height, lat and long etc). Came across this package which seems to be dormant now (https://github.com/NZRS/...
0
votes
1answer
13 views
2
votes
1answer
18 views

Data preprocessing framework/library alternatives

I am currently working on some python machine learning projects that are soon to be deployed to production. As such, in our team we are interested in doing this the most "correct" way, ...
0
votes
0answers
11 views

Association rule mining for continuous variables

I'm trying to study the relationships between several numerical variables, eg. electricity generation between different stations at 30min intervals over several months. My data has the format I want ...
0
votes
0answers
7 views

chat bot nfL6.json training data

I am experimenting with chatterbot and the nfL6.json training data set. The data looks like this below, should I be using an nbestanswers or ...
0
votes
1answer
16 views

Can a Transformers be used for a classification problem?

I want to perform binary classification on a sequential data. I want to leverage the PyTorch nn.TransformerEncoder module, is there a way I can do this? If not, are Transformers only used NLP? If yes, ...
1
vote
0answers
31 views

Can numerical encoding really replace one-hot encoding?

I am reading these articles (see below), which advocate the use of numerical encoding rather than one hot encoding for better interpretability of feature importance output from ensemble models. This ...
1
vote
0answers
16 views

How to convert varians of a character to the same character in python? [closed]

How to convert variants of a character to a same character in python. For example, the character a with the U+0061 Unicode has ...
1
vote
0answers
29 views

Error parsing message with type 'tensorflow.SavedModel'

This is the error message i get- ...
0
votes
0answers
18 views

Correctly plotting CCDF of network one-way delay

I have a histogram of values of test setup network. Values are from iperf 2.1.6. I send stream of data and get how many packets are in a bin of microseconds. bin(w=100us) I lose some packets sometimes....
0
votes
1answer
28 views

Building a relevant predictive model [closed]

I have a dataset on Covid and diet: https://www.kaggle.com/mariaren/covid19-healthy-diet-dataset This dataset includes percentage of fat, food quantity, energy, protein intake from different types of ...
0
votes
1answer
29 views

Custom vectorizer transformer in sklearn with cross validation

I created a custom transformer class called Vectorizer() that inherits from sklearn's ...
2
votes
2answers
33 views

Estimating the uncertainty of regression models

Given a regression model, with n features, how can I measure the uncertainty or confidence of the model for each prediction? Suppose for a specific prediction the accuracy is amazing, but for another ...
1
vote
0answers
13 views

Handling imbalanced Feature (X) not lavbel (Y) in machine learning

I am very new to this field and have done a decent amount of research on this, but every time, I stumble upon handling the imbalanced label by using f1 score, recall, precision as metrics, and using ...
0
votes
1answer
38 views

How to perform regression on image data using Tensorflow?

Overview I understand the surface of the mathematics* of simple neural networks. I went through single label image clasification problems (ie using MNIST & fashion-MNIST datasets) using native ...
0
votes
1answer
14 views

Cross Validation with One Class Classification in Python

I'm trying to do cross-validation with One Class Classification - I'm using the PyOD lib - but I don't know if I'm doing it right. The precision is too low and I'm also not able to bring up the mean ...
1
vote
0answers
30 views

Naive Bayes ValueError: Dimension Mismatch

I am attempting to make predictions of categories of text data, one of which is Naive Bayes. The training data contains 7 categories, 802 data points. After balancing with SMOTE all 7 categories now ...
2
votes
0answers
120 views

How to use the keras.layers.AdditiveAttention correctly?

My understanding on the topic is superficial at best, so do bear with me. I have a couple questions (specifically on how to use keras.layers.AdditiveAttention) which I hope is suitable to be asked ...
0
votes
1answer
39 views

How to combine preprocessor/estimator selection with hyperparameter tuning using sklearn pipelines?

I'm aware of how to use sklearn.pipeline.Pipeline() for simple and slightly more complicated use cases alike. I know how to set up pipelines for homogeneous as well ...
0
votes
0answers
14 views

Sampling from Discrete Space using Latin Hypercube

I wanted to use Discrete Latin Hypercube to sample 20 samples from a space created by an array that has 4096 values. So my desired output would be something like the following: ...
0
votes
1answer
12 views

Could I use some elements of my target variable to predict it?

I'm trying to predict if a company will bankrupt, I use a dataset of 2020 and I manually created my target variable with the status of the company the status date, status reason to create my target ...
0
votes
0answers
5 views

Methods to extract / merge signal from three noisy time series (of same event)

I have three time series of same length, all containing magnitude measurements of the same event "A". But each time series is using a different method of measurement. My goal is to merge the ...
0
votes
0answers
12 views

Training a ML model with a table for each observation

I have several csv's which are inputs for a row of outputs. A sample input dataset can look something like this: whose output would be as follows: The task is to train a model by reading each csv as ...
1
vote
1answer
27 views

Average of metrics using 10-fold

I'm working with 10 k-fold cross validation and I'm wanting to average the metrics, but I'm not getting it with sklearn. This is the way I am doing it and the metrics are being printed by fold. ...
1
vote
1answer
30 views

Trouble understanding regression line learned by SGDRegressor

I am working on a demonstration notebook to better understand online (incremental) learning. I read in sklearn documentation that the number of regression models that support online learning via the <...
0
votes
0answers
6 views

model loss is less but prediction is wrong

I have 100 samples having following data ...
0
votes
0answers
19 views

Setting Time and Date Format in Python

I have a dataset with a time column and a separate date column in .xlsx format. time column has values in the below format: 12:32:21.499145197 12:32:21.499145197 date column has values in the below ...
1
vote
1answer
35 views

should text augmentation take place before or after splitting the dataset?

I've a text dataset with ~20000 samples (which is not enough). I used text augmentation to "invent" more samples so essentially I've multiplied each sample by 10 - ending up with ~200000 ...
0
votes
1answer
26 views

Get the keywords from positive and negative reviews

I have trained a classifier algorithm on a sentiment analysis model which classifies the reviews scraped off Amazon as Positive or Negative. Now for each class, I want to get the keywords from the ...
0
votes
1answer
23 views

Extracting the metadata form Json file making it columns

I have the following json data file which I have converted to pandas dataframe. The columns are as follows ...
0
votes
0answers
34 views

What is wrong with my RNN-LSTM model?

I have built an RNN-LSTM where I try to forecast time-series on temperature. I have read this: https://machinelearningmastery.com/diagnose-overfitting-underfitting-lstm-models/ but my problem does not ...