Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (which should be asked on Stack Overflow).

Filter by
Sorted by
Tagged with
0 votes
0 answers
30 views

How can I use two different datasets to train an ML model?

I am trying to create a machine learning model that takes in two different pandas data-frames from a basketball stats website and given multiple variables, will output a prediction of how many points ...
jshargo7's user avatar
0 votes
1 answer
34 views

What ML model is best suited for an intelligent search assistant?

I'm working on my thesis project, and want to make an intelligent search assistant that understands context and, of course, processes and repsonds in natural languaje. The data I want to train this ...
Bito's user avatar
  • 1
0 votes
0 answers
8 views

kernal failure on data science ICA algorithm for independent component analysis

ICA algorithm Problem Statement The goal is to have the ICA algorithm for independent component analysis custom python code perform convergence on many independent audio wave files, e.g., 10, 15, 20, ...
Data Science Analytics Manager's user avatar
1 vote
1 answer
125 views

What do special tokens used for in Roberta?

When I use this code: ...
user avatar
0 votes
0 answers
39 views

Quantifying heat map data

I have heatmap data (the x and y axis take in angles as inputs, and the color of that particular spot on the grid represents the "divergence.") I am unsure on how to numerically describe my ...
MaximeJaccon's user avatar
0 votes
1 answer
37 views

Why was the learning rate decreased for Roberta compared to LSTM?

I'm reading the codebase of a project that uses Bidirectional-LSTM. The learning rate for it is 0.02. Later, someone improved the project by replacing LSTM with Roberta and decreased the learning rate ...
user avatar
0 votes
0 answers
50 views

How to find adjacent neighbours using Python?

Full problem description at stackoverflow I need to find the adjacent neighbours (not necessarily nearest neighbours) to a given point in a multidimensional space. As shown in the screenshot below, I ...
skm's user avatar
  • 101
0 votes
0 answers
16 views

sampling points on an interpolated curve

I have this interpolated curve (black) using black data points. I used scipy.interpolate.griddata to obtain the curve, I was wondering if there's a way to sample function values from this ...
user159958's user avatar
0 votes
0 answers
16 views

Are there any data processing documentation tools?

I was responsible of generating a table report to a commercial area of my job. It is currently being updated daily by DAG in airtomation that runs a python script. The problem is that now I have to ...
Gustavo's user avatar
2 votes
1 answer
206 views

What do these terms mean in the context of Roberta?

When I read articles about Roberta, I often read the terms "transfer learning" and "fine-tuning". Additionally, they also mention "feature extraction". What are the ...
user avatar
0 votes
1 answer
71 views

What are the differences between Embedding Layer and Roberta Embedding?

I'm reading an article about the Embedding Layer: The Embedding Layer learns word embeddings from raw text. It is initialized with small random numbers and can be learned simultaneously with a neural ...
user avatar
0 votes
0 answers
27 views

sklearn Random Forest classifier vs R’s Random Forest classifier

I’m trying to implement the R’s random forest classifier equivalent in python- ...
Mark W's user avatar
  • 1
0 votes
2 answers
111 views

What are the differences between contextual embeddings of Bidirectional-LSTM and Transformer?

A Transformer, like Roberta, can generate contextual embeddings using the encoder part, similar to a Bidirectional-LSTM that concatenates hidden states. What are the differences between them ? Are ...
user avatar
-1 votes
1 answer
39 views

TypeError: unhashable type: 'slice' K-Means; Custom code for K-Means

Problem Statement The goal is to have the K-Means customer code run for clusters and not use scikit-learn libraries. Learning exercise. This K-means has the standard predict, fix, centroids, cluster ...
Data Science Analytics Manager's user avatar
0 votes
1 answer
53 views

Seaborn distplot and countplot displaying incorrect y-axis

I have a simple dataset created using NLTK's FreqDist function, and have successfully converted the distribution to a pandas dataframe. When visualizing that dataframe using seaborn, I get a weird ...
NotHead's user avatar
0 votes
1 answer
64 views

Illegal action reward strategy for reinforcement learning : reward shaping and termination / truncation

I have some questions about strategy to adopt regarding illagal action handling in reinforcement learning (Stable Baselines 3 / SAC algo). First is about reward shaping, second is about terminating / ...
GerardL's user avatar
  • 13
1 vote
1 answer
98 views

Predict average duration of a contract

I would like to predict the average duration of a contract. I have worked with different machine learning models before, especially from Scikit-learn, but this task seems to be somehow different for ...
Julian's user avatar
  • 11
0 votes
1 answer
46 views

Is it usual to obtain very different values of mutual information score using sci-kit learn?

Good day to everyone! I have a little question related to the function mutual_information_score of sci-kit learn. I've been using it to obtain the mutual information between two datasets A and B; ...
gengar123's user avatar
0 votes
0 answers
31 views

How to Determine the Minimum Value of a Continuous Variable for Predicting Categorical variable using Logistic Regression?

I am using logistic regression to predict df['MortSubiteCardiaque'], which contains 0 and 1, based on my continuous variable df['NTProBNP']. I would like to determine the threshold for df['NTProBNP'], ...
Mohamed kenani's user avatar
1 vote
1 answer
92 views

When would pandas.get_dummies()'s parameter of drop_first=False be appropriate to use?

While working on some case studies that use various machine learning models, I came across a project for predicting churn in the telecom industry. The Jupyter Notebook I saw had the following lines of ...
Rakesh Poluri's user avatar
0 votes
1 answer
40 views

How to optimize my CNN classification architecture

I have this CNN based model architecture that takes an RGB image. Now I'm trying to change it for a color classification case on an object (10 color classes: white, black, yellow, etc). This current ...
Mary's user avatar
  • 217
0 votes
1 answer
36 views

GMM expectation maximization ValueError: could not broadcast input array from shape (2700,1) into shape (2700,)

Problem Statement The goal is to have the Gaussian Mixture Model has_prediction() function to execute after the Gaussian Mixture Model fix() function that will invoke methods on this custom ...
Data Science Analytics Manager's user avatar
1 vote
0 answers
40 views

How do I know the appropriate number of iterations when using Miceforest for imputation?

I want to know how to avoid overfitting without having to increase the number of iterations excessively in Python with the Miceforest library. I know you can make a correlation map of data sets but I ...
Eduardo Dimas's user avatar
0 votes
0 answers
21 views

Differentiating between runs that start at different timestamps

I have a CSV file of the latitude and longitude coordinates of trains in the format [timestamp, runID, latitude, longitude]. The problem is that I have tracked ...
make nice rhombus's user avatar
0 votes
1 answer
62 views

Which ML algorithm is suitable for a dataset that has seasonality and trend?

I have a small dataset from 2006 to 2023, I would like to predict monthly sales for the next year. This is my data: I already tried Prophet and NeuralProphet, but unfortunately they don't work well ...
Hadis's user avatar
  • 1
1 vote
1 answer
43 views

Feature selection or hyperparameter tuning first for 30 feature data

I have about 30 variables and trying to create a Random Forest model. All the variables are expected to be predictors of outcome. I want to find the best model based on a C-stat score with any number ...
user2704338's user avatar
0 votes
0 answers
22 views

Beginner Question: How to automate desk allocation

I am not sure whether this is the right place, but I have my first project and don't know where to begin. I want to get into data science/ practical programming and a family friend gave me my first ...
Orange Pukeko's user avatar
0 votes
0 answers
38 views

AWSWrangler chucksize with data type conversion and conversion to Parquet

I have a lambda function that reads in json files and then converts to parquet from S3. I came across an issue where if the files are greater than 1GB, lambda runs out of memory. I'm using awswrangler ...
nahmed's user avatar
  • 1
2 votes
1 answer
844 views

What are the differences between BPE and byte-level BPE?

In Roberta, I'm not sure if the model use BPE or byte-level BPE tokenization, are these techniques different or the same ? Can someone explain ? Thanks
user avatar
-2 votes
1 answer
43 views

How to decide your first self-project for Data Science/Analysis [closed]

Suppose a complete beginner(in the field of data analysis/data science), which is my case indeed, decided to make a project for his/her resume. These are the following skills he have learnt: Python(...
Swayam Jha's user avatar
0 votes
1 answer
59 views

Sale Forecasting Problem -- Is it legit to use inventory level as a feature?

I'm working on a project to predict future sales for our company's products so that the supply chain can have better idea how much to restock. Detail about the model I'm working on: Model: LGBM (from ...
user159479's user avatar
0 votes
0 answers
17 views

Bulding Deep Learning model for multiclassification case

I am soo confused i read a lot of information in forumas and still cna't get what is wrong. my data is around 500.000 rows and 32 columns. my target variables consists of 3 classes (0, 1, 2). Hyperopt ...
Shamkhal Mammadov's user avatar
0 votes
2 answers
75 views

Is there a way to check if a time-series is seasonal without manual inspection of its decomposition?

While using various models to generate a forecast of a time-series it is needed to know if a time-series is seasonal or not. I have searched about this topic but could only find people mentioning ...
Mathias Hillmann's user avatar
0 votes
0 answers
9 views

How to find the minimum data point that predicts the target class in longitudinal data

I am working on medical data where a screening is done regularly for 200 days. I need to know the minimum number of screenings that can predict the outcome. I also need to know the best time/times to ...
Ghof-90's user avatar
2 votes
1 answer
122 views

What is the difference between hidden states in RNN and Transformers model?

I'm very terrible at NLP and I have searched for these questions but didn't find any answer, my question is, in RNNs, there are hidden states to remember information for processing the next state, and ...
user avatar
0 votes
1 answer
116 views

Calculate the best alpha, beta and gamma that should be used in the holt-winters exponential smoothing formula

With the following code I can find the best forecast value for a series by iterating over the alpha, beta and gamma and saving the result with the best RMSE: ...
Mathias Hillmann's user avatar
0 votes
0 answers
16 views

pyspark combined standard deviation?

I have a data frame which consists of multiple standard deviations with mean and count of each. my df is as follows ...
userDuR's user avatar
  • 101
0 votes
0 answers
723 views

AI tools/LLM models to analyze excel data

I have a bunch of excel files containing information of employee attendance. I want to use NLP-based search to ask questions like: Which employee has taken most leaves?; What dates witness a high ...
Apoorva's user avatar
  • 307
0 votes
0 answers
23 views

I am trying to do mle for school assignment, why could I be getting this error:

...
mobiusstrip's user avatar
0 votes
1 answer
84 views

What is the best practice to detect constant and nearly constant data over time?

I'm experimenting with the characterization of data over time. I generated some synthesized data with certain periodic patterns over time every 5mins (granularity of 5mins= data generated with the ...
Mario's user avatar
  • 400
1 vote
1 answer
73 views

Packages for Density Estimation using K-Nearest Neighbor

I would like to have suggestions for packages that provide K-Nearest Neighbor density estimator, I've already searched the web (to not bother you guys with my question :) ), but most results were ...
Neyo Goldsmith's user avatar
0 votes
1 answer
60 views

How to automate the restarting of training of deep learning model in TensorFlow

I am trying to automate the (recursively) restart of a finished deep-learning training session in TensorFlow. Currently, to restart I am manually restarting my kernel and re-running the training code. ...
user10529827's user avatar
0 votes
0 answers
17 views

Email Segregation

I am working on an email parsing project using machine learning, and our goal is to extract only those emails that require quotation. I am struggling to find a solution on how to automatically ...
Cupcake's user avatar
0 votes
1 answer
23 views

Why is there a kmeans and kmeans_plusplus function in scikit-learn?

I want to use the kmeans method in the scikit-learn library and I was reading the documentation to see if there was a parameter to use kmeans++. It turns out that this is the default behavior. However,...
Adrian Fletcher's user avatar
0 votes
0 answers
40 views

Need Expert Advice for using sklearn pipeline to create a composite estimator with multiple models & features. Column Transformer with Mixed Types

I am trying to use sklearn pipeline to create a composite estimator. Please check attached image for model blueprint. Can anyone help me understand how this can be done in python. Column Transformer ...
prafull ghare's user avatar
0 votes
1 answer
29 views

Customer Lifetime expectancy

I'm trying to compute Customer Lifetime Expectancy based on a table of subscriptions, that have 3 columns : user_id, user_subscription_start, user_subscription_end ...
ed WSA's user avatar
  • 1
0 votes
0 answers
22 views

How to treat "ignore" boxes during object detector evaluation?

I want to evaluate the performance of multiple object detectors for comparison, each of them has been trained on this data. This dataset has images labeled, where each image can have multiple labels. ...
hafiz031's user avatar
  • 121
0 votes
0 answers
52 views

Is there a way to achieve a transparent basemap with px.density_mapbox?

I'm creating a heatmap with Plotly density_mapbox, and I need the background to be fully transparent so that the heatmap can be exported to geotiff and used as an overlay for an external map. The ...
user18937561's user avatar
0 votes
0 answers
37 views

What feature selection method is ideal for a large dimensional data frame after the result of one hot encoding?

I am trying to solve a sports related multi class classification problem in Python, I aim to train a custom neural network and also a SVM. I have performed prior data cleaning and encoded my data ...
pastybake2002's user avatar
0 votes
0 answers
27 views

What is the difference between two different Keras sequential models?

I run two different Keras sequential models on IMDB data: truncate the input data by the first 100 words and translating to float32 array convert the input data using categorical array The "1&...
user1312837's user avatar