2022 Developer Survey is open! Take survey.

Questions tagged [data-science-model]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

Unable to generate useful insights on a highly cardinal data

I'm working on CRM data, did some cleaning, encoding and ran a decision tree classifier from which i plotted a feature_importance graph From that I found that Sales person column is one of the ...
user avatar
0 votes
0 answers
11 views

How to access dataframe created or upadted inside if statement outside? [closed]

A dataframe is created/updated isnide an if loop which itself is within a for loop. The DF need to be accessed outside the if statement. But outside the scope it is not updating. ...
user avatar
  • 1
0 votes
0 answers
10 views

ValueError: continuous is not supported

I am working on a regression problem and building a model using Random Forest Regressor but while trying to get the accuracy I am getting ValueError: continuous is not supported. ...
user avatar
1 vote
1 answer
15 views

Combining two separate confusion matrix results from two seperate machine learning model to overall increase the True Positive accuracy

What are the steps involved if it is possible to add two confusion matrix results together to get a better final prediction. we have calculated two confusion matrixs as follows from naive bayes and ...
user avatar
0 votes
0 answers
11 views

List of main statistics models

I am not able to find some list of main statistics models. Is is possible to devide statistics models into categories as supervised (regression,classification) x unsupervised (clustering) or is it ...
user avatar
0 votes
0 answers
7 views

In Python, how can I transfer/remove duplicate columns from one dataset, such that the rows and columns of all datasets would be equal?

So I've been trying to improve my Random Decision Tree model for the Titanic Challenge on Kaggle by introducing a Validation Dataset, and now I encounter this roadblock, as shown by the images below: ...
user avatar
1 vote
0 answers
23 views

massively imbalanced data

I am dealing with time series data with +200K (every minute for 6 months)record of gas turbine I am trying to early detect the fault (0 or 1-fault). The issues with the data are: 1.the fault occurred ...
user avatar
  • 11
0 votes
0 answers
8 views

Where should I find electrolytic capacitor ageing data

I am trying to get a dataset of Electrolytic capacitors ageing and I am not being able to find one that shows the ripple current and the voltage in order to calculate its Equivalent Series Resistance (...
user avatar
0 votes
0 answers
7 views

Time Series forecasting for 20,000 products using python

I am using timeseries forecasting(ARIMA) to forecast the future demands of products of a store handleing 20,000 variety of products. Currently different models are developed and used to forecast ...
user avatar
0 votes
1 answer
33 views

Is there any way to artificially create a probability calibration for data coming from another model?

I have predictions, which come from a survival model, this model gives me very low probabilities, and I am not sure if they fulfill the real probability of the phenomenon. For example, I calculate $P\...
user avatar
0 votes
1 answer
15 views

How to count words in a dataframe?

I would like to count how many Male and female who answer (ex. Biking / Cycling). Below is the sample data:
user avatar
0 votes
2 answers
27 views

NameError: name 'librosa' is not defined [closed]

i'm working on Arabic Speech Recognition using Wav2Vec XLSR model. While fine-tuning the model it gives the error shown in the picture below. i can't understand what's the problem with librosa it's ...
user avatar
0 votes
0 answers
18 views

Can we extract data types of features/variables from pickled model for Logistic Regression, Decision Tree, Random Forest?

I am trying to extract data types of variables/features from a pickled ML model file. I could see there is no information of the data types of variables in pickle file except for XG Boost. Is there ...
user avatar
0 votes
2 answers
22 views

Create new rows based on a value in a column

My dateset is generated like the example df = {'event':['A','B','C','D'], 'budget':['123','433','1000','1299'], 'duration_days':['6','3','4','2']} I need ...
user avatar
-1 votes
0 answers
14 views

What are the biggest risks about the ethical dilemmas in data science?

What is your opinion about ethical dilemmas in data science?, what are some things that may happen when the privacy rights are not respected?
user avatar
0 votes
0 answers
14 views

Python DataScience Library vs SQL

I am working on a project where there is a necessity to store considerable data. I was wondering what is the difference between using SQL and the datascience library in python. I intend to use SQL ...
user avatar
0 votes
0 answers
14 views

Model recalibration on different dataset

I have a large dataset approximately 150k rows and 1500 of positive labels on which I can train my model for binary classification. And also I have the other dataset which is smaller and is comprised ...
user avatar
0 votes
0 answers
11 views

Predicting value using LSTM

I'm currently learning about LSTM and want to make a prediction using an array as an input and have an output as a single value. I currently trying to do that by using this model: ...
user avatar
0 votes
0 answers
18 views

Calculate days for each years between two dates in a pandas dataframe

I have a usecase wherein the from and to dates are provided. I need to calculate the total_days in each year and the total orders for each year. I'm currently coding in python Below is an example <...
user avatar
  • 1
0 votes
2 answers
21 views

How to retrain a model if a subset of predictions needs improvement

In my work, sometimes my client complain about a subset of predictions not being accurate. Despite I know it's nearly impossible to just change the model for fitting that subgroup, while other ...
user avatar
  • 359
0 votes
0 answers
8 views

How are ROC curves constructed? [duplicate]

I would like to understand how to build a ROC curve of a model. For example, if we would like to draw it by hand, what steps should we do? Thank you.
user avatar
  • 320
0 votes
0 answers
12 views

Machine Learning: Predict No. of bid winning

I am trying find an approach to build a Machine Learning model capable of predicting number of bids that can be won from the number of bids placed by a company given other features. The domain of the ...
user avatar
0 votes
0 answers
12 views

Good values for GridSearchCV testing

Good morning, I'm solving a data problem in several and I'm testing the different models between them: Ridge, Lasso and ElasticNet. I wanted the best parameters for my L1 and L2, but how do I choose ...
user avatar
0 votes
0 answers
5 views

Model transfer with limit to none label information

I have this problem I hope to get some help here. Say I have a type of product A whose measurements are X_A and an outcome property is ...
user avatar
  • 21
0 votes
0 answers
10 views

How to test if a curve is well described by an ellipse?

I have a set of data points in 2D, and I am trying to come up with some sort of statistical to determine if the points fall along an ellipse. My idea so far is to fit an ellipse to the points, take ...
user avatar
  • 1
0 votes
0 answers
14 views

Why has only sense to show the Separation criteria in the ROC plot?

I am dealing with the issue of fairness in machine learning models. One of the group fairness criteria is separation. I have read that it only makes sense to show the separation criterion using ROC ...
user avatar
  • 320
0 votes
1 answer
36 views

regression model behaves (predicts) like classification

I have a simple data: ...
user avatar
  • 1
0 votes
0 answers
7 views

Calculating Demand confidence

I recently was asked to work on a business problem related to sales using Data Science. This required me to come up with a Demand confidence number that would specify how confident we are on ...
user avatar
1 vote
0 answers
8 views

references on how to use shap values without the shap package

I am familiar with the shap python package and how to use it, I also have a pretty good idea about shap values in general, but it is still new to me. What I'm requesting are references (ideally python ...
user avatar
0 votes
1 answer
76 views

model.fit vs model.evaluate gives different results?

The following is a small snippet of the code, but I'm trying to understand the results of model.fit with train and test dataset vs the model.evaluate results. I'm not sure if they do not match up or ...
user avatar
  • 101
1 vote
1 answer
25 views

HyperOpt: Finding the best modeling based on precision or f1 score

I have been using the hyperopt for 2 days now and I am trying to create logistic regression models using the hyperopt and choosing the best combination of parameters by their f1 scores. However, ...
user avatar
  • 11
0 votes
1 answer
15 views

Correlation Wikipedia translated pages vs number of in links is weird (scatterplot)?

I'm trying to find a correlation measure for the number of Wikipedia pages an entity (an article) has been translated to vs number of links that point to that page (both measures that can point to the ...
user avatar
0 votes
0 answers
46 views

Predicting Opportunity Win/Loss using Machine Learning

I have a dataset as below. These are closed opportunities where we have the outcome (won/lost). I want to predict whether the opportunity would be won/lost based on these features and also the time ...
user avatar
  • 235
0 votes
0 answers
7 views

Feature engineering after train and test splitting [duplicate]

I am new in data science, I am doing a project in which I have completed EDA part . Before splitting data into train and test I did feature engineering and now I don't have test data at all, what ...
user avatar
0 votes
0 answers
30 views

Add an empty row in dataframe with condition using python

i have this 2 datafram and when i compare this 2 datafram i want to add an empty row where the row is in the first one and is not in the second i write this code but is not working for the empty row <...
user avatar
1 vote
0 answers
27 views

How do I calculate the New W1 and New W2?

You are training the following perceptron. The neuron in this perceptron has a sigmoid activation function. The sigmoid function is represented by the following equation: Using the update function ...
user avatar
  • 111
1 vote
1 answer
23 views

Lagged Features

Lets look for example, at the forecast the sales of a retail outlet. If I understood the concept correctly, than a lagged feature would be the sales of a previous month t−1. Would it make sense/is it ...
user avatar
0 votes
0 answers
9 views

Generate Legal Advice (Text Generation)

Somebody came to me with this use case/question and as I am not an expert yet on NLP, NLTK, spacy, GPT-x Then I dont know if this is possible and I would like to get some feedback. We have hundreds of ...
user avatar
0 votes
0 answers
9 views

Collaborative predictive modeling

How do you share modeling work among several programmers? Our team has split apart the work of writing the SQL code to create our dataset. However, we will soon need to build a machine learning model. ...
user avatar
0 votes
1 answer
9 views

Do we need to define model everytime we need to train data in LSTM?

Suppose if I have two datasets where 1st dataset is AAPL stock price and 2nd dataset is GOOGL stock price. Now if I define the model as ...
user avatar
2 votes
0 answers
59 views

MinMaxScaler makes my prediction flat

I am trying to do univariate forecasting. But when i try to use MinMax Scaler my predictions are being flat (tried to use different activation functions) but when i use Standart Scaler my predictions ...
user avatar
  • 23
1 vote
0 answers
12 views

How to compare new model to current production model?

Given new data, I trained the same model architecture and same hyperparameters (for example a random forest) as the current production model. How do I know that the new model that I trained is better ...
user avatar
1 vote
0 answers
43 views

How to tune a Catboost Regressor

I have been trying to study about hyperparameter tuning for CatBoost regressor for my regression problem. The only issue being I can't figure out what all parameters should I tune for my use case out ...
user avatar
0 votes
0 answers
38 views

Why coefficients from logistic regression are not proportional to bad rate?

I am building a logistic regression model in Python with statsmodels.api.Logit. The model contains 12 features that are encoded using ...
user avatar
0 votes
1 answer
34 views

What the difference between a flattening validation curve and one that increases again?

I know that we monitor the validation loss to investigate overfitting. I am familiar with the validation curve that first decreases and then increases again. The increasing part means the model starts ...
user avatar
  • 113
1 vote
0 answers
17 views

What is the formula of gradient boosting trees model?

I have been reading about gradient boosting trees (GBT) in some machine learning books and papers, but the references seem to only describe the training algorithms of GBT, but they do not describe the ...
user avatar
0 votes
0 answers
37 views

How to interpret the score output by a binary classifier when using a threshold < 0.5?

My understanding is that a score output by a binary classifier e.g. logistic regression for an input instance, is interpreted as the probability of the instance belonging to class 1. The threshold 0.5 ...
user avatar
1 vote
0 answers
16 views

Why are there different technique to measure distance between two vector?

While learning for ML, came across different ways to means the distance between vector Like L1 norms, L2 norms, Lp norms and Cosine. Beside the formula of each of these methods, how are these ...
user avatar
  • 11
0 votes
0 answers
12 views

Concatenate different image bands in same input channel

Take a multispectral image with many bands (pixel matrices values). I am thinking about concatenating the pixels values in the following way to be inputted into a CNN: 1st Channel: Red, Green and ...
user avatar
0 votes
1 answer
29 views

Can I include a quotient as dependent variable and independent variables with same denominator in a linear model? How do we interpret such models?

I want to create a model in a food processing plant where my dependent variable is Electricity (KWhr) consumption per kg. Plant produce different food items with varying electricity consumption. I'm ...
user avatar
  • 109

1
2 3 4 5
11