Questions tagged [data-science-model]
The data-science-model tag has no usage guidance.
534
questions
0
votes
0
answers
10
views
Unable to generate useful insights on a highly cardinal data
I'm working on CRM data, did some cleaning, encoding and ran a decision tree classifier from which i plotted a feature_importance graph
From that I found that Sales person column is one of the ...
0
votes
0
answers
11
views
How to access dataframe created or upadted inside if statement outside? [closed]
A dataframe is created/updated isnide an if loop which itself is within a for loop. The DF need to be accessed outside the if statement. But outside the scope it is not updating.
...
- 1
0
votes
0
answers
10
views
ValueError: continuous is not supported
I am working on a regression problem and building a model using Random
Forest Regressor but while trying to get the accuracy I am getting
ValueError: continuous is not supported.
...
1
vote
1
answer
15
views
Combining two separate confusion matrix results from two seperate machine learning model to overall increase the True Positive accuracy
What are the steps involved if it is possible to add two confusion matrix results together to get a better final prediction. we have calculated two confusion matrixs as follows from naive bayes and ...
0
votes
0
answers
11
views
List of main statistics models
I am not able to find some list of main statistics models. Is is possible to devide statistics models into categories as supervised (regression,classification) x unsupervised (clustering) or is it ...
0
votes
0
answers
7
views
In Python, how can I transfer/remove duplicate columns from one dataset, such that the rows and columns of all datasets would be equal?
So I've been trying to improve my Random Decision Tree model for the Titanic Challenge on Kaggle by introducing a Validation Dataset, and now I encounter this roadblock, as shown by the images below:
...
1
vote
0
answers
23
views
massively imbalanced data
I am dealing with time series data with +200K (every minute for 6 months)record of gas turbine
I am trying to early detect the fault (0 or 1-fault). The issues with the data are:
1.the fault occurred ...
- 11
0
votes
0
answers
8
views
Where should I find electrolytic capacitor ageing data
I am trying to get a dataset of Electrolytic capacitors ageing and I am not being able to find one that shows the ripple current and the voltage in order to calculate its Equivalent Series Resistance (...
0
votes
0
answers
7
views
Time Series forecasting for 20,000 products using python
I am using timeseries forecasting(ARIMA) to forecast the future demands of products of a store handleing 20,000 variety of products. Currently different models are developed and used to forecast ...
0
votes
1
answer
33
views
Is there any way to artificially create a probability calibration for data coming from another model?
I have predictions, which come from a survival model, this model gives me very low probabilities, and I am not sure if they fulfill the real probability of the phenomenon.
For example, I calculate $P\...
- 2,182
0
votes
1
answer
15
views
How to count words in a dataframe?
I would like to count how many Male and female who answer (ex. Biking / Cycling).
Below is the sample data:
- 1
0
votes
2
answers
27
views
NameError: name 'librosa' is not defined [closed]
i'm working on Arabic Speech Recognition using Wav2Vec XLSR model.
While fine-tuning the model it gives the error shown in the picture below.
i can't understand what's the problem with librosa it's ...
0
votes
0
answers
18
views
Can we extract data types of features/variables from pickled model for Logistic Regression, Decision Tree, Random Forest?
I am trying to extract data types of variables/features from a pickled ML model file. I could see there is no information of the data types of variables in pickle file except for XG Boost. Is there ...
0
votes
2
answers
22
views
Create new rows based on a value in a column
My dateset is generated like the example
df = {'event':['A','B','C','D'],
'budget':['123','433','1000','1299'],
'duration_days':['6','3','4','2']}
I need ...
-1
votes
0
answers
14
views
What are the biggest risks about the ethical dilemmas in data science?
What is your opinion about ethical dilemmas in data science?, what are some things that may happen when the privacy rights are not respected?
0
votes
0
answers
14
views
Python DataScience Library vs SQL
I am working on a project where there is a necessity to store considerable data. I was wondering what is the difference between using SQL and the datascience library in python. I intend to use SQL ...
0
votes
0
answers
14
views
Model recalibration on different dataset
I have a large dataset approximately 150k rows and 1500 of positive labels on which I can train my model for binary classification.
And also I have the other dataset which is smaller and is comprised ...
- 291
0
votes
0
answers
11
views
Predicting value using LSTM
I'm currently learning about LSTM and want to make a prediction using an array as an input and have an output as a single value. I currently trying to do that by using this model:
...
0
votes
0
answers
18
views
Calculate days for each years between two dates in a pandas dataframe
I have a usecase wherein the from and to dates are provided. I need to calculate the total_days in each year and the total orders for each year. I'm currently coding in python
Below is an example
<...
- 1
0
votes
2
answers
21
views
How to retrain a model if a subset of predictions needs improvement
In my work, sometimes my client complain about a subset of predictions not being accurate. Despite I know it's nearly impossible to just change the model for fitting that subgroup, while other ...
- 359
0
votes
0
answers
8
views
How are ROC curves constructed? [duplicate]
I would like to understand how to build a ROC curve of a model. For example, if we would like to draw it by hand, what steps should we do?
Thank you.
- 320
0
votes
0
answers
12
views
Machine Learning: Predict No. of bid winning
I am trying find an approach to build a Machine Learning model capable of predicting number of bids that can be won from the number of bids placed by a company given other features. The domain of the ...
0
votes
0
answers
12
views
Good values for GridSearchCV testing
Good morning,
I'm solving a data problem in several and I'm testing the different models between them: Ridge, Lasso and ElasticNet.
I wanted the best parameters for my L1 and L2, but how do I choose ...
- 1
0
votes
0
answers
5
views
Model transfer with limit to none label information
I have this problem I hope to get some help here.
Say I have a type of product A whose measurements are X_A and an outcome property is ...
- 21
0
votes
0
answers
10
views
How to test if a curve is well described by an ellipse?
I have a set of data points in 2D, and I am trying to come up with some sort of statistical to determine if the points fall along an ellipse. My idea so far is to fit an ellipse to the points, take ...
- 1
0
votes
0
answers
14
views
Why has only sense to show the Separation criteria in the ROC plot?
I am dealing with the issue of fairness in machine learning models. One of the group fairness criteria is separation. I have read that it only makes sense to show the separation criterion using ROC ...
- 320
0
votes
1
answer
36
views
0
votes
0
answers
7
views
Calculating Demand confidence
I recently was asked to work on a business problem related to sales using Data Science. This required me to come up with a Demand confidence number that would specify how confident we are on ...
1
vote
0
answers
8
views
references on how to use shap values without the shap package
I am familiar with the shap python package and how to use it, I also have a pretty good idea about shap values in general, but it is still new to me. What I'm requesting are references (ideally python ...
0
votes
1
answer
76
views
model.fit vs model.evaluate gives different results?
The following is a small snippet of the code, but I'm trying to understand the results of model.fit with train and test dataset vs the model.evaluate results. I'm not sure if they do not match up or ...
- 101
1
vote
1
answer
25
views
HyperOpt: Finding the best modeling based on precision or f1 score
I have been using the hyperopt for 2 days now and I am trying to create logistic regression models using the hyperopt and choosing the best combination of parameters by their f1 scores. However, ...
- 11
0
votes
1
answer
15
views
Correlation Wikipedia translated pages vs number of in links is weird (scatterplot)?
I'm trying to find a correlation measure for the number of Wikipedia pages an entity (an article) has been translated to vs number of links that point to that page (both measures that can point to the ...
0
votes
0
answers
46
views
Predicting Opportunity Win/Loss using Machine Learning
I have a dataset as below.
These are closed opportunities where we have the outcome (won/lost). I want to predict whether the opportunity would be won/lost based on these features and also the time ...
- 235
0
votes
0
answers
7
views
Feature engineering after train and test splitting [duplicate]
I am new in data science, I am doing a project in which I have completed EDA part . Before splitting data into train and test I did feature engineering and now I don't have test data at all, what ...
0
votes
0
answers
30
views
Add an empty row in dataframe with condition using python
i have this 2 datafram and when i compare this 2 datafram i want to add an empty row where the row is in the first one and is not in the second i write this code but is not working for the empty row
<...
1
vote
0
answers
27
views
How do I calculate the New W1 and New W2?
You are training the following perceptron. The neuron in this perceptron has a sigmoid activation function. The sigmoid function is represented by the following equation:
Using the update function ...
- 111
1
vote
1
answer
23
views
Lagged Features
Lets look for example, at the forecast the sales of a retail outlet.
If I understood the concept correctly, than a lagged feature would be the sales of a previous month t−1.
Would it make sense/is it ...
0
votes
0
answers
9
views
Generate Legal Advice (Text Generation)
Somebody came to me with this use case/question and as I am not an expert yet on NLP, NLTK, spacy, GPT-x Then I dont know if this is possible and I would like to get some feedback.
We have hundreds of ...
- 111
0
votes
0
answers
9
views
Collaborative predictive modeling
How do you share modeling work among several programmers?
Our team has split apart the work of writing the SQL code to create our dataset. However, we will soon need to build a machine learning model.
...
0
votes
1
answer
9
views
Do we need to define model everytime we need to train data in LSTM?
Suppose if I have two datasets where 1st dataset is AAPL stock price and 2nd dataset is GOOGL stock price.
Now if I define the model as
...
- 157
2
votes
0
answers
59
views
MinMaxScaler makes my prediction flat
I am trying to do univariate forecasting. But when i try to use MinMax Scaler my predictions are being flat (tried to use different activation functions) but when i use Standart Scaler my predictions ...
- 23
1
vote
0
answers
12
views
How to compare new model to current production model?
Given new data, I trained the same model architecture and same hyperparameters (for example a random forest) as the current production model. How do I know that the new model that I trained is better ...
- 111
1
vote
0
answers
43
views
How to tune a Catboost Regressor
I have been trying to study about hyperparameter tuning for CatBoost regressor for my regression problem. The only issue being I can't figure out what all parameters should I tune for my use case out ...
- 11
0
votes
0
answers
38
views
Why coefficients from logistic regression are not proportional to bad rate?
I am building a logistic regression model in Python with statsmodels.api.Logit. The model contains 12 features that are encoded using ...
- 1
0
votes
1
answer
34
views
What the difference between a flattening validation curve and one that increases again?
I know that we monitor the validation loss to investigate overfitting. I am familiar with the validation curve that first decreases and then increases again. The increasing part means the model starts ...
- 113
1
vote
0
answers
17
views
What is the formula of gradient boosting trees model?
I have been reading about gradient boosting trees (GBT) in some machine learning books and papers, but the references seem to only describe the training algorithms of GBT, but they do not describe the ...
- 66
0
votes
0
answers
37
views
How to interpret the score output by a binary classifier when using a threshold < 0.5?
My understanding is that a score output by a binary classifier e.g. logistic regression for an input instance, is interpreted as the probability of the instance belonging to class 1. The threshold 0.5 ...
- 66
1
vote
0
answers
16
views
Why are there different technique to measure distance between two vector?
While learning for ML, came across different ways to means the distance between vector Like L1 norms, L2 norms, Lp norms and Cosine. Beside the formula of each of these methods, how are these ...
- 11
0
votes
0
answers
12
views
Concatenate different image bands in same input channel
Take a multispectral image with many bands (pixel matrices values).
I am thinking about concatenating the pixels values in the following way to be inputted into a CNN:
1st Channel: Red, Green and ...
0
votes
1
answer
29
views
Can I include a quotient as dependent variable and independent variables with same denominator in a linear model? How do we interpret such models?
I want to create a model in a food processing plant where my dependent variable is Electricity (KWhr) consumption per kg. Plant produce different food items with varying electricity consumption. I'm ...
- 109