Questions tagged [data-science-model]
The data-science-model tag has no usage guidance.
324
questions
1
vote
2answers
42 views
Extract information using NLP and store it in csv file
I have a text file that stores the pickup, drops, and time. SMS text is a dummy file that is used to train a cab service model. The text is like in this format:
...
0
votes
0answers
3 views
Schedule Sagemaker with boto3 comprehend
I have a jupyter notebook running on SageMaker which I have to run daily to output CSV files. I am using boto3 'comprehend' to analyze the text in my workflow. I want to be able to schedule this daily ...
0
votes
1answer
15 views
Continuous VS Categorical variable
In the dataset I have a continuous variable AGE and categorical variable AGE_CATEGORY as well. They are highly correlated.
Which method shall I proceed to identify the feature to be dropped AGE OR ...
0
votes
0answers
10 views
Choosing a non-linear regression model and predicting
I'm new to data science and machine learning. I was working on a project and I happened to get this graph.
I want to build a predictive model using this, for each of the boroughs. I do understand ...
0
votes
0answers
9 views
What is the best possible method/methods to determine best possible branch(rule) in a decision tree plot for the positive cases only?
I have a dataset, which I am using for loan prediction. Thus, it is pretty much clear that my dataset is imbalanced. I have used Decision Tree to plot the tree structure. Now, I want to find the ...
0
votes
0answers
14 views
Why i am getting this error? [closed]
I am trying to apply NaiveBayes algorithm on covid19 data sample but this give me a error, if you have any solution kindly share with me
1
vote
1answer
21 views
WHY or WHEN to convert numeric data to a categorical data?
This is an open ended WHY TO or WHEN TO question rather than a question on HOW TO encode numeric to categorical data.
I am currently working on Telco Customer Churn dataset from kaggle.
This is ...
0
votes
0answers
9 views
Predict users $ value based on selections
I want to predict the chance of a user becoming a customer, and subsequently predict their customer value based on inputs from a new user onboarding questionnaire.
In the onboarding a new user can ...
1
vote
1answer
24 views
How do I train a model with pre-saved weights?
I need to train a model but the data after preprocessing is too large to fit in the RAM. Is it possible to preprocess the data in batches and train a model for the small data set and then save the ...
1
vote
0answers
30 views
How to group various similar search keywords and find top 100 keywords from big dataset
I have search keywords in one of my database table. These are the keywords searched by users on a website.
My requirement is to find the top 100 search keywords after consolidating various similar ...
2
votes
1answer
24 views
Text classification based on n-grams and similarity
I have tried to cluster hundred texts using k-means clustering. I would like to consider other algorithms to group text based on their content and try to spot news not related to other news (topic ...
1
vote
2answers
33 views
What's my target variable?
I am beginner in data science. I have this "aids" dataset from "mdhglm" package in R.
dataset = aids, info = Repatead Measures on AIDS Data
...
-1
votes
0answers
38 views
0
votes
0answers
13 views
Variance Inflation Factor or Correlation Coefficient which one to choose?
In a machine learning model, while removing the correlated variables, which one is better whether we have to go with Variance Inflation Factor(VIF) or Correlation Coefficient like Pearson or spearmans ...
0
votes
4answers
46 views
How to view entire 5788 rows and 7 columns.?
I want to see entire 5788 rows and 7 columns of my CSV data. I used below code and it is giving little data by stating [5788 rows x 7 columns] but showing only 10 rows and 5 columns in python prompt. ...
0
votes
1answer
43 views
Can we use one hot encoding instead of for loops?
I have an anomaly detection model, that I run per store with a bunch of features.
I intend to run this code, everyday, per store. Now, lets say I have 8000 stores, I would imagine, I should write a ...
1
vote
1answer
20 views
Model assessment for EXTREMELY imbalanced dataset
I am dealing with an extremely imbalanced dataset, with about 10,000 negative samples for each positive sample.
I am now trying to come up with an adequate measurement of model accuracy but none seem ...
1
vote
2answers
85 views
What are advantages or disadvantages of training deep learning model from scratch? [closed]
I want to know advantages and disadvantages.Also, What are advantages and disadvantages of transfer leraning?
1
vote
1answer
35 views
what is the difference between Bias and overfilling and under fitting? [closed]
I am confused about Bias and overfitting as I see it sounds similar but not able to explain differene?
0
votes
1answer
18 views
Text classification and predictive model
I have collected a lot of data that I would like to analyse and classified. Unfortunately, they are not already labelled, so I am going to do manually.
The dataset consists of texts in Italian and I ...
0
votes
0answers
17 views
How to estimate a heterogeneous asset pricing model in eviews?
I'am trying to estimate a heterogeneous asset pricing model for my thesis in eviews. It is a model where the fundamental price is determined by the sum of discounted future dividends where two type of ...
0
votes
0answers
9 views
Merging/Consolidation of duplicate records
I have large number of groups of duplicate records, which i need to merge to get a single master record per group.
Input dataset contains columns like - FirstName, LastName, FullName, AddrLine1, City,...
0
votes
1answer
23 views
plotting a decision tree based on gridsearchcv
i was trying to plot the decision tree which is formed with GridSearchCV, but its giving me an Attribute error.
...
0
votes
0answers
17 views
English Word popularity scoring
I'm new to data science and looking for the word popularity ranking algorithm.
Given a list of words, What would the best and easy to implement algorithm to score each word based on the word ...
0
votes
0answers
21 views
H2O distributed Environment Setup for 70million transaction data ~ 80GB
We have approximately 70 million records (around 90 GB of Data) and we need to ingest it into H2O and then we need to do modeling in H2O.
How can we estimate the approximate H2O distributed ...
0
votes
1answer
26 views
Getting a best k in KNN Algorithm
So, i was learning the KNN Algorithm and there i learnt cross Validation to find a optimal value of k.Now i want to apply grid search to get the optimal value.I found an answer on stack overflow where ...
0
votes
1answer
18 views
How to measure Covid impact by analysing credit card transaction of customer
I Want to know how can I identify that is the customer is in financial distress due to the COVID situation using its credit card transactions.
I have a daily transaction of customers till current ...
-1
votes
0answers
9 views
how to approach Kaggle Challenge for Restaurant Revenue Prediction ? which models gives best prediction and why?
project for restaurant revenue prediction
approach for
1. Random forest works the best on our prepossessed data.?
2. what is mean by the P1....P37??
3. how feature contribute on revenue?
https://www....
0
votes
0answers
20 views
Simple revenue predictor
I'm trying to create a program which should estimate the total revenue for current year by analyzing the payment data from this and previous years.
Generally, it's about a subscription-based SaaS ...
1
vote
1answer
45 views
will increasing threshold always increase precision?
here precision at threshold 0.85 > precision at threshold 0.90. shouldnt it be the other way round? increasing threshold will reduce False positive and precision will be greater than before?
0
votes
0answers
17 views
Orientation on a Genetic Algorithm approach for the Financial Forecasting
So there is these article: Brown, M. S., Pelosi, M. & Dirska, H. (2013). Dynamic-radius Species-conserving Genetic Algorithm for
the Financial Forecasting of Dow Jones Index Stocks. Machine ...
0
votes
0answers
7 views
How to feed 3 dimensional data to Naive Bayes classifier and OneVsRestClassifier?
I have 3 dimensional data. I want to feed 3D data to Naive Bayes classifier and OneVsRestClassifier. When I tried to do that I got error "ValueError: Found array with dim 3. Estimator expected <= 2"...
0
votes
0answers
21 views
A/B testing methodology
I am learning A/B testing and so far I have learn't the theory and am trying to solve some use cases. I am working on a website's data which has total visits and clicks at day x country x test/control ...
0
votes
2answers
39 views
Can Random Forest regressor or Decision trees handle missing values and outliers? [closed]
I have below assumptions about RF & Decision trees in general, please correct me if the assumptions are incorrect.
It takes care of missing values
It handles outliers
It handles skewness in ...
0
votes
0answers
22 views
Keras - Custom loss function with multiple output and different weights
I'm trying to train a network with multiple outputs, in the below code: the loss from X, Y logits are weighted equally in the loss function. I want X weighted twice than Y. I tried Loss weight when ...
1
vote
0answers
19 views
Negative loss, 100% accuracy
I have a very small dataset with 567 images. I've used pretrained model resnet50 in transfer learning. But whenever i fit my model it's giving me 100% accuracy , 100% validation accuracy and in per ...
1
vote
0answers
17 views
Different Decision Tree pruning method
I am trying to learn different pruning methods for decision trees. I have put together a list of methods below.
Reduced Error Pruning
Cost Complexity pruning
Minimum error pruning
Pessimistic Error ...
0
votes
1answer
23 views
Topic Similarity Measure | Multi-class Text Classification Model
I am trying to build a multi class text-classifier that classifies whether the tweet belongs to one of the categories ( Advise or Science or others )
let the input be any tweet like this ,
Input :
<...
2
votes
1answer
39 views
High error machine learning regressor algorithm in Python - XGBOOST Regressor
I have a dataframe with real state data from florida, it includes single apartments and buildings data:
...
0
votes
0answers
9 views
Estimate machine count for API dependency
I have a SDK, with many APIs, which is used by many apps. Those apps are installed on many machines. I get data containing machine Id, app name and API info . The data is sampled such that 2% of the ...
0
votes
2answers
39 views
Which ML algorithm should I use for following use case for classification and Why?
I have data in table format having total 3 columns. One column for label, other two columns are features. So, such 30 rows(1 row contains 2 feature and 1 label) make one set of data with all 30 rows ...
1
vote
0answers
14 views
I Have Issues Installing Basemap
I tried to install Basemap and it gives me this:
preparing transaction: done
verifying transaction: done
executing transaction: failed
ERROR conda.core.link:_execute(507): An error occurred while ...
0
votes
0answers
46 views
Is it a good practice to exclude the noisy data for training a model?
I executed a linear search on an array containing all unique elements in range [1, 10000], sorted in increasing order with all search values i.e., from 1 to 10000 and plotted the runtime vs search ...
0
votes
1answer
46 views
Machine Learning classification problem
I'm trying to do this classification problem, depicted in the following Figure. The task is to separate the blue elements from the red elements in a cartesian (x,y) coordinate system.
I have to:
ā ...
0
votes
0answers
17 views
Is this the case of data leakage?
Let's say, I have the dataframe with numerical features A, B, C. I do not have the target variable but I extract the target variable Y from the features A, B, and C. E.g.
...
0
votes
0answers
12 views
Guidance on sequencing the data science professional courses
I am planning to start data science online professional courses at Harvard University, but I don't which course should I begin with . I request for help in sequencing these courses below so that I can ...
-2
votes
1answer
26 views
what is learning rate in neural network?
When I am creating a model using Keras we should define the learning rate(lr) in that optimizer method Please refer to the below code.
...
0
votes
0answers
38 views
Workng of LME model used for a set of category variable(s) and a continuous variable?
LME models are being used to analyze the effect of continuos data and category data. Is
this model appropriate for checking the effect of two independent variables - one with continuous values and ...
0
votes
0answers
9 views
KS-Score Methods
I came across 2 methods to calculate KS-Score and select best probability threshold.
Decible Method
TPR - FPR
Is there any specific scenario on which it depends which method to select.
or we cna ...
1
vote
1answer
23 views
What is neural structure learning in tensorflow?
What is neural structure learning?
what is the difference between neural network vs neural structure
learning?
