Questions tagged [data-science-model]
Questions about the organization of elements of data, and the standardization of their relations.
654
questions
0
votes
1
answer
11
views
Any Interface/Library that can take the Python ML code and run on spark cluster without learning PySpark?
I have been working with Python for machine learning and have a fair amount of code written in Python using libraries such as scikit-learn, pandas, and numpy. Recently, I’ve been faced with larger ...
- 452
0
votes
0
answers
13
views
How to deal with missing values in the output for XGBoost Regressor
So I am trying to create a regression model that takes two arrays as the input features and an array as an Output. However, some of the point in this dataset do not contain any value.
This is because ...
- 1
1
vote
1
answer
39
views
Should I apply the same data transformations in production for my classification model's inference steps
I am now moving my best classification model to production and doing tests currently.
Should I use the same scaler() I used in training during my inference in ...
0
votes
0
answers
16
views
How to make a dataset for Audio Speech Recognition of Arabic Language
I am trying to map Quranic Verses for the identification of errors. Different audios for the same verse have different durations as it depends on the reciter. I used Audacity to normalize but the ...
- 1
0
votes
0
answers
5
views
How to handle multiple training jobs in AWS Sagemaker Pipelines?
I am trying to create a pipeline for training an image dataset via Sagemaker Pipelines. Based on the examples I understood that for all distinct stages like data preparation, model training, ...
0
votes
0
answers
6
views
Measuring Product Search effectiveness
I want to measure the effectiveness of my search engine, one of the ways i can do that is by measuring the rate at which a customer reformulates the previous query. Hence, I need to quantify inter-...
- 23
0
votes
0
answers
30
views
Optimal ML classification approach
Background: I have an app data (impressions, user activities) that I can use as features for a multiclass classifier (5 classes).
I just want to discuss about some things that our team is having a ...
0
votes
0
answers
45
views
Actual values vs Preditec values plot
I was working on a project and I got a 0.98 R^2 score on both the training and test data sets and 0.91 training mse and 1.02 test mse, But my Actual values vs Predicted values looks like this, I was ...
0
votes
0
answers
23
views
Cannot load audio files using AudioSegment
I am trying to load an mp3 file using pydub's AudioSegment method with below code.
...
1
vote
0
answers
21
views
Determining the start and stop of vehicle movement by GPS
There is a large fleet, slightly more than a million vehicles, which is constantly growing. Each vehicle sends GPS coordinates to the server, as well as events (ignition is on, door is open, parking ...
0
votes
1
answer
40
views
Workflow when making a machine learning model
I'm new to data science, and kinda confused with the workflow and steps to make a model. Before learning the math and concepts behind the algorithms like SVM, linear regressions, etc, I would just ...
0
votes
0
answers
19
views
overfitting or not
Hello so i'm building a classification model i train my on various models and these are the metrices so i want to know if ther's an overfitting or not
0
votes
1
answer
76
views
Best practices on encoding on an increasing number of categorical variables
I'm currently using Gradient Boosting Regressor as my model to predict production risk based off a set number of features as a side-project. One of these features, ...
0
votes
0
answers
9
views
How to add Lasso regularization to midas regression?
This is the code I am running now:
m17 <- midas_r(mls(bip_2018_2023_NA[0:60], 1:1, 1) ~mls(log_diff_load_seasonally_adjusted[0:240]^2, 4:8, 4)+ mls(log_diff_load_seasonally_adjusted[0:240], 4:8, 4),...
- 1
1
vote
1
answer
207
views
issue loading the ckpt file PytorchStreamReader failed reading zip archive: failed finding central directory
I am trying to load the ckpt file and getting error
PytorchStreamReader failed reading zip archive: failed finding central directory
Here is the code
...
- 35
0
votes
0
answers
9
views
0
votes
0
answers
8
views
How to model noisy time series data? Does non-linear modelling help?
Is it possible to model time series data that fluctuates. The main solution is to take first differences and make it easier to fit conventional models. What if non-linear models are built? Can they ...
- 1
0
votes
0
answers
7
views
how to convert these string columns so that i can put the data to the regressor
https://drive.google.com/file/d/1yQR1xJvzWd2UGgGQS2bHhdywZEE_Nu0J/view?usp=sharing
I have to change the name of the columns: city, batting_team, bowling_team, batsman, non_striker, bowler. These ...
1
vote
2
answers
89
views
Training model using BERT
I have generated dataset using chat gpt. Dataset has 9000 data recodes. It's 6 class sentiment analysis. classes are 0,1,2,3,4,5
I used 3000 recodes for training, 1200 recods for validation and ...
0
votes
1
answer
252
views
Help with R - funModeling package no longer available, need help downloading older version
I have posted on GitHub but I'm not getting a response as quickly as I would like so I figured I'd seek help on multiple forums/websites. I am trying to do a few sample mean exercises from a textbook '...
- 101
0
votes
1
answer
44
views
Models vs algorithms
What are basic difference between algorithms and models
Is regression SVM random forest decision trees algorithms or model
- 101
0
votes
0
answers
14
views
Ranking risky routes
i'm looking to get the top 10 rank of the most dangerous routes. I have a routes table where each row is a route and it has features such as
avg daily traffic for past three years
No. of times where ...
- 1
0
votes
0
answers
30
views
Torchvision Faster-RCNN, modified loss function
I'm trying to solve a problem of table detection in spreadsheets in Excels. I've came across this paper, which suggests to use modified version of Faster RCNN to do object detection on the ...
- 101
0
votes
0
answers
42
views
Low silhouette score yet better clustering result
I applied k-means and k-medoids clustering techniques on iris dataset, in particular I clustered with respect to sepal length and sepal width features across 4 classes. With k=3 using k-means I ...
0
votes
0
answers
7
views
How to handle multiple values which have multiple subsets?
I am new to datascience. I am trying to predict tab position in msword using features like paragraph text, font name, tab count, tab index etc. In tab position there may be 0 values or multiple ...
1
vote
1
answer
13
views
Classifier learn from model or data
I'm new in data science. I read context . It saids "The clf (for classifier) estimator instance is first fitted to the model; that is, it must learn from the model. ...."
and example shows:
<...
- 13
0
votes
0
answers
10
views
Updating Predictive Model to Forecast from Today’s Date
I have a historical dataset with four numerical columns: c1, c2, c3, and c4. Column c4 is calculated and represents the number of days between the active date and the closed date. I have created a ...
- 1
1
vote
1
answer
82
views
Which statistical procedure to use?
I'm new to Hypothesis testing. I can't really think of how to answer this -
Five students take a test one year and another test the next year. Their overall grades(in %) are given below for the test-1 ...
- 197
2
votes
2
answers
383
views
Parameter estimation in linear regression
Another test Q I couldn't answer -
We have marks of students belonging to 3 sections - A,B,C and two genders - M & F. Which regression model will not be able to estimate all the parameters?
1 ) ...
- 197
0
votes
1
answer
288
views
How many Non-frauds would be blocked to stop one Fraud based on the given confusion matrix?
One of the questions in a recent online test that I couldn't answer is below(I couldn't copy paste exact question, so this one is totally from memory. Kindly excuse in case of poor wording.)
We have a ...
- 197
0
votes
0
answers
30
views
How to predict binary values for time series models?
How one can predict 100 new binary values (0s and 1s) if there are 3200 previous values?
Note that there is no seasonality, but just a set of numbers. I used Arima, lgbm, Markova but it predicts ...
0
votes
0
answers
9
views
Architectures for continuous vector labels prediction
I have a small dataset of feature vectors of size 200 to each corresponds a larger vector of size 1000. I would like to "predict" a large vector for every small one. The task sounds like ...
0
votes
1
answer
24
views
A model for recognizing suspicious user activity on a forum
I have a log of user actions on my website. Some users were blocked for spamming manually. This is also shown in the log.
Here is an example of the dataset:
...
- 13
0
votes
2
answers
31
views
Optimal Data Split
I have a multiclass problem (3 classes) that looks to predict if someone will buy a product, neutral or not. I have initial features of in-app activity data such as likes, share, bookmark, share, ...
0
votes
0
answers
7
views
YOLOV4 model training
I can't start my model tutorial. I ran it correctly, I tried it before, but now it doesn't even start. why could it be? Did the update come?
1
vote
1
answer
72
views
How to Justify Anomalies Detected by Unsupervised Anomaly Detection Models? [closed]
I'm working on an unsupervised anomaly detection project involving a large sensor dataset, where I aim to identify anomalies without the aid of labeled data. While I've implemented several ...
0
votes
2
answers
67
views
Been stuck on a DS problem. Just need to know whether this problem statement is solvable or not
I've been stuck on the following problem for weeks now. To be clear I'm not asking the community to provide a full solution. Just a few ideas or at least confirmation on whether this problem statement ...
- 173
0
votes
1
answer
28
views
Overfitting in the trained model
For my project on the classification problem of predicting churn customers, I trained various base models using k-fold validation on the training dataset and out of which random forest gave the best ...
0
votes
0
answers
10
views
type error in making model using MultinonialNB
TypeError Traceback (most recent call last)
Cell In[90], line 2
1 mb=MultinomialNB()
----> 2 mb.fit(x_train_idf,y_train)
File ~\anaconda3\lib\site-packages\sklearn\...
1
vote
0
answers
46
views
How does HDBSCAN generate cluster hierarchy?
HDBSCAN generates the minimum spanning tree where each vertex represents the data point whereas the edges represent the mutual reachability distance.
But how does it generate the cluster hierarchy?
I ...
0
votes
0
answers
75
views
Error in tsfresh feature extraction: TypeError: ufunc 'isnan' not supported for the input types
I'm trying to use the tsfresh library for feature extraction from time series data. Specifically, I'm using the extract_features function with the EfficientFCParameters as the default feature ...
0
votes
0
answers
9
views
What are the Bidirectional methodoly based pre-trained models in NLP
BERT model is useful to solve many NLP tasks (11+) like Sentiment analysis, Q&A, summarization and NER.
what are the other predefined model similar to BERT?
what is best way to find them in NLP?
- 57
1
vote
1
answer
40
views
0
votes
1
answer
28
views
How should I proceed with this type of dataset and requirement?
Suppose I have a dataset where I want to give input of up-to 30 lines (can be variable with min 5 available) these lines are timestamps of every minute and want my model to predict output of next 5 ...
0
votes
0
answers
13
views
Deciding on a data model based on something’s attributes and their properties
Considering something with these fields:
A category
A channel
A post
Where a post has:
A title
A body
Tags
The ability to write messages/comments on the post
The ability to link to other posts in ...
- 131
0
votes
0
answers
16
views
Help creating a SCORING model/formula.......am I on the right track?
Trying to create a basic machine scoring model, that takes in 4 parameters:
Number of maintenance events
Years of life left
Manufacturer support (bit - either yes or no)
Visual condition
The ...
- 1
6
votes
2
answers
1k
views
How to remove the hotspots from given image by using Python and opencv?
In the picture below there are some regions which are very bright (i.e. more white). Some bright regions are wide and some are narrow or thin. The red box covers one such wide bright spot, and blue ...
- 173
1
vote
2
answers
61
views
Looking for a couple of ideas please
I’ve got some data by postal zone that includes:
Postal zone code
Average rent value per square foot
Brand affinity 1
Brand affinity 2
Brand affinity 3
Brand affinity 4
…and so on
The brand affinity ...
- 11
0
votes
2
answers
1k
views
corr() is giving an error. please help out of this problem and tell me what is this error about
when I am trying to run sns.heatmap(df.corr(),annot=True) this code in my jupyter notebook. this error is occuring. I cannot understand this problem. please help me.
0
votes
2
answers
631
views
Input to reshape is a tensor with 4096 values, but the requested shape requires a multiple of 6400. (Having problem with shapes)
I am trying to make a caption generator model.
(Having problem with shapes)
I am getting error as
...
- 135