Questions tagged [data-science-model]

Questions about the organization of elements of data, and the standardization of their relations.

Filter by
Sorted by
Tagged with
0 votes
1 answer
11 views

Any Interface/Library that can take the Python ML code and run on spark cluster without learning PySpark?

I have been working with Python for machine learning and have a fair amount of code written in Python using libraries such as scikit-learn, pandas, and numpy. Recently, I’ve been faced with larger ...
Mohith7548's user avatar
0 votes
0 answers
13 views

How to deal with missing values in the output for XGBoost Regressor

So I am trying to create a regression model that takes two arrays as the input features and an array as an Output. However, some of the point in this dataset do not contain any value. This is because ...
RM25's user avatar
  • 1
1 vote
1 answer
39 views

Should I apply the same data transformations in production for my classification model's inference steps

I am now moving my best classification model to production and doing tests currently. Should I use the same scaler() I used in training during my inference in ...
Marc Atanante's user avatar
0 votes
0 answers
16 views

How to make a dataset for Audio Speech Recognition of Arabic Language

I am trying to map Quranic Verses for the identification of errors. Different audios for the same verse have different durations as it depends on the reciter. I used Audacity to normalize but the ...
Sijran's user avatar
  • 1
0 votes
0 answers
5 views

How to handle multiple training jobs in AWS Sagemaker Pipelines?

I am trying to create a pipeline for training an image dataset via Sagemaker Pipelines. Based on the examples I understood that for all distinct stages like data preparation, model training, ...
Mimansa Maheshwari's user avatar
0 votes
0 answers
6 views

Measuring Product Search effectiveness

I want to measure the effectiveness of my search engine, one of the ways i can do that is by measuring the rate at which a customer reformulates the previous query. Hence, I need to quantify inter-...
ricardo's user avatar
  • 23
0 votes
0 answers
30 views

Optimal ML classification approach

Background: I have an app data (impressions, user activities) that I can use as features for a multiclass classifier (5 classes). I just want to discuss about some things that our team is having a ...
Marc Atanante's user avatar
0 votes
0 answers
45 views

Actual values vs Preditec values plot

I was working on a project and I got a 0.98 R^2 score on both the training and test data sets and 0.91 training mse and 1.02 test mse, But my Actual values vs Predicted values looks like this, I was ...
Ario Alavi's user avatar
0 votes
0 answers
23 views

Cannot load audio files using AudioSegment

I am trying to load an mp3 file using pydub's AudioSegment method with below code. ...
Kevin Shah's user avatar
1 vote
0 answers
21 views

Determining the start and stop of vehicle movement by GPS

There is a large fleet, slightly more than a million vehicles, which is constantly growing. Each vehicle sends GPS coordinates to the server, as well as events (ignition is on, door is open, parking ...
Abraham Lincoln's user avatar
0 votes
1 answer
40 views

Workflow when making a machine learning model

I'm new to data science, and kinda confused with the workflow and steps to make a model. Before learning the math and concepts behind the algorithms like SVM, linear regressions, etc, I would just ...
Justin Jonany's user avatar
0 votes
0 answers
19 views

overfitting or not

Hello so i'm building a classification model i train my on various models and these are the metrices so i want to know if ther's an overfitting or not
Bilel kort's user avatar
0 votes
1 answer
76 views

Best practices on encoding on an increasing number of categorical variables

I'm currently using Gradient Boosting Regressor as my model to predict production risk based off a set number of features as a side-project. One of these features, ...
Andrew Narvaez's user avatar
0 votes
0 answers
9 views

How to add Lasso regularization to midas regression?

This is the code I am running now: m17 <- midas_r(mls(bip_2018_2023_NA[0:60], 1:1, 1) ~mls(log_diff_load_seasonally_adjusted[0:240]^2, 4:8, 4)+ mls(log_diff_load_seasonally_adjusted[0:240], 4:8, 4),...
J_Bake's user avatar
  • 1
1 vote
1 answer
207 views

issue loading the ckpt file PytorchStreamReader failed reading zip archive: failed finding central directory

I am trying to load the ckpt file and getting error PytorchStreamReader failed reading zip archive: failed finding central directory Here is the code ...
Shruti's user avatar
  • 35
0 votes
0 answers
9 views

How to solve starting values problem in midas in R?

I am running the following code. ...
J_Bake's user avatar
  • 1
0 votes
0 answers
8 views

How to model noisy time series data? Does non-linear modelling help?

Is it possible to model time series data that fluctuates. The main solution is to take first differences and make it easier to fit conventional models. What if non-linear models are built? Can they ...
J_Bake's user avatar
  • 1
0 votes
0 answers
7 views

how to convert these string columns so that i can put the data to the regressor

https://drive.google.com/file/d/1yQR1xJvzWd2UGgGQS2bHhdywZEE_Nu0J/view?usp=sharing I have to change the name of the columns: city, batting_team, bowling_team, batsman, non_striker, bowler. These ...
gaurav najpande's user avatar
1 vote
2 answers
89 views

Training model using BERT

I have generated dataset using chat gpt. Dataset has 9000 data recodes. It's 6 class sentiment analysis. classes are 0,1,2,3,4,5 I used 3000 recodes for training, 1200 recods for validation and ...
Sandun Tharaka's user avatar
0 votes
1 answer
252 views

Help with R - funModeling package no longer available, need help downloading older version

I have posted on GitHub but I'm not getting a response as quickly as I would like so I figured I'd seek help on multiple forums/websites. I am trying to do a few sample mean exercises from a textbook '...
JohnColtraneisJC's user avatar
0 votes
1 answer
44 views

Models vs algorithms

What are basic difference between algorithms and models Is regression SVM random forest decision trees algorithms or model
quanity's user avatar
  • 101
0 votes
0 answers
14 views

Ranking risky routes

i'm looking to get the top 10 rank of the most dangerous routes. I have a routes table where each row is a route and it has features such as avg daily traffic for past three years No. of times where ...
Joe's user avatar
  • 1
0 votes
0 answers
30 views

Torchvision Faster-RCNN, modified loss function

I'm trying to solve a problem of table detection in spreadsheets in Excels. I've came across this paper, which suggests to use modified version of Faster RCNN to do object detection on the ...
sagi's user avatar
  • 101
0 votes
0 answers
42 views

Low silhouette score yet better clustering result

I applied k-means and k-medoids clustering techniques on iris dataset, in particular I clustered with respect to sepal length and sepal width features across 4 classes. With k=3 using k-means I ...
Harshavardhana Srinivasan's user avatar
0 votes
0 answers
7 views

How to handle multiple values which have multiple subsets?

I am new to datascience. I am trying to predict tab position in msword using features like paragraph text, font name, tab count, tab index etc. In tab position there may be 0 values or multiple ...
Kamal Budhathoki's user avatar
1 vote
1 answer
13 views

Classifier learn from model or data

I'm new in data science. I read context . It saids "The clf (for classifier) estimator instance is first fitted to the model; that is, it must learn from the model. ...." and example shows: <...
Mike Liu's user avatar
0 votes
0 answers
10 views

Updating Predictive Model to Forecast from Today’s Date

I have a historical dataset with four numerical columns: c1, c2, c3, and c4. Column c4 is calculated and represents the number of days between the active date and the closed date. I have created a ...
Tahmeed's user avatar
1 vote
1 answer
82 views

Which statistical procedure to use?

I'm new to Hypothesis testing. I can't really think of how to answer this - Five students take a test one year and another test the next year. Their overall grades(in %) are given below for the test-1 ...
a_jelly_fish's user avatar
2 votes
2 answers
383 views

Parameter estimation in linear regression

Another test Q I couldn't answer - We have marks of students belonging to 3 sections - A,B,C and two genders - M & F. Which regression model will not be able to estimate all the parameters? 1 ) ...
a_jelly_fish's user avatar
0 votes
1 answer
288 views

How many Non-frauds would be blocked to stop one Fraud based on the given confusion matrix?

One of the questions in a recent online test that I couldn't answer is below(I couldn't copy paste exact question, so this one is totally from memory. Kindly excuse in case of poor wording.) We have a ...
a_jelly_fish's user avatar
0 votes
0 answers
30 views

How to predict binary values for time series models?

How one can predict 100 new binary values (0s and 1s) if there are 3200 previous values? Note that there is no seasonality, but just a set of numbers. I used Arima, lgbm, Markova but it predicts ...
Marko Radu's user avatar
0 votes
0 answers
9 views

Architectures for continuous vector labels prediction

I have a small dataset of feature vectors of size 200 to each corresponds a larger vector of size 1000. I would like to "predict" a large vector for every small one. The task sounds like ...
Denis Marcinkov's user avatar
0 votes
1 answer
24 views

A model for recognizing suspicious user activity on a forum

I have a log of user actions on my website. Some users were blocked for spamming manually. This is also shown in the log. Here is an example of the dataset: ...
Quaw's user avatar
  • 13
0 votes
2 answers
31 views

Optimal Data Split

I have a multiclass problem (3 classes) that looks to predict if someone will buy a product, neutral or not. I have initial features of in-app activity data such as likes, share, bookmark, share, ...
Marc Atanante's user avatar
0 votes
0 answers
7 views

YOLOV4 model training

I can't start my model tutorial. I ran it correctly, I tried it before, but now it doesn't even start. why could it be? Did the update come?
Elif Deniz's user avatar
1 vote
1 answer
72 views

How to Justify Anomalies Detected by Unsupervised Anomaly Detection Models? [closed]

I'm working on an unsupervised anomaly detection project involving a large sensor dataset, where I aim to identify anomalies without the aid of labeled data. While I've implemented several ...
Jais Varghese Joseph's user avatar
0 votes
2 answers
67 views

Been stuck on a DS problem. Just need to know whether this problem statement is solvable or not

I've been stuck on the following problem for weeks now. To be clear I'm not asking the community to provide a full solution. Just a few ideas or at least confirmation on whether this problem statement ...
Aakash Dusane's user avatar
0 votes
1 answer
28 views

Overfitting in the trained model

For my project on the classification problem of predicting churn customers, I trained various base models using k-fold validation on the training dataset and out of which random forest gave the best ...
Yuvika Yadav's user avatar
0 votes
0 answers
10 views

type error in making model using MultinonialNB

TypeError Traceback (most recent call last) Cell In[90], line 2 1 mb=MultinomialNB() ----> 2 mb.fit(x_train_idf,y_train) File ~\anaconda3\lib\site-packages\sklearn\...
Gourav singh thakur's user avatar
1 vote
0 answers
46 views

How does HDBSCAN generate cluster hierarchy?

HDBSCAN generates the minimum spanning tree where each vertex represents the data point whereas the edges represent the mutual reachability distance. But how does it generate the cluster hierarchy? I ...
Sushil Khadka's user avatar
0 votes
0 answers
75 views

Error in tsfresh feature extraction: TypeError: ufunc 'isnan' not supported for the input types

I'm trying to use the tsfresh library for feature extraction from time series data. Specifically, I'm using the extract_features function with the EfficientFCParameters as the default feature ...
Anas Salah El-din's user avatar
0 votes
0 answers
9 views

What are the Bidirectional methodoly based pre-trained models in NLP

BERT model is useful to solve many NLP tasks (11+) like Sentiment analysis, Q&A, summarization and NER. what are the other predefined model similar to BERT? what is best way to find them in NLP?
tovijayak's user avatar
1 vote
1 answer
40 views

LogisticRegression Loading problem

Saving Model Part ...
Micheal T. Atienza's user avatar
0 votes
1 answer
28 views

How should I proceed with this type of dataset and requirement?

Suppose I have a dataset where I want to give input of up-to 30 lines (can be variable with min 5 available) these lines are timestamps of every minute and want my model to predict output of next 5 ...
Pratik Bhadane's user avatar
0 votes
0 answers
13 views

Deciding on a data model based on something’s attributes and their properties

Considering something with these fields: A category A channel A post Where a post has: A title A body Tags The ability to write messages/comments on the post The ability to link to other posts in ...
Julius H.'s user avatar
  • 131
0 votes
0 answers
16 views

Help creating a SCORING model/formula.......am I on the right track?

Trying to create a basic machine scoring model, that takes in 4 parameters: Number of maintenance events Years of life left Manufacturer support (bit - either yes or no) Visual condition The ...
manavjn's user avatar
6 votes
2 answers
1k views

How to remove the hotspots from given image by using Python and opencv?

In the picture below there are some regions which are very bright (i.e. more white). Some bright regions are wide and some are narrow or thin. The red box covers one such wide bright spot, and blue ...
S. M.'s user avatar
  • 173
1 vote
2 answers
61 views

Looking for a couple of ideas please

I’ve got some data by postal zone that includes: Postal zone code Average rent value per square foot Brand affinity 1 Brand affinity 2 Brand affinity 3 Brand affinity 4 …and so on The brand affinity ...
Jake's user avatar
  • 11
0 votes
2 answers
1k views

corr() is giving an error. please help out of this problem and tell me what is this error about

when I am trying to run sns.heatmap(df.corr(),annot=True) this code in my jupyter notebook. this error is occuring. I cannot understand this problem. please help me.
Subhajit Sarkar's user avatar
0 votes
2 answers
631 views

Input to reshape is a tensor with 4096 values, but the requested shape requires a multiple of 6400. (Having problem with shapes)

I am trying to make a caption generator model. (Having problem with shapes) I am getting error as ...
tikendraw's user avatar
  • 135

1
2 3 4 5
14