Questions tagged [data-science-model]
Questions about the organization of elements of data, and the standardization of their relations.
686
questions
0
votes
0
answers
7
views
Understanding the `model.fit` function in keras and imbalanced datasets
As an exercise, I'm trying to translate a model written in Keras (https://github.com/CVxTz/ECG_Heartbeat_Classification/blob/master/code/baseline_mitbih.py) into Pytorch code. I realize in Keras much ...
0
votes
1
answer
12
views
TypeError: argument of type 'SimpleImputer' is not iterable [closed]
Please help
I keep getting an error (TypeError: argument of type 'SimpleImputer' is not iterable) when I run the code below. I have done google searches on how to resolve it but failed.
...
- 1
0
votes
0
answers
17
views
How to approach Machine Learning with Circular distribution (Classification problem)
I have 41 continuous columns and all are distributed roundly to each pair:
I used:
SMOTE for resampling data ( my dataset is imbalanced)
Test dataset: last month in the data dataset. Train dataset: ...
0
votes
0
answers
24
views
SARIMA - 1 day gap between prediction vs observed data after removing seasonality
I need to identify outliers in my data, ranging from 2023-01-01 to 2024-08-31. My data is stationary and has seasonality of 7 days. I've decided to use the SARIMA model, by using SARIMAX in Python to ...
- 1
0
votes
0
answers
7
views
Custom NER Model
I have develop a custom NER model with custom labels,but whenever I am giving sentence that is out of training data it is giving either empty output or output with wrong label.Also,the model is not ...
0
votes
1
answer
17
views
Where should I learn pytorch from?
I'm a undergraduate student. I've coded a three-node neural network (that works) based on my professor's guidance. However, I'd like to pursue a career in AI and Data Science, and I'd like to teach ...
0
votes
1
answer
41
views
Analyzing Range of Features for Classification
I have developed a classifier with 5 ordinal classes, and I’m looking to do a detailed analysis on the range of values required for each class (1 through 5).
For example, I want to understand how a ...
0
votes
0
answers
4
views
Disaggregation algorithm that maintains mass balance?
I am working with monthly timeseries data that I need to represent as daily data. I am looking for a disaggregation algorithm or Python/R package where:
Daily values smoothly increase from one month ...
- 101
0
votes
1
answer
25
views
How to Classify Blueberries as "Crunchy", "Juicy" or "Soft" using Acoustic Signal Processing and Machine Learning?
I'm working on a project to classify blueberries based on their texture—specifically, whether they are soft, juicy, or crunchy—using the sounds they produce when crushed. I have about 1100 audio ...
-1
votes
1
answer
26
views
machine learning - data science - data analysis
I have a research project in the machine learning area. In this study, dataset contains more than 4000 numbers categorized in four columns. I am going to find or predict a possible relation between ...
- 1
2
votes
1
answer
144
views
What is appropriate Individual KPI for AI projects?
I work in the sales department of electronics component manufacturing company and we do data science projects using traditional algorithm like Random forests (success likelihood of design project), ...
- 2,607
0
votes
0
answers
6
views
Within-subject analysis of the impact of energetic music on video game performance: Wilcoxon or Friedman?
I am conducting an amateur study of the impact of music on video game performance across multiple rounds of gameplay.
The gameplay loop is simple: Arrows are falling down, and you have to hit them at ...
- 1
0
votes
0
answers
34
views
How to predict price behavior according to model predictions for a week ahead?
I wrote the simplest linear regression model (I'm a noob, please don't scold me; this is my first model) to predict the price of solana, I would like to get some advice or tips on how to improve. The ...
0
votes
1
answer
69
views
What programming language is better for data science?
I'm a student who is currently learning Python and Java. I'm very interested to learn about data science and machine learning as well as deep learning.
Which of Java and Python is more useful for me?
0
votes
0
answers
67
views
Confidence and prediction intervals for polynomial regression in a non-normally distributed dataset?
I'm applying polynomial regression (Python—polyfit) to a dataset and have been asked to include confidence intervals, and as this is well outside my field, I have concerns. Mainly that this may not be ...
- 1
0
votes
0
answers
14
views
Identifying Underperforming Movies Using Bayesian Metrics or Time Series Approach?
I have two years' worth of movie rental sales data.
I'm interested in identifying underperforming movies based on these variables. So far, I'm considering creating a Bayesian product metric that will ...
- 1
0
votes
0
answers
60
views
Seeking Feedback on Data Science Portfolio
I hope you're doing well. I've recently started building my data science portfolio on GitHub with the goal of securing a role as a data scientist. I would greatly appreciate your feedback to help me ...
0
votes
0
answers
13
views
can we build sequential model?
Like is it possible to train a model on X1_i Inputs Y1_i Output and then the second one is running on X1_i + Y1_i to give output Y2_i ??
Context : (I am just getting my hand into ML and trying to ...
- 1
0
votes
1
answer
57
views
1
vote
0
answers
32
views
Linear Discriminant Analysis (LDA) determining the class for test data after transforming with eigenvectors
Suppose we are given two classes class-1 and class-2 and the mean of the two classes are $μ_1$ and $μ_2$ respectively before projection. Both of their variance-covariance matrix is also provided ...
1
vote
0
answers
40
views
I need some advice about working as Mathematical Modeler/Data scientist [closed]
I have 10 years experience in studying mathematics in University, and I've just finished my Masters degree
I have some experience with LaTeX and Mathematica, what do I need to study to be able to ...
- 11
0
votes
0
answers
14
views
ML for lot sizing task
I am solving the problem of automatic lot sizing, which consist of different products. I receive data about a certain product and I have to decide which products to combine into lots. And then ...
- 11
1
vote
1
answer
31
views
I have some topics from python out of which I want to know the one which actually needs to be studied for data science roles specifically
The topics are as follows:
Collections
Itertools
Regex and parsing
Error and exceptions
Date and time
- 13
2
votes
1
answer
67
views
How to analyze time series data and create time series model in Python?
I am trying to understand time-series data and model. In youtube tutorial and others, mostly univariate examples are shown. And they are applicable or suitable for those conditions. What if our ...
- 23
0
votes
0
answers
16
views
What to include in fact and dimension table from election database
I am working with election dataset of India of year 2014
and data for 2019
I also have table for party names and descriptions
and finally state name with code.
I am not getting how do i create a ...
1
vote
1
answer
35
views
Feature Engineering a Recency feature
I have a customer scoring problem I'm working on specifically on predicting conversion and coming up with a probability score on conversion (using xgboost classifier atm). There's a feature I want to ...
- 11
0
votes
0
answers
30
views
Input 0 of layer "sequential_2" is incompatible with the layer: expected shape=(None, 3, 3), found shape=(1, 3, 11)
I am trying to predict stock closing price using news sentiment analysis with the help LSTM neural network but after I do model.fit(). I am encountered with the error message:
ValueError: Input 0 of ...
2
votes
1
answer
65
views
How to measure different models' feature importance using a generic and common standard?
I want to measure the feature importance of a series of models after training them. Most models have some built-in APIs that allow me to access their feature importance, but as far as I know, these ...
- 41
-2
votes
1
answer
37
views
I need suggestion for a project
I want to make a forecasting system which will forecast how much quantity will be sold next year based on the previous 5 years' data from 2019 to 2023 and want to predict for future years. Now the ...
- 11
1
vote
0
answers
42
views
Xgboost custom objective function. How to modify the weights?
I have a custom objective function to xgboost:
...
- 33
0
votes
0
answers
23
views
Question regarding model inference consistency
I am doing some research on gaze estimation model L2CS provided in here.
When you look at the inference predictions on provided image, when face/eye direction barely changes, inference predictions ...
- 1
1
vote
1
answer
27
views
How can a citation dataset (like Cora) have strongly connected components?
This website https://snap.stanford.edu/data/cit-HepPh.html shows that the High-energy physics citation network has strongly connected components and it's driving me crazy.
A SCC would mean that you ...
- 111
0
votes
0
answers
13
views
Are there any Python (sklearn or other) ML libraries that can handle missing data?
I am running a model using a sparse matrix for the X. It can not be imputed. So are there any libraries who modify the main ML models to deal with this?
- 1
1
vote
1
answer
24
views
Need inputs on logging Machine Learning models and their versions in logs for my application
So i have a web application that recommends movies based on the subject and question that the user puts in , it also uses NLP and ML models like Named Entity Recognition(NER) model to extract keywords ...
- 11
0
votes
0
answers
22
views
Variable Selection and model prediction
In a supervised problem, I used randomForest for variable selection to identify the most important features.
Question: am I required to use a random forest model for subsequent predictions, or can I ...
0
votes
1
answer
32
views
Missing data in train set and test set
I have a dataset of N columns. Now I'm able to preprocess data and find a subset of features that I can use to train a model and make predictions. In the case where the train data has missing feature ...
- 1
0
votes
0
answers
17
views
Missing Value in a dataset
i'm curently cleaning a dataset and theres a column called " Institution", it is needed for encoding and training the classification model later so it needs to be cleaned.
In that column, ...
0
votes
0
answers
26
views
Using Reinforcement learning for minimisation
I would like to use reinforcement learning for the optimisation of a given function under some contraints.
Take for example the following problems:
...
- 1
0
votes
0
answers
26
views
Calculation of Covariance Matrices for a QDA classifier in Python Numpy
For a school project, I have to design a QDA classifier for 28x28 pixel images of letters in sign language. I have been given 27455 images for training, which have to be flattened to a 784 pixel ...
0
votes
0
answers
16
views
Does hours spent training early in an Associates career impact the amount of revenue they generate in the future?
I can use some help with project design and choosing the correct method to answer this question.
I have a dataset that contains associate ID, associate specialization, hours worked, revenue, period, ...
0
votes
0
answers
31
views
What is the best library for time-series analysis?
Recently, I've been using statsmodels, but I would like to know if others work fine for you.
- 13
0
votes
0
answers
11
views
how to obtain predictions for all observations in a dataset?
I have a dataset comprising 1864 roads with various traffic and safety features, along with the number of accidents at each road. The objective is to rank these roads based on the number of accidents. ...
- 1,307
1
vote
0
answers
39
views
Training Models Directly with Transformer Attention Weights: A Viable Strategy?
I'm currently using pre-trained transformers to extract embeddings for sequence analysis, which are then used in downstream tasks. My process involves using the extracted embeddings as features for ...
- 422
0
votes
0
answers
16
views
How to use pairwise data to cluster users into group?
I have a table_1 shows the phone call history, with two columns 'ID_1' and 'ID_2' (id1 calls id2), how can I find the clusters where each user in the cluster in the calls with at least 2 other people ...
- 1
0
votes
0
answers
15
views
How to correctly use gstar package of R for spatio-tempral analysis?
I want to perform a spatio-temporal analysis by highlighting spatial as well as temporal dependencies of the data (I have a 'weight matrix' highlighting spatial dependencies of the counties) on the ...
0
votes
0
answers
74
views
How to Detect and Identify German Compound Words in a Text using Python
I would like to know if there is any python libraries or packages available which can help me detect and extract German compound words. I took a look at nltk and spacy packages and they don't seem to ...
1
vote
1
answer
60
views
Data Science book with exercices
I'm looking for a data science book that covers statistics, probabilities, machine learning, and includes exercises for practice. I believe that having exercises will greatly help in understanding and ...
- 13
0
votes
0
answers
22
views
Cluster size distribution
In unsupervized matrix clustering (i.e., no images, no graphs), has anyone ever noticed an issue with clustering algorithms where often the clusters are extremely imbalanced, to the point of the ...
0
votes
0
answers
48
views
Interpreting Perplexity, U_mass coherence and Cv score trends for a Latent Dirichlet Allocation Model
I'm running an LDA model through gensim. To my understanding, closer the u_mass coherence score is to zero, higher is the interpretability of the topics that come up. I'm getting the u_mass coherence ...
- 1
0
votes
0
answers
25
views
5-fold cross validation in R: getting error Age variable different lengths
I am tasked to do a 5-fold cross validation for my R grad course with pga golf data. I continually get an error for a certain variable, Age, saying different lengths. Here is the error code:
...
