Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [data-science-model]

Questions about the organization of elements of data, and the standardization of their relations.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

Linear Discriminant Analysis (LDA) determining the class for test data after transforming with eigenvectors

Suppose we are given two classes class-1 and class-2 and the mean of the two classes are $μ_1$ and $μ_2$ respectively before projection. Both of their variance-covariance matrix is also provided ...
Suman Debnath's user avatar
1 vote
0 answers
33 views

I need some advice about working as Mathematical Modeler/Data scientist

I have 10 years experience in studying mathematics in University, and I've just finished my Masters degree I have some experience with LaTeX and Mathematica, what do I need to study to be able to ...
Malmo's user avatar
  • 11
0 votes
0 answers
11 views

ML for lot sizing task

I am solving the problem of automatic lot sizing, which consist of different products. I receive data about a certain product and I have to decide which products to combine into lots. And then ...
user162857's user avatar
0 votes
1 answer
25 views

I have some topics from python out of which I want to know the one which actually needs to be studied for data science roles specifically

The topics are as follows: Collections Itertools Regex and parsing Error and exceptions Date and time
Kanika Dixit's user avatar
2 votes
1 answer
25 views

How to analyze time series data and create time series model in Python?

I am trying to understand time-series data and model. In youtube tutorial and others, mostly univariate examples are shown. And they are applicable or suitable for those conditions. What if our ...
Bad Coder's user avatar
0 votes
0 answers
11 views

What to include in fact and dimension table from election database

I am working with election dataset of India of year 2014 and data for 2019 I also have table for party names and descriptions and finally state name with code. I am not getting how do i create a ...
Sugam Sharma's user avatar
1 vote
1 answer
22 views

Feature Engineering a Recency feature

I have a customer scoring problem I'm working on specifically on predicting conversion and coming up with a probability score on conversion (using xgboost classifier atm). There's a feature I want to ...
MetalicSt33l's user avatar
0 votes
0 answers
19 views

Input 0 of layer "sequential_2" is incompatible with the layer: expected shape=(None, 3, 3), found shape=(1, 3, 11)

I am trying to predict stock closing price using news sentiment analysis with the help LSTM neural network but after I do model.fit(). I am encountered with the error message: ValueError: Input 0 of ...
pankaj yadav's user avatar
1 vote
1 answer
37 views

How to measure different models' feature importance using a generic and common standard?

I want to measure the feature importance of a series of models after training them. Most models have some built-in APIs that allow me to access their feature importance, but as far as I know, these ...
Yuuya's user avatar
  • 11
-1 votes
0 answers
19 views

Why Python is considered the perfect choice for healthcare technology

"Versatile Language: Python's flexibility allows for a wide range of applications in healthcare technology, from data analysis to machine learning algorithms.Rapid Prototyping: The ease of ...
Sakshi Gaba's user avatar
-1 votes
1 answer
33 views

I need suggestion for a project

I want to make a forecasting system which will forecast how much quantity will be sold next year based on the previous 5 years' data from 2019 to 2023 and want to predict for future years. Now the ...
theunknown's user avatar
1 vote
0 answers
18 views

Xgboost custom objective function. How to modify the weights?

I have a custom objective function to xgboost: ...
Gábor B's user avatar
0 votes
0 answers
20 views

Question regarding model inference consistency

I am doing some research on gaze estimation model L2CS provided in here. When you look at the inference predictions on provided image, when face/eye direction barely changes, inference predictions ...
JohnDoe's user avatar
1 vote
1 answer
25 views

How can a citation dataset (like Cora) have strongly connected components?

This website https://snap.stanford.edu/data/cit-HepPh.html shows that the High-energy physics citation network has strongly connected components and it's driving me crazy. A SCC would mean that you ...
StackExchanger's user avatar
0 votes
0 answers
13 views

Are there any Python (sklearn or other) ML libraries that can handle missing data?

I am running a model using a sparse matrix for the X. It can not be imputed. So are there any libraries who modify the main ML models to deal with this?
J_Bake's user avatar
  • 1
1 vote
1 answer
18 views

Need inputs on logging Machine Learning models and their versions in logs for my application

So i have a web application that recommends movies based on the subject and question that the user puts in , it also uses NLP and ML models like Named Entity Recognition(NER) model to extract keywords ...
nOhAr's user avatar
  • 11
0 votes
0 answers
17 views

Variable Selection and model prediction

In a supervised problem, I used randomForest for variable selection to identify the most important features. Question: am I required to use a random forest model for subsequent predictions, or can I ...
Zakaria Faouzi's user avatar
0 votes
1 answer
19 views

Missing data in train set and test set

I have a dataset of N columns. Now I'm able to preprocess data and find a subset of features that I can use to train a model and make predictions. In the case where the train data has missing feature ...
0-0's user avatar
  • 1
0 votes
0 answers
10 views

Missing Value in a dataset

i'm curently cleaning a dataset and theres a column called " Institution", it is needed for encoding and training the classification model later so it needs to be cleaned. In that column, ...
Harry lou's user avatar
0 votes
0 answers
23 views

Using Reinforcement learning for minimisation

I would like to use reinforcement learning for the optimisation of a given function under some contraints. Take for example the following problems: ...
Ach Raf's user avatar
0 votes
0 answers
19 views

Calculation of Covariance Matrices for a QDA classifier in Python Numpy

For a school project, I have to design a QDA classifier for 28x28 pixel images of letters in sign language. I have been given 27455 images for training, which have to be flattened to a 784 pixel ...
Neev Penkar's user avatar
0 votes
0 answers
16 views

Does hours spent training early in an Associates career impact the amount of revenue they generate in the future?

I can use some help with project design and choosing the correct method to answer this question. I have a dataset that contains associate ID, associate specialization, hours worked, revenue, period, ...
Danielle DeLosa's user avatar
0 votes
0 answers
20 views

What is the best library for time-series analysis?

Recently, I've been using statsmodels, but I would like to know if others work fine for you.
L1rola's user avatar
  • 13
0 votes
0 answers
11 views

how to obtain predictions for all observations in a dataset?

I have a dataset comprising 1864 roads with various traffic and safety features, along with the number of accidents at each road. The objective is to rank these roads based on the number of accidents. ...
ebrahimi's user avatar
  • 1,307
1 vote
0 answers
38 views

Training Models Directly with Transformer Attention Weights: A Viable Strategy?

I'm currently using pre-trained transformers to extract embeddings for sequence analysis, which are then used in downstream tasks. My process involves using the extracted embeddings as features for ...
pparker's user avatar
  • 392
0 votes
0 answers
15 views

How to use pairwise data to cluster users into group?

I have a table_1 shows the phone call history, with two columns 'ID_1' and 'ID_2' (id1 calls id2), how can I find the clusters where each user in the cluster in the calls with at least 2 other people ...
kumalu's user avatar
  • 1
0 votes
0 answers
13 views

How to correctly use gstar package of R for spatio-tempral analysis?

I want to perform a spatio-temporal analysis by highlighting spatial as well as temporal dependencies of the data (I have a 'weight matrix' highlighting spatial dependencies of the counties) on the ...
Shashank Gupta's user avatar
0 votes
0 answers
34 views

How to Detect and Identify German Compound Words in a Text using Python

I would like to know if there is any python libraries or packages available which can help me detect and extract German compound words. I took a look at nltk and spacy packages and they don't seem to ...
Ali Mazhar Sadat's user avatar
1 vote
1 answer
50 views

Data Science book with exercices

I'm looking for a data science book that covers statistics, probabilities, machine learning, and includes exercises for practice. I believe that having exercises will greatly help in understanding and ...
krm fordev's user avatar
0 votes
0 answers
22 views

Cluster size distribution

In unsupervized matrix clustering (i.e., no images, no graphs), has anyone ever noticed an issue with clustering algorithms where often the clusters are extremely imbalanced, to the point of the ...
Yugen Omer Korat's user avatar
0 votes
0 answers
37 views

Interpreting Perplexity, U_mass coherence and Cv score trends for a Latent Dirichlet Allocation Model

I'm running an LDA model through gensim. To my understanding, closer the u_mass coherence score is to zero, higher is the interpretability of the topics that come up. I'm getting the u_mass coherence ...
Sarthak's user avatar
0 votes
0 answers
17 views

5-fold cross validation in R: getting error Age variable different lengths

I am tasked to do a 5-fold cross validation for my R grad course with pga golf data. I continually get an error for a certain variable, Age, saying different lengths. Here is the error code: ...
Heather S.'s user avatar
1 vote
0 answers
23 views

How to Yield a Better AUC / Lift Score?

I have a dataset with 200k records and 173 features focused on binary classification. Class proportion is around 98.7:1.3 (1.3% target=1). Currently, I am trying to increase the performance of my ...
DM_FCP's user avatar
  • 11
2 votes
2 answers
101 views

Improve text classification accuracy

Task: I am building a text classification for salary prediction for data science jobs. I want to achieve at least 70 percent accuracy. Data: Features: Consists of job descriptions of data science, ...
Sendhan's user avatar
  • 21
0 votes
0 answers
17 views

Is it possible to calculate a GRU RNN in its entirety by hand on a small dataset?

I want to see whether my code works and compare it to the results I do myself
J_Bake's user avatar
  • 1
0 votes
0 answers
11 views

Where can I find a good dataset for Customer Journey Analysis (Multi-Touch Attribution)?

For my thesis I need to find a dataset regarding customer journey analysis, hereby I assign credits to touchpoints of certain channels to see which channels are the most efficient. For example a ...
Mr.John.Doe2nd's user avatar
0 votes
0 answers
51 views

Heavy right skewed target

I scrapped Data science job postings of almost 1000 jobs for salary prediction. I wanted to bin the salary column and build a classification model from the job descriptions. But the (salary)target is ...
Sendhan's user avatar
  • 21
1 vote
0 answers
43 views

Designing a data science workflow for a small company

I work at a company with eight employees, and was recently hired as a data/business analyst. The company uses an in-house software to manage online retail sales. There is a huge amount of data ...
Kelsey's user avatar
  • 11
0 votes
0 answers
8 views

Best model for predicting user ranking of a bag of (travel) options without a fixed size

I need to predict which flights users's will choose, as a %. For example a user searches for flights LHR->JFK. There will be a variable set of available options each with a price, number of legs, ...
zs1's user avatar
  • 101
0 votes
0 answers
12 views

How to find the parameters of custom regression in python code?

I want to make a time series regression that has a custom formula lets say: Yt=Beta+1/(1+(some parameter sigma))Y(t-1)+(1/(1+(some parameter sigma))**2)Y(t-2)+Beta1X(t-1)+Beta2X(t-2) how to code this ...
J_Bake's user avatar
  • 1
0 votes
0 answers
58 views

Improving the performance of gradient boosting classifier

I am training a gradient boosting classifier on an imbalanced data but the model is not performing very well. These are the things I have done to improve the model's performance. Balanced the data ...
Toluwalope Owolabi's user avatar
0 votes
0 answers
35 views

combining fields in two files

I'm a newbie when it comes to machine learning and I have the following problem: I have two files that contain similar information and have the following structure: A: 10, B: 5, C: 5, D: 9, E: 10, … ...
Jared's user avatar
  • 1
1 vote
0 answers
31 views

Integrating MLFlow and SageMaker for a More Robust ML Model Deployment Pipeline

I'm seeking advice on enhancing the deployment pipeline of a machine learning model that's accessed via a FastApi in production. My goal is to replace the existing setup with a more robust and ...
Daniel Ben Zaken's user avatar
0 votes
0 answers
6 views

Connect and publish with Tableau and Azure Analysis Services

My organization is migrating to another platform and we need to replicate the reports we have made with Power BI in Tableau, but the problem is that our server data is in Azure Analysis Services, but ...
Ozdran Yankovic's user avatar
0 votes
0 answers
16 views

Working of 'Interpolation' to impute missing values & the importance of the order of data points for the process

Definition of interpolation - Interpolation predicts values at a point by studying its neighbouring points (within the same column), as opposed to data modeling where all the columns are taken into ...
Apoorva's user avatar
  • 307
0 votes
0 answers
13 views

Demand Forecasting methods for hundreds of time series

I have TFL cycling dataset for the time period from JAN 2023 to JUNE 2023. I would like to forecast the demand or expected no. of trips for each station at every hour of the day. Post some data ...
a_jelly_fish's user avatar
0 votes
1 answer
28 views

Any Interface/Library that can take the Python ML code and run on spark cluster without learning PySpark?

I have been working with Python for machine learning and have a fair amount of code written in Python using libraries such as scikit-learn, pandas, and numpy. Recently, I’ve been faced with larger ...
Mohith7548's user avatar
0 votes
0 answers
63 views

How to deal with missing values in the output for XGBoost Regressor

So I am trying to create a regression model that takes two arrays as the input features and an array as an Output. However, some of the point in this dataset do not contain any value. This is because ...
RM25's user avatar
  • 1
1 vote
1 answer
142 views

Should I apply the same data transformations in production for my classification model's inference steps

I am now moving my best classification model to production and doing tests currently. Should I use the same scaler() I used in training during my inference in ...
easymoneysniper's user avatar
1 vote
0 answers
28 views

How to make a dataset for Audio Speech Recognition of Arabic Language

I am trying to map Quranic Verses for the identification of errors. Different audios for the same verse have different durations as it depends on the reciter. I used Audacity to normalize but the ...
Sijran's user avatar
  • 11

1
2 3 4 5
14