All Questions
35,972
questions
0
votes
0
answers
5
views
Why is UMAP used in combination with other Clustering Algorithm?
I've noticed that UMAP is often used in combination with other clustering algorithms, such as K-means, DBSCAN, HDBSCAN. However, from what I've understood, UMAP can be used for clustering tasks. So ...
- 101
0
votes
0
answers
4
views
For a node in an undirected graph - does the node affect itself if its markov blanket is known?
Consider the following Markov Random Field.
Question 1: Which of the following nodes will have no effect on H given the Markov Blanket of H?
Question 2: Will node H itself have any effect on itself, ...
0
votes
0
answers
3
views
Gaining insights into commonly co-bought categories of products using association rules (frequent itemset mining)
Let's say I have a small database of supermarket transactions.
Each records consists of a tx with several bought products, each having a unique identifier (eg, 1,2,3,4,5 etc.). Running A-priori on ...
0
votes
0
answers
9
views
Averaging subregion correlation coefficients into a single measure
I've got a 257x257 correlation matrix of functional connectivity (fMRI) data. It is a symmetric matrix where each value is the Pearsons correlation of the brain area in the row with the brain area in ...
- 101
0
votes
1
answer
18
views
Overfitted model [duplicate]
A classic question with an unclear answer, is it better to have an overfitted model performing better on a Cross-Validation setting, or a non-overfitted model performing worse?
In this context, higher ...
- 133
0
votes
0
answers
9
views
How do you know which is the closest class example(observation) to a particular class in Self Organizing Map?
For example there are two classes C1 and C2,
distance1 and distance2,
The rule tells which class is:
If distance1 > distance2 than class C1 (Node1)
else class C2 (Node2)
Here are some data examples:...
- 1
0
votes
0
answers
9
views
Is the node in undirected graph itself included in the set of its own Markov Blanket?
Consider an undirected graph with nodes set {a,b,c,d,e}, and edge set {(a,b), (a,c), (a,d), (a,e)}.
From the above info, you will clearly visualise that the node a ...
0
votes
0
answers
6
views
Time series : add constants for each time series
I have N time series of shape (30*36) (time step, feature). For each time series 500 parameters that can be seen as the history of the time series have to be added. ...
- 29
0
votes
1
answer
15
views
How sklearn logistic regression computes accuracy, recall etc if we don't provide threshold?
It might be a stupid question, but I just realized that calling score function on logistic regression model shouldn't make any sense - as far as I know in sklearn ...
- 103
3
votes
1
answer
320
views
What is Data Scientist's Salary In 2023?
I searched for information on data scientist pay by country but was unable to find any.
I am aware that the US pays data scientists an absurdly high salary (well, maybe not "insanely high" ...
- 31
0
votes
0
answers
8
views
Voting classifier ensemble error: 'ValueError: Unknown metric function: 'function'.'
I am trying to create a hard voting ensemble of three neural networks. I've already converted them to Keras Classifiers.
Here is the code:
...
0
votes
0
answers
8
views
ANN regressor with multiple outputs in place of single data output
I built an ANN model(Keras) to handle a regression problem with one output. By mistake I filled 10 units at the output layer (instead of 1). To my surprise, there was no error. Later on this was ...
- 101
0
votes
0
answers
8
views
Multi output regression model with non categorical text data
I am doing a multioutput regression model where 1 input and 6 output. My dataset contain all non categorical text data. I used countvectorization in the preprocessing but got an error while fitting ...
0
votes
1
answer
15
views
StandardScaler and MinMaxScaler vs RobustScaler
I've recently read that Standard Scaler functions best in situations where the distribution of the features are approximately normal.
MinMaxScaler works in a way that it preserves the features' ...
1
vote
1
answer
9
views
What is the difference between State Value function and Return for Markov Reward process ( MRP)?
I have been going through Stanford Lecture on RL. I see in MRP that Return function is same as State Value function. Both are getting expected sum of reward keeping discount factor in mind.
Although ...
- 415
0
votes
1
answer
9
views
How ReLU is bringing non linearity and why it is not an alternative to dropout?
The differentiation of ReLU function is 1 when input is greater than 0, and 0, when input is less than or equal to 0. In the backpropagation process it doesn’t change the value of d(error)/d(weight) ...
0
votes
0
answers
9
views
My linear regression doesn't work when i try to calculate theta1
I want to create my own linear regression. But my formula of the coefficient theta1 doesn't work i have big values :
...
0
votes
0
answers
10
views
Convert csv with row and column headers into nested json
I've searched here and on Stackoverflow and read through the list of potentially duplicate questions based on my header, but no luck. This is after many hours of googling.
I have a spreadsheet that I ...
- 1
0
votes
1
answer
8
views
How to make Train-Test split on multivariate timeseries data
I am building a model for the purpose of forcasting when someone is going into a stressful state.
I am using the WESAD dataset which has electrodermal activity (EDA) data on 11 subjects. I take this ...
- 101
0
votes
0
answers
5
views
Best ML Model for Embedded to detect anamoly over the time utilizing Accelerometer data (x, y, z)
I am trying to design a product related to predictive maintenance for the motor pump, where I need to utilize the accelerometer data to detect any upcoming anomaly in the future through Machine ...
0
votes
1
answer
16
views
0
votes
0
answers
8
views
Prophet generating negative seasonalities
Observed some negative values in the seasonality variable generated by Prophet. Have not observed that when using STL decomposition to extract seasonality from the same time series. Is this ...
- 1
0
votes
0
answers
9
views
NLP + Elasticsearch
I want to integrate NLP in my project ( medical data visualization using Elasticsearch and kibana)
have you any idea about that
0
votes
0
answers
11
views
RNN/LSTMs underfitting massively on spectrogram data - is a CNN encoder a prerequisite?
I am prototyping a pipeline on the FSDD dataset (audio/10-class classification); the audio data are loaded with librosa, 0-padded/trimmed to 0.5 sec (4000-dimensioned numpy vectors) each and converted ...
- 148
0
votes
0
answers
7
views
Can we set a maximum distance between pairs in a cluster with DBSCAN?
I'm trying to cluster embedded words (embedded using FastText) into words that are very similar e.g helloworld, heloworl and <...
- 440
0
votes
0
answers
5
views
Combine several pyod model into one scikit learn pipeline
I recently discovered pyod for outlier detection in python. Many outlier detection algorithms are implemented in pyod, and the package also comes with several combination function to combine the ...
0
votes
0
answers
16
views
Time-series LSTM using current/past exogenous variables and past outcomes
I want to predict a time-dependent outcome (y) using current/past features (exogenous variables x) and past outcomes (y). The features also change with time.
In other words, for each sample (different ...
- 1
0
votes
0
answers
15
views
How to prepare my dataset for T5?
I want to use T5 to do sentiment analysis on IMDB dataset. My dataset is of the following format:
...
- 1,267
0
votes
0
answers
6
views
In abstractive document summarization task, if I have multiple target sequences for one input document, what is the ideal loss form?
I use autoregressive model like T5 to tackle the abstractive document summarization task. In my case there are multiple target summarizations for one input document. Is there some related works about ...
- 103
0
votes
0
answers
10
views
YoloV5 results rendered lead to different picture
I basically trained a custom model to detect peg-solitaire games.
In my example I use the results of the detection to render a board with matplotlib
...
- 157
-1
votes
1
answer
21
views
0
votes
1
answer
35
views
Why use tanh (or any other activation function)?
In machine learning, it is common to use activation functions like
tanh, sigmoid, or ReLU to introduce non-linearity into a neural
network. These non-linearities help the network learn complex
...
- 103
0
votes
0
answers
12
views
Do exist Multi-output Deep Reinforcement Learning?
I need a deep reinforcement learning algorithm where the action is an array of N integer values. The idea is that given a state, the action that the agent will decide should be an array of N numbers, ...
- 1
0
votes
2
answers
26
views
Regression model doesn't handle negative values
I'm trying to create a model that, given a feature $x_i$, predicts $y_i$ such that $y_i=ax^2_i+bx_i+c$ by using backpropagation.
To do this, I'm using the ReLU activation function for each layer.
The ...
- 146
0
votes
0
answers
7
views
RNN using multiple time series for forecasting
I am trying to create a Rnn model to suppose, forecast weather. I have the weather data for a year recorded at multiple places and the dataset also contains other factors such as humidity, daylight ...
0
votes
0
answers
9
views
Techniques for choosing datasets from pool of datasets to create priors(time-series data, DTW? spike and slab?)
Let us consider a scenario where we have a pool of 100 datasets from various customers, with varying sizes containing sales and budgeting(mutiple channels) data per date. The datasets can range from ...
0
votes
1
answer
23
views
What is the relationship between robustness to Adversarial Attacks and Out-Of-Distribution OOD generalization?
You hear the word Robustness (defined as a model's robustness to spurious correlations ) a lot associated with Adversarial-Attacks and OOD Generalization and wonder how are these concepts interrelated?...
0
votes
0
answers
9
views
How can I separate and follow the plants that are very similar to each other with the YOLOv5 algorithm?
I want to differentiate between fern and mint using the YOLOv5 algorithm. Now I can take pictures of fern and mint, mark them on LabelImg, and train them in collaboration with Google. However, since ...
0
votes
0
answers
8
views
Is there a rule of thumb for the initial value of loss function in algorithm like word2vec?
There is a rule of thumb for the initial value of the loss function in multi-class classification tasks:
$$loss=-ln(1/number\_of\_unique\_class)$$
By comparing the initial loss with the loss after ...
0
votes
1
answer
27
views
Is my model overfitted?
I am using a naive bayes classifier to classify 20 newsgroup dataset. My accuracy on the training set is 97 and on the testing set is 89. Is my model overfitted?
If it is what steps can I take to ...
0
votes
1
answer
6
views
Training a Neural Network on standard deviation
Right now I have a training dataset that kinda looks like this (Image, Float) array, where the Image is independent variable and the float is the dependent variable:
...
0
votes
1
answer
27
views
Can I use LLM to explain codebase?
I am a Data Engineer, and I am currently assigned a task to refactor an outdated code and rectify any bugs present. However, I am unable to comprehend the code written in the existing codebase. ...
- 3
0
votes
0
answers
7
views
Is it possible to encourage a constitutional network to reduce the range of activations
I am trying to implement a statically-quantized convolutional network. A problem I am having is that the convolutional layers all tend to produce an activation tensor with a large range (roughly -10.0 ...
0
votes
0
answers
16
views
Reinforcement learning with Q-learning doesn't seem to be learning
I'm learning reinforcement learning with Q-learning and I made a training script for Flappy Bird. The problem is that it doesn't seem to be learning and I'm not sure why.
My guess is that there's ...
- 1
0
votes
1
answer
12
views
Best practice for variables that only have answer if yes in previous column
I currently have a dataset that consists of survey data that has several columns that have answers dependent on the previous question. For example, I may have a question that says "Did you take ...
- 23
0
votes
0
answers
12
views
How to label a transactions dataset made from scratch?
I have a question regarding creating a transactions dataset from scratch.
I've created customer profiles and am generating transactions based on these profiles. The way I do this is based on the ...
- 1
0
votes
1
answer
15
views
What is the best way to store large data when training a model?
This is rather a technical question, when training the model I import data locally to Google Colab runtime which does not seem to be efficient, when the runtime ...
- 101
0
votes
1
answer
24
views
Matching items in a recommender system
I would like to ask for a proposal for a machine learning model that would be suitable for the following problem:
I have a training set where each element of type A corresponds to a certain number of ...
- 41
0
votes
0
answers
16
views
ValueError: Found input variables with inconsistent numbers of samples: [283, 943]
I am trying yo split the data using train_test_split(), but I got this error:
...
- 1
1
vote
1
answer
15
views
How to interpret a Bayesian neural network prediction for binary classification in comparison to deterministic neural network?
Allow me to clarify my current understanding:
Interpreting a binary classification prediction made by a deterministic neural network
On one hand, point estimates fall on a sigmoid curve (between 0-1, ...
- 489