# All Questions

35,972 questions
Filter by
Sorted by
Tagged with
5 views

### Why is UMAP used in combination with other Clustering Algorithm?

I've noticed that UMAP is often used in combination with other clustering algorithms, such as K-means, DBSCAN, HDBSCAN. However, from what I've understood, UMAP can be used for clustering tasks. So ...
• 101
4 views

### For a node in an undirected graph - does the node affect itself if its markov blanket is known?

Consider the following Markov Random Field. Question 1: Which of the following nodes will have no effect on H given the Markov Blanket of H? Question 2: Will node H itself have any effect on itself, ...
3 views

### Gaining insights into commonly co-bought categories of products using association rules (frequent itemset mining)

Let's say I have a small database of supermarket transactions. Each records consists of a tx with several bought products, each having a unique identifier (eg, 1,2,3,4,5 etc.). Running A-priori on ...
9 views

### Averaging subregion correlation coefficients into a single measure

I've got a 257x257 correlation matrix of functional connectivity (fMRI) data. It is a symmetric matrix where each value is the Pearsons correlation of the brain area in the row with the brain area in ...
• 101
18 views

### Overfitted model [duplicate]

A classic question with an unclear answer, is it better to have an overfitted model performing better on a Cross-Validation setting, or a non-overfitted model performing worse? In this context, higher ...
• 133
9 views

### How do you know which is the closest class example(observation) to a particular class in Self Organizing Map?

For example there are two classes C1 and C2, distance1 and distance2, The rule tells which class is: If distance1 > distance2 than class C1 (Node1) else class C2 (Node2) Here are some data examples:...
9 views

### Is the node in undirected graph itself included in the set of its own Markov Blanket?

Consider an undirected graph with nodes set {a,b,c,d,e}, and edge set {(a,b), (a,c), (a,d), (a,e)}. From the above info, you will clearly visualise that the node a ...
6 views

### Time series : add constants for each time series

I have N time series of shape (30*36) (time step, feature). For each time series 500 parameters that can be seen as the history of the time series have to be added. ...
• 29
15 views

### How sklearn logistic regression computes accuracy, recall etc if we don't provide threshold?

It might be a stupid question, but I just realized that calling score function on logistic regression model shouldn't make any sense - as far as I know in sklearn ...
• 103
320 views

### What is Data Scientist's Salary In 2023?

I searched for information on data scientist pay by country but was unable to find any. I am aware that the US pays data scientists an absurdly high salary (well, maybe not "insanely high" ...
8 views

### Voting classifier ensemble error: 'ValueError: Unknown metric function: 'function'.'

I am trying to create a hard voting ensemble of three neural networks. I've already converted them to Keras Classifiers. Here is the code: ...
8 views

### ANN regressor with multiple outputs in place of single data output

I built an ANN model(Keras) to handle a regression problem with one output. By mistake I filled 10 units at the output layer (instead of 1). To my surprise, there was no error. Later on this was ...
• 101
8 views

### Multi output regression model with non categorical text data

I am doing a multioutput regression model where 1 input and 6 output. My dataset contain all non categorical text data. I used countvectorization in the preprocessing but got an error while fitting ...
15 views

### StandardScaler and MinMaxScaler vs RobustScaler

I've recently read that Standard Scaler functions best in situations where the distribution of the features are approximately normal. MinMaxScaler works in a way that it preserves the features' ...
1 vote
9 views

### What is the difference between State Value function and Return for Markov Reward process ( MRP)?

I have been going through Stanford Lecture on RL. I see in MRP that Return function is same as State Value function. Both are getting expected sum of reward keeping discount factor in mind. Although ...
9 views

### How ReLU is bringing non linearity and why it is not an alternative to dropout?

The differentiation of ReLU function is 1 when input is greater than 0, and 0, when input is less than or equal to 0. In the backpropagation process it doesn’t change the value of d(error)/d(weight) ...
9 views

### My linear regression doesn't work when i try to calculate theta1

I want to create my own linear regression. But my formula of the coefficient theta1 doesn't work i have big values : ...
10 views

### Convert csv with row and column headers into nested json

I've searched here and on Stackoverflow and read through the list of potentially duplicate questions based on my header, but no luck. This is after many hours of googling. I have a spreadsheet that I ...
8 views

### How to make Train-Test split on multivariate timeseries data

I am building a model for the purpose of forcasting when someone is going into a stressful state. I am using the WESAD dataset which has electrodermal activity (EDA) data on 11 subjects. I take this ...
• 101
5 views

### Best ML Model for Embedded to detect anamoly over the time utilizing Accelerometer data (x, y, z)

I am trying to design a product related to predictive maintenance for the motor pump, where I need to utilize the accelerometer data to detect any upcoming anomaly in the future through Machine ...
16 views

### Is it good if during training the model my test accuracy much higher then train accuracy. How can i prevent this?

my training code: ...
• 1
8 views

### Prophet generating negative seasonalities

Observed some negative values in the seasonality variable generated by Prophet. Have not observed that when using STL decomposition to extract seasonality from the same time series. Is this ...
9 views

### NLP + Elasticsearch

I want to integrate NLP in my project ( medical data visualization using Elasticsearch and kibana) have you any idea about that
11 views

### RNN/LSTMs underfitting massively on spectrogram data - is a CNN encoder a prerequisite?

I am prototyping a pipeline on the FSDD dataset (audio/10-class classification); the audio data are loaded with librosa, 0-padded/trimmed to 0.5 sec (4000-dimensioned numpy vectors) each and converted ...
• 148
7 views

### Can we set a maximum distance between pairs in a cluster with DBSCAN?

I'm trying to cluster embedded words (embedded using FastText) into words that are very similar e.g helloworld, heloworl and <...
• 440
5 views

### Combine several pyod model into one scikit learn pipeline

I recently discovered pyod for outlier detection in python. Many outlier detection algorithms are implemented in pyod, and the package also comes with several combination function to combine the ...
16 views

### Time-series LSTM using current/past exogenous variables and past outcomes

I want to predict a time-dependent outcome (y) using current/past features (exogenous variables x) and past outcomes (y). The features also change with time. In other words, for each sample (different ...
15 views

### How to prepare my dataset for T5?

I want to use T5 to do sentiment analysis on IMDB dataset. My dataset is of the following format: ...
• 1,267
6 views

### In abstractive document summarization task, if I have multiple target sequences for one input document, what is the ideal loss form?

I use autoregressive model like T5 to tackle the abstractive document summarization task. In my case there are multiple target summarizations for one input document. Is there some related works about ...
• 103
10 views

### YoloV5 results rendered lead to different picture

I basically trained a custom model to detect peg-solitaire games. In my example I use the results of the detection to render a board with matplotlib ...
• 157
21 views

...
35 views

### Why use tanh (or any other activation function)?

In machine learning, it is common to use activation functions like tanh, sigmoid, or ReLU to introduce non-linearity into a neural network. These non-linearities help the network learn complex ...
• 103
12 views

### Do exist Multi-output Deep Reinforcement Learning?

I need a deep reinforcement learning algorithm where the action is an array of N integer values. The idea is that given a state, the action that the agent will decide should be an array of N numbers, ...
26 views

### Regression model doesn't handle negative values

I'm trying to create a model that, given a feature $x_i$, predicts $y_i$ such that $y_i=ax^2_i+bx_i+c$ by using backpropagation. To do this, I'm using the ReLU activation function for each layer. The ...
• 146
7 views

### RNN using multiple time series for forecasting

I am trying to create a Rnn model to suppose, forecast weather. I have the weather data for a year recorded at multiple places and the dataset also contains other factors such as humidity, daylight ...
9 views

### Techniques for choosing datasets from pool of datasets to create priors(time-series data, DTW? spike and slab?)

Let us consider a scenario where we have a pool of 100 datasets from various customers, with varying sizes containing sales and budgeting(mutiple channels) data per date. The datasets can range from ...
23 views

### What is the relationship between robustness to Adversarial Attacks and Out-Of-Distribution OOD generalization?

You hear the word Robustness (defined as a model's robustness to spurious correlations ) a lot associated with Adversarial-Attacks and OOD Generalization and wonder how are these concepts interrelated?...
9 views

### How can I separate and follow the plants that are very similar to each other with the YOLOv5 algorithm?

I want to differentiate between fern and mint using the YOLOv5 algorithm. Now I can take pictures of fern and mint, mark them on LabelImg, and train them in collaboration with Google. However, since ...
8 views

### Is there a rule of thumb for the initial value of loss function in algorithm like word2vec?

There is a rule of thumb for the initial value of the loss function in multi-class classification tasks: $$loss=-ln(1/number\_of\_unique\_class)$$ By comparing the initial loss with the loss after ...
27 views

### Is my model overfitted?

I am using a naive bayes classifier to classify 20 newsgroup dataset. My accuracy on the training set is 97 and on the testing set is 89. Is my model overfitted? If it is what steps can I take to ...
6 views

### Training a Neural Network on standard deviation

Right now I have a training dataset that kinda looks like this (Image, Float) array, where the Image is independent variable and the float is the dependent variable: ...
27 views

### Can I use LLM to explain codebase?

I am a Data Engineer, and I am currently assigned a task to refactor an outdated code and rectify any bugs present. However, I am unable to comprehend the code written in the existing codebase. ...
7 views

### Is it possible to encourage a constitutional network to reduce the range of activations

I am trying to implement a statically-quantized convolutional network. A problem I am having is that the convolutional layers all tend to produce an activation tensor with a large range (roughly -10.0 ...
16 views

### Reinforcement learning with Q-learning doesn't seem to be learning

I'm learning reinforcement learning with Q-learning and I made a training script for Flappy Bird. The problem is that it doesn't seem to be learning and I'm not sure why. My guess is that there's ...
12 views

### Best practice for variables that only have answer if yes in previous column

I currently have a dataset that consists of survey data that has several columns that have answers dependent on the previous question. For example, I may have a question that says "Did you take ...
• 23
12 views

### How to label a transactions dataset made from scratch?

I have a question regarding creating a transactions dataset from scratch. I've created customer profiles and am generating transactions based on these profiles. The way I do this is based on the ...
15 views

### What is the best way to store large data when training a model?

This is rather a technical question, when training the model I import data locally to Google Colab runtime which does not seem to be efficient, when the runtime ...
• 101
24 views

### Matching items in a recommender system

I would like to ask for a proposal for a machine learning model that would be suitable for the following problem: I have a training set where each element of type A corresponds to a certain number of ...
• 41
16 views

### ValueError: Found input variables with inconsistent numbers of samples: [283, 943]

I am trying yo split the data using train_test_split(), but I got this error: ...
• 1
1 vote