# All Questions

29,343 questions
Filter by
Sorted by
Tagged with
5 views

### ML model to forecast time series data

This question has three sub-parts, answering each of which probably doesn't require huge text. I hope that is okay. I'm trying to understand time series prediction using ML. I have the target variable ...
5 views

### Tableau - Take row-wise data and pivot it to have an output table based on separate table of product hierarchies

I have sales data with a unique customer ID on each row and their corresponding sales by Product on columns. I have a separate table for product hierarchy, which has a mapping of all products (column ...
3 views

### Visualization suggestion:-

I have a data frame like below:- Here I have 194 countries and the columns are fan_out values which is in percent of the total population. Like for country AD, the total fan_out value is 2.24 -06 % of ...
9 views

### Prediction of rain using DBSCAN and Spectral Clustering

I wanted to do a prediction model of rainfall after clustering of data. I don't know how to proceed.Is it possible? May be using csv data.
5 views

### Is it possible to use a different metric in early_stopping and ModelCheckpoint?

I'm using the tensorflow functions checkpoint and early stopping. Can I use 'val accuracy' in checkpoint to  and 'val loss' in early stopping? or  Should I use the same metric in both functions? ...
12 views

### How does K-Means clustering help in the analysis of Word2vec embeddings

I have a yelp-review dataset. I have done a word2vector embedding on the text column of the yelp-review. I am using unsupervised leaning K-means and PCA to visualise the data. I have got 6 clusters ...
10 views

### How can I prepare my discrete batches of data for training?

I'm trying to calculate effect of parameters of an operation on the thickness of a wall. Each operation is thinning the wall thickness and at some point the wall is replaced and operation starts again....
18 views

### Data cleaning how to detect it is enough

We are working on 5 columns data with 1 column output as you can see below: So we are working with LSTM and time series classification methods, and we need to clean the data and the data cleaning. ...
7 views

### Evaluation Metrics for Multiclass

How to obtain the Accuracy, Detection_Rate, False_Positive_Rate and ...
11 views

### Studying and choosing between different neural network structures

I wish like to develop a model that uses convolutional neural networks for image classification. From the many different networks structures described on papers and articles online, I want to choose ...
12 views

### introduction to stochastic neural networks

I need to know more about stochastic neural networks as used here but I really don't understand where they are coming from. Here's my understanding of normal neural networks: Given an unknown function ...
16 views

### Quantitative measure of the smoothness of learning curves

$\DeclareMathOperator{\loss}{loss}$ $\DeclareMathOperator{\AvgVar}{AvgVar}$ Lat's say we have some deep learning task. We have our model and two sets of hyperparameters $A$ and $B$. We train both ...
5 views

### How to incorporate multi-task in CTR/recommendation model (deep & wide/ xDeepFM etc)?

I am building a rank algorithm for an e-commerce website that ranks the product based on likely hood of purchase and I have formulated this problem into a binary classification problem. Given each ...
13 views

### Tips for clustering rows of a gigantic “distance” matrix

I've been assigned to the following task: I was given 1,000,000 data points and was asked to create a sort of distance matrix and to cluster the rows. So this matrix is 1,000,000 x 1,000,000 which is ...
7 views

### Detect Bias in YOLO format object detection dataset

I work on YOLO data annotation. I want to know how to detect bias in the data before training in this kind of datasets. What are the metrics and analysis methods needed to use to detect bias in ...
9 views

### Question regarding weights in a Model

Let's take a Linear regression model.I just want to know are weights the same for every row once the model is trained? Or are weights vector or an array for eg I have data X= [4,5] four rows and 5 ...
12 views

### Memorization in deep neural networks, random vs. properly labelled datasets

From about 19:20 in the video here: https://www.youtube.com/watch?v=IHZwWFHWa-w it shows the difference in value of the cost function for randomly labelled data vs. properly labelled data. What do ...
18 views

### How to detect outliers to lable my unsupervised data?

I have unsupervised sensor data and i want to lable each row of data as anomaly or normal. Here K-mean clustering will work?
8 views

### How to perform transfer learning on tabular data?

Is it possible to apply transfer learning on tabular data?
13 views

### How to represent a “switch”-like behavior in a neural network?

I have three input variables $x_1$, $x_2$ and $d$, where $x_1$ and $x_2$ are numerical variables and $d$ is a dummy variable that takes the value of 1 or 2. How to represent the part of a neural ...
6 views

### In Keras, how to organize multiple input features using pre-trained embedding mapping?

Let's say the goal is to predict weather given multiple features (temp, humidity) in the past 3 days. weather (y) can be: Sunny, Cloudy, Rainy. Temp (X1) can be : Hot, Cool, Cold. Humidity (X2) can be:...
17 views

### The proper way of resampling data to be used for clustering when the outliers are the important data points

I have a dataset that is bigger than I need it to be. In fact, bigger than my hardware can handle. So I'm trying to lower the number of samples. And I'm not sure what is the right approach to do so. ...
13 views

### GridSearchCV and time complexity

So, I was learning and trying to implement a GridSearch. I have a question regarding the following code, which I wrote: ...
12 views

### Panel data classification and regression?

I am very new to time series/panel/longituginal data. From my understanding Panel data = multi-object time series and I have some panel data in long format (objects have multiple rows corresponding to ...
8 views

### Tsallis entropy - advice needed regarding obtaining probability distribution

As is always the way I stumbled across Tsallis entropy on SO whilst looking for something completely different. This soon lead me reading all sorts of interesting but terse academic papers. I am ...
4 views

### How does bayesian optimization with gaussian processes work?

Could someone explain in simple words what are gaussian processes how does bayesian optimization work and their combination?
11 views

### What to do when facing bias in LSTM?

I have created an LSTM model which is trained on an 8-hour time frame for a cryptocurrency. When the training is finished I see that it is learning the pattern but there is some bias in it. How to ...
13 views

### Comparing Baysain hyper-parameter Optimization algorithms over sampled space

I want to compare Bayesian hyper-parameter Optimization algorithms for optimizing application performance. The problem is that each sample (from that space) takes long time to compute and is noisy. So ...
11 views

### How to use genetic algorithm to generate association rules from a transaction dataset using Python?

Please help! I really need to use GA to obtain association rules. My dataset looks like this: ...
15 views

### How to input a list into my model and not have it care about order

I'm trying to predict a list of numbers, e.g: [23,55,198,200,64] The data I have includes multiple things, along with: The numbers from the previous run (These ...
14 views

### How to apply class weights for imbalanced data and given misclassification costs?

Say I have a dataset for binary classification with 1000 samples in the minority class and 100000 samples in the majority class and use XGBoost to output probabilities. From an expert I know that ...
32 views

### How to structure unstructured data

I am analysing tweets and have collected them in an unstructured format. What is the best way to structure this data so I can begin the data mining processes? Somebody suggested using python packages ...
22 views

### How to obtain Accuracy of Feature Selection methods?

I used the following methods: Variance_Threshold: selecto_vth = VarianceThreshold(threshold=1.0) ANOVA: ...
23 views

### Ethical consequences of non-deterministic learning processes?

Most advanced supervised learning techniques are non-deterministic by construction. The final output of the model usually depends on some random parts of the learning process. (Random weight ...
10 views

### Pytorch DataLoader returns iterable , how and when to convert into a Tensor for Model Training

I have coded up two DataSet classes (one map-style and one iterator-style) to be used with DataLoader (either one is okay, but I just wanted to experiment and learn myself) on TEXT data. The data + ...
24 views

### XgBoost given targets its only feature but fails when test targets are outside the range of training targets?

I'm learning to use XgBoost, and I'm doing an exercise involving predicting prices. However I'm noticing some weird behavior where XgBoost's predictions deviate from the target value even if I'm ...
19 views

### How can a complex function be approximated using deep learning?

The insurance company I work for has a computationally intensive process to estimate future earnings based on tables of assumptions regarding price and probability of cancelation. I would like to ...
15 views

### Evaluation metric for time-series anomaly detection

I have a question for AI or data experts. I'm writing a paper My dataset are time-series sensor data and anomaly ratio is between 5% and 6% 1. For time-series anomaly detection evaluation, which one ...
14 views

### Label data set for sentiment analysis

I am a beginner in this field. I have a scrapped review data set. It contains review socre (1 - 10) and review content. I am going to label the reviews according to the review score like below : 0-2 -&...
19 views

### How to generate images from numerical data [closed]

I'm a beginner at GAN's and have to perform a task for my job. I am basically required to generate images from a set of input variables. I have a dataset with a few images that correlate to the input ...
18 views

### Any recomended Neural Network structure to classify similar images?

I have a number of very similar images, and I want to classify them into two groups. I used Resnet50 before, but the network does not seem to be learning at all. As the training progresses, the ...
20 views

### How to handle “year” variable for classification

I am writing to you because I need to create a model that tells me whether or not a company will pay its taxes the next month. For that I have data from 2017 to 2020, with characteristics such as size ...
14 views

### When training a cGAN on a X (B/W image) to Y (RGB image) “paired” dataset, how much will slight differences in the images matter in terms of results?

Right now I am working on a colorization project with GANs, and had originally settled on using a CycleGAN because I considered structuring my dataset in an unpaired manner. I've since been able to ...
15 views

### Top 5 useful Data Science skills [closed]

I am a beginner in Data Science, exploring and learning how to operate Data Science stack exchange as well. I would like to know the top 5 useful Data Science skills to acquire in 2021.
12 views

### Counter-intuitive weight in logistic regression

I have a question about logistic regression. One of the features I built is a binary feature which produces either 0 or 1. The number of the positive instances with the feature activated is ~110K ...
6 views

### Ordering of sklearn's confusion_matrix()

Trying to figure out how to sort integer labels on a 102X102 (oxford102 UK flower dataset) confusion matrix graph that I plot with plot_confusion_matrix from mlxtend. I see that basically conf_mat and ...
11 views

### Is it possible for class activation maps where the predicted class has no activations?

I'm trying out grad CAM from the captum library in pytorch for a 3 class image classifier that uses transfer learning on resnet50. The model has good accuracy, but when I was looking at the generated ...
9 views

### How does Keras optimization for a network with multiple outputs

I currently have a neural network that takes in 3 numbers as inputs and outputs 3 numbers. I've attached a picture of the network below and my code is accessible through the following link: [Google ...