All Questions

This question has three sub-parts, answering each of which probably doesn't require huge text. I hope that is okay. I'm trying to understand time series prediction using ML. I have the target variable ...

I have sales data with a unique customer ID on each row and their corresponding sales by Product on columns. I have a separate table for product hierarchy, which has a mapping of all products (column ...

I have a data frame like below:- Here I have 194 countries and the columns are fan_out values which is in percent of the total population. Like for country AD, the total fan_out value is 2.24 -06 % of ...

I wanted to do a prediction model of rainfall after clustering of data. I don't know how to proceed.Is it possible? May be using csv data.

I'm using the tensorflow functions checkpoint and early stopping. Can I use 'val accuracy' in checkpoint to  and 'val loss' in early stopping? or  Should I use the same metric in both functions? ...

I have a yelp-review dataset. I have done a word2vector embedding on the text column of the yelp-review. I am using unsupervised leaning K-means and PCA to visualise the data. I have got 6 clusters ...

I'm trying to calculate effect of parameters of an operation on the thickness of a wall. Each operation is thinning the wall thickness and at some point the wall is replaced and operation starts again....

We are working on 5 columns data with 1 column output as you can see below: So we are working with LSTM and time series classification methods, and we need to clean the data and the data cleaning. ...

How to obtain the Accuracy, Detection_Rate, False_Positive_Rate and ...

I wish like to develop a model that uses convolutional neural networks for image classification. From the many different networks structures described on papers and articles online, I want to choose ...

I need to know more about stochastic neural networks as used here but I really don't understand where they are coming from. Here's my understanding of normal neural networks: Given an unknown function ...

$\DeclareMathOperator{\loss}{loss}$ $\DeclareMathOperator{\AvgVar}{AvgVar}$ Lat's say we have some deep learning task. We have our model and two sets of hyperparameters $A$ and $B$. We train both ...

I am building a rank algorithm for an e-commerce website that ranks the product based on likely hood of purchase and I have formulated this problem into a binary classification problem. Given each ...

I've been assigned to the following task: I was given 1,000,000 data points and was asked to create a sort of distance matrix and to cluster the rows. So this matrix is 1,000,000 x 1,000,000 which is ...

I work on YOLO data annotation. I want to know how to detect bias in the data before training in this kind of datasets. What are the metrics and analysis methods needed to use to detect bias in ...

Let's take a Linear regression model.I just want to know are weights the same for every row once the model is trained? Or are weights vector or an array for eg I have data X= [4,5] four rows and 5 ...

From about 19:20 in the video here: https://www.youtube.com/watch?v=IHZwWFHWa-w it shows the difference in value of the cost function for randomly labelled data vs. properly labelled data. What do ...

I have unsupervised sensor data and i want to lable each row of data as anomaly or normal. Here K-mean clustering will work?

Is it possible to apply transfer learning on tabular data?

I have three input variables $x_1$, $x_2$ and $d$, where $x_1$ and $x_2$ are numerical variables and $d$ is a dummy variable that takes the value of 1 or 2. How to represent the part of a neural ...

Let's say the goal is to predict weather given multiple features (temp, humidity) in the past 3 days. weather (y) can be: Sunny, Cloudy, Rainy. Temp (X1) can be : Hot, Cool, Cold. Humidity (X2) can be:...

I have a dataset that is bigger than I need it to be. In fact, bigger than my hardware can handle. So I'm trying to lower the number of samples. And I'm not sure what is the right approach to do so. ...

So, I was learning and trying to implement a GridSearch. I have a question regarding the following code, which I wrote: ...

I am very new to time series/panel/longituginal data. From my understanding Panel data = multi-object time series and I have some panel data in long format (objects have multiple rows corresponding to ...

As is always the way I stumbled across Tsallis entropy on SO whilst looking for something completely different. This soon lead me reading all sorts of interesting but terse academic papers. I am ...

Could someone explain in simple words what are gaussian processes how does bayesian optimization work and their combination?

I have created an LSTM model which is trained on an 8-hour time frame for a cryptocurrency. When the training is finished I see that it is learning the pattern but there is some bias in it. How to ...

I want to compare Bayesian hyper-parameter Optimization algorithms for optimizing application performance. The problem is that each sample (from that space) takes long time to compute and is noisy. So ...

Please help! I really need to use GA to obtain association rules. My dataset looks like this: ...

I'm trying to predict a list of numbers, e.g: [23,55,198,200,64] The data I have includes multiple things, along with: The numbers from the previous run (These ...

Say I have a dataset for binary classification with 1000 samples in the minority class and 100000 samples in the majority class and use XGBoost to output probabilities. From an expert I know that ...

I am analysing tweets and have collected them in an unstructured format. What is the best way to structure this data so I can begin the data mining processes? Somebody suggested using python packages ...

I used the following methods: Variance_Threshold: selecto_vth = VarianceThreshold(threshold=1.0) ANOVA: ...

Most advanced supervised learning techniques are non-deterministic by construction. The final output of the model usually depends on some random parts of the learning process. (Random weight ...

I have coded up two DataSet classes (one map-style and one iterator-style) to be used with DataLoader (either one is okay, but I just wanted to experiment and learn myself) on TEXT data. The data + ...

I'm learning to use XgBoost, and I'm doing an exercise involving predicting prices. However I'm noticing some weird behavior where XgBoost's predictions deviate from the target value even if I'm ...

The insurance company I work for has a computationally intensive process to estimate future earnings based on tables of assumptions regarding price and probability of cancelation. I would like to ...

I have a question for AI or data experts. I'm writing a paper My dataset are time-series sensor data and anomaly ratio is between 5% and 6% 1. For time-series anomaly detection evaluation, which one ...

I am a beginner in this field. I have a scrapped review data set. It contains review socre (1 - 10) and review content. I am going to label the reviews according to the review score like below : 0-2 -&...

I'm a beginner at GAN's and have to perform a task for my job. I am basically required to generate images from a set of input variables. I have a dataset with a few images that correlate to the input ...

I have a number of very similar images, and I want to classify them into two groups. I used Resnet50 before, but the network does not seem to be learning at all. As the training progresses, the ...

I am writing to you because I need to create a model that tells me whether or not a company will pay its taxes the next month. For that I have data from 2017 to 2020, with characteristics such as size ...

Right now I am working on a colorization project with GANs, and had originally settled on using a CycleGAN because I considered structuring my dataset in an unpaired manner. I've since been able to ...

I am a beginner in Data Science, exploring and learning how to operate Data Science stack exchange as well. I would like to know the top 5 useful Data Science skills to acquire in 2021.

I have a question about logistic regression. One of the features I built is a binary feature which produces either 0 or 1. The number of the positive instances with the feature activated is ~110K ...

Trying to figure out how to sort integer labels on a 102X102 (oxford102 UK flower dataset) confusion matrix graph that I plot with plot_confusion_matrix from mlxtend. I see that basically conf_mat and ...

I'm trying out grad CAM from the captum library in pytorch for a 3 class image classifier that uses transfer learning on resnet50. The model has good accuracy, but when I was looking at the generated ...

I currently have a neural network that takes in 3 numbers as inputs and outputs 3 numbers. I've attached a picture of the network below and my code is accessible through the following link: [Google ...

So I'm new to data science and was trying to solve a few problems that my mentor gave to me. I came across this question where there are multiple coin tosses and ten of them are recorded. I am ...

The entrys are real numbers (each row is a channel output, where blocklength n= 128). The accuracy is approx. 50%. The labels contain elements of the set {0,1}. The dataset is balanced. For smaller ...