All Questions

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

ML model to forecast time series data

This question has three sub-parts, answering each of which probably doesn't require huge text. I hope that is okay. I'm trying to understand time series prediction using ML. I have the target variable ...
0
votes
0answers
5 views

Tableau - Take row-wise data and pivot it to have an output table based on separate table of product hierarchies

I have sales data with a unique customer ID on each row and their corresponding sales by Product on columns. I have a separate table for product hierarchy, which has a mapping of all products (column ...
0
votes
0answers
3 views

Visualization suggestion:-

I have a data frame like below:- Here I have 194 countries and the columns are fan_out values which is in percent of the total population. Like for country AD, the total fan_out value is 2.24 -06 % of ...
1
vote
0answers
9 views

Prediction of rain using DBSCAN and Spectral Clustering

I wanted to do a prediction model of rainfall after clustering of data. I don't know how to proceed.Is it possible? May be using csv data.
0
votes
0answers
5 views

Is it possible to use a different metric in early_stopping and ModelCheckpoint?

I'm using the tensorflow functions checkpoint and early stopping. Can I use 'val accuracy' in checkpoint to  and 'val loss' in early stopping? or  Should I use the same metric in both functions? ...
1
vote
1answer
12 views

How does K-Means clustering help in the analysis of Word2vec embeddings

I have a yelp-review dataset. I have done a word2vector embedding on the text column of the yelp-review. I am using unsupervised leaning K-means and PCA to visualise the data. I have got 6 clusters ...
0
votes
0answers
10 views

How can I prepare my discrete batches of data for training?

I'm trying to calculate effect of parameters of an operation on the thickness of a wall. Each operation is thinning the wall thickness and at some point the wall is replaced and operation starts again....
1
vote
1answer
18 views

Data cleaning how to detect it is enough

We are working on 5 columns data with 1 column output as you can see below: So we are working with LSTM and time series classification methods, and we need to clean the data and the data cleaning. ...
0
votes
1answer
7 views

Evaluation Metrics for Multiclass

How to obtain the Accuracy, Detection_Rate, False_Positive_Rate and ...
1
vote
1answer
11 views

Studying and choosing between different neural network structures

I wish like to develop a model that uses convolutional neural networks for image classification. From the many different networks structures described on papers and articles online, I want to choose ...
0
votes
0answers
12 views

introduction to stochastic neural networks

I need to know more about stochastic neural networks as used here but I really don't understand where they are coming from. Here's my understanding of normal neural networks: Given an unknown function ...
1
vote
1answer
16 views

Quantitative measure of the smoothness of learning curves

$\DeclareMathOperator{\loss}{loss}$ $\DeclareMathOperator{\AvgVar}{AvgVar}$ Lat's say we have some deep learning task. We have our model and two sets of hyperparameters $A$ and $B$. We train both ...
0
votes
0answers
5 views

How to incorporate multi-task in CTR/recommendation model (deep & wide/ xDeepFM etc)?

I am building a rank algorithm for an e-commerce website that ranks the product based on likely hood of purchase and I have formulated this problem into a binary classification problem. Given each ...
1
vote
1answer
13 views

Tips for clustering rows of a gigantic “distance” matrix

I've been assigned to the following task: I was given 1,000,000 data points and was asked to create a sort of distance matrix and to cluster the rows. So this matrix is 1,000,000 x 1,000,000 which is ...
1
vote
0answers
7 views

Detect Bias in YOLO format object detection dataset

I work on YOLO data annotation. I want to know how to detect bias in the data before training in this kind of datasets. What are the metrics and analysis methods needed to use to detect bias in ...
0
votes
1answer
9 views

Question regarding weights in a Model

Let's take a Linear regression model.I just want to know are weights the same for every row once the model is trained? Or are weights vector or an array for eg I have data X= [4,5] four rows and 5 ...
0
votes
1answer
12 views

Memorization in deep neural networks, random vs. properly labelled datasets

From about 19:20 in the video here: https://www.youtube.com/watch?v=IHZwWFHWa-w it shows the difference in value of the cost function for randomly labelled data vs. properly labelled data. What do ...
1
vote
3answers
18 views

How to detect outliers to lable my unsupervised data?

I have unsupervised sensor data and i want to lable each row of data as anomaly or normal. Here K-mean clustering will work?
0
votes
0answers
8 views

How to perform transfer learning on tabular data?

Is it possible to apply transfer learning on tabular data?
0
votes
0answers
13 views

How to represent a “switch”-like behavior in a neural network?

I have three input variables $x_1$, $x_2$ and $d$, where $x_1$ and $x_2$ are numerical variables and $d$ is a dummy variable that takes the value of 1 or 2. How to represent the part of a neural ...
1
vote
0answers
6 views

In Keras, how to organize multiple input features using pre-trained embedding mapping?

Let's say the goal is to predict weather given multiple features (temp, humidity) in the past 3 days. weather (y) can be: Sunny, Cloudy, Rainy. Temp (X1) can be : Hot, Cool, Cold. Humidity (X2) can be:...
0
votes
0answers
17 views

The proper way of resampling data to be used for clustering when the outliers are the important data points

I have a dataset that is bigger than I need it to be. In fact, bigger than my hardware can handle. So I'm trying to lower the number of samples. And I'm not sure what is the right approach to do so. ...
1
vote
1answer
13 views

GridSearchCV and time complexity

So, I was learning and trying to implement a GridSearch. I have a question regarding the following code, which I wrote: ...
1
vote
0answers
12 views

Panel data classification and regression?

I am very new to time series/panel/longituginal data. From my understanding Panel data = multi-object time series and I have some panel data in long format (objects have multiple rows corresponding to ...
1
vote
0answers
8 views

Tsallis entropy - advice needed regarding obtaining probability distribution

As is always the way I stumbled across Tsallis entropy on SO whilst looking for something completely different. This soon lead me reading all sorts of interesting but terse academic papers. I am ...
0
votes
0answers
4 views

How does bayesian optimization with gaussian processes work?

Could someone explain in simple words what are gaussian processes how does bayesian optimization work and their combination?
0
votes
0answers
11 views

What to do when facing bias in LSTM?

I have created an LSTM model which is trained on an 8-hour time frame for a cryptocurrency. When the training is finished I see that it is learning the pattern but there is some bias in it. How to ...
0
votes
0answers
13 views

Comparing Baysain hyper-parameter Optimization algorithms over sampled space

I want to compare Bayesian hyper-parameter Optimization algorithms for optimizing application performance. The problem is that each sample (from that space) takes long time to compute and is noisy. So ...
0
votes
0answers
11 views

How to use genetic algorithm to generate association rules from a transaction dataset using Python?

Please help! I really need to use GA to obtain association rules. My dataset looks like this: ...
1
vote
1answer
15 views

How to input a list into my model and not have it care about order

I'm trying to predict a list of numbers, e.g: [23,55,198,200,64] The data I have includes multiple things, along with: The numbers from the previous run (These ...
0
votes
0answers
14 views

How to apply class weights for imbalanced data and given misclassification costs?

Say I have a dataset for binary classification with 1000 samples in the minority class and 100000 samples in the majority class and use XGBoost to output probabilities. From an expert I know that ...
1
vote
1answer
32 views

How to structure unstructured data

I am analysing tweets and have collected them in an unstructured format. What is the best way to structure this data so I can begin the data mining processes? Somebody suggested using python packages ...
1
vote
1answer
22 views

How to obtain Accuracy of Feature Selection methods?

I used the following methods: Variance_Threshold: selecto_vth = VarianceThreshold(threshold=1.0) ANOVA: ...
4
votes
0answers
23 views

Ethical consequences of non-deterministic learning processes?

Most advanced supervised learning techniques are non-deterministic by construction. The final output of the model usually depends on some random parts of the learning process. (Random weight ...
0
votes
0answers
10 views

Pytorch DataLoader returns iterable , how and when to convert into a Tensor for Model Training

I have coded up two DataSet classes (one map-style and one iterator-style) to be used with DataLoader (either one is okay, but I just wanted to experiment and learn myself) on TEXT data. The data + ...
0
votes
2answers
24 views

XgBoost given targets its only feature but fails when test targets are outside the range of training targets?

I'm learning to use XgBoost, and I'm doing an exercise involving predicting prices. However I'm noticing some weird behavior where XgBoost's predictions deviate from the target value even if I'm ...
0
votes
0answers
19 views

How can a complex function be approximated using deep learning?

The insurance company I work for has a computationally intensive process to estimate future earnings based on tables of assumptions regarding price and probability of cancelation. I would like to ...
1
vote
1answer
15 views

Evaluation metric for time-series anomaly detection

I have a question for AI or data experts. I'm writing a paper My dataset are time-series sensor data and anomaly ratio is between 5% and 6% 1. For time-series anomaly detection evaluation, which one ...
1
vote
1answer
14 views

Label data set for sentiment analysis

I am a beginner in this field. I have a scrapped review data set. It contains review socre (1 - 10) and review content. I am going to label the reviews according to the review score like below : 0-2 -&...
0
votes
0answers
19 views

How to generate images from numerical data [closed]

I'm a beginner at GAN's and have to perform a task for my job. I am basically required to generate images from a set of input variables. I have a dataset with a few images that correlate to the input ...
0
votes
0answers
18 views

Any recomended Neural Network structure to classify similar images?

I have a number of very similar images, and I want to classify them into two groups. I used Resnet50 before, but the network does not seem to be learning at all. As the training progresses, the ...
0
votes
1answer
20 views

How to handle “year” variable for classification

I am writing to you because I need to create a model that tells me whether or not a company will pay its taxes the next month. For that I have data from 2017 to 2020, with characteristics such as size ...
0
votes
0answers
14 views

When training a cGAN on a X (B/W image) to Y (RGB image) “paired” dataset, how much will slight differences in the images matter in terms of results?

Right now I am working on a colorization project with GANs, and had originally settled on using a CycleGAN because I considered structuring my dataset in an unpaired manner. I've since been able to ...
0
votes
0answers
15 views

Top 5 useful Data Science skills [closed]

I am a beginner in Data Science, exploring and learning how to operate Data Science stack exchange as well. I would like to know the top 5 useful Data Science skills to acquire in 2021.
0
votes
0answers
12 views

Counter-intuitive weight in logistic regression

I have a question about logistic regression. One of the features I built is a binary feature which produces either 0 or 1. The number of the positive instances with the feature activated is ~110K ...
0
votes
0answers
6 views

Ordering of sklearn's confusion_matrix()

Trying to figure out how to sort integer labels on a 102X102 (oxford102 UK flower dataset) confusion matrix graph that I plot with plot_confusion_matrix from mlxtend. I see that basically conf_mat and ...
1
vote
0answers
11 views

Is it possible for class activation maps where the predicted class has no activations?

I'm trying out grad CAM from the captum library in pytorch for a 3 class image classifier that uses transfer learning on resnet50. The model has good accuracy, but when I was looking at the generated ...
1
vote
0answers
9 views

How does Keras optimization for a network with multiple outputs

I currently have a neural network that takes in 3 numbers as inputs and outputs 3 numbers. I've attached a picture of the network below and my code is accessible through the following link: [Google ...
1
vote
2answers
34 views

How do I decide on my X and Y variables for the prediction of a coin toss?

So I'm new to data science and was trying to solve a few problems that my mentor gave to me. I came across this question where there are multiple coin tosses and ten of them are recorded. I am ...
0
votes
0answers
18 views

Low accuracy of trained model when dataset contains 10^5 observations and 128 features using “Statistic and machine learning toolbox” of Matlab

The entrys are real numbers (each row is a channel output, where blocklength n= 128). The accuracy is approx. 50%. The labels contain elements of the set {0,1}. The dataset is balanced. For smaller ...

15 30 50 per page
1
2 3 4 5
587