All Questions

Filter by
Sorted by
Tagged with
0
votes
0answers
15 views

Constrained ANN

In my model, I use a twin network (Siamese) for which the training datasets are called positive class (u__{i,pos}) and negative class (u__{i,neg}). The anchor (u__{anchor}) is a fixed training ...
3
votes
1answer
72 views

Distributed training with low level Tensorflow API

I am using low level Tensorflow API's for my model training. When I say low level it means I'm defining the tf.Session() object of the graph and evaluate graph with ...
6
votes
3answers
4k views

Isolation forest sklearn contamination param

I am working on an unsupervised anomaly detection task on time series data using an isolation forest algorithm. I am developing it in Python, more in detail using ...
5
votes
0answers
659 views

XGBoost custom objective for regression in R

I implemented a custom objective and metric for a xgboost regression. In order to see if I'm doing this correctly, I started with a quadratic loss. The ...
2
votes
1answer
41 views

Transferring the hidden state of a RNN to another RNN

I am using Reinforcement Learning to teach an AI an Austrian Card Game with imperfect information called Schnapsen. For different states of the game, I have different neural networks (which use ...
1
vote
1answer
317 views

Predict the probability that a customer buy today

My company sells a single product that is a commodity. If you buy it, I can be sure that you will buy it again in the near future from me or from my competitors. The demand is affected by the weather. ...
1
vote
1answer
16 views

spot/stain growth in image classification problems

I am working on a problem with images where we are monitoring development of spot in certain region of image. We are able to classify spot present(NOK) or not present(OK) successfully if initially ...
0
votes
0answers
6 views

each class distribution in a classification problem

Can we tell that the distribution of each class differs and the distribution of dataset is the combination of the classes distribution? What can we infer about the class distribution?
0
votes
2answers
33 views

How to calculate steady/incremental growth?

I have timeseries data for stocks at minute intervals. What is the best way to calculate incremental growth, for example if I have a stock's price from 9am to 2pm at minute intervals, how can calcule ...
0
votes
0answers
4 views

KL divergence as loss for classification

I want to use KL-divergence as a loss function for classification problems. For example, consider the dog and cat image classification problem where I designed a CNN model with Softmax as output layer ...
1
vote
1answer
47 views

Does it make sense to build a ROC for a decision tree where there are multiple threshold you can adjust?

I understand building a ROC curve when the output is a probability, say, from a logistic regression model. You can build a ROC curve by varying the cutoff threshold. But what about decision trees of ...
0
votes
0answers
6 views

Pytorch different result when using `torch.matmul` and `for-loop` to pass input through linear layers

I have been at this for about two days now. I am working on a model that takes on an input x and passes it through several linear layers, and concatenates the ...
0
votes
1answer
54 views

Handling Imbalanced Datasets in Orange

I work in the medical domain, so class imbalance is the rule and not the exception. While I know Python has packages for class imbalance, I don't see an option in Orange for e.g. a SMOTE widget. I ...
1
vote
1answer
31 views

Making an ensemble model for high F1 score

I presently have 2 algorithms that have a numerical output. Using a threshold of 0.9, I get the classification output. Let's say they are: P (high precision, low recall) R (high recall, low precision)...
2
votes
2answers
1k views

pandas data frame doesn't show any thing ,when view as data frame in pycharm

import pandas as pd; dataSet = pd.read_csv("winequality-red.csv"); dataSet.describe(include = 'all'); When view data set as data frame ,it show empty table.But ...
0
votes
0answers
4 views

How to revert np.log(data) and data.diff()?

I have used np.log(data) and then applied data.diff() to transform my data in timeseries model. I have the predictions. How do I ...
1
vote
2answers
69 views

Training Object Detection model on just 10 images

I am trying to train an object detection model using Mask-RCNN with Resnet50 as backbone. I am using the pre-trained models from PyTorch's Torchvision library. I have only 10 images that I can use to ...
2
votes
1answer
34 views

How to create a multi label classification network in Keras if I have the training data with various accuracy?

I'm trying to create a neural network that finds the most effective treatment for each patient. I have a medical database for training. The inputs are histological and pathological data (mostly 0/1 ...
0
votes
1answer
759 views

Low memory error while performing degree 2 polynomial regression on (3000*1835) sized array

I am working on a problem to predict the revenue, a film will generate. Some of the features available in the data set are json collection for the crew, cast which worked in the film. I applied ...
1
vote
0answers
5 views

Why does a linear regression in time series forecasting does not provide a line in predictions?

I'm reading the TensorFlow Time Series forecasting Tutorial 1 trying to perform my own time series prediction. However, specifically on single-shot models section for multiple time steps, the Linear ...
97
votes
9answers
124k views

When should I use Gini Impurity as opposed to Information Gain (Entropy)?

Can someone practically explain the rationale behind Gini impurity vs Information gain (based on Entropy)? Which metric is better to use in different scenarios while using decision trees?
0
votes
1answer
11 views

Increase accuracy through "overfitting" multiple models?

I am currently trying to create a model to classify 5 specific classes from the coco dataset. I am using the object detection app from tensorflow. My question is: Will it be better if i: -Train one ...
1
vote
1answer
2k views

Error with pandas dataframe (needs to be 1-dimensional)

I am trying to determine the conformal predictions for my model with my data. But it gives me following error that occurs at icp.calibrate(X_cal, y_cal) : ...
0
votes
2answers
25 views

How to implement ID3

I'm trying to follow the suggested outline form implementing ID3 ...
0
votes
0answers
9 views

Relationship of cost function to regularisation

lets say my cost function C(w)=DataLos(w)+1.Regt (w). Assume that the global minimum of C(w) when , 1 is 23. Now we change à to be equal to 10 and again minimize C(w). Does my optimal cost function ...
0
votes
0answers
6 views

BERT vs GPT architectural, conceptual and implemetational differences

In the BERT paper, I learnt that BERT is encoder-only model, that is it involves only transformer encoder blocks. In the GPT paper, I learnt that GPT is decoder-only model, that is it involves only ...
0
votes
1answer
8 views

error: (-215:Assertion failed)

I was testing OpenCV with their code provided within the documentation. Unfortunately, I'm facing an unexpected error. I think, it might be due to the newer version that I've. I've tried the fix of ...
0
votes
1answer
86 views

How to plot a table with multiple columns as a box plot

I am trying to plot a box plot with the Trinucleotide as the x axis (so 64 trinucleotides on the x axis) and the frequency of each trinucleotide in each of 6 samples then color code the plot according ...
0
votes
1answer
81 views

Naive bayes expectation maximization vs logistic regression for binary classification

Assuming I'm dealing with binary classification. For what kind of data Naive bayes using expectation maximization would give a better solution and for what kind of data logistic regression would be ...
2
votes
2answers
1k views

Why is performance worse when my time-series data is not shuffled prior to a train/test split vs. when it is shuffled prior to the split?

We are running RandomForest model on a time-series data. The model is run in real time and is refit every time a new row is added. Since it is a timeseries data, we set shuffle to false while ...
0
votes
0answers
7 views

I am not sure whether I am being asked to calculate the variance or the irreducible error

The question: Suppose we randomly sample a training set D from some unknown distribution. For each training set D we sample, we train a regression model to predict $y$ from $x$. We repeat this 10 ...
1
vote
1answer
825 views

Generate a balanced batch with ImageDataGenerator() and flow_from_directory()

Hi I am new to python and deep learning. I am doing a multiclass classification. My 3-classes dataset is imbalanced, the classes take about 50%, 40%, and 20%. I am trying to generate mini batches with ...
0
votes
0answers
6 views

How to extract rules for decision tree from ID3 classification

I am working on a program to implement the ID3 algorithm. The program takes in user input for setting a threshold and creating a decision tree. ...
9
votes
2answers
11k views

How can I find out what class each of the columns in the probabilities output correspond to using Keras for a multi-class classification problem?

I'm using transfer learning to build an image recognition model using a pre-trained VGG network in Keras and excluding the final fully-connected layer to get the output weights. I'm then using these ...
0
votes
1answer
40 views

Is it possible to build a regression model for predicting movie gross using sections on their wikipedia pages?

I got this as an assignment from a company recruiter and I've successfully scraped a dataset of about 650 movies with their 'Plot', 'Music' and 'Marketing' sections and gross. I've tried tfidf and ...
0
votes
0answers
8 views

Siamese netwroks - how to choose loss function?

I have read several articles about siamese netwroks, and I understand that there are 3 different types of loss functions: ...
2
votes
1answer
82 views

what does the standard deviation plot around my learning curve indicate?

I plotted a learning curve below. There is a thick red band around the top portion of my training score. Why is it so high at the beginning? Below is a snippet of the code used: ...
0
votes
1answer
14 views

K-Fold cross validating with random forest - how to correctly fit model to every fold?

So I have created K-Folds from my data using this code: ...
0
votes
1answer
12 views

How to preprocess heavy MRI images?

I have a large MRI dataset for an image segmentation task that cannot directly fit in memory in Colab, you can access the data with the link I put at the end. They are brain MRI images: 484 training ...
1
vote
0answers
5 views

Measuring the distance between data points based on mutual linkages

How to measure the distance between two data points (or: nodes?) based on their mutual share of linkages? I don't know the technical term for that, so here is a fictitious example from scientific ...
0
votes
1answer
11 views

Correct Machine Learning approach for prediction using multiple timelines

I am wondering what would be the correct ML approach in order to predict the upcoming value of a time serie based on the previous behaviours of various time series for the same period. To explain ...
1
vote
1answer
41 views

Find feature categories associated with a specific target category

I have a Dataset with three columns. Products (up to 200). The quality checks that have not been conducted at the final quality check. (Up to 70 different Quality Control Measures) The result of the ...
0
votes
0answers
12 views

What should I visualize for humor detection model to gain some useful insight?

I was going through bunch (1,2,3) of humor detection paper. But most papers don't include any visualizations, say some graph related to model being trained. I was thinking to train some language ...
1
vote
1answer
26 views

Points to remember when embarking on an organization-wide turn to AI solutions

In our organization, we are currently in the phase of building up team, skills to automate and implement AI based solutions. So, we are very early in this AI journey. Right now, we are also working on ...
0
votes
2answers
33 views

XgBoost given targets its only feature but fails when test targets are outside the range of training targets?

I'm learning to use XgBoost, and I'm doing an exercise involving predicting prices. However I'm noticing some weird behavior where XgBoost's predictions deviate from the target value even if I'm ...
1
vote
1answer
45 views

Anomaly detection using LSTM AutoEncoder

Having a sequence of 10 days of sensors events, and a true / false label, specifying if the sensor triggered an alert within the 10 days duration: sensor_id timestamp feature_1 feature_2 ...
0
votes
0answers
7 views

Expected input for time-series transformers

I am trying to create a model starting from the Attention is all you need paper. Specifically I want to setup the Encoder/Decoder architecture to predict time-series. I would like to implement it in ...
0
votes
2answers
32 views

What are the “training error” and “test error” used in deep learning papers?

I have heard of the terms "training" and "test error" in the context of classification quite often, but I am not sure I know what they mean. This article writes: Training Error: ...
2
votes
1answer
488 views

Overfitting in K-means

How do you test your results for overfitting in a k-means run? Some people have said use a training set. I have about 1500 records and about 20 fields.

15 30 50 per page
1
2 3 4 5
630