All Questions

33,341 questions
Filter by
Sorted by
Tagged with
2 views

How to perform crossover between models of different sizes in deep genetic algorithms?

I'm working building a genetic algorithm that will learn to play snake. I've worked out how to add/remove layers and neurons in the model, allowing the model's size to change through mutation. But ...
2 views

Do cost-complexity pruned trees perform better relative to unpruned trees?

I came across an adapted question from the famous ISLR book and realise I am unsure of the answer. Does anyone know? Interested in the intuition here! Cost-complexity pruned trees with $\alpha=1$ ...
3 views

Can I "fit" a k-nearest neighbors classifier without precomputing anything?

I am currently trying to fit a KNeighborsClassifier (scikit-learn implementation) to about a gigabyte of training data. From every resource I've read online, a k-nearest-neighbors classifier is a &...
3 views

Consider the following scenario: Suppose two lists of words $L_{1}$ and $L_{2}$ are given. $L_{1}$ contains just bad-written phrases (like 'age' instead of '4ge' or 'blwe' instead of 'blue' etc.). On ...
1 vote
3 views

In which way GAN generator transforms the data(for transforming a noise to the data)?

I have the problem: I understood how GAN works in general, but I need information how it work detailed. The part I don't understand is how the random noise at input is transformed to data on the ...
• 11
2 views

Optimally sample from multiple distributions

I have two datasets both of the form from the table below. I am interested in downselecting from dataset A by sampling from the distribution of values from dataset B. However, I want to consider both ...
2 views

Forecasting future point with partial future data already available

Working on a forecast model that should output an End of monthly value, the interesting part is that we already have partial (90%) of that data available at the prediction point (max 30 days away). ...
6 views

...
7 views

Sampling Highly Imbalanced Large Dataset

I am working on a model which will run monthly on 8M users. I've snapshot-wise data in training set, eg: Jan, 21 Snapshot : 8M Total : 233 Positives Rest Negative Feb, 21 Snapshot : 8M Total : 599 ...
1 vote
7 views

sklearn - StandardScaler - Use in Production

I transformed my input data using StandardScaler as given here: https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html Code looks like this: ...
• 111
3 views

I want instructions to transform angled image of a document to vertically straight format

To perform an OCR task, I want to transform angled image into a proper rectangle shape. As an example, Input: Expected output:
6 views

Predicting a signal based on other signals

I want to predict a signal based on other related signals, how would I go about doing this? My current approach is to do some feature extraction (in the time and frequency domain) on both the ground ...
22 views

...
6 views

data ...
4 views

Store preprocessing function along with model in mlflow.keras

The following is a simplified code snipet that is relevant to storing keras LSTM models in MLFlow. ...
• 79
9 views

Are spiking neural networks considered a low hanging fruit in machine learning research

Does anyone have experience with spiking neural networks? My question is would you consider this a field (or any specifics in this field) as low-hanging fruit to do research in?
11 views

NNs for fitting highly oscillatory functions

in a scientific computing application of neural networks, I have to maximize several neural networks with scalar output with respect to a target/loss function (coming from a weak form of a PDE). It is ...
• 1
1 vote
11 views

Combining two separate confusion matrix results from two seperate machine learning model to overall increase the True Positive accuracy

What are the steps involved if it is possible to add two confusion matrix results together to get a better final prediction. we have calculated two confusion matrixs as follows from naive bayes and ...
6 views

How is the input given to the NeuMF architecture?

I was going through this neural recommendation paper (Fig. 2). I want to implement it from scratch in Tensorflow. The thing I don't understand is how is the input given to this architecture. Can ...
1 vote
15 views

How do I design a random forest split with a "not sure" category?

Let's say I have data with two target labels, A and B. I want to design a random forest that has three outputs: A, B and Not sure. Items in the Not sure category would be a mix of A and B that would ...
• 247
6 views

Why calculating how much removed sentences with most contributing words to the result helps to show that a model is "*faithful*"?

I don't understand how the calculation score taking out the sentences where the words contribute the most of to the result helps to show to what extent a model is "faithful" to a reasoning ...
8 views

Multi task cpu might be faster than gpu ? for classification using deep neural network

Multi core operation on CPU might be faster than GPU? For classification using deep neural networks. I focus on inference process here.
13 views

For feature selection, do we use Chi-squared with Mutual Information together?

Or do we only choose one out of two for categorical data.
• 1
1 vote
9 views

Is there anyway to classify the category on give amazon reviews using python

I am trying to find a model or way to classify text which falls into a category and its a positive or negative feedback. For ex. we have three columns Review : Camera's not good battery backup is not ...
1 vote
14 views

Whether to use LDA or QDA

I'm trying to determine whether it's best to use linear or quadratic discriminant analysis for an analysis that I'm working on. It's my understanding that one of the motivations for using QDA over LDA ...
• 11
7 views

Exact Shap calculations for logistic regression?

Given the relatively simple form of the model of standard logistic regression. I was wondering if there is an exact calculation of shap values for logistic regressions. To be clear I am looking for a ...
• 2,107
14 views

How to train ML model for time series data

I am trying to build a machine learning model in python. I used pytorch and sklearn to make the model. My model is a bit ...
• 101
1 vote
18 views

Regularizing the intercept

I am reading The Elements of Statistical Learning and regarding regularized logistic regression it says: "As with the lasso, we typically do not penalize the intercept term" and I am ...
1 vote
13 views

How can I use a confusion matrix in image captioning?

I read that a confusion matrix is used with image classification but if I need to draw it with image captioning how to use it or ...
• 45
11 views

VGG-16 Model getting 11% accuracy classifying MNIST, HELP!

I have built a VGG-16 model which looks correct to me but I am only getting 11% accuracy classifying the 10 digits (10% of getting it correct by chance). Can someone please help me!! ...
15 views

Finding data with transformation applied

Is there a way to find relatedness between data and the data obtained after some transformation applied to it? i.e. given a data I need to find the most related data(most of the values in that data ...
• 1
5 views

How to read a table from MySQL work bench in colab

I have referred to various articles on Stack Overflow and external sources but somehow am unable to get answer for this. I would like to read a table from MySQL workbench database into a dataframe in ...
• 26
17 views

How can deep learning be applied to association rule mining?

Association rule mining is considered to be an old technique of AI. Rules are mined on statistical support. How can deep learning be applied to this? What are approaches for structured data (in a ...
• 439
21 views

Using Sci-Kit Learn Clustering and/or Random-Forest Classification on String Data with Multiple Sub-Classifications

I have a set of data with some numerical features and some string data. The string data is essentially a set of classes that are not inherently related. For example: ...
24 views

Why Do a Set of 3 Clusters Across 1 Dimension and a Set of 3 Clusters Across 2 Dimensions Form 9 Apparent Clusters in 3 Dimensions?

I am sorry if this is a well-known phenomenon but I can't quite wrap my head around this. I have a related question: How To Develop Cluster Models Where the Clusters Occur Along Subsets of Dimensions ...
17 views

What is the purpose of Sequence Length parameter in RNN (specifically on PyTorch)?

I am trying to understand RNN. I got a good sense of how it works on theory. But then on PyTorch you have two extra dimensions to your input data: batch size (number of batches) and sequence length. ...
11 views

Slice NumPy arrays differently along axes (without looping)

I am trying to analyze a temporal signal sampled by a 2D sensor. Effectively, this means integrating the signal values for each sensor pixel (array row/column coordinate) at the times each pixel is ...
4 views

I need to plot only training curve in the fastai library using the learner.recorder.plot_losses() function . FASTAI devs pls help

I have a task where I need to only plot the training loss and not the validation loss of the plot_losses function in the fastai library with learner object having ...
5 views

How to disentangle non-mutually exclusive items coded in same question?

I have to work with a dataset where people ostensibly had the option to check several options to a question (eg "check all that apply"). But in the data, all of the options when selected are ...
• 1
1 vote
11 views

Contextual word embeddings from pretrained word2vec vectors

I would like to create word embeddings that take context into account, so the vector of the word Jaguar [animal] would be different from the word Jaguar [car brand]. As you know, word2vec only gives ...
• 11
1 vote
11 views

Dealing with little available data: transfer learning

Suppose I seek to predict a certain numerical value, whereby the data set which contains the predetermined correct labels is only very small. However, I'm also provided a large data set with a label ...
• 11
1 vote
44 views

How can i deal with this overfitting?

I trained my data over 40 epochs but got finally this shape. How can I deal with this problem? Please as I used 30.000 for training and 5000 for testing and ...
• 45
10 views

Is it possible to "fine-tune" a pre-trained logistic regression model?

Fine tuning is a concept commonly used in deep learning. We may have a pre-trained model and then fine-tune it to our specific task. Does that apply to simple models, such as logistic regression? For ...
• 101
22 views

Could a quadratic function be empolyed instead of a linear one, for piecewise approximation to learn indexes?

It has been evaluated the use of learned piecewise segments in order to create compressed indexes that substitute classical B+-Tree structures, in order to optimize space and have higher query ...
13 views

Can we use an independent t-test as a metric for feature importance?

I have a supervised binary classification problem. I tuned an xgboost model on the training set and achieved a reasonably high accuracy on the test set. Now I want to interpret the results of the ...
• 165
8 views

'Solvers' in Machine Learning

What role do 'Solvers' play in optimization problems? Surprisingly, I could not find any definition for 'Solvers' online. All the sources I've referred to just explain the types of solvers & the ...
• 313
1 vote
36 views

Does it make sense to use target encoding together with tree-based models?

I'm working on a regression problem with a few high-cardinality categorical features (Forecasting different items with a single model). Someone suggested to use target-encoding (mean/median of the ...
• 11
17 views

How does ExtraTrees (Extremely Randomized Trees) learn?

I'm trying to understand the difference between random forests and extremely randomized trees (https://orbi.uliege.be/bitstream/2268/9357/1/geurts-mlj-advance.pdf) I understand that extratrees uses ...
• 247
9 views

List of main statistics models

I am not able to find some list of main statistics models. Is is possible to devide statistics models into categories as supervised (regression,classification) x unsupervised (clustering) or is it ...