All Questions
33,341
questions
0
votes
0
answers
2
views
How to perform crossover between models of different sizes in deep genetic algorithms?
I'm working building a genetic algorithm that will learn to play snake. I've worked out how to add/remove layers and neurons in the model, allowing the model's size to change through mutation. But ...
- 23
0
votes
0
answers
2
views
Do cost-complexity pruned trees perform better relative to unpruned trees?
I came across an adapted question from the famous ISLR book and realise I am unsure of the answer. Does anyone know? Interested in the intuition here!
Cost-complexity pruned trees with $\alpha=1$ ...
0
votes
0
answers
3
views
Can I "fit" a k-nearest neighbors classifier without precomputing anything?
I am currently trying to fit a KNeighborsClassifier (scikit-learn implementation) to about a gigabyte of training data. From every resource I've read online, a k-nearest-neighbors classifier is a &...
- 101
0
votes
0
answers
3
views
Text similarity for bad-written text
Consider the following scenario:
Suppose two lists of words $L_{1}$ and $L_{2}$ are given. $L_{1}$ contains just bad-written phrases (like 'age' instead of '4ge' or 'blwe' instead of 'blue' etc.). On ...
- 113
1
vote
0
answers
3
views
In which way GAN generator transforms the data(for transforming a noise to the data)?
I have the problem: I understood how GAN works in general, but I need information how it work detailed. The part I don't understand is how the random noise at input is transformed to data on the ...
- 11
0
votes
0
answers
2
views
Optimally sample from multiple distributions
I have two datasets both of the form from the table below. I am interested in downselecting from dataset A by sampling from the distribution of values from dataset B. However, I want to consider both ...
0
votes
0
answers
2
views
Forecasting future point with partial future data already available
Working on a forecast model that should output an End of monthly value, the interesting part is that we already have partial (90%) of that data available at the prediction point (max 30 days away).
...
- 1
0
votes
0
answers
6
views
0
votes
0
answers
7
views
Sampling Highly Imbalanced Large Dataset
I am working on a model which will run monthly on 8M users. I've snapshot-wise data in training set, eg:
Jan, 21 Snapshot : 8M Total : 233 Positives Rest Negative
Feb, 21 Snapshot : 8M Total : 599 ...
1
vote
0
answers
7
views
sklearn - StandardScaler - Use in Production
I transformed my input data using StandardScaler as given here:
https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html
Code looks like this:
...
- 111
0
votes
0
answers
3
views
I want instructions to transform angled image of a document to vertically straight format
To perform an OCR task, I want to transform angled image into a proper rectangle shape.
As an example,
Input:
Expected output:
- 1
0
votes
0
answers
6
views
Predicting a signal based on other signals
I want to predict a signal based on other related signals, how would I go about doing this?
My current approach is to do some feature extraction (in the time and frequency domain) on both the ground ...
0
votes
2
answers
22
views
0
votes
0
answers
6
views
0
votes
0
answers
4
views
Store preprocessing function along with model in mlflow.keras
The following is a simplified code snipet that is relevant to storing keras LSTM models in MLFlow.
...
- 79
0
votes
0
answers
9
views
Are spiking neural networks considered a low hanging fruit in machine learning research
Does anyone have experience with spiking neural networks? My question is would you consider this a field (or any specifics in this field) as low-hanging fruit to do research in?
0
votes
2
answers
11
views
NNs for fitting highly oscillatory functions
in a scientific computing application of neural networks, I have to maximize several neural networks with scalar output with respect to a target/loss function (coming from a weak form of a PDE).
It is ...
- 1
1
vote
1
answer
11
views
Combining two separate confusion matrix results from two seperate machine learning model to overall increase the True Positive accuracy
What are the steps involved if it is possible to add two confusion matrix results together to get a better final prediction. we have calculated two confusion matrixs as follows from naive bayes and ...
0
votes
0
answers
6
views
How is the input given to the NeuMF architecture?
I was going through this neural recommendation paper (Fig. 2). I want to implement it from scratch in Tensorflow. The thing I don't understand is how is the input given to this architecture. Can ...
- 23
1
vote
2
answers
15
views
How do I design a random forest split with a "not sure" category?
Let's say I have data with two target labels, A and B.
I want to design a random forest that has three outputs:
A, B and Not sure.
Items in the Not sure category would be a mix of A and B that would ...
- 247
0
votes
0
answers
6
views
Why calculating how much removed sentences with most contributing words to the result helps to show that a model is "*faithful*"?
I don't understand how the calculation score taking out the sentences where the words contribute the most of to the result helps to show to what extent a model is "faithful" to a reasoning ...
0
votes
0
answers
8
views
Multi task cpu might be faster than gpu ? for classification using deep neural network
Multi core operation on CPU might be faster than GPU? For classification using deep neural networks.
I focus on inference process here.
0
votes
1
answer
13
views
For feature selection, do we use Chi-squared with Mutual Information together?
Or do we only choose one out of two for categorical data.
- 1
1
vote
0
answers
9
views
Is there anyway to classify the category on give amazon reviews using python
I am trying to find a model or way to classify text which falls into a category and its a positive or negative feedback.
For ex. we have three columns
Review : Camera's not good battery backup is not ...
- 111
1
vote
0
answers
14
views
Whether to use LDA or QDA
I'm trying to determine whether it's best to use linear or quadratic discriminant analysis for an analysis that I'm working on. It's my understanding that one of the motivations for using QDA over LDA ...
- 11
0
votes
0
answers
7
views
Exact Shap calculations for logistic regression?
Given the relatively simple form of the model of standard logistic regression. I was wondering if there is an exact calculation of shap values for logistic regressions. To be clear I am looking for a ...
- 2,107
0
votes
0
answers
14
views
How to train ML model for time series data
I am trying to build a machine learning model in python. I used pytorch and sklearn to make the model. My model is a bit ...
- 101
1
vote
1
answer
18
views
Regularizing the intercept
I am reading The Elements of Statistical Learning and regarding regularized logistic regression it says:
"As with the lasso, we typically do not penalize the intercept term"
and I am ...
- 13
1
vote
1
answer
13
views
How can I use a confusion matrix in image captioning?
I read that a confusion matrix is used with image classification but if I need to draw it with image captioning how to use it or ...
- 45
0
votes
1
answer
11
views
VGG-16 Model getting 11% accuracy classifying MNIST, HELP!
I have built a VGG-16 model which looks correct to me but I am only getting 11% accuracy classifying the 10 digits (10% of getting it correct by chance). Can someone please help me!!
...
0
votes
0
answers
15
views
Finding data with transformation applied
Is there a way to find relatedness between data and the data obtained after some transformation applied to it? i.e. given a data I need to find the most related data(most of the values in that data ...
- 1
0
votes
0
answers
5
views
How to read a table from MySQL work bench in colab
I have referred to various articles on Stack Overflow and external sources but somehow am unable to get answer for this. I would like to read a table from MySQL workbench database into a dataframe in ...
- 26
0
votes
0
answers
17
views
How can deep learning be applied to association rule mining?
Association rule mining is considered to be an old technique of AI. Rules are mined on statistical support. How can deep learning be applied to this? What are approaches for structured data (in a ...
- 439
0
votes
1
answer
21
views
Using Sci-Kit Learn Clustering and/or Random-Forest Classification on String Data with Multiple Sub-Classifications
I have a set of data with some numerical features and some string data. The string data is essentially a set of classes that are not inherently related. For example:
...
- 3
-2
votes
1
answer
24
views
Why Do a Set of 3 Clusters Across 1 Dimension and a Set of 3 Clusters Across 2 Dimensions Form 9 Apparent Clusters in 3 Dimensions?
I am sorry if this is a well-known phenomenon but I can't quite wrap my head around this. I have a related question: How To Develop Cluster Models Where the Clusters Occur Along Subsets of Dimensions ...
0
votes
1
answer
17
views
What is the purpose of Sequence Length parameter in RNN (specifically on PyTorch)?
I am trying to understand RNN. I got a good sense of how it works on theory. But then on PyTorch you have two extra dimensions to your input data: batch size (number of batches) and sequence length. ...
- 3
0
votes
0
answers
11
views
Slice NumPy arrays differently along axes (without looping)
I am trying to analyze a temporal signal sampled by a 2D sensor. Effectively, this means integrating the signal values for each sensor pixel (array row/column coordinate) at the times each pixel is ...
0
votes
0
answers
4
views
I need to plot only training curve in the fastai library using the learner.recorder.plot_losses() function . FASTAI devs pls help
I have a task where I need to only plot the training loss and not the validation loss of the plot_losses function in the fastai library with learner object having ...
0
votes
0
answers
5
views
How to disentangle non-mutually exclusive items coded in same question?
I have to work with a dataset where people ostensibly had the option to check several options to a question (eg "check all that apply"). But in the data, all of the options when selected are ...
- 1
1
vote
0
answers
11
views
Contextual word embeddings from pretrained word2vec vectors
I would like to create word embeddings that take context into account, so the vector of the word Jaguar [animal] would be different from the word Jaguar [car brand].
As you know, word2vec only gives ...
- 11
1
vote
0
answers
11
views
Dealing with little available data: transfer learning
Suppose I seek to predict a certain numerical value, whereby the data set which contains the predetermined correct labels is only very small. However, I'm also provided a large data set with a label ...
- 11
1
vote
1
answer
44
views
How can i deal with this overfitting?
I trained my data over 40 epochs but got finally this shape. How can I deal with this problem? Please as I used 30.000 for training and 5000 for testing and
...
- 45
0
votes
0
answers
10
views
Is it possible to "fine-tune" a pre-trained logistic regression model?
Fine tuning is a concept commonly used in deep learning. We may have a pre-trained model and then fine-tune it to our specific task.
Does that apply to simple models, such as logistic regression?
For ...
- 101
0
votes
0
answers
22
views
Could a quadratic function be empolyed instead of a linear one, for piecewise approximation to learn indexes?
It has been evaluated the use of learned piecewise segments in order to
create compressed indexes that substitute classical B+-Tree structures,
in order to optimize space and have higher query ...
0
votes
0
answers
13
views
Can we use an independent t-test as a metric for feature importance?
I have a supervised binary classification problem. I tuned an xgboost model on the training set and achieved a reasonably high accuracy on the test set. Now I want to interpret the results of the ...
- 165
0
votes
0
answers
8
views
'Solvers' in Machine Learning
What role do 'Solvers' play in optimization problems? Surprisingly, I could not find any definition for 'Solvers' online. All the sources I've referred to just explain the types of solvers & the ...
- 313
1
vote
0
answers
36
views
Does it make sense to use target encoding together with tree-based models?
I'm working on a regression problem with a few high-cardinality categorical features (Forecasting different items with a single model).
Someone suggested to use target-encoding (mean/median of the ...
- 11
0
votes
1
answer
17
views
How does ExtraTrees (Extremely Randomized Trees) learn?
I'm trying to understand the difference between random forests and extremely randomized trees (https://orbi.uliege.be/bitstream/2268/9357/1/geurts-mlj-advance.pdf)
I understand that extratrees uses ...
- 247
0
votes
0
answers
9
views
List of main statistics models
I am not able to find some list of main statistics models. Is is possible to devide statistics models into categories as supervised (regression,classification) x unsupervised (clustering) or is it ...
0
votes
0
answers
9
views
Accuracy drops when adding a fully connected layer for dimensionality reduction to a ResNet50
I'm training a ResNet50 for image classification and I'm interested in decreasing the dimensionality of the embedded layer, in order to apply some clustering techniques.
The suggested dimension is ...
- 101