New answers tagged classification
0
votes
Accepted
Binary Classification [Text] based on Embedding Distance?
The are two levels to your question:
Conceptual - Yes, you can perform an approximate nearest neighbor search on text documents that have been embedded. What you call binary classification is more ...
- 18.8k
1
vote
How to calculate accuracy of a logistic regression?
You need a threshold value $t$ to assign a class based on the probability, so that if $p < t$ you assign it to class 0, and if $p >= t$ you assign it to class 1. Then, you can compute the ...
- 17.7k
0
votes
Classification of sequential data
The definition of your problem is univariate timeseries classification. Univariate because there is only one variable evolving through a time-axis, whereas multivariate is the case when you have more-...
- 554
0
votes
What method should I use to see whether my categories are well-encoded by my variable?
Why don't you split your data into train, validation and test sets and then train a model to predict the category?
The model could be an LSTM or a transformer encoder. It would receive the words and, ...
- 17.7k
1
vote
Accepted
How to present a statistical justification for the choice of models with approximate accuracies?
A widely accepted approach for your problem is using the Critical Difference Diagram. At this time, the original paper has 12274 citations (and counting).
Implementations:
Python;
Julia;
R.
- 554
1
vote
Probability distribution of probabilities
The landscape of the output probabilities depends entirely on the training data. If the data itself is sampled from a normal distribution, then the learned probabilities will reflect that. Otherwise, ...
- 17.7k
0
votes
Classify E-commerce URLs into predefined classes
There is not enough information for a comprehensive answer, but I will try to assist you. You can try to hard code it without using any machine learning algorithms. Try looking at specific parts, ...
0
votes
Can a recommendation system be used as a binary classifier?
Please do not build a recommendation system in order to solve a binary classification problem. If you like the idea with the ratings, you can always create a new feature that is the ranking. Consider ...
1
vote
Accepted
How to perform a classification experiment with data augmentation?
Data augmentation must be done after splitting your data into training and testing sets to avoid data leakage.
- 17.7k
1
vote
How to increase retention?
You can get inspiration from reading the following papers: 1, 2, 3, 4, 5
In any case, I recommend you investigate more on your data. In particular, you should be able to tell a story before employing ...
- 554
2
votes
How to automatically classify a sentence or text based on its context?
If you don't have training data, "classification" may not be the best name. But it could be if you consider this task as like in zero-shot learning. There are some pretrained models in ...
- 280
0
votes
Applying the model on validation data achieves higher performance than on test set. Is this possible?
Yes, it is unlikely, but it is possible. If we assume that you have no mistakes in the code, this simply means that the unseen data is easier for the model to classify compared to the test data. This ...
2
votes
How to automatically classify a sentence or text based on its context?
Extending Solomon Ucko's comment, I would propose trying to label a few thousand examples and fine tune one of the available large language models, for example, the Google's T5 or BERT using your ...
- 840
0
votes
Is it possible to reverse the layers of a convolutional neural network?
Neural networks in general are not bijective. This means that inputs and outputs do not map onto each other in a 1-to-1 fashion and you cannot simply reverse an output to obtain the input it was ...
- 159
12
votes
How to automatically classify a sentence or text based on its context?
Me: Please give 2 semantic tags for the sentence "The area of a circle is pi time the radius squared"
ChatGPT: 1. Mathematics. 2. Geometry
I'm not sure it's a robust and scalable solution ...
- 331
9
votes
Accepted
How to automatically classify a sentence or text based on its context?
To my knowledge, there is no such library or pre-trained model.
Imho there is an important issue in the task as defined in the question, more exactly in the example: these tags seem natural for a ...
- 24.4k
0
votes
Combining results from classifiers trained on different test/train splits results in higher accuracy
A number of observations as per your use case:
Please use a k-fold validation scheme, for more reliable numbers of accuracy.
The model or data need some more hyper-parameter tuning since ...
0
votes
Using human created small groups to identify entirely new small groups
Welcome to the Data Science World.
Note: I am not 100% sure about your project and details are not known. So I will only point you in some directions.
As I understand your question, your problem is to ...
- 146
0
votes
Best way to compare classification output between different locations
Reducing 0.25 to something like 0.15 would help you find the similar predictions. Similar prediction arrays have high correlation. If you can get a set of arrays with correlation just above 0.15 (for ...
2
votes
What can be done about mislabeled data points in the training set of a binary classification model?
A good method for identifying mislabeled data is Confident Learning. It can use predictions from any trained classifier to automatically identify which data is incorrectly labeled. Since Confident ...
0
votes
Dataset with some mislabeled data (around 1%)
In case it's useful, I recently published a paper on this very topic:
Identifying Incorrect Annotations in Multi-Label Classification Data
TLDR: what you are doing is not ideal because your model ...
0
votes
Accepted
In DBSCAN, can the distance between a Noise Point and Border Point be less than Epsilon?
Yes you are correct.
I just gave an example check this image
1
vote
How to use 5 different datasets to re-train & test your ML Classification models multiple times
Your code appears correct for this project. Additionally, the performance of the model appears acceptable for the nature of the project. It is very difficult to predict stock returns based on ...
- 18.8k
Top 50 recent answers are included