5
If you want to move from theory to application then I suggest to do exactly that: get your handy "dirty"!
UCI Machine Learning Repository has some easier datasets to get started. Kaggle is great too but before going for any competition look for an easier dataset from their repository.
If you prefer something with more guidance the book "Introduction to
...
4
Doing Kaggle problems is a good way to test your skills, and it is a good way to improve your skills. There are problems that don't require advanced techniques. For example, Titanic is an introductory problem. Also, solutions for many problems are available. You can do a problem yourself and then check how other people did it.
4
TL;DR
Yes, with overfitting all data becomes (non-linearly) separable (as long as the points don't precisely overlap).
Explanation
The problem with your argument is that you are using circles on a 2D plane, which is very difficult to learn. However, I think your argument can be made stronger with a decision-tree.
(0.2, 3.1)? --> yes -> star
...
3
There cannot be a unique answer to your question. There is a discrepancy in your question though -
I am aware that this is a classification problem on which I am working on.
Could you please help me with the right step by step guide that I should follow in order to achieve an efficient clustering at the end?
However, I am assuming that you are ...
2
Your data set is unbalanced since 28432 out of 28481 examples belong to class 0 (that is 99.8%). Therefore, your predictor almost always predicts any given sample as belonging to class 0 and thereby achieves very high scores like precision and recall for class 0 and very low scores for class 1.
In the case of weighted average the performance metrics are ...
2
Yes, we can use the Discriminator of the GAN to classify images. But we should make sure that the images produced by the Generator are real looking.
If you have trained your GAN on a large number of images and it is performing pretty well on the dataset then I insist you to treat the Discriminator model as a pretrained model ( like we do in transfer ...
2
Accuracy is commonly defined as total nbr of correct predictions / total nbr of predictions.
Imagine a binary classification of color in: White or not White.
Since it's binary, all the white Classifications - a True Positive classification are as important for accuracy as a - True Negative classification, after all, the classification is also correct, hence,...
2
Having consulted my professor, the person that wrote the question from the exercise book featured in the OP, here is their perspective:
Groups of data points can always be separated. The exception is when two points are at the same location.
However, the thing to consider is whether or not your decision boundary can separate unseen data, generated by the ...
1
It depends…
The general rule of thumb is that there should be at least 40 occurrences of an item to train an embedding model to find a robust representation. If most follower IDs repeat then an embedding model can learn which ones co-occur. If follower IDs are sparse then hashing (which randomly assigns numbers) is a better choice.
Which method is better ...
1
This is a standard problem with distance/similarity measures between texts of different length. I'm not aware of any standard way to solve it, but in your case I would simply try to remove any email shorter than a certain length from the training set (you can experiment with different thresholds). This would hopefully force the centroids to be more specific, ...
1
Question 1) Is my calculation for each case above correct?
No, it is not correct. The formula to calculate the spatial dimensions (height and width) of a (square shaped) convolutional layer is
$$O = \frac{I - K + 2P}S + 1$$ with $I$ being the spatial input size, $K$ being the kernel size, $P$ the padding and $S$ the stride. For your two cases that is (...
1
It sounds like a great opportunity for feature engineering. You're contemplating this in your last paragraph so you're on the right track, but I'll elaborate on a possible solution here.
You could use the features that you know will exist in the test set to construct synthetic features based on the context information. For example, you could predict the ...
1
In step 1, it's no need to use MCMC, instead, we can compute the posterior with some assumptions in closed form, it's $N(E(f_*),Var(f_*))$. In the second step, we use approximation method to compute the probs by sampling from the posterior distribution.
1
Improvement in the evaluation metric by rebalancing depends on the type of classifier algorithm. Certain algorithms are very sensitive to rate imbalances, thus adjusting the respective levels of support will change the algorithm's performance.
One way to empirically verify that only under-sampling is driving the improvement is to run an experiment - ...
1
This is actually much more about feature engineering than just finding any representation. Therefore, I'd think through which variables might help your algorithm.
Here are some more ideas for features based on your time ranges which might be helpful:
duration
start time
end time
morning, day time, evening or night activity (categorical, i.e. not for LDA)
...
1
Option 1
Convert range to numerical features. You can create 2 features from it.
range
-------------
7:00 to 14:00
becomes
percentage of day in school | hours in school
----------------------------|----------------
0.29 | 7
Option 2
Convert range to a hot-encoding. You can create 24 features from it.
range
--------...
1
Not knowing your data in detail, my intuition is that you could go for dummy (one hot) encoding. You could split each day in (say) 10 min. intervals $x$ (144 columns) and attach labels (up/down) $y$. Each time interval $x$ would be one dummy encoded column (true/false).
The model would be a binary classification (logistic) like:
$$ y = \beta_0 + \beta_1 ...
1
It would depend on use case that you are addressing. Say in a healthcare where you are using it for medical diagnosis recall might take precedence since false negatives can be quite detrimental there however in the use case you mentioend fraud detection both false positives and false negatives can have dire impact you would look at F1-score
1
When implementing a model from a paper to reproduce their results, it is very important to pay attention to all the details. For this case there are some important differences when comparing to the CIFAR10 results of ResNet:
You are using the Adam optimizer, while the ResNet paper uses SGD with a learning rate schedule. Adam is known to have issues ...
1
Just a couple thoughts:
It looks like these "regimes" could be represented as a latent variable: you could probably design a bayesian model in which the OLS model depends on the value of this latent variable. This means that the model would still be trained only with the observed features, but would internally predict the value of the regime and this value ...
1
In simple terms: Linearly separable = a linear classifier could do the job. You could fit one straight line to correctly classify your data.
Technically, any problem can be broken down to a multitude of small linear decision surfaces; i.e. you approximate a non-linear function with a high number of small linear boundaries. That's what Neural Networks are ...
1
Mutually exclusive classes are usually integers; in your case, win=0, draw=1, loss=2 (for instance).
It is mostly a convention and is convenient for implementation.
1
This is indeed the expected behavior.
(N.B. You should not balance test sets, since they are supposed to inform you about performance on unseen, original-distribution data.
https://stats.stackexchange.com/a/258974/232706
In a binary classification, should the test dataset be balanced?
)
A quick example may be clearer than being general. Let's fix a ...
1
I think what you are dealing with is a type I vs. type II error discussion. It seems that you want to avoid Type II error.
One way to go is to consider $F_\beta$ for your performance metric. It is a modified version of $F_1$. As seen here : https://en.wikipedia.org/wiki/F1_score, $F_\beta$ can be formulated in terms of type I /type II error. You then need ...
Only top voted, non community-wiki answers of a minimum length are eligible
