Questions tagged [vc-theory]
The vc-theory tag has no usage guidance.
25
questions
26
votes
3
answers
32k
views
What is the exact definition of VC dimension?
I'm studying machine learning from Andrew Ng Stanford lectures and just came across the theory of VC dimensions. According to the lectures and what I understood, the definition of VC dimension can be ...
- 363
16
votes
2
answers
32k
views
How to calculate VC-dimension?
Im studying machine learning, and I would like to know how to calculate VC-dimension.
For example:
$h(x)=\begin{cases} 1 &\mbox{if } a\leq x \leq b \\
0 & \mbox{else } \end{cases} $, with ...
- 173
10
votes
1
answer
10k
views
With regards to VC-dimension, why can you shatter 3 points with circles but not 4 points?
When using VC-dimensions to estimate the capability of a binary classifier, you can find 3 points in R2 that can be shattered, e.g.:
But you can not shatter any 4 points with a circle.
This is ...
- 19.8k
4
votes
1
answer
2k
views
VC dimension of hypothesis space of finite union of intervals
I have the following concept:
$$C = \left\{\bigcup_{i=1}^{k}(a_i, b_i): a_i, b_i \in {\Bbb R}, a_i < b_i, i=1,2,..,k\right\}
$$
and was wondering how to determine the VC dimension of C?
2
votes
1
answer
726
views
Why is a lower bound necessary in proofs of VC-dimensions for various examples of hypotheses?
In the book "Foundations of Machine Learning" there are examples of proving the VC dimensions for various hypotheses, e.g., for axis-aligned rectangles, convex polygons, sine functions, hyperplanes, ...
- 151
2
votes
2
answers
4k
views
Why the VC dimension to this linear hypothesis equal to 3?
I am trying hard to understand this. Here is the scenario:
X = R^2
H = { h(x) = x + 10 }
I need to calculate the VC dimension for the above linear separator. ...
- 591
2
votes
1
answer
83
views
A question on realizable sample complexity
I came across the following exercise, and I just can't seem to crack it:
Let $l$ be some loss function such that $l \leq 1$. Let $H$ be some hypothesis class, and let $A$ be a learning algorithm. ...
2
votes
0
answers
144
views
calculate the VC-dimension [closed]
I have a question about VC-dimension. I have this claim and I need to find out what its VC-dimension is
$ H\subseteq\{0,1\}^n $ collection of Boolean functions over n
In my opinion the answer should ...
- 121
2
votes
0
answers
96
views
VC Dimensions in Machine Learning
Hello I'm learning about VC dimensions in machine learning. The class of classifiers $H$ where $h \in H$ if $h \in \mathbb{R} \rightarrow \{0,1\}$ is what I believe is simply binary classifiers (with ...
- 21
2
votes
0
answers
171
views
Find VC dimension
I'm studying theoretical machine learning at university, and I have this problem in textbook, that I have no Idea how to start.
In space $X=R^2$ are given two models $H_1$ (rectangle with sides ...
- 29
2
votes
0
answers
155
views
Intuition behind Occam's Learner Algorithm using VC-Dimension
So I'm learning about Occam's Learning algorithm and PAC-Learning where for a given hypothesis space $H$, if we want to have a model/hypothesis $h$ that has an True error of $error_D \leq \epsilon$, ...
- 101
2
votes
0
answers
506
views
How to determine the VC Dimension of axis-aligned, origin-centered ellipses?
I've been trying to determine the VC dimension of ellipses which are origin centered and axis aligned. My first approach was to find some equivalence to a threshold classifier function family of the ...
- 21
1
vote
1
answer
203
views
VC-dimension of the infinite set of convex bodies?
I try to find and prove the VC-dimension of the infinite set of (uni-directional) convex bodies. From intuition, it's clear to me that it goes to infinity, but I don't know the correct way to prove it ...
- 113
1
vote
1
answer
36
views
Disproving or proving claim that if VCdim is "n" then it is possible that a set of smaller size is not shattered
Today in the lecture the lecturer said something I found peculiar, and I felt very inconvenient when I heard it:
He claimed, that if the maximal VCdim of some hypothesis class is $n\in\mathbb N$, then ...
- 11
1
vote
1
answer
417
views
VC dimension of half spaces over the real line
I'm studying VC dimension and I'm having a little difficulty understanding it. I read lots of explanations, but when I come across this simple exercise I did not get a good intuition. The problem is ...
- 113
1
vote
0
answers
112
views
1
vote
1
answer
343
views
Calculation of VC dimension of simple neural network
Suppose I have a perceptron with one-hidden layer, with the input - one real number $x \in \mathbb{R}$, and the activation function of the output layers - threshold functions:
$$
\theta(x) =
\begin{...
1
vote
1
answer
185
views
Growth function of a 6-dimensional linear classifier
In our course, we are dealing with a d-dimensional classification problem
($\chi = \mathbb{R}^{d}$ as our input space, and $y = \{-1,+1\}$). Our hypothesis class $H$ consists of all hypotheses of the ...
- 13
0
votes
1
answer
120
views
VC dimension for Gaussian Process Regression [closed]
In neural networks, the VC dimension $d_{VC}$ equals approximately the number of parameters (weights) of the network. The rule of thump for good generalization is then $N \geq 10 d_{VC} \approx 10 * (\...
- 111
0
votes
1
answer
70
views
Vapnik Chervonenkis dimension of a classifier from the Wikipedia page
The Vapnik Chervonenkis dimension is defined by the wikipedia page here for a classification model as:
A classification model $f$ with some parameter vector $\theta$ is said to shatter a set of data ...
- 109
0
votes
1
answer
144
views
VC-dimension of the class of hypotheses that assign label $1$ to exactly $k$ points of some finite domain $\mathcal{X}$
Let $\mathcal{X}$ be a finite domain and $k$ a number such that $k\leq|\mathcal{X}|$. Consider the hypothesis class
$\mathcal{H}:=\big\{h:|\{\mathbf{x}\in\mathcal{X}:h(\mathbf{x})=1\}|=k\bigr\}$;
that ...
- 3
0
votes
0
answers
10
views
How is PCA applied in CV with multiple channels?
I am currently working with a CNN with a physical magnitude dataset. The goal is to downscaling one of them based on the others, but previously I would like to do dimensionality reduction.
If I have ...
- 1
0
votes
1
answer
103
views
VC- dimension calculate
Let X = {1, 2, 3, ... , 100}. Let H be the class of all subsets of X that contain at least 20 and at most 80 elements. What is the VC-dimension of H?
0
votes
0
answers
44
views
VC Dimension of a Countably Infinite Class
I know that there are many examples of classes where the VC Dimension is finite/infinite even though the size of the class is Uncountably Infinite.
However, I could not argue if the VC Dimension of a ...
0
votes
0
answers
84
views
Growth function of the class of all circles in the plane
I know that the VC dimension for this problem is 3. My concern is about the growth function. The following bound is obtained using the VC dimension:
$$m_{\mathcal{H}}(n)\le \sum_{k=0}^3\,{n\choose k}$$...
- 1