Questions tagged [vc-theory]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
0 answers
42 views

Occam's factor and the VC dimension

I was watching this lecture by Prof. Dr. Philipp Hennig (Probabilistic ML) and when he reached this formula which is the type two maximum log likelihood I had the following question: The Occam's ...
HAMDI ABDERRAHMENE's user avatar
0 votes
1 answer
316 views

VC-dimension of the class of hypotheses that assign label $1$ to exactly $k$ points of some finite domain $\mathcal{X}$

Let $\mathcal{X}$ be a finite domain and $k$ a number such that $k\leq|\mathcal{X}|$. Consider the hypothesis class $\mathcal{H}:=\big\{h:|\{\mathbf{x}\in\mathcal{X}:h(\mathbf{x})=1\}|=k\bigr\}$; that ...
VK88's user avatar
  • 3
0 votes
1 answer
194 views

VC- dimension calculate

Let X = {1, 2, 3, ... , 100}. Let H be the class of all subsets of X that contain at least 20 and at most 80 elements. What is the VC-dimension of H?
tomkeruse's user avatar
0 votes
0 answers
68 views

VC Dimension of a Countably Infinite Class

I know that there are many examples of classes where the VC Dimension is finite/infinite even though the size of the class is Uncountably Infinite. However, I could not argue if the VC Dimension of a ...
Chen Reddy Sundeep's user avatar
1 vote
0 answers
154 views

Find VC dimension of 1D data

...
Tuhin Dutta's user avatar
2 votes
0 answers
172 views

calculate the VC-dimension [closed]

I have a question about VC-dimension. I have this claim and I need to find out what its VC-dimension is $ H\subseteq\{0,1\}^n $ collection of Boolean functions over n In my opinion the answer should ...
hch's user avatar
  • 121
2 votes
0 answers
115 views

VC Dimensions in Machine Learning

Hello I'm learning about VC dimensions in machine learning. The class of classifiers $H$ where $h \in H$ if $h \in \mathbb{R} \rightarrow \{0,1\}$ is what I believe is simply binary classifiers (with ...
M. Fire's user avatar
  • 21
1 vote
1 answer
245 views

VC-dimension of the infinite set of convex bodies?

I try to find and prove the VC-dimension of the infinite set of (uni-directional) convex bodies. From intuition, it's clear to me that it goes to infinity, but I don't know the correct way to prove it ...
RD7's user avatar
  • 113
2 votes
0 answers
216 views

Find VC dimension

I'm studying theoretical machine learning at university, and I have this problem in textbook, that I have no Idea how to start. In space $X=R^2$ are given two models $H_1$ (rectangle with sides ...
user779537's user avatar
0 votes
1 answer
144 views

VC dimension for Gaussian Process Regression [closed]

In neural networks, the VC dimension $d_{VC}$ equals approximately the number of parameters (weights) of the network. The rule of thump for good generalization is then $N \geq 10 d_{VC} \approx 10 * (\...
kot's user avatar
  • 111
1 vote
1 answer
405 views

Calculation of VC dimension of simple neural network

Suppose I have a perceptron with one-hidden layer, with the input - one real number $x \in \mathbb{R}$, and the activation function of the output layers - threshold functions: $$ \theta(x) = \begin{...
spiridon_the_sun_rotator's user avatar
0 votes
1 answer
99 views

Vapnik Chervonenkis dimension of a classifier from the Wikipedia page

The Vapnik Chervonenkis dimension is defined by the wikipedia page here for a classification model as: A classification model $f$ with some parameter vector $\theta$ is said to shatter a set of data ...
IntegrateThis's user avatar
1 vote
1 answer
37 views

Disproving or proving claim that if VCdim is "n" then it is possible that a set of smaller size is not shattered

Today in the lecture the lecturer said something I found peculiar, and I felt very inconvenient when I heard it: He claimed, that if the maximal VCdim of some hypothesis class is $n\in\mathbb N$, then ...
C.H.'s user avatar
  • 11
0 votes
0 answers
104 views

Growth function of the class of all circles in the plane

I know that the VC dimension for this problem is 3. My concern is about the growth function. The following bound is obtained using the VC dimension: $$m_{\mathcal{H}}(n)\le \sum_{k=0}^3\,{n\choose k}$$...
Ali Kara's user avatar
1 vote
1 answer
215 views

Growth function of a 6-dimensional linear classifier

In our course, we are dealing with a d-dimensional classification problem ($\chi = \mathbb{R}^{d}$ as our input space, and $y = \{-1,+1\}$). Our hypothesis class $H$ consists of all hypotheses of the ...
drommedaris's user avatar
1 vote
1 answer
478 views

VC dimension of half spaces over the real line

I'm studying VC dimension and I'm having a little difficulty understanding it. I read lots of explanations, but when I come across this simple exercise I did not get a good intuition. The problem is ...
Marcelo de Sousa's user avatar
2 votes
1 answer
817 views

Why is a lower bound necessary in proofs of VC-dimensions for various examples of hypotheses?

In the book "Foundations of Machine Learning" there are examples of proving the VC dimensions for various hypotheses, e.g., for axis-aligned rectangles, convex polygons, sine functions, hyperplanes, ...
learning machine's user avatar
4 votes
1 answer
2k views

VC dimension of hypothesis space of finite union of intervals

I have the following concept: $$C = \left\{\bigcup_{i=1}^{k}(a_i, b_i): a_i, b_i \in {\Bbb R}, a_i < b_i, i=1,2,..,k\right\} $$ and was wondering how to determine the VC dimension of C?
Lars Erik Fagernaes's user avatar
2 votes
1 answer
85 views

A question on realizable sample complexity

I came across the following exercise, and I just can't seem to crack it: Let $l$ be some loss function such that $l \leq 1$. Let $H$ be some hypothesis class, and let $A$ be a learning algorithm. ...
Nadav Schweiger's user avatar
2 votes
2 answers
5k views

Why the VC dimension to this linear hypothesis equal to 3?

I am trying hard to understand this. Here is the scenario: X = R^2 H = { h(x) = x + 10 } I need to calculate the VC dimension for the above linear separator. ...
Suhail Gupta's user avatar
2 votes
0 answers
156 views

Intuition behind Occam's Learner Algorithm using VC-Dimension

So I'm learning about Occam's Learning algorithm and PAC-Learning where for a given hypothesis space $H$, if we want to have a model/hypothesis $h$ that has an True error of $error_D \leq \epsilon$, ...
JoeVictor's user avatar
  • 101
27 votes
3 answers
35k views

What is the exact definition of VC dimension?

I'm studying machine learning from Andrew Ng Stanford lectures and just came across the theory of VC dimensions. According to the lectures and what I understood, the definition of VC dimension can be ...
Kaushal28's user avatar
  • 373
2 votes
0 answers
570 views

How to determine the VC Dimension of axis-aligned, origin-centered ellipses?

I've been trying to determine the VC dimension of ellipses which are origin centered and axis aligned. My first approach was to find some equivalence to a threshold classifier function family of the ...
AlexFink's user avatar
10 votes
1 answer
11k views

With regards to VC-dimension, why can you shatter 3 points with circles but not 4 points?

When using VC-dimensions to estimate the capability of a binary classifier, you can find 3 points in R2 that can be shattered, e.g.: But you can not shatter any 4 points with a circle. This is ...
Brian Spiering's user avatar
16 votes
2 answers
35k views

How to calculate VC-dimension?

Im studying machine learning, and I would like to know how to calculate VC-dimension. For example: $h(x)=\begin{cases} 1 &\mbox{if } a\leq x \leq b \\ 0 & \mbox{else } \end{cases} $, with ...
铭声孙's user avatar
  • 173