I am confused about the Parzen Window question.
Suppose we have two training data points located at 0.5 and 0.7, and we use 0.3 as its rectangle window width. How do we estimate its probability density?
According to the definition, the probability density is $\frac{k/n}{V}$, where $k$ is the number of patterns, $n$ is the total number of points, and $V$ is the window region.
Therefore, is the density for this question $(1/2)/1$?
Then what if we use a triangle instead of a rectangle?
In kernel density estimation, rectangle, triangle, or Gaussian kernels assign weight to positions around query point $x$. For rectangle and square, the weight is steady, for triangle, weight drops linearly with distance, and for Gaussian, weight drops exponentially with distance. Here is an image for square, rectangle, and triangle kernels (one dimensional).
Here is the formula for kernel density estimation in one dimension:
$$\hat{f}_{h}(x) =\frac{1}{nh}\sum_{i=1}^{n}K\left(\frac{x-x_i}{h}\right)=\frac{1}{n}\sum_{i=1}^{n}K_h\left(x-x_i\right)$$
where scaled kernel $K_h$ is defined as: $$K_h(x-x_i) := \frac{1}{h}K(\frac{x-x_i}{h})$$
Example
We have two training points located at $0.5$, and $0.7$ in one dimension.
If we assume query point is $x=0.5$, and $h$ is 0.3 (for rectangle and triangle, the scaled width is 0.6 from -0.3 to 0.3), for rectangle kernel we have: $$\begin{align*} \hat{f}_{0.3}(0.5) &=\frac{1}{2*0.3}\left(rect \left(\frac{0.5-\color{blue}{0.5}}{0.3} \right)+rect \left(\frac{0.5-\color{blue}{0.7}}{0.3} \right)\right) \\ &= \frac{1}{0.6}(0.5+0.5) = 1.67 \end{align*}$$ and for triangle kernel we have: $$\begin{align*} \hat{f}_{0.3}(0.5) &=\frac{1}{2*0.3}\left(tri \left(\frac{0.5-\color{blue}{0.5}}{0.3} \right)+tri \left(\frac{0.5-\color{blue}{0.7}}{0.3} \right)\right) \\ & = \frac{1}{0.6}(1+0.33) = 2.22 \end{align*}$$
For square kernel $sqr$, kernel outputs 1 for each point inside the hyper-cube, and $h$ would be the width of scaled hyper-cube (from $-h/2$ to $h/2$). For the square kernel, estimation in $d$ dimension is: $$\hat{f}_{h}(x) =\frac{1}{nV}\sum_{i=1}^{n}sqr\left(\frac{x-x_i}{h}\right)=\frac{1}{nV}k=\frac{\frac{k}{n}}{V}$$
where $k$ is the number of points inside a hyper-cube centered at $x$ with width $h$, and $V=h^d$ is the size of hyper-cube in $d$ dimensions (length of a segment in one dimension).
The previous example for $h=0.3$ would be: $$\begin{align*} \hat{f}_{0.3}(0.5) &=\frac{1}{2*0.3}\left(sqr \left(\frac{0.5-\color{blue}{0.5}}{0.3} \right)+sqr \left(\frac{0.5-\color{blue}{0.7}}{0.3} \right)\right) \\ & = \frac{1}{0.6}(1+0) = \frac{\frac{1}{2}}{0.3} = 1.67 \end{align*}$$ Note that these are estimations for density not probability, thus, they are allowed to be larger than one (imagine a rectangle with width 0.5 and height 2 as pdf of a continuous random variable).
