I've read some interpreting "information bottleneck" as the loss of information of X regarding a random variable Y when a compressed random variable T is used.
Could you please explain why the definition uses:
$$ I(X;T) - \beta I(T;Y) $$
and not
$$ I(X;Y) - \beta I(T;Y) $$
To make sure we're understanding the variables correctly, you are already given the two variables $X$ and $Y$ and know they are correlated. You are trying to come up with the optimal compression of $X$ (called $T$) that still has the relevant information about $Y$.
So, $I(X; Y)$ is fixed. It doesn't make sense to include it in your optimization.
Where does the bottleneck come from? You want to maximize the amount of information that gets transmitted, $I(T; Y)$. You could just put $T=X$, but this wastes bits on features of $X$ that might not be relevant. So you also want to maximize $H(X|T)$, which equals $H(X) - I(X; T)$. As $H(X)$ is fixed your overall problem is $$\text{Maximize}\quad I(T; Y),\qquad\text{and minimize}\quad I(X;T).$$ Letting $\beta$ be the tradeoff parameter, you get $$\text{Maximize}\quad\beta I(T; Y) - I(X; T).$$
I'd also recommend reading the original paper, "The Information Bottleneck Method" by Tishby et al.
