> Are flat and non-flat geometry a legit terminology in machine learning
> and statistics?

These are terminologies from Mathematics, they are valid in any field.

> What is the mathematical definition?

In mathematics, a (Riemannian) [manifold](https://en.wikipedia.org/wiki/Manifold) is said to be [flat](https://en.wikipedia.org/wiki/Flat_manifold) if its [curvature](https://en.wikipedia.org/wiki/Curvature) is everywhere zero; otherwise non-flat. This is very different than the definition of [flat in geometry](https://en.wikipedia.org/wiki/Flat_(geometry)) that you have referenced. According to that definition, only points, lines, and hyper-planes are flat.

For example, set $\left\{(t,t):t\in(-1,1)\right\}$ is a 1D flat manifold in ${\Bbb R}^2$, set $\left\{(t,t^2):t\in(-1,1)\right\}$ is a 1D non-flat (positively curved) manifold in ${\Bbb R}^2$, and a hypersphere is an $n$D non-flat (positively curved) manifold in ${\Bbb R}^{n+1}$.

Accordingly, a cluster with a (non) flat shape corresponds to a (non) flat manifold.

Here are some examples from the document.

[![enter image description here][1]][1]

Points are concentrated around (A) two 1D non-flat manifolds (circles) which are non-convex, (B) two 1D non-flat manifolds (arcs) which are non-convex, (C) three 1D flat manifolds (segments) which are convex, (D) three 0D flat manifolds (centers as points) which are convex.


  [1]: https://i.sstatic.net/mzN3O.png