> Are flat and non-flat geometry a legit terminology in machine learning
> and statistics?

These are terminologies from Mathematics, they are valid in any field.

> What is the mathematical definition?

### (Non) flat manifold

In mathematics, a (Riemannian) [manifold](https://en.wikipedia.org/wiki/Manifold) is said to be [flat](https://en.wikipedia.org/wiki/Flat_manifold) if its [curvature](https://en.wikipedia.org/wiki/Curvature) is everywhere zero; otherwise non-flat. This is very different than the definition of [flat object in geometry](https://en.wikipedia.org/wiki/Flat_(geometry)). According to that definition, only points, lines, and hyper-planes are flat (not for example hyperspheres or segments).

For example, set $\left\{(t,t):t\in(-1,1)\right\}$ is a 1D flat manifold in ${\Bbb R}^2$, set $\left\{(t,t^2):t\in(-1,1)\right\}$ is a 1D non-flat (positively curved) manifold in ${\Bbb R}^2$, and a hypersphere is an $n$D non-flat (positively curved) manifold in ${\Bbb R}^{n+1}$.

Accordingly, a cluster with a (non) flat shape corresponds to a (non) flat manifold.

Here are some examples from the document.

[![enter image description here][1]][1]

Points are concentrated around (A) two 1D non-flat manifolds (circles) which are non-convex, (B) two 1D non-flat manifolds (arcs) which are non-convex, (C) three 1D flat manifolds (segments) which are convex, (D) three 0D flat manifolds (centers as points) which are convex.

### (Non) flat geometry vs a Flat
 
By "flat geometry" the document means "[Euclidean geometry](https://en.wikipedia.org/wiki/Euclidean_geometry)" not "[a flat object defined in geometry](https://en.wikipedia.org/wiki/Flat_(geometry))". If we measure distances (consequently lengths, areas, volumes, etc.) via Euclidean distance we are obeying the Euclidean geometry, otherwise, we are obeying a non-Euclidean geometry. For example, if we measure a distance between two points by following a non-flat manifold ([a geodesic](https://en.wikipedia.org/wiki/Geodesic)), we are obeying a non-Euclidean geometry. As an illustration,

[![enter image description here][2]][2]

In (A), the red line measures a distance obeying a flat geometry, the blue line measures the distance obeying a non-flat geometry (by moving along the non-flat manifold). If an appropriate map of manifold to a lower dimension is possible (B), obeying a flat geometry would be equivalent to obeying the non-flat geometry before mapping.





  [1]: https://i.sstatic.net/mzN3O.png
  [2]: https://i.sstatic.net/f3HyX.png