# Is it appropriate to use polar images as input to CNNs? Or must they be Cartesian transformed first?

I have built a convolutional neural net which trains on data originally represented in polar space (measurements are a function of angle and distance). My pipeline begins by converting coordinates to a Cartesian grid and re-projecting the images. This however results in a significant increase in the data dimensions (i.e. pixel size of the input images) and requires interpolation steps which I would prefer to avoid.

Is there an appropriate method (implemented in keras/tensorflow) I can use for performing (2D) convolutions on the raw polar data? I have followed the cs231n course, so I have some background but am not yet an expert, especially on the theory side. Thanks!

• What format are your polar coordinates? Are they just numpy arrays? What shape do they have? – n1k31t4 Aug 15 '18 at 15:57
• Each scan originates as a 30x200 matrix. I stack them into a 3D numpy.ndarray with shape for example (2000,30,200) before the rest of my pre-processing. The rows represent azimuth angles and columns are radial distances. So the data format is rectangular but the actual volume is not. – Elliot Simon Aug 15 '18 at 18:44
• You need to construct a loss function using metrics appropriate for your coordinate system. Could be polar coordinates for 2D data or spherical coordinates for #d data. – 42- Nov 20 '18 at 18:01

At the basic level, CNNs work by finding spatially-linked correlations i.e. places in the input which often appear close together. For this reason, creating cartesian projections of your data from the polar information sounds like the natural way to go about it.

If there is some inherent structure to your raw data, it might be possible to use something like a CNN, but you'd have to think carefully about the architecture.

Here are a couple of other thoughts that cross my mind that might help you brainstorm or find related research papers:

• Casting the polar data into images, as you describe your method, not only increases the dimensions of the data, but usually the sparsity of the data. This makes it very difficult to train something like a CNN, which generally works best for dense chunks of information, such as a photograph. If your data is sparse, you might consider some other pre-processing such as adding blur to the projections.

• Polar coordinates with a distance metric is essentially the same as a point cloud. If you represent a series as scans, instead of a set of images, but rather as a 3d pointcloud, you could look into models such a PointNet, PointNet++, VoxelNet (see the example projects at the bottom of that linked webpage). There are examples of object detection or segmentation in 3d pointclouds, which might give you other ideas for your case - all available openly in Tensorflow or another DL framework.

• How well do you understand your data? Perhaps some further exploration or visualisation might help spark some ideas or at least provide a better feeling for approaches that could work. Try plotting the cartesian coordinates of your data (if you can map many frames to a global origin), using something like PyntCloud (an example).

• Adjacency (or it's converse: loss) should be possible to compute in circular or spherical coordinates. It won't be sqrt((x1-x2)^2+(y1-y2)^2) , but rather sqrt( r1^2+r2^2 -2R1*r2*cos(theta2-theta1) ). – 42- Nov 20 '18 at 18:06

You can perfectly use polar data with 2D convolution with no need to cartesian conversion (I don't mean it will work better)

• Make a grid (image) in which rows represent elevation angle, and columns represent azimuth
• Then, with two of the following three parameters you get the third one: angle limits, angle resolution, number of pixels
• Map your polar coordinate $$p(\phi,\theta)$$ to a pixel coordinate $$(h,w)$$

Example:

• elevation ($$\phi$$) limits = [-45,45]º
• azimuht_limits ($$\theta$$) = [0,180]º
• elevation_resolution $$\Delta\phi$$ = 1º
• azimuth resolution $$\Delta\theta$$ = 0.5º

thus,

• Number of rows H = 90/1 = 90 pixels
• Number of columns W = 180/0.5 = 360 pixels

You end up with an image of 90x360 pixels.

Then you need a mapping function that maps $$p(\phi,\theta)$$ polar coordinates to $$(h,w)$$ pixel position, the function is given by:

$$\newcommand{\floor}{\left\lfloor #1 \right\rfloor} h = \floor{\frac{\phi}{\Delta\phi}} ,$$

$$w = \floor{\frac{\theta}{\Delta\theta}}$$

Let's take in our example,

$$p(\phi,\theta) = (32.1º, 125.2º) \rightarrow (h,w) = 32, 250$$