I am working on neural network models for 3D skeletal character animation, where I learn joint positions and orientations. The problem comes with the orientations. There are several ways I can choose to represent a 3D rotation, but all of them have some form of discontinuity that is making my model unable to produce stable correct rotations in all cases. The alternatives I have considered are:

  • Quaternions. Quaternions (or unit quaternions) are very commonly used for 3D orientation, and are generally fine except a unit quaternion and its opposite represent the same orientation. It is not hard to come up with a loss formula the accounts for this (1 - abs(sum(componentwise_product(q1, q2))) works fine), but the problem is that in some cases the network somehow learns to produce the same quaternions with both signs, and kind of randomly "flips" from one sign to the other at times, with some frames in between, resulting in a sort of "flickering". One possibility would be to take only "half" of the quaternions, e.g. by fixing one component to be positive (then learn by squared differences for example). It doesn't always work, though. For example, suppose I fix the first component to be positive and I have a quaternion like [0.018, 0.743, 0.557, 0.371], and another one like [-0.017, 0.870, 0.348, 0.348]; these are rather similar quaternions, but if I make the first component positive then the second one will be [0.017, -0.870, -0.348, -0.348], which is numerically very different, and the result is that sometimes the network cannot learn the value properly and comes up with some numbers in between (which are incorrect).
  • Rotation vectors. A rotation vector has the direction of the axis of rotation and the size of the angle of the rotation. They are used for example in the original paper of phase-functioned neural networks, but they still have bad cases. Suppose vector sizes go from 0 to π (greater angles would be taken along the opposite axis). If you have a rotation that is just around the value of π, the rotation vector will be "swinging" from one direction to the opposite one, again making it hard for the network to learn it.
  • Euler angles. These are just three rotation angles that are applied along specific axes (X, Y or Z) in specific order. They are generally discouraged because they are not very stable and suffer from the infamous gimbal lock, but geometry aside, they still have the same problems. If my angles go from -180º to 180º (or -π to π in radians), values in the "frontier" will always cause instability.
  • Extensive rotation encodings. One stable way of encoding rotations is giving both sine and cosine values. So I could for example have both the sine and cosine of each Euler angle, or just the nine values of a 3D rotation matrix (which are not sines and cosines but are in a way derived from these, and also a stable representation). However this obviously increases the number of values to learn significantly, and I would be trying to learn independent values that actually have a relationship among them.

I haven't found relevant literature addressing this problem specifically, although maybe I am not searching with the right terms. Has anyone faced this issue before? Or has some idea or alternative that I could consider?

  • 2
    $\begingroup$ Very thorough question, welcome to the site! $\endgroup$
    – kbrose
    Commented Aug 2, 2018 at 13:45
  • $\begingroup$ Have you tried any of your solutions apart from quaternions? $\endgroup$
    – kbrose
    Commented Aug 2, 2018 at 14:20
  • $\begingroup$ @kbrose I have tried quaternions (in several ways) and rotation vectors and have found "bad cases" for both with my data. I didn't actually try Euler angles, but even if that worked with my data ideally I would like a solution that eliminates bad cases in general. I didn't try with a full rotation matrix or similar because of the problems mentioned (although I may at one point), I was hoping to find a more "compact" way to solve it. $\endgroup$
    – javidcf
    Commented Aug 2, 2018 at 14:28

2 Answers 2


Every rotation representation with four or less parameters (e.g. Euler angles, Quaternions, Axis-angle vectors, Exponential coordinates) inevitably has singular points in its map to SO(3) aka the space of 3D rotation matrices. Consider the figure below, here we map features $a$ to quaternions $q$. Quaternions double cover SO(3) that is, $q$ and $-q$ correspond to the same rotation matrix. In turn, large distances in quaternion space can correspond to small distances between two rotation matrices $R_1$ and $R_2$. As a consequence, if your target function has as outputs quaternions then there exist points in feature space where the output may jump through the full width of quaternion space. Learning these discontinuities is difficult.

Quaternion example

If your 3D rotations are small, that is $\|I_3 - R\|_{\mathbb{F}} \leq \sqrt{2}$, then mapping the rotation matrices to quaternion space does not introduce jumps into the target function.


For example, consider the MoCap human skeleton data. Most of the 3D rotations in this data set have a small distance to the "unit rotation" $I_3$. In this case, learning with quaternions in the output of your model is fine.

enter image description here

However, if your 3D rotations are not small, e.g. contain all possible rotations, learning with low-dimensional representations in your model output will reduce the model's data-efficiency. Instead, you need to use representations with more than four parameters. Currently, two commonly used representations of rotation with more than four parameters are:

Bregier (https://arxiv.org/abs/2103.16317) pointed out that the GSO representation is a special case of the Orthogonal-Procustes problem that yields the SVD representation. Implementations of these rotation representation in PyTorch you find here: https://github.com/naver/roma

The above figures are from a detailed guide on learning with 3D rotations that I have been working on: https://arxiv.org/abs/2404.11735

  • $\begingroup$ Thank you for your detailed answer to this old (but still relevant to me) question, and for the link to your publication and the other references. That is a good point about quaternions being fine for small rotations. I think in the model I was considering at the time I posted the question I was using joint rotations in "global" space (i.e. relative to the character root, as opposed to "local" space relative to the parent joint), so they could take all kinds of values. But your explanations and figures are very useful. $\endgroup$
    – javidcf
    Commented Jun 4 at 11:17

I found an intermediate solution between minimal representation and full rotation matrix, which is using two unit vectors indicating two orthogonal directions (e.g. forward and right) with respect to the orientation. These can be learned simply through squared differences and are simple to use (in my case the animation runtime already had functions to compute an orientation given two directions, taking into account that they may not be perfectly orthogonal). This representation takes 6 values per orientation which is not as good as 3 or 4 but not too terrible either, and in my experiments it seems to work just as fine, only without the unstable cases.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.