From the Montana article "Kinematics of Contact and Grasp", if I have a ball roll on the plane without sliding, the motion equation is described below: \begin{equation*} \begin{bmatrix} \dot{u}_{2} \\ \psi \end{bmatrix} = \begin{bmatrix} M^{-1}_{2}R_{\psi}\\ T_{1}+T_{2}R_{\psi} \end{bmatrix} M_{1}\dot{u}_{1} \end{equation*}
${(1)}$ for Ball, ${(2)}$ for Plane. ${\psi}$ is the angle of contact between ${x}$ axis of coordinates of ball and plane. ${K, T, M}$ are the curvature form, torsion and metric tensor at time $t$ relative to coordinates of ball and plane.
We have \begin{equation*} R_{\psi} = \begin{bmatrix} \cos_{\psi} -\sin_{\psi}\\ -\sin_{\psi} -\cos_{\psi} \end{bmatrix} \end{equation*}
How can I use the Bellman equation by Sutton and Barto to discrete path planning of the motion of the ball?
\begin{align} v_{\pi}(s) = \sum_a \pi(a|s) \sum_{s'} P_{ss'}^a(R_{ss'}^a + \gamma v_{\pi}(s')) \end{align}
From my understanding that agent is a ball, environment is the plane, action is rolling ball without sliding and the achieved goal is motion planning from one point to another point.
The goal is how to optimize the path planning of the point in the motion.
I do not know how to determine the policy $\pi,$ and how to build up a function from Bellman equation. Please leave comments.
