Since the disparity map above is a representation of the object's distance from the camera's origin, is it reasonable to assume that a network (perhaps a convolutional LSTM) could be trained to perform a infer the depth of the object in the Z coordinate of camera frame?
As you can see above the brightness of the object increases as the car moves closer to it, and the size of the 2D bounding box also increases.
Thus the depth in the Z direction becomes a function of two things:
the change in the diagonal/size of the 2D bounding box
1.1. More or less the object's size as the car approaches it because it can also be moving across from that car as it is stopped at a street light. In which case the 2D box is mostly constant in size.
the change of the brightness of the pixels inside the 2D box.
Before going down this rabbit hole, I was hoping to get feedback on the feasibility of this idea, what "gotcha's" there might be that are not immediately apparent.
A couple of things are immediately noticeable about why this would be too difficult:
- The occlusion of parked cars on the right hand side there makes their shapes very hard to distinguish, although maybe the 2D boxes could help with that
- It will be difficult to track the same object through time steps, and would probably require a SLAM network as well.
Any help is very much appreciated!