This is a huge topic, so I will just give you a high-level overview and some pointers to more info.
Yes, there are definitely ways to work with video that are different from working with individual still images.
At the simplest level, it could be running an object detector such as HoG (or a sliding-window convnet) on each frame, and then some means of assigning nearby detections in adjacent frames to be the same object, and discarding detections which do not seem to have continuity in time. Many of the algorithms in this field seem to look at a single frame as a building block to looking at the whole sequence, where perhaps the data from adjacent frames is combined, aggregated, and/or used to disambiguate the current frame.
Another approach is to first estimate object motion between frames (using optical flow, phase correlation, pyramid block matching or another method) and then treat areas of multiple frames that are collocated after accounting for motion as being the same object. This is very powerful but limited by the accuracy of the motion estimation.
In newer research, there is a back-and-forth between finding where things are (detection) and how things move (tracking), where each task can help the other one, for example (Kalal 2010) or (Andriluka 2008), to the point that the two parts of the algorithm are no longer separable. Kalal's TLD algorithm is a recently famous version of this.
There are also algorithms which work directly in the spatiotemporal (or sometimes just temporal) domain. An example of purely temporal would be detecting vehicles by the periodicity of variation of their wheel spokes.
Some of the frequently studied model problems are:
People, vehicles or other objects detection and tracking throughout a video sequence. For example, using the CalTech Pedestrians dataset or other standard benchmarks.
Hand gesture recognition, either for games, user interface, or sometimes sign language recognition. Often using depth data, for example Kinect video.
Activity recognition, for example walking vs standing. Due to the nature of this problem, it is much more common to see pure spatiotemporal algorithms here, for example (Sadanand and Corso 2012).
Simultaneous location and mapping (SLAM). Often found in robotics, this problem is basically to build a 3D model of the environment from video of a single camera (or stereo or depth video) which is moving around; the environment is often assumed to be static. This is often done by running some kind of feature detector such as SIFT or SURF, matching up features in successive frames, and then building point clouds from the relative 3D motion implied by features.
Here are some more references:
Shah, Mubarak, and Ramesh Jain, eds. Motion-based recognition. Vol. 9. Springer Science & Business Media, 2013.
Turk, Matthew. "Gesture recognition." Computer Vision: A Reference Guide (2014): 346-349.
Rosenfeld, Azriel, David Doermann, and Daniel DeMenthon, eds. Video mining. Vol. 6. Springer Science & Business Media, 2013.
Kalal, Zdenek, Krystian Mikolajczyk, and Jiri Matas. "Tracking-learning-detection." Pattern Analysis and Machine Intelligence, IEEE Transactions on 34.7 (2012): 1409-1422.
M. Andriluka, S. Roth, B. Schiele. People-Tracking-by-Detection and People-Detection-by-Tracking. Computer Vision and Pattern Recognition (CVPR) 2008
Sadanand, Sreemanananth, and Jason J. Corso. "Action bank: A high-level representation of activity in video." Computer Vision and Pattern Recognition (CVPR), 2012
