The book you are reading is being somewhat lax with terms. It uses the terms "actor" and "critic", but there is another algorithm called actor-critic which is very popular recently and is quite different from Q learning. Actor-critic does have two function estimators with the roles suggested in the quote. Q-learning has one such estimator*.
I have looked at the chapter in more detail, and where it says:
one will be used to drive Ms. Pac-Man during training (the actor), and the other will watch the actor and learn from its trials and errors (the critic).
I would substitute:
one will be used learn from current actions, and the other will remember results from some time steps ago in order to estimate the values for next actions.
This is not something that's inherently part of Q-learning, but it is part of DQN's adjustments when combining Q-learning with neural networks. Both experience replay and having two copies of the learning network (one a temporarily "frozen" version of the other) are important for stabilising the learning algorithm. Without them it can become numerically unstable.
Is this a typical Q-learning implementation?
It's a typical implementation of basic DQN, which is how many people nowadays would implement Q-learning with neural networks.
You can ignore the references to "actor" and "critic". Instead it is easier to consider that there is just one "action value" network, and you keep an old copy of it around to help with stability.
* Generally in RL, the term "model" is reserved for a model of the environment - which neither Q-learning nor actor-critic provide. So you will also read that Q-learning is a "model free" algorithm. For the rest of the book, you will have seen "model" to refer to any statistical learning algorithm (or the architecture and learned parameters) . . . what you will see in RL texts is the careful use of "function estimator" or other terms for networks which learn something else other than how the environment behaves.
