Auxiliary losses are additional terms added to your global loss to induce learning further upstream in your model and can be seen as a way to combat vanishing gradients.
An early reference would be how Google's Inception model uses auxiliary classifiers. The general idea is that you take an intermediate output of your model and use it as the prediction in a separate loss function. So during training, your model is acting as both a global model (all parameters subject to that final global loss) and a model composed of sub-models with specific terms applying exclusively to your auxiliary tasks.
In practical terms, you can follow this example. If the link ever dies, here's a generic view:
# let's say we have two auxiliary losses, and thus two auxiliary outputs
final_task_prediction, predictions = model.forward(x)
global_loss = loss(final_task_prediction, y)
intermediate_loss_1 = aux_loss_1(predictions, aux_y_1)
intermediate_loss_2 = aux_loss_2(predictions, aux_y_2)
# can weight auxiliary losses however we choose
total_loss = global_loss + .4*intermediate_loss_1 + .8*intermediate_loss_2
As an example, say we have an input sequence ABCDEF. To quickly run through the auxiliary losses in the Al-Rfou et al. paper
Multiple positions works as an auxiliary task of "given A, predict B, then given B (and our internal memory of A), predict C" (etc)
Intermediate hidden representations works the same as multiple positions, except intermediate layers also get their own loss terms
Multiple predictions breaks the target prediction sequence into multiple subsequences, so our auxiliary task is "given A predict B,C ; given B, predict C,D" (etc)
Hope this helps!