When applied to machine learning (ML), these terms could all mean the same thing or not, depending on the context.
From the optimization standpoint, one would always like to have them minimized (or maximized) in order to find the solution to ML problem.
Each term came from a different field (optimization, statistics, decision theory, information theory, etc.) and brought some overlapping to the mixture:
it is quite common to have a loss function, composed of the error + some other cost term, used as the objective function in some optimization algorithm :-)
When dealing with modern neural networks, almost any error function could be eventually called a cost/loss/objective and the criterion at the same time. Therefore, it is important to distinguish between their usages:
functions optimized directly while training: usually referred to as loss functions,
but it is quite common to see the term "cost", "objective" or simply "error" used
as well. These functions can be combinations of several other loss or functions,
including different error terms and regularizes (e.g., mean-squared error + L1 norm of
functions optimized indirectly: usually referred to as metrics. These are used as
criteria for performance evaluation and for other heuristics (e.g., early
stopping, cross-validation). Almost any loss function can be used as a metric, which is
quite common. The opposite, however, may not work well since
commonly used metric functions (such as F1, AUC, IoU and even binary accuracy) are not
suitable to be optimized directly.