I've seen information retrieval systems that return some results from a query, and then the user rates these results as either "relevant" or "not relevant".
What can you do if you do not have user feedback?
E.g. suppose your system returns some ranked results from a query. Suppose you have no pre-defined notion of what is relevant, and suppose you cannot receive any kind of user feedback. What can you do?
This is important, because information retrieval evaluation metrics are based on relevance. Maybe it isn't possible to define relevance without user feedback, if so, maybe some information retrieval evaluation metrics without dependency on user feedback can be suggested?