0
$\begingroup$

Take a RNN network fed with Shakespeare and generating Shakespeare-like text.

Once a model seems mathematically fine, as can be assessed by observing its loss and accuracy over training epochs, how can one assess and refine the goodness of the result ?

Only human eyes can judge of the readable character of a text, its creativity, its grammatical correctness etc.

QUESTION : Which systematic approach can be used to refine a generative model (text) ?

$\endgroup$

1 Answer 1

2
$\begingroup$

The answer is in the question :)

Only human eyes can judge of the readable character of a text, its creativity, its grammatical correctness etc.

In the example of a model trained on Shakespeare's writing, take a group of human annotators (preferably literature experts) and ask them to annotate texts as likely authored by Shakespeare or not (variant: mark texts according to how close they are to Shakespeare's style). The texts provided to them should contain actual texts by Shakespeare and texts generated by the model, of course. Btw this is the principle of the well known Turing Test.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.