Perplexity
Statistical metric that measures how well a language model predicts a text array.
Perplexity is a statistical metric derived from the Knowledge theory, measuring how well a language model predicts a text array. Mathematically, the probability that the model is assigned to the test data is calculated by taking the base of the negative log-average; intuitively, it means that the model is hesitated between the average few options when choosing the next word in every step. For example, if the perplexity value of a model is 20, this model means it remains unstable as they choose from an average of 20 equal possibilities while predicting each word; the more low the value the model means it makes sure and hit predictions.
Perplexity is a widely used measure to monitor progress especially in the educational process of language models and compare different models or architectures objectively; it is one of the basic metrics continuously monitored in the process of developing models such as GPT and BERT. However, low perplexity does not always mean that the model will be better done by the human, so today large language models are evaluated with evaluations or task-based criteria based on perplexity as well as human feedback (soru response accuracy, reasoning tests).
Perplexity is a widely used measure to monitor progress especially in the educational process of language models and compare different models or architectures objectively; it is one of the basic metrics continuously monitored in the process of developing models such as GPT and BERT. However, low perplexity does not always mean that the model will be better done by the human, so today large language models are evaluated with evaluations or task-based criteria based on perplexity as well as human feedback (soru response accuracy, reasoning tests).