Top-p / Top-kSample
sampling strategies that limit the potential distribution when choosing the next word.
Top-k and ball-p (nucleus sampling) is two sampling strategies aimed at balancing both diversity and consistency by limiting the possibility distribution when choosing the next word of a language model. The model in the ball-k sampling takes only the most possible part of the words sorted according to the potential, all the low probability words that are left completely elers and selection makes it from the cluster of fixed size. A weakness of this method is that even if the distribution is too "sivri", it is difficult to include the word, if the distribution is very "flat", it is not allowed to be sufficiently varied.
Top-p (nucleus sampling), this problem solves the problem by determining a dynamic threshold according to the cumulative potential rather than a fixed number: words are sorted according to the potential and the cumulative possibilities are added to the cluster until they exceed the specified p value (e.g. 0.9); the choice is made only from this "core". In this way, if the dispersion spikes are small, the distribution flats are larger than the cluster. Two are often used with the temperature parameter, and the modern large language model APIs are standard parameters offered to enable developers to finely adjust their output variety.
Top-p (nucleus sampling), this problem solves the problem by determining a dynamic threshold according to the cumulative potential rather than a fixed number: words are sorted according to the potential and the cumulative possibilities are added to the cluster until they exceed the specified p value (e.g. 0.9); the choice is made only from this "core". In this way, if the dispersion spikes are small, the distribution flats are larger than the cluster. Two are often used with the temperature parameter, and the modern large language model APIs are standard parameters offered to enable developers to finely adjust their output variety.