Top-P (Nucleus Sampling)

Top-P, also known as nucleus sampling, is another way to control how random or focused an AI model's responses are. While temperature adjusts the randomness of all word choices equally, top-P works by limiting which words the model is even allowed to consider.

Here is how it works: at each step, the model calculates the probability of every possible next word. Top-P then takes only the most likely words whose probabilities add up to P percent, and throws out the rest. If top-P is set to 0.9, the model considers the smallest group of words whose combined probability is 90%, ignoring the unlikely 10%. If top-P is set to 0.1, the model only looks at the very top candidates whose probabilities add up to 10%.

In practice, top-P and temperature achieve similar goals -- controlling the creativity versus predictability trade-off -- but they work differently under the hood. Most experts recommend adjusting one or the other, not both at the same time, to avoid unpredictable interactions.

For everyday users, the key takeaway is this: if you have access to a top-P setting, lowering it makes responses more focused and predictable, while raising it allows more variety. A top-P of 0.9 or 1.0 is a common default. If you need very precise, factual responses, try lowering it to 0.3 or 0.5. Most casual users can leave top-P at its default and just adjust temperature, which tends to be more intuitive.