pytrio.SamplingParams

class SamplingParams(BaseModel):
    max_tokens: Optional[int] = None
    seed: Optional[int] = None
    stop: Union[str, Sequence[str], Sequence[int], None] = None
    temperature: float = 1
    top_k: int = -1
    top_p: float = 1

SamplingParams controls text generation behavior in SamplingClient.sample().

future = sampling_client.sample(
    prompt=prompt,
    num_samples=4,
    sampling_params=SamplingParams(temperature=0.8, max_tokens=256),
)

Parameters

Parameter	Type	Default	Description
`max_tokens`	`int \| None`	`None`	Maximum number of tokens to generate. `None` means no explicit limit
`seed`	`int \| None`	`None`	Sampling random seed for reproducible generation
`stop`	`str \| Sequence[str] \| Sequence[int] \| None`	`None`	Stop condition. Supports a string, a list of strings, or a list of token IDs; generation stops when matched
`temperature`	`float`	`1`	Controls sampling randomness. Higher values make output more random; `0` means greedy decoding
`top_k`	`int`	`-1`	Top-K sampling. Samples only from the K highest-probability tokens; `-1` means disabled
`top_p`	`float`	`1`	Top-P (nucleus) sampling. Samples only from candidate tokens whose cumulative probability is within this value; `1` means disabled

Examples

Greedy decoding

SamplingParams(temperature=0, max_tokens=128)

Stop words

SamplingParams(temperature=1.0, max_tokens=256, stop=["</s>", "\n\n"])

Fixed random seed

SamplingParams(temperature=0.8, max_tokens=128, seed=42)

pytrio.SamplingParams

Parameters

Examples

On this page