What to record
Model architecture, training data version, hyperparameters, training date, evaluation results at the point of approval, and change metadata. Each is there for a reason: without the training data version a result cannot be reproduced, and without the evaluation results at approval time there is no baseline to establish that anything later regressed.
Why reproducibility is the point
The same model version on the same input should produce the same output. That property is what makes an audit trail meaningful, an incident investigable, and an improvement attributable. Where it does not hold — because the system is non-deterministic, or because the model is hosted and mutable — governance has to shift from reproducibility to observability: if you cannot re-run it, you must have recorded it at the time.
Versioning what you do not control
For a hosted model, almost nothing in the list above is available to you. What is, and what must therefore be versioned instead, is your side of the assembly: the model identifier and version string as reported by the provider at the time of each call, the prompt version, the retrieval corpus state, the tool permission set, and the evaluation results current when the deployment was approved.
Two practical notes. Record the version the provider reports rather than the one you configured — aliases such as "latest" resolve differently over time, and the configured value tells you nothing about what actually ran. And retain evaluation results with the assembly they were run against; a floating score cannot establish that anything regressed.
The test
For any output produced last month, can you reconstruct exactly what produced it? If the model version was not captured per call, the answer is no, and every subsequent investigation is inference rather than evidence. This is the single question that distinguishes a versioning practice from a versioning intention.
As a governance control
Versioning is what lets model drift be distinguished from a performance regression caused by a change someone made. Without it the two are indistinguishable, which means the response cannot be matched to the cause. It also underpins rollback: a rollback path that has never been tested against a recorded version is an assumption. In hosted-model deployments, "roll back the model" may not be an option the organisation owns at all — which is worth establishing before it is needed rather than during an incident.