What do you think about product delivery of non-deterministic systems? Same processes used, or are there changes that require the PM to think beyond the release?
Great question, Elena. Pre-launch iteration matters a lot more for probabilistic systems. You can't just define a feature and build it, because there are so many permutations to evaluate. The team needs extra time in development to play with the feature, refine it, and cover as wide a range of use cases as possible before they're confident putting it in front of customers.
I'd split it two ways:
- Product Delivery: internal iteration, with the goal of exploring as much of the surface area as possible.
- Product Quality (next week): AI evals, so once the feature is live there's a continuous signal on how it's doing.
Agree that the PM has to think well beyond the release. I'm going to update the article to make this clearer. Thanks for flagging this.
What do you think about product delivery of non-deterministic systems? Same processes used, or are there changes that require the PM to think beyond the release?
Great question, Elena. Pre-launch iteration matters a lot more for probabilistic systems. You can't just define a feature and build it, because there are so many permutations to evaluate. The team needs extra time in development to play with the feature, refine it, and cover as wide a range of use cases as possible before they're confident putting it in front of customers.
I'd split it two ways:
- Product Delivery: internal iteration, with the goal of exploring as much of the surface area as possible.
- Product Quality (next week): AI evals, so once the feature is live there's a continuous signal on how it's doing.
Agree that the PM has to think well beyond the release. I'm going to update the article to make this clearer. Thanks for flagging this.