The PM-to-engineer ratio point is spot on, and AI makes the leverage gap even wider. The best PMs now also judge model output, not just features. Disclosure: I built a freemium AI PM course from 604 real PM job postings on exactly those skills: https://www.allthingspm.app/
Thanks, Arpit! Judging model output is quickly becoming a core PM skill, which is why AI evals now sit inside the Product Competency Toolkit. I'm building an in-depth assessment so people can see where they land on each of the 12 competencies, and product quality evals are a key part of it. Mining 604 real job postings for those skills is a smart way to ground the course. Congrats on shipping it!
Good article, I like the graphics. Especially the one showing how urgent operational tasks eat up on what's meaningful long term. A question. While it's evident that AI can automate a lot of documentation, it's less evident to me how that should be done reliably. The output cannot be fully trusted and it always need a review that might as much time as filling up a document template you know. What's your take on that? How do you use AI to free up time for slow judgement tasks?
Thanks, Antonio! I spent way too much time on those graphics, so I'm glad they landed.
Great question. I think it matters more than people realize, because teams are drowning in documentation now. Much of it is too long and not quite accurate, so nobody reads it. That's just as bad as having none.
I recently saw Clay's AI writing guidelines, and one rule stuck with me: if it takes someone longer to read a document than it took to write it, something is off. It's a good rubric. If AI makes a document faster to produce but slower to review, you haven't saved any time -- its just shifted it to review.
I've used a couple of approaches for helping with document accuracy. The first and msot important is to use the minimum amount of relevant context. I avoid skills as much as possible. Too much context leads to context rot which harms accuracy. I ask for citations and then will typically do an eval pass to check citations and claims. I also go through every word... I use AI a lot in my posts, but it doesn't save me much time. Posts take about the same amount of time as before but I think they turn out better.
The PM-to-engineer ratio point is spot on, and AI makes the leverage gap even wider. The best PMs now also judge model output, not just features. Disclosure: I built a freemium AI PM course from 604 real PM job postings on exactly those skills: https://www.allthingspm.app/
Thanks, Arpit! Judging model output is quickly becoming a core PM skill, which is why AI evals now sit inside the Product Competency Toolkit. I'm building an in-depth assessment so people can see where they land on each of the 12 competencies, and product quality evals are a key part of it. Mining 604 real job postings for those skills is a smart way to ground the course. Congrats on shipping it!
Good article, I like the graphics. Especially the one showing how urgent operational tasks eat up on what's meaningful long term. A question. While it's evident that AI can automate a lot of documentation, it's less evident to me how that should be done reliably. The output cannot be fully trusted and it always need a review that might as much time as filling up a document template you know. What's your take on that? How do you use AI to free up time for slow judgement tasks?
Thanks, Antonio! I spent way too much time on those graphics, so I'm glad they landed.
Great question. I think it matters more than people realize, because teams are drowning in documentation now. Much of it is too long and not quite accurate, so nobody reads it. That's just as bad as having none.
I recently saw Clay's AI writing guidelines, and one rule stuck with me: if it takes someone longer to read a document than it took to write it, something is off. It's a good rubric. If AI makes a document faster to produce but slower to review, you haven't saved any time -- its just shifted it to review.
I've used a couple of approaches for helping with document accuracy. The first and msot important is to use the minimum amount of relevant context. I avoid skills as much as possible. Too much context leads to context rot which harms accuracy. I ask for citations and then will typically do an eval pass to check citations and claims. I also go through every word... I use AI a lot in my posts, but it doesn't save me much time. Posts take about the same amount of time as before but I think they turn out better.