XDR Project Management #1: User stories, Estimating, Acceptance criteria, Definition of done
In this post, I will be covering User Stories, Estimating and Monitoring tasks, how well that worked for us, why our estimations were off, and the key improvements that we will be making to do a better job in the production phase. There will not be an explicit section of what I would do differently in this entry, as I will break down what went wrong and offer solutions as I go along for each of the problems individually.
Since this pre-production phase was 3 two-week sprints, it meant we only had three opportunities to learn how to do Sprint Planning right and to improve on what we’ve been doing wrong. The first estimating session was a bit of a mess, as we effectively did it wrong. We wrote user stories for what needs to be done, and then assigned story points to these. We never really broke it down to actual tasks and we didn’t estimate hours of work needed for that. Therefore, there’s effectively not much that I can discuss in terms of the first sprint, since I can’t discuss our accuracy in estimating story points.
In the second and third sprints, we started properly breaking down tasks and estimating in hours how long these would take to complete. Our estimations ended up being wrong in both of these sprints, and we seem to have achieved less than we set out to both times.
The first reason for why this happened, I think, is because our sprint planning sessions were rushed quite a lot, meaning that quite often we wouldn’t have enough time to properly think through and break down our tasks into pieces that would be better understood. Furthermore, we were naïve at times with our estimations, such as assigning only 6 hours to implementing locally saved leader boards, which, thinking about it now, sounds completely based on nothing. The reason why this task got assigned 6 hours was because we thought that either it would be as simple as using File I/O to create and write a file with the data we need, or that it won’t be as easy as that on the PS4 and we’ll need to research a proper way to do this. If it were the first case, the task would take a couple of hours at most. If it were the second case, we have absolutely no idea. Therefore, we picked a sort of mid-point, and when it turned out that I need to use the PlayStation 4 SDK to do this (which I had never worked with directly before), the task ended up taking me a couple of days to do, meaning that my other assigned tasks, such as implementing propeller animations on the drone, weren’t done.
The second reason for our estimations being wrong, was that we always ended up with more stuff to do in our sprint than we’d planned for. This meant various troubleshooting we had to do for the art/design team when their level editors wouldn’t work, or when one file or another got corrupted uploading to Perforce, and so on. What’s worse, we would find out during the sprint, that an artist is blocked because they are relying on the code team to do something, and the code team doesn’t have that task in the sprint! All of this stemmed from the lack of communication during sprint planning, as we would split off to our separate teams, such as Gameplay Code, Framework Code, and Gameplay Art to do our planning separately. The idea was that once we’ve done that, we would come together and sort out any inter-dependencies. However, that never truly happened, since by the time we finished our own separate planning, everyone seemed tired of it and proper attention wasn’t really given to aligning dependencies and goals. There’s really no excuse for this happening, as rushing to get planning done would affect the whole 2 weeks afterwards, and unexpected tasks cropping up mid-sprint turns the whole thing into a chaotic situation where no-one can really be sure what’s going to get done by the end of the sprint. Even if dependencies can not be foreseen in sprint planning, we should have then reduced the total available hours for the code team in the sprint, in order to take into account sorting these dependencies out. Finally, we all talked about this and agreed that the divide between Gameplay and Framework teams really isn’t working, since there’s too many inter-dependencies. Instead, we decided to try and split up into teams based on goals for the production phase, such as Camera team, Particle effects team, Controls team and so on. This should ensure that the people working on common goals would always be in communication and would be able to plan cooperatively.
Finally, our estimations were wrong because in a lot of cases, we quite frankly had no clue when a task would be done. For example, Alvaro had a task called “Make AI move like a drone”. In this case, we can’t really tell how long it is going to take to achieve this. After all, this is kind of the goal for the whole game, meaning it’s an epic. We didn’t immediately realise this, thinking that Alvaro would work to make it move more like a drone than it is not, but we had no end goal for the task in sight. I only realised this towards the end of the pre-production phase, so I did not have much of a chance to offer changes to our planning process up until now, but at least for the first production sprint, we took the User Stories written by the team, reviewed the ones that require code implementation, and wrote Acceptance Conditions on the backs of these cards. This should gradually solve the problem of us estimating without knowing what we’re estimating, since we would define precisely what we’d expect from a task. Since we know what we have right now, and we clearly know what we want to have when the task is done, we can estimate better how long it will take to do it. Hopefully, this should be the missing link in improving our estimations.
When writing the Acceptance Conditions, we also finally came up with our definition of “done”. During the whole pre-production phase, we had a lot of situations where some implementation would work on the implementer’s machine, but not work anywhere else. In addition to that, we wanted to include peer-reviews for all of the code, and enforce testing on the PlayStation 4, so that we don’t end up with a task that is considered done, when really, the solution is flaky at best, works in only a very specific set of conditions, and might well be untested on our target hardware! Therefore, I proposed that we always get a sign-off for each task by at least two people. If it’s a design task, that is a coder and a designer. If it’s a code task, then 2 coders and if it’s an art task, then 1 artist and 1 coder. By sign-off, we mean that it works on at least two other people’s machines (to alleviate the “works on mine, but no one else’s machine” problem), and that the two people are happy with the way it works. Furthermore, at least one coder will need to peer-review the code. Finally, the task can not be marked as done if it has not been tested to work as expected inside the Level Editor (to reduce number of crashes on design machines), PC and the PlayStation 4.
A combination of all of these improvements should solve the problems mentioned, improve estimations, and eliminate the unfortunate situation we are in right now, where tasks are being presented as done, but they do not work on the PlayStation 4 (such as the VFX), or when they are obviously buggy on everyone’s machines apart from the person who implemented it (such as the camera).
XDR Project Management #1: User stories, Estimating, Acceptance criteria, Definition of done
In this post, I will be covering User Stories, Estimating and Monitoring tasks, how well that worked for us, why our estimations were off, and the key improvements that we will be making to do a better job in the production phase. There will not be an explicit section of what I would do differently in this entry, as I will break down what went wrong and offer solutions as I go along for each of the problems individually.
Since this pre-production phase was 3 two-week sprints, it meant we only had three opportunities to learn how to do Sprint Planning right and to improve on what we’ve been doing wrong. The first estimating session was a bit of a mess, as we effectively did it wrong. We wrote user stories for what needs to be done, and then assigned story points to these. We never really broke it down to actual tasks and we didn’t estimate hours of work needed for that. Therefore, there’s effectively not much that I can discuss in terms of the first sprint, since I can’t discuss our accuracy in estimating story points.
In the second and third sprints, we started properly breaking down tasks and estimating in hours how long these would take to complete. Our estimations ended up being wrong in both of these sprints, and we seem to have achieved less than we set out to both times.
The first reason for why this happened, I think, is because our sprint planning sessions were rushed quite a lot, meaning that quite often we wouldn’t have enough time to properly think through and break down our tasks into pieces that would be better understood. Furthermore, we were naïve at times with our estimations, such as assigning only 6 hours to implementing locally saved leader boards, which, thinking about it now, sounds completely based on nothing. The reason why this task got assigned 6 hours was because we thought that either it would be as simple as using File I/O to create and write a file with the data we need, or that it won’t be as easy as that on the PS4 and we’ll need to research a proper way to do this. If it were the first case, the task would take a couple of hours at most. If it were the second case, we have absolutely no idea. Therefore, we picked a sort of mid-point, and when it turned out that I need to use the PlayStation 4 SDK to do this (which I had never worked with directly before), the task ended up taking me a couple of days to do, meaning that my other assigned tasks, such as implementing propeller animations on the drone, weren’t done.
The second reason for our estimations being wrong, was that we always ended up with more stuff to do in our sprint than we’d planned for. This meant various troubleshooting we had to do for the art/design team when their level editors wouldn’t work, or when one file or another got corrupted uploading to Perforce, and so on. What’s worse, we would find out during the sprint, that an artist is blocked because they are relying on the code team to do something, and the code team doesn’t have that task in the sprint! All of this stemmed from the lack of communication during sprint planning, as we would split off to our separate teams, such as Gameplay Code, Framework Code, and Gameplay Art to do our planning separately. The idea was that once we’ve done that, we would come together and sort out any inter-dependencies. However, that never truly happened, since by the time we finished our own separate planning, everyone seemed tired of it and proper attention wasn’t really given to aligning dependencies and goals. There’s really no excuse for this happening, as rushing to get planning done would affect the whole 2 weeks afterwards, and unexpected tasks cropping up mid-sprint turns the whole thing into a chaotic situation where no-one can really be sure what’s going to get done by the end of the sprint. Even if dependencies can not be foreseen in sprint planning, we should have then reduced the total available hours for the code team in the sprint, in order to take into account sorting these dependencies out. Finally, we all talked about this and agreed that the divide between Gameplay and Framework teams really isn’t working, since there’s too many inter-dependencies. Instead, we decided to try and split up into teams based on goals for the production phase, such as Camera team, Particle effects team, Controls team and so on. This should ensure that the people working on common goals would always be in communication and would be able to plan cooperatively.
Finally, our estimations were wrong because in a lot of cases, we quite frankly had no clue when a task would be done. For example, Alvaro had a task called “Make AI move like a drone”. In this case, we can’t really tell how long it is going to take to achieve this. After all, this is kind of the goal for the whole game, meaning it’s an epic. We didn’t immediately realise this, thinking that Alvaro would work to make it move more like a drone than it is not, but we had no end goal for the task in sight. I only realised this towards the end of the pre-production phase, so I did not have much of a chance to offer changes to our planning process up until now, but at least for the first production sprint, we took the User Stories written by the team, reviewed the ones that require code implementation, and wrote Acceptance Conditions on the backs of these cards. This should gradually solve the problem of us estimating without knowing what we’re estimating, since we would define precisely what we’d expect from a task. Since we know what we have right now, and we clearly know what we want to have when the task is done, we can estimate better how long it will take to do it. Hopefully, this should be the missing link in improving our estimations.
When writing the Acceptance Conditions, we also finally came up with our definition of “done”. During the whole pre-production phase, we had a lot of situations where some implementation would work on the implementer’s machine, but not work anywhere else. In addition to that, we wanted to include peer-reviews for all of the code, and enforce testing on the PlayStation 4, so that we don’t end up with a task that is considered done, when really, the solution is flaky at best, works in only a very specific set of conditions, and might well be untested on our target hardware! Therefore, I proposed that we always get a sign-off for each task by at least two people. If it’s a design task, that is a coder and a designer. If it’s a code task, then 2 coders and if it’s an art task, then 1 artist and 1 coder. By sign-off, we mean that it works on at least two other people’s machines (to alleviate the “works on mine, but no one else’s machine” problem), and that the two people are happy with the way it works. Furthermore, at least one coder will need to peer-review the code. Finally, the task can not be marked as done if it has not been tested to work as expected inside the Level Editor (to reduce number of crashes on design machines), PC and the PlayStation 4.
A combination of all of these improvements should solve the problems mentioned, improve estimations, and eliminate the unfortunate situation we are in right now, where tasks are being presented as done, but they do not work on the PlayStation 4 (such as the VFX), or when they are obviously buggy on everyone’s machines apart from the person who implemented it (such as the camera).
Related files
None.