There’s a good reason SIMUL8’s KPI summary loads up blank. We’ve had thousands of requests from users over the years to auto populate it with results but we’ve firmly declined.

Why? One of the most common errors we see users make with simulation is going into result overload. If you have so many results you can no longer see the critical insights, you will never be able to make a decision. The simulation is there to help make a decision – the simulation is not the end goal!

Less is more.

Let me tell you about one of my least successful projects from my early consulting days. We were making an operational tool that would be used to plan out staffing for a call center. Our clients were used to getting system performance results in 5 minute intervals so it seemed sensible to agree to their request to make the simulation match this. That was bad decision #1.

Bad decision #2 was letting them specify over 50 results that they felt they might need (i.e. were terrified they would need in the future after the consulting project was finished, so they wanted to build in anything and everything they could conceivably see a potential use for in the future). The fact that each result then had to be able to filtered by each agent and call type took the results into thousands, all with 5 minute interval results with each simulation run lasting 1 year.

Initially I was very proud of the fact I had managed to make the simulation run super-fast whilst collecting all this data, there many air punches as I shaved seconds off each run, again and again. My elation was short lived as we moved into client testing. “There are so many results I just don’t know what I’m looking at any more.” Aargh, but that’s what you wanted! But it wasn’t my job to give them what they wanted, I should have given them what they needed. It’s not easy to achieve but as the simulation builders it’s our job to educate our end users that endless results will not necessarily get them to their goal.

Three results.

Typically about 3 results, 5 maximum, are all you need to evaluate the success of your “what if” scenario. This doesn’t include the validation and verification phases (when you’re checking the simulation matches what you’ve understood the process is, and then checking the simulation performs as the process does). During that phase you’ll quickly be checking lots and lots of different results but not analyzing them in detail, you’re only looking for signs that you’ve built your simulation incorrectly.

Once your simulation is verified and validated and you’re into the analysis phase, this is where you need to restrict your results. First, establish the question your simulation is trying to answer, for example “how do I increase my production line throughout to 600 units per day?” or “how do I reduce patient waiting time to 18 weeks?”. Then pick the results that help you answer that. Let’s take our production line throughput example, for that I would look at:

• Throughput per hour. I’d chart this, because I want to see this is steady rate or whether it varies depending on other factors in the system, e.g. staff scheduling, whether its vulnerable to machine downtime, or if deteriorates as the simulation gets into a steady state.
• Average time in system. I also want to know how quickly I’ll be able to get orders through my system, I want to look at the confidence intervals on this because I want to see if there’s a wide range which might indicate instability in the process.
• Machine and/or staff utilization. Again, it’s about testing my confidence in that throughput number. If all my machines and staff are working at 100% utilization to achieve that then I’m vulnerable to machines breaking down, staff being ill or staff needing a bathroom break!

So what I’m testing with my results is “does my “what if” scenario reach my goal?” and “is it a valid/good solution?”. The latter question is particularly key as there will often be many good solutions to the same problem. It’s also in helping answer that question that you’ll often provide the most value to your client.

Depending on these results I will want to dive deep into parts of my simulation. And that’s fine, that’s what all the visual interactive results and the detailed all object reports in SIMUL8 are for. But don’t get lost in these. Train yourself to use them for in-depth analysis but then take a step back and reflect on your simulation question. Then use your 3 main results you’ve added to the KPI summary to keep you and – most importantly – your client on track.

And then change them!

No, I’m not trying to be deliberately confusing. When you’ve answered your initial question, it’s most likely you’ll move onto the next question. “Yes I can hit my throughput goal, now what does my machine maintenance schedule need to look like to hit that always?”. The same 3 results may not be the right ones for your next question. Again, train yourself to recognize you’re moving into a new iteration, a new learning cycle, pause reflect, what is my question now and what 3 key results will help me answer that question?