A friend and fellow simulationist posed the following four questions to me. After spending the better part of the morning preparing my response to him, I thought there might be others who could benefit from this.
-
What is the answer for ROI when everything is at zero?
There’s an old saying: “There’s always a cost; there’s not always a value.” In other words, the cost is never zero. Your time doing anything has to be compensated so – if all other aspects are provided for nothing -- you are the cost. -
Should the student hours in the lab be tracked?
Yes, and whatever tangible assets you have should be tracked as well. And staff are tangible assets! -
If the student pays x dollars for y hours per semester and the student spends z hours in the lab, do we then get the hours in the lab represented in dollars?
Nice idea, but it isn’t always easy to determine what the “x dollars” is. Perhaps you are fortunate to work where simulation activities are added into specific course costs, but in my experience, this is rarely the case. So, how do you actually calculate what the student pays per hour of instruction? If you have your simulations as part of a clinical course, you can say the tuition for that course is the price they pay, but what about lab fees, insurance costs, materials the student pays for to use in clinical/simulation? What other costs might they incur for a course? -
If there is no money spent, is the return on investment all profit?
There is money spent. See #1 above. The scenario had to be written and the setting created. That has a cost. Whatever is in the simulation had to be obtained somehow and at some expense. Don’t know what something cost back when it was purchased or donated? Look at the cost of replacing the “free” equipment should it fail. Apply the cost of a replacement to the one you have and treat it as if you had bought it for that price. What about the spaces where you run your simulations? Are you paying for the space? If you weren’t doing a simulation in that space, what would it be used for? Would the organization be able to generate revenue from that space during the hours you are in there simulating? If so, the cost of the simulation space is what the company didn’t make off that space because you were in there. And finally, there is always what you cost as an employee. There is no such thing as a free lunch or a free simulation.
So, in answering the above questions I have raised even more questions. But you can determine your simulation costs. You just do it using what my friend, Eddie Luevano1, calls the Mythical 2080. Its mythical because it really doesn’t exist, but it is still a useful construct. Assuming you could run simulations 8 hours a day every day of the work week and do it for each week of the year, you can simulate a total of 2080 hours per year (8*5*52=2080). This is not practical or even possible, yet it is the maximum number of hours of simulation per year using a 40-hour work week. Using this theoretical upper bound, if you are paid annually, that means your compensation can be divided by 2080 to get your hourly wage. For example, if your annual salary is $80,000/year, your hourly wage is $38.46/hour. Now, every hour you are involved in simulation can be costed at $38.46. This includes all setup time, execution time – including prebriefs and debriefs, if you are involved in those aspects -- and teardown time.
Your equipment can be costed out the same way. A $5,000 IV pump has a useful life of, say, five years. Five years at 2080 hours per year is 10,400 hours. Over 5 years, $5,000 / 10,400 = $0.48 / hour, if you used that pump every day of every week for all five years. But you won’t do that, will you? No, you’ll use it maybe three times a week for four two-hour long sessions during a semester. If we assume that you will use it for 12 of the 15 weeks of that semester, and you have two semesters per year, that’s 768 hours of use per year, not the Mythical 2080 hours. Over 5 years, that’s 3,840 hours of use (3*8*12*2=768; 768*5=3,840). Now, your $5,000 pump is costing you $1.32 per hour of use.
Okay, now you can see how to determine the cost of a simulation: Add up the hourly cost of your staff, equipment, and space (if you have to pay for that space) and divide that by the number of hours of simulation where you use that staff, equipment, and space. That’s what your simulation program is costing.
You now have to add in the learners to your cost calculations. Often, the best you can do for your organization is just define the cost factors and determine how best to delineate the activities. For our shop, we settled on defining a simulation session as the same set of learners doing the same set of activities. Thus, a session could be two students in a room with a manikin while a facilitator and an operator observed, or it could be eight students in two rooms – a bay and an observation room – with only a facilitator observing. It could be manikin-based or an SP experience. It could also be skills training and have only an instructor and, say, four students participating. The definition is based on the students involved and the activity or activities being the same. If in a single day we do the same activities twice with different students, that’s two simulation sessions logged for that day.
Once you can say what a simulation is and how many learners are in that session, you can figure out what costs to apply to that session and then divide that by the number of learners to get a per-learner cost. Per-learner costing is simply counting “butts in the seats”. For example, if you ran 10 sessions in a week with eight learners per session, then you served 80 learners. These are called learner encounters (LE). It doesn’t matter if you had 80 different learners that week or you had 40 learners and had them each in twice that week. Still a total of 80 LE.
Another way to look at costing is by learner contact hours (LCH). Here, you don’t just count the learners in the rooms, you count how long they are in those rooms. Taking our earlier example, let’s assume that in six of the ten sessions you had four learners in each session and eight students in each of the four remaining sessions. Let’s also assume that the sessions with four students were each six hours and the sessions with eight students were two hours long. Doing the math: 4 students * 6 hours * 6 sessions = 144 LCH; 8 students * 2 hours * 4 sessions = 64 LCH; 144 + 64 = 208 LCH. Using LCHs makes more sense in academia because counting hours of instruction times number of students is how schools measure instructor workload.
But what does all that mean in terms of value to learners? In simple terms, not much. The value is what the learners get out of simulation, which is often very hard to measure with any reasonable reliability and accuracy. Using the Mythical 2080 you can calculate the hourly staff costs and an approximate cost per hour of any given piece of equipment or of the space being used during any given simulation session. Dividing that total session cost by the number of LCHs provided in that session will give you a dollar cost per LCH. That will make your management happy, but it doesn’t say squat about the value provided by those dollars. For that, you would need to look to something like the Kirkpatrick Model and/or one of the many simulation evaluation instruments out there.
If you are lucky enough to have a “before simulation” expense model, you can then compare it to an “after simulation” model. Let’s say your hospital has a given issue, like an in-hospital falls, that results in two additional days of hospitalization on average. After training the staff in simulation to avoid patient falls, the number of falls decreases by 50% over a six-month period. This would indicate that simulation training on fall risk mitigation has resulted in fewer extended patient stays with a reduction in the costs associated with those extended stays. If you know what it costs to keep a person in the hospital for a day, then you can say the value of your simulation training is equal to the money saved by not having as many patients hospitalized after having fallen. Reduction in hospital cost minus the cost of the training to reduce that cost equals the value of that training.
Okay, that’s my take on how to quantify costs when discussing Return on Investment (ROI) and a little on determining the value of the investment. Another factor, one we don’t always consider, is the Return on Expectations (ROE). What do stakeholders expect of your simulation program and are those expectations being met? That’s a topic for another post.
Bottom Line on ROI:
- Determine who are your learners and how you serve them.
- Determine what tangible assets – people, equipment, spaces, etc., -- you use to serve your learners.
- Determine the cost of each tangible asset in the same units you use to serve your learners. (Hint: This is usually going to be measured in hours.)
- Report your results in costs per LE and LCH, preferably separated into the types of simulation activities you provide.
1Eddie Luevano is Associate Director of Administration at the Training and Educational Center for Healthcare Simulation (TECHS), which is part of the Texas Tech University Health Sciences Center, El Paso. (See https://elpaso.ttuhsc.edu/TECHS/Staff.aspx.)
Ed Rovera