In my latest HealthStream Research employee survey
report, I noticed that the employees at our hospital are
described as being significantly more satisfied with the hospital
as a place to work than they were last year.  What does this mean?  This result sounds good, but I am confused.  What is meant by a significant difference?  I have to present the results of the employee survey to the executive team at my hospital.  My presentation will be significantly better if I understand what is meant by a significant difference between two things.

Response:

This is a common question and much confusion exists around this topic.  Let’s try to shed some light upon it and in the process help you be better equipped to make those presentations in front of the executive team. 

If a significant difference exists between two things it means that the difference is thought to be real and not a result of a fluke or a result of chance or sampling error.  Basically, there are three levels of significance used by researchers.  These are the .05, .01, and .001 levels.  These significance levels mean that the probability of obtaining a difference that large by chance is less than 5 in 100, 1 in 100, or 1 in 1000, respectively.  You may see these levels of significance in the footnotes of your reports.  This is statistical talk that lets you know the level of confidence you can have in your results. 

It is therefore extremely likely that your employees are more satisfied with their jobs on this survey than on the survey last year.  Whether at the .05, .01, or .001 levels, the difference is trustworthy and there is only a remote possibility that it is a fluke. 

So, in general, if you have a significant difference in your report, it means that it is highly unlikely to have occurred by chance or sampling error.  Thus, it is probably not a fluke and the difference is real.