A pre-test with 107 professional employees allowed us to evaluate and refine the scale. We assessed the construct validity through our main study with 278 employees who were experiencing IT change across a range of organizations (i.e., main study). Finally, we conducted a follow-up study of 68 instructors anticipating the introduction of a new learning management system at a university.
In our pre-test, we randomly sampled alumni from the business school of a Canadian public university and invited 1445 individuals to participate in an online survey. To identify those individuals who were anticipating a new technology, we asked a screening question: “Is your company currently introducing to you a new information technology that you might use for your job later?” 294
responses were received. 137 individuals (46.6%) answered “no” to the screening question, 155 individuals (52.7%) answered “yes,” and 2 individuals did not answer that screening question but completed the rest of the survey. Among the 157 individuals, 26 abandoned the survey immediately after the screening question. Among the 131 individuals who answered some portion of the survey,
24 missed 5% or more of the 48 questions, leaving 107 usable responses. This falls just short of the needed cases (n = 113) to achieve 80% power for a small effect size (R2 = 10%) at the 5% significance level but is well above the sample (n = 41) needed for a moderate effect (Hair et al., 2016).