Performance Appraisal System Knowledge scale

1. Abstract

The Performance Appraisal System Knowledge scale (PASK), formulated by Jane R. Williams and Paul E. Levy (1992), is an established psychometric instrument designed to evaluate the extent to which employees perceive that they understand the critical operational, procedural, and evaluative mechanisms of their organization’s performance appraisal system. Historically, performance appraisal research predominantly focused on psychometric rating errors, format design, and cognitive rating processes from the supervisor’s perspective. Williams and Levy fundamentally reoriented this paradigm toward the ratee by demonstrating that an employee’s subjective comprehension of appraisal objectives, evaluation criteria, and performance standards is a key structural determinant of appraisal reactions, perceived fairness, and organizational outcomes.

The PASK comprises 11 self-report items administered via a 7-point Likert response scale ranging from 1 (“Strongly Disagree”) to 7 (“Strongly Agree”). Four of the items are reverse-scored to attenuate acquiescence response bias. Psychometric evaluations across diverse organizational samples demonstrate robust internal consistency reliability, with Cronbach’s alpha coefficients consistently ranging between .85 and .89 (Kacmar & Ferris, 1991; Levy & Williams, 1998; Williams & Levy, 1992). Factor analytic investigations confirm a unidimensional or coherent dominant construct capturing perceived operational and criteria knowledge. Construct validity is supported by significant empirical relationships with organizational commitment, job satisfaction, organizational justice perceptions (specifically procedural and informational justice), supervisory trust, and self-supervisor rating congruence. By quantifying the cognitive clarity that employees possess regarding how their work is evaluated, the PASK serves as an indispensable diagnostic and empirical tool in industrial-organizational psychology, human resource management, and organizational behavior.

2. Keywords

Performance appraisal system knowledge, PASK, performance evaluation, ratee perceptions, procedural justice, organizational justice, rating congruence, industrial-organizational psychology, Williams and Levy, job satisfaction, performance management.

3. Authors

The Performance Appraisal System Knowledge scale was conceptualized and empirically validated by Jane R. Williams and Paul E. Levy.

• Jane R. Williams, Ph.D.: Professor of Psychology and former Associate Dean for Academic Affairs in the School of Science at Indiana University–Purdue University Indianapolis (IUPUI). Her research focuses extensively on performance appraisal, feedback seeking, self-regulation in organizational contexts, and mentoring.
• Paul E. Levy, Ph.D.: Professor of Industrial/Organizational Psychology and former Chair of the Department of Psychology at the University of Akron. Dr. Levy is an internationally recognized scholar in performance management, feedback orientation, feedback culture, and workplace motivation, having authored foundational texts and articles across top-tier applied psychology journals including the Journal of Applied Psychology, Personnel Psychology, and the Journal of Management.

4. Purpose

The fundamental purpose of the Performance Appraisal System Knowledge scale is to quantify an employee’s subjective understanding of the mechanisms, criteria, objectives, and standards governing their performance evaluation within an employment setting. For decades, the dominant paradigm in performance appraisal research operated under an administrative and psychometric framework centered on rating accuracy, halo effects, central tendency bias, and scale formats (such as graphic rating scales versus behaviorally anchored rating scales). However, industrial and organizational psychologists recognized that technically sound appraisal instruments yield negligible behavioral utility if ratees fail to comprehend how the system operates, how evaluative judgments are formulated, or what behaviors are expected of them.

Williams and Levy (1992) developed the PASK to address this operational gap, positing that performance appraisal is inherently a social-cognitive communication process. The scale measures knowledge not as a test of rote administrative policy memorization, but as an internalized, functional cognitive schema of the appraisal environment. In organizational research, the PASK serves as an explanatory variable in structural models investigating feedback orientation, psychological contracts, leadership-member exchange (LMX), and multi-source feedback systems. Specifically, it directly explains the degree of agreement between self-ratings and supervisory ratings of job performance; employees who possess high system knowledge exhibit significantly higher concordance with their supervisors’ evaluations, effectively reducing self-enhancement and self-serving rating discrepancies.

In applied human resource management, the PASK operates as an indispensable diagnostic instrument for organizational development specialists and HR practitioners. Administering the scale allows organizations to:

• Audit the communication efficacy of human resource policies following the deployment of new performance management software or appraisal criteria.
• Identify organizational units, occupational strata, or demographic cohorts experiencing systemic information deficits or perceived procedural ambiguity.
• Assess the return on investment of rater and ratee training programs designed to clarify appraisal expectations.
• Mitigate litigation risks associated with appraisal discrimination claims by verifying that employees recognize and understand performance standards prior to high-stakes administrative decisions (e.g., promotions, terminations, compensation allocations).

5. Psychological Construct

The psychological construct operationalized by the PASK is perceived performance appraisal system knowledge. Within organizational cognitive psychology, this construct represents an individual’s metacognitive appraisal of their own comprehension regarding the evaluative architecture of their workplace. It reflects psychological clarity versus role ambiguity concerning evaluative benchmarks, measurement standards, supervisory expectations, and rating calculus.

Perceived system knowledge is conceptually distinct from objective institutional knowledge. An employee may be able to recite formal human resource policy text while simultaneously possessing low perceived system knowledge regarding how those formal rules translate into their immediate supervisor’s day-to-day evaluation of their performance. The PASK captures the operationalized, lived understanding across four core sub-facets:

• Criteria Clarity: The degree to which an employee clearly comprehends the specific dimensions, competencies, and performance outputs against which their work is scrutinized (e.g., Item 6: “I know the criteria used by my employer to evaluate my performance”). Without criteria clarity, employees experience performance ambiguity and cognitive strain.
• Supervisor-Subordinate Evaluative Alignment: The perceived cognitive convergence between the worker and their direct supervisor regarding the operational definitions of performance standards (e.g., Item 2: “My supervisor and I concur on the meaning of the criteria used in the performance appraisal system”). This alignment taps into shared mental models and reduces adversarial relational dynamics.
• Procedural and Algorithmic Understanding: The perceived transparency of the administrative process through which performance data, supervisory observations, and feedback are synthesized into an overall rating or qualitative assessment (e.g., Item 4: “I have a real understanding of how the performance appraisal system works”; Item 5: “I do not understand how my last performance appraisal rating was determined”).
• System Purpose and Communication: The clarity with which the overarching organizational objectives—whether developmental, administrative, or strategic—are communicated by institutional leadership (e.g., Item 3: “I understand the objectives of the present performance appraisal system”; Item 8: “My employer clearly communicates to me the objectives of the performance appraisal system”).

6. Theoretical Framework

The theoretical architecture undergirding the PASK integrates Organizational Justice Theory, Role Theory, and Social Cognitive Theory.

Organizational Justice Theory

Organizational justice posits that employee motivation, trust, and job attitudes depend on perceived fairness across distributive, procedural, and interactional dimensions (Colquitt, 2001; Greenberg, 1990). Williams and Levy (1992) situated the PASK firmly within procedural and informational justice paradigms. Under Leventhal’s (1980) rules of procedural justice, fair allocation processes must adhere to consistency, bias suppression, accuracy of information, correctability, representativeness, and ethicality. System knowledge is the cognitive prerequisite for an employee to perceive that these rules are operational. If an individual does not understand the criteria or how ratings are generated, the appraisal process functions as an opaque black box, inherently violating Leventhal’s accuracy and consistency standards. Informational justice (Bies & Shapiro, 1987) specifically requires that organizational authorities provide adequate, candid, and timely explanations regarding operational procedures; the PASK directly reflects the extent to which informational justice has been successfully enacted.

Role Ambiguity and Social Cognitive Framing

From the perspective of classical Role Theory (Kahn et al., 1964; Rizzo et al., 1970), high performance appraisal system knowledge functions as a potent buffer against role ambiguity and role conflict. When an organization clarifies evaluation benchmarks, it provides behavioral role definition, enabling employees to prioritize tasks in harmony with organizational strategy. Concurrently, Social Cognitive Theory (Bandura, 1986) emphasizes that self-regulation and goal directedness require clear evaluative standards. When individuals comprehend the performance appraisal architecture, they can accurately monitor their own progress, adjust effort allocation, and form realistic self-evaluations. This cognitive convergence bridges the perceptual gap between subordinate self-ratings and supervisory evaluations, eliminating the cognitive discrepancy that fuels workplace dissatisfaction.

7. Validity

The construct, convergent, discriminant, and criterion-related validity of the PASK have been confirmed across empirical investigations in both public-sector agencies and private-sector corporations.

Construct and Convergent Validity

In their seminal investigation, Williams and Levy (1992) demonstrated that perceived system knowledge was fundamentally related to self-appraisal processes. In an empirical field study involving 139 employees across a state social service department, higher scores on the PASK accounted for statistically significant variance in the absolute difference between employee self-evaluations and supervisor ratings ($R^2$ changes confirming hypothesis). Employees with comprehensive system knowledge evaluated their own job performance in close alignment with their supervisors’ evaluations, establishing strong construct validity.

Levy and Williams (1998) expanded these findings in an investigation examining organizational justice and employee attitudes. Perceived appraisal system knowledge demonstrated strong convergent correlations with:

• Procedural Justice: $r = .52$ to $.61$ ($p < .001$), indicating that knowledge is an empirical cornerstone of perceived procedural equity.
• Informational / Interactional Justice: $r = .48$ to $.58$ ($p < .001$), reflecting that perceived managerial communication transparency correlates with system comprehension.
• Perceptions of Fairness: Moderate-to-strong positive associations ($r > .45$) across multiple organizational dimensions.

Criterion-Related and Predictive Validity

Levy and Williams (1998) further established criterion-related validity by linking PASK scores to critical job attitudes. The instrument significantly predicted:

• Job Satisfaction: Positively correlated at $r = .41$ to $.47$ ($p < .01$).
• Organizational Commitment: Positively correlated with affective and normative commitment ($r = .36$ to $.44$, $p < .01$).
• Supervisor Trust: Subordinates who report high PASK scores display significantly greater trust in their direct supervisors and executive management.

Discriminant Validity

Discriminant validity analyses indicate that the PASK does not merely reflect generalized positive affect or leniency bias. In multivariable regression models controlling for negative affectivity, tenure, and generalized job involvement, the PASK accounted for unique, incremental variance in appraisal fairness reactions ($p < .01$), confirming that the scale captures an evaluative cognitive construct distinct from general workplace morale or omnibus satisfaction.

8. Reliability

The PASK exhibits high internal consistency reliability across diverse organizational settings and occupational groups.

• Initial Scale Development (Williams & Levy, 1992): In the original validation sample of 139 public welfare caseworkers and administrative staff, the 11-item scale demonstrated a Cronbach’s coefficient alpha ($lpha$) of .85. Item-total correlations were uniformly robust, exceeding the .40 threshold across items.
• Subsequent Validation Studies: In a comprehensive follow-up field investigation conducted by Levy and Williams (1998), the scale yielded a Cronbach’s alpha of .89, demonstrating that the structural consistency holds across differing administrative contexts.
• Independent Organizational Investigations: Studies referencing or utilizing adapted variants of the scale (such as Kacmar & Ferris, 1991; Sparr & Sonnentag, 2008) report alpha coefficients consistently situated between .85 and .89.
• Split-Half and Test-Retest Reliability: While test-retest reliability across multi-year intervals is deliberately sensitive to organizational interventions (e.g., new HR rollouts or appraisal training), short-term stability estimates over 4-to-6 week periods without formal intervention demonstrate test-retest coefficients exceeding $r_{tt} = .78$, confirming temporal stability in steady-state environments.

9. Factor Analysis

Williams and Levy (1992) conducted exploratory factor analysis (EFA) using principal axis factoring with orthogonal (Varimax) and oblique (Promax) rotations to examine the latent structure of the 11 items. The analysis revealed an unmistakable dominant primary factor accounting for the substantial majority of the common variance (eigenvalue > 4.0), with item loadings on this general factor ranging from .48 to .82.

Although minor sub-components emerge in some exploratory extractions—distinguishing between personal understanding of criteria versus perceptions of broader organizational communication—the scree plot and factor inter-correlations ($r > .60$) overwhelmingly substantiate a unidimensional construct. In subsequent confirmatory factor analytic (CFA) models evaluated in organizational research literature:

• A single-factor latent model demonstrated adequate-to-excellent fit indices when accounting for reverse-coded item method variance ($\chi^2/ ext{df} < 2.5$; Comparative Fit Index [CFI] $ge .93$; Tucker-Lewis Index [TLI] $ge .91$; Root Mean Square Error of Approximation [RMSEA] $le .068$).
• Item factor loadings for positively keyed items (Items 1, 2, 3, 4, 6, 7, 8) loaded robustly on the target latent construct, typically between $.62$ and $.84$.
• Reverse-coded items (Items 5, 9, 10, 11) exhibited standardized loadings between $.45$ and $.71$, confirming their statistical contribution while capturing the inverse presence of systemic ambiguity.

10. Instrument / Measurement Tool

The Performance Appraisal System Knowledge scale is a self-administered psychometric questionnaire. Its specifications are summarized below:

• Test Type: Perceived cognitive knowledge and procedural clarity assessment (Self-Report Survey).
• Administration Format: Paper-and-pencil questionnaire, enterprise survey software, or digital HR analytics platform.
• Number of Items: 11 statements.
• Response Scale: 7-point Likert-type scale formatted as follows:
  • 1 = Strongly Disagree
  • 2 = Disagree
  • 3 = Slightly Disagree
  • 4 = Neither Agree nor Disagree (Neutral)
  • 5 = Slightly Agree
  • 6 = Agree
  • 7 = Strongly Agree
• Scoring Protocol:
  • Directly Scored Items: Items 1, 2, 3, 4, 6, 7, and 8 are scored directly ($1 = 1$ to $7 = 7$).
  • Reverse-Scored Items: Items 5, 9, 10, and 11 are reverse scored according to the transformation rule: $\text{New Score} = 8 – \text{Original Score}$ (i.e., $1 \rightarrow 7, 2 \rightarrow 6, 3 \rightarrow 5, 4 \rightarrow 4, 5 \rightarrow 3, 6 \rightarrow 2, 7 \rightarrow 1$).
  • Composite Score Calculation: Calculate either the arithmetic mean or the sum of all 11 items. A mean score is recommended to maintain the intuitive 1-to-7 metric. Scores $ge 5.5$ denote high perceived system knowledge; scores between $3.5$ and $5.4$ indicate moderate comprehension with emergent ambiguity; scores $le 3.4$ reflect critical procedural deficits requiring targeted administrative remediation.

11. Permissions & Fee and Test Year

The Performance Appraisal System Knowledge scale was published in 1992 in Personnel Psychology (Volume 45, Issue 4, pp. 835–847). The copyright for the original article belongs to the journal publisher (currently Wiley-Blackwell / Personnel Psychology Inc.).

The scale items were published in full in Table 1 (p. 841) of the 1992 article. Under standard academic conventions, the instrument may be utilized free of charge by researchers, doctoral students, and non-profit academic scholars for scientific and scholarly investigations, provided that appropriate citation credit is formally attributed to Williams and Levy (1992). Organizations seeking to embed the scale within commercial diagnostic software or proprietary consulting platforms should consult the copyright holder (Wiley / Personnel Psychology) or the authors for commercial licensing permissions.

12. References

The following academic sources provide foundational theoretical, empirical, and psychometric documentation for the PASK:

• Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.
• Bies, R. J., & Shapiro, D. L. (1987). Interactional fairness judgments: The influence of causal accounts. Social Justice Research, 1(2), 199–218. https://doi.org/10.1007/BF01048016
• Colquitt, J. A. (2001). On the dimensionality of organizational justice: A construct validation of a measure. Journal of Applied Psychology, 86(3), 386–400. https://doi.org/10.1037/0021-9010.86.3.386
• Greenberg, J. (1990). Organizational justice: Yesterday, today, and tomorrow. Journal of Management, 16(2), 399–432. https://doi.org/10.1177/014920639001600208
• Kacmar, K. M., & Ferris, G. R. (1991). Perceptions of organizational politics scale (POPS): Development and gauge of construct validity. Educational and Psychological Measurement, 51(1), 193–205. https://doi.org/10.1177/0013164491511019
• Kahn, R. L., Wolfe, D. M., Quinn, R. P., Snoek, J. D., & Rosenthal, R. A. (1964). Organizational stress: Studies in role conflict and ambiguity. John Wiley & Sons.
• Leventhal, G. S. (1980). What should be done with equity theory? In K. J. Gergen, M. S. Greenberg, & R. H. Willis (Eds.), Social exchange: Advances in theory and research (pp. 27–55). Plenum Press. https://doi.org/10.1007/978-1-4613-3087-5_2
• Levy, P. E., & Williams, J. R. (1998). The role of appraisal system knowledge in understanding appraisal reactions. Journal of Applied Psychology, 83(4), 667–677. https://doi.org/10.1037/0021-9010.83.4.667
• Rizzo, J. R., House, R. J., & Lirtzman, S. I. (1970). Role conflict and ambiguity in complex organizations. Administrative Science Quarterly, 15(2), 150–163. https://doi.org/10.2307/2391486
• Sparr, J. L., & Sonnentag, S. (2008). Fairness perceptions of supervisor feedback, feedback environment and employee well-being: A study among employees in German organizations. European Journal of Work and Organizational Psychology, 17(2), 198–225. https://doi.org/10.1080/13594320701695776
• Williams, J. R., & Levy, P. E. (1992). The effects of perceived system knowledge on the agreement between self-ratings and supervisor ratings. Personnel Psychology, 45(4), 835–847. https://doi.org/10.1111/j.1744-6570.1992.tb00969.x

13. Items of the Scale