Principal Component Analysis (Chapter 1)

Literature › Works

Anon. (n.d.). Principal Component Analysis (Chapter 1).

Type	book chapter
Scope	Comparative and methods
Method	theoretical or conceptual
Period studied	-
Themes	methods data sources
Citation key	anonnda

Argument

A didactic textbook chapter introducing principal component analysis (PCA) as a variable-reduction procedure that turns many correlated observed variables into fewer orthogonal artificial variables (components) accounting for most of the variance. It separates PCA from exploratory factor analysis: PCA assumes no latent causal structure. It sets out a seven-step workflow in SAS PROC FACTOR: extraction, deciding how many components to retain (eigenvalue-one, scree, proportion of variance, interpretability), varimax rotation, interpretation, factor or factor-based scores, tabulation and write-up. The steps are illustrated with fictitious questionnaire data: a 6-item Prosocial Orientation Inventory and a 12-item investment-model instrument.

Evidence and method

Fictitious illustrative datasets only: a 7-item job satisfaction correlation matrix (N=200), 6-item Prosocial Orientation Inventory raw data (N=50), and a 12-item investment-model correlation matrix (N=300). Annotated SAS output is included. No empirical or historical data.

Findings

• PCA defined as reducing many observed variables to fewer artificial variables capturing most of their variance. (p. 2)

  wish to develop a smaller number of artificial variables (called principal components) that will account for most of the variance in the observed variables.

• Unlike factor analysis, PCA does not assume an underlying causal model. (p. 10)

  In contrast, principal component analysis makes no assumption about an underlying causal model.

• Minimum sample-size rule of thumb for PCA. (p. 13)

  the minimal number of subjects providing usable data for the analysis should be the larger of 100 subjects or five times the number of variables being analyzed.

• At least three variables should load on each retained component. (p. 12)

  It is highly desirable to have at least three (and preferably more) variables loading on each retained component when the principal component analysis is complete.

• Conditions under which the Kaiser eigenvalue-one criterion is reliable, citing Stevens. (p. 23)

  recommends its use when less than 30 variables are being analyzed and communalities are greater than .70, or when the analysis is based on over 250 observations and the mean communality is greater than or equal to .60.

• Conventional cumulative-variance threshold for retaining components. (p. 26)

  they usually retain enough components so that the cumulative percent of variance accounted for at least 70% (and sometimes 80%).

• Recommends a combined, sequential use of all four retention criteria. (p. 27)

  We recommend combining all four in a structured sequence.

• Introductory threshold for a meaningful loading. (p. 29)

  simply consider a loading to be “large” if its absolute value exceeds .40.

• Simple summed (factor-based) scales lose the orthogonality of true component scores. (p. 38)

  This is why factor-based scales will often demonstrate nonzero correlations with one another, while true principal components (from an orthogonal solution) will not.

• In the worked POI example, two retained components explain 71% of variance. (p. 41)

  Combined, components 1 and 2 accounted for 71% of the total variance.