Questionnaire Reliability, Validity, and Structural Equation Modeling (SEM)

About Trust and Effectiveness

The questions in this article can also be addressedLinear regression base: linear model, minimum 2x2 estimate and regression diagnosis、Linear regression step: proposed alignment, model selection and co-line SexHow the concept of a relatively close read together is developed in different contexts.

In the questionnaire study,TrustandEffectThese are two measures of the quality of the questionnaire. The analysis and conclusions based on the data collected from a questionnaire are unreliable if the letter and effectiveness of the questionnaire are not met.

Letter Analysis Concerned that “Does my questionnaire measure stable, consistent and reliable?” It cares.Measurement tool itselfThe question is not whether it measured the right thing.
Effect Analysis Focusing on "Did my questionnaire measure what I wanted to measure?" It cares.Content and purpose of measurementsCorrect.

Assuming that we are looking at “staff satisfaction”, this is a typical abstract concept that needs to be measured through a number of specific issues. We divide it into two dimensions:Pay satisfactionandWorking environment satisfactionThe following general questionnaire was developed.

Part I: Remuneration satisfaction

$X_1$I think my pay matches my work. (1 - 2 - 3 - 4 - 5)
$X_2$I am satisfied with my level of remuneration compared to other colleagues in the company. (1 - 2 - 3 - 4 - 5)
$X_3$The company ' s welfare benefits (such as leave, subsidies, etc.) are attractive. (1 - 2 - 3 - 4 - 5)

Part II: Satisfactory working environment degrees

$Y_1$I have a good relationship with my colleagues and have a good cooperation. (1 - 2 - 3 - 4 - 5)
$Y_2$I work in a positive, positive atmosphere. (1 - 2 - 3 - 4 - 5)
$Y_3$. I often feel tremendous pressure from work. ( 1 - 2 - 3 - 4 - 5 ) (Note: This is a reverse score.)

Letter Analysis

Letter AnalysisTest the results of the questionnaire.Stability and consistencyI don't know. This is the main concern.Internal consistency trust, i.e. the measurement of “consistent calibre” of multiple topics of the same concept. The usual method.Klonbach. $\alpha$ Coefficient (Cronbach)'s alpha)。

Because of the title $Y_3$ It's a reverse score, while the rest is a positive description. In order to be consistent, we must $Y_3$ . That is, in the 5-point scale, use 6 - 原始得分, the 5-point system is maintained but converted to a positive measure.

The analysis of the level of confidence is directed at the “dimensional dimension” rather than the entire questionnaire. We need separate analyses of “pay satisfaction” and “work environment satisfaction”.

$\alpha$ The coefficient measures a set of topicsInternal coherenceThe extent. If these topics are indeed measuring the same concept, there should be a strong positive correlation between scores.$\alpha$ The coefficient does not directly calculate the correlation between all topics, but is adoptedDifferenceThis larger indicator indirectly measures this relevance. The formula is as follows:

$$ \alpha = \frac{k}{k - 1} \left( 1 - \frac{\sum_{i=1}^{k} \sigma_{Y_i}^2}{\sigma_X^2} \right) $$

of which $k$ It's the number of topics.$\sigma_{Y_i}^2$ I'm going to have to calculate the total difference of all the topics.$\sigma_{X}^2$ Calculates the total division difference for the entire dimension. If there's a high correlation between the topics, the difference in the total score will be sustained by the “coordinate difference”, resulting in (各题目方差之和 / 总分方差) This is a small margin.$\alpha$ Value is high. If the subject is not relevant, then it should be the same.$\alpha$ Value is low. coefficient $k / (k-1)$ It's an amendment to make sure that the subject is completely irrelevant.$\alpha$ The theoretical value of the coefficient is 0.

Clonbach for each dimension $\alpha$ A coefficient that measures the internal consistency of the questionnaire and supports the determination that the answer is stable.

Effect Analysis

Validity analysis to test the questionnaireMeasure accuratelyThe concept we want to measure. This is the main concern.Structure Effectis the theoretical structure of the questionnaire (two dimensions) consistent with the structure of the data actually collected. The usual method.Validity factor analysis (CFA), to verify the consistency of data and theoretical assumptions.

The CFA is a common tool to test structural effectiveness. It's used to verify the model that we've designed. $X_1, X_2, X_3$ It's a pay factor.$Y_1, Y_2, Y_3$ “Environmental” factor) and the degree to which the data are proposed.

The first step is to develop theoretical assumptions that give us an understanding of the problem.Assumptions modele.g.:

There are two potential factors (substantial variables): “salary satisfaction” and “work environment satisfaction”.
Title $X_1, X_2, X_3$ It is an observation indicator of the “pay satisfaction” factor.
Title $Y_1, Y_2, Y_3$ (reversed) is the observation indicator for the “work environment satisfaction” factor.
There may be a correlation between these two factors.

The entire CFA workflow can be summarized as a comparison of the two matrixes:

Sample Arguments$S$ - Sample Covariance Matrix)：Actual data collectedThe agreed matrix calculated reflects the true correlation between all the topics in the questionnaire.
Model Implicit Difference Matrix ()$\Sigma(\theta)$ - Model-implied Covariance Matrix): Based on youPredefined theoretical model, by mathematical formulaFigureA collusive matrix. It represents “if theoretical models are established, the data should be presented”.

CFA By Adjusting All in ModelParametersThe parameters to be adjusted to bring the two matrices closer together include:

Factor load: submersible variables (e.g. “pay satisfaction”) for observational variables (e.g. $X_1$The degree of impact.
Because of the difference between the twoLevel of correlation between different submersible variables (e.g. “remuneration” and “environment”).
Error: Part of each subject that cannot be explained by subvariant (i.e. measuring error).

We need to map the theoretical structure in the CFA software, and then rely on software to optimize the parameters:

UseEllipseOrganisationSubtract Variables。
UseRectangleOrganisationObservation variables。
UseOne-way ArrowFrom submersible to its corresponding observation variable,Factor load。
UseDouble ArrowConnect to different submersible variables, representingBecause of the difference between the two。
Each observation variable will have a one-way arrow pointing at itself, representingError。

The CFA tests the structural effectiveness through three levels of evidence:

Card-check: test whether there are significant differences between matrices, but $p > 0.05 million can only be considered ready. Large samples are almost always significant, so only for reference purposes.
RMSEA: The smaller the average error of the model in each degree of freedom, 0.08.
SRMR: The average of the differences in the relevant coefficients, the smaller the better, 0.08.
CFI / TLI: Compare your model with a baseline model that is "not relevant to all variables" and see how much better it is. The closer one, the better.> 0.90 (Acceptable).

ObservationStandardised Factor LoadAnswer: "Is my subject measuring the subvariant I want?" Because the subload should be high enough and $p$ Significant value. Usually. > 0.5, ideal > 0.7。

ResearchDifferentiate EffectAnswer: "Is something really different between the different submersible variables I've measured?" If the relevant coefficient is too high (if > 0.85), indicates that the two factors may measure the same concept and are not sufficiently differentiated.

Validity factor analysis (CFA) and structural equation model (SEM)

The CFA is the foundation and exception of the SEM, the extension and extension of the CFAI don't know. The former only includesMeasurement modelI don't know. It's about...Validation of measurement toolsI don't know. The question it answered was, "Did my questionnaire title accurately measure the abstract concepts that I wanted to measure? "

SEM (structure equation model) It's also included.Structure Model, to analyse causal relationships between submersible variables (abstract concepts). The answer to the question was: “Are my theoretical assumptions (e.g., that pay satisfaction will affect work environment satisfaction and, in turn, exit trends) supported by data?”

It's...Just add a causal path between submersible variables to the CFA.I don't know. When we use it, we mainly answer the following questions:

Path coefficient: the direct effect of one submersible variable on the size and direction of another submersible variable (similar to the regression factor), typically between -1 and +1 and provided $p\text{-value}$。
Indirect effects/intermediaries Response: Variable A influences the extent of variable C through variable B.
Model alignment: Does the entire theoretical model of causality that I designed fit the actual data?

SEM is a return equation with some advantages over traditional multi-linear regression. Although the technical details have been better sealed, they can still be used in many academic studies. It includes the following advantages:

Process measurement errorsThis is the main advantage of SEM. SEM recognizes that there are errors in each observation variable and separates it from the model, thus estimating more purely the relationship between submersible variables.
Addressing multiple causality at the same time: A regression analysis can only test one cause variable at a time. SEM can test a complex network of multiple variables and self-variant in a model.
Estimated brokering and reconciliation effects: SEM is the one that tests the intermediate effect (A through B) and the regulatory effect (B by B)Best ToolsI don't know. It can be clearly deciphered.Direct effects、Indirect effectsandTotal effect。
Provide overall model alignment: A regression analysis can only tell you whether a single path is significant, but it is not possible to evaluate the good or bad of the whole “theory model”. SEM offers a series of composite indices (e.g., CFI, RMSEA) that allow you to judge the degree to which your entire theoretical blueprint matches reality.
Comparative competition theory: You can propose two different theoretical models, and then use SEM data to determine which model can better interpret the data.