The two most important indicators for the scientific nature and performance of a model are validity and reliability. As an organisation, it is extremely important that the measurement results are based on an as accurate as possible reflection of reality.
A model is:
- valid if it achieves its objective and
- reliable if the measurement results are an accurate reflection of the variables being measured.
A ZebraZone model contains one or more concepts (e.g. the concept of “employee satisfaction”) each consisting of several KPIs (Key Performance Indicators). Every KPI is defined on the basis of one or more sub-sectors called factors. Factors consist of one or more items or questions.
Figure: General structure of ZebraZone models
Every ZebraZone model is translated into a questionnaire, the validity and reliability of which are guaranteed.
Validity
Validity is described as the extent to which a measurement instrument attains its objective. A questionnaire that does not measure what it should measure is worthless.
1. Construct validity
Construct validity refers to the degree to which a measurement instrument measures a non-observable concept (e.g., “employee satisfaction”) in the way it ought to be measured. In examining construct validity we are looking to see whether all aspects that distinguish the concept are also correctly operationalized. The theoretical relations between the concept and other related concepts should also be matched (statistically) in the data (for example via correlational examination).ZebraZone has extensive datasets (e.g. N=8290) enabling statistical validation (and cross validation) of ZebraZone models through, among others, correlational analyses, but also Structural Equation Modelling (see “Reliability” below).
2. Content validity
Content validity is a non-statistical form of validity, whereby a measuring tool is assessed in terms of its content correctness. We examine here whether a measuring tool covers all facets of a particular concept and whether the items therefore provide a representative reflection of the concept to be measured. In most case, this form of validity is assessed (subjectively) by a panel of experts, who have to reach agreement.[1]The content validity of ZebraZone questionnaires is in the first place guaranteed by the way in which the questionnaires are developed. More specifically, the concept is first defined and then specific sub-sectors (KPIs and factors) are distinguished by experts. The links between these KPIs and the concept must be both theoretically and statistically proven in specialised scientific literature. Finally, representative items for these KPIs are selected. In the second place, the questionnaire (broken down into KPIs, factors and items) is critically examined by researchers (from the practical and academic worlds), consultants and HR managers.
3. Face validity
Face validity is closely related to content validity, but is really not validity in the technical sense of the term. Whilst content validity is dependent on theoretically considerations, face validity refers to the degree to which one has the impression that a model or measuring tool is valid and that the measuring tool therefore appears to measure what needs to be measured.
What guarantees the face validity of ZebraZone questionnaires are the personal opinions of our researchers and customers and their numerous conversations among themselves and with the academic world, etc.
Reliability
Reliability is linked to the accuracy and precision of a measurement instrument. It is the extent to which a measurement is free from coincidental errors. It is a necessary but insufficient condition for validity. The reliability of a measurement instrument can be defined in various ways. A commonly used description is the extent to which research can be repeated and still generate the same end results.
A reliability analysis is carried out when a ZebraZone model is developed and after every measurement within an organisation. Depending on the model, the reliability analysis is carried out using one of the two methods described below:
- Analysis of the internal consistency or homogeneity based on correlations and Cronbach’s alpha or,
- Analysis of the entire model structure using structural equation modelling.
[1] A measuring tool can be content-valid but not construct-valid, or vice-versa. Where all relevant KPIs are included in the measuring tool (or a good selection has been made), but have been wrongly operationalized (i.e., items not measuring what they are supposed to measure), the tool is content-valid but not construct-valid. Where all KPIs have been correctly operationalized, but certain key KPIs are missing, then the tool is construct-valid but not content-valid.