Development of an individual well-being scores assessment

Abstract

Background

This study describes the development and validation of an individual-level well-being assessment and scoring method (IWBS) adapted from the population-based Gallup-Healthways Well-being Index across six domains (physical health, emotional health, healthy behaviors, work environment, basic access and overall life-evaluation).

Methods

Exploratory analyses were conducted on half the sample (n = 2036) using principal component analyses (PCA) with varimax rotation and confirmatory factor analysis was conducted on the second half of the sample (n = 2100) using structural equation modeling to validate the measurement model found by the PCA.

Results

Optimal results in the exploratory sample were achieved for a seven-factor solution, accounting for 52.0% of the variance. All domains displayed adequate reliability, ranging from .42 to .79.

Conclusions

The IWBS met each of the criteria that were established for measurement development. Findings indicated that there was initial support for using the IWBS to assess well-being at the individual level. The IWBS has acceptable psychometrics of reliability, internal and external validity.

Background

There is growing recognition that national indicators of well-being are important for informing policy debates and for evaluating practices in many areas, including healthcare, public health, social services, parks and recreation, work life, transportation and the environment (Diener and Seligman 2004; Hall et al. 2010; OECD 2011a). Indices of national well-being often include macro-economic indicators such as gross domestic product (GDP) and health-related statistics such as life expectancy (Helliwell and Barrington-Leigh 2010). There is a growing awareness however, that these broad indicators do not adequately reflect well-being at the citizen-level, as they do not account for differential perceptions of these circumstances and offer only a limited view of the many factors that influence individuals. For example, correlations between GDP and subjective life-satisfaction are complex in nature and such measures do not account for variability within countries (Howell and Howell 2008; OECD 2011b). Reviewing three decades of research, Diener et al. (1999) point out that personal well-being is only modestly related to economic prosperity, demographic factors, and other social indicators.

Experts clearly recognize that subjective well-being of individual citizens is of unique importance and can complement objective indices. This awareness, in combination with other shifts in the field of economics and psychology, led to the study of subjective well-being and attempts to develop national measures (Diener and Seligman 2004; Gallup, Inc. 2009; OECD 2011a; Kahneman and Krueger 2006; World Health Organization 1997). Subjective well-being refers to “all of the various types of evaluations, both positive and negative, that people make of their lives. It includes reflective cognitive evaluations, such as life satisfaction and work satisfaction, interest and engagement, and affective reactions to life events, such as joy and sadness”. (Diener, 2006 page 153). Essentially, the field of subjective well-being is based on the notion that “people’s experience of a set of circumstance is as important as the circumstances themselves, and the people are the best judges of how their lives are going”. (OECD, 2011b, page 265). For this reason, measuring well-being at the subjective level is important.

Subjective well-being cannot be considered a specific unidimensional construct (Diener et al. 1999). Diener defines subjective well-being as “all of the various types of evaluations, both positive and negative, that people make of their lives. It includes reflective cognitive evaluations, such as life satisfaction and work satisfaction, interest and engagement, and affective reactions to life events, such as joy and sadness”. (Diener et al. 2006). Unfortunately, most researchers assess only one or two well-being variables and rarely include a broader range of concepts. In a review of the literature Diener and Seligman (2004)) found that 2,158 publications discussed “positive affect” but only 93 of those mentioned “life satisfaction”. This has led to variable and piecemeal construct definitions and measures of well-being. Fortunately, there appears to be a growing consensus on variables that constitute the core of psychological or subjective well-being in the literature. Theorists seem to agree that definitions of subjective well-being should include life satisfaction, positive affect, and negative affect (Diener et al. 1999; Diener and Seligman 2004; DeNeve and Cooper 1998) as well as domain satisfaction which are “the judgments people make when evaluating major life domains such as physical and mental health, leisure, social relationships and family” (Diener 2006). While there is no universal agreement on which life domains are central to well-being, national surveys tend to include, in addition to life satisfaction, subjective measures of physical and mental health, financial stability, personal safety, and work satisfaction, among others (Gallup, Inc. 2009; World Health Organization 1997; World Values Survey, 2011).

Two methods of assessing subjective well-being have also emerged through research and include “evaluative” and “experienced” well-being. The work of Diener and colleagues focuses largely on evaluative well-being, which assesses people’s judgments and global impressions of their experiences. Kahneman and colleagues, emphasize the importance of assessing “experienced” well-being, which is concerned with momentary affective states and the way people feel about experiences in real-time (Kahneman and Krueger 2006 ; Kahneman and Riis 2005).

The Gallup-Healthways Well-being Index

The present study investigated whether one of the leading national indicators of well-being, the Gallup-Healthways Well-being Index (WBI; Gallup, Inc. 2009), could be adapted to assess individual level well-being across multiple domains. The WBI survey was developed to establish official statistics on the state of well-being in regions across the United States. Prior work by Diener (2006) and Kahneman and Riis (2005) was particularly influential in defining well-being as measured by the WBI, which encompasses both evaluative and experienced well-being. The full details of WBI survey development are outlined in a report on the Well-Being Index website (http://well-beingindex.com/files/Gallup-Healthways%20Index%20Methodology%20Report%20FINAL%203-25-08.pdf). In brief, a collection of survey items aligned with previous research on well-being was compiled by experts in the field and reviews of the literature (see Diener and Seligman 2004; Kahneman and Riis 2005). Final survey content and structure was determined empirically by conducting factor analysis on data collected from a large national sample. Criterion-related validity was established through correlation to health and socioeconomic indicators by geographic region. The six domains represent the key aspects of well-being for states, communities and other geographic entities (Gallup, Inc 2009).

The WBI has overlap with domains assessed in other national measures of well-being such as physical health, emotional health, environmental factors (e.g. access to resources), and general life satisfaction (World Health Organization 1997). Findings on life satisfaction from the WBI are broadly consistent with other world surveys (Deaton 2008). In addition, each of the six domains has been found to be related to independent indicators of health, happiness, quality of life, functioning, health care costs and/or lost productivity. This evidence for each of the domains, domain definitions, and related research are delineated below.

Life evaluation

Life evaluation, also referred to as life satisfaction, represents how a person evaluates his or her life as a whole (Diener 2000). It can refer to a broad appraisal of life in general or life during a particular time period. The WBI assesses life evaluation using combined responses from the Cantril Self-Anchoring Scale (Cantril, 1965). Participants are asked to imagine a ladder with steps numbered from zero at the bottom to ten at the top, with the top of the ladder representing the best possible life and the bottom representing the worst possible life. They are then asked where on the ladder they feel their life falls currently and where it will stand in five years, yielding a present and future evaluation. Research indicates that a positive evaluation of one’s life is related to physical and emotional health, higher quality of life, lower behavior risks, and greater productivity (Diener and Seligman 2004).

Emotional health

Positive and negative affect are central elements of subjective well-being (Diener et al. 1999; Kahneman and Krueger 2006).The WBI assesses the experience of both positive and negative emotions, stress, depression, and happiness. Self-reports of emotional health problems are related to the presence of mental health problems, decreased quality of life, and lost productivity (e.g., Murray and Lopez, 1997; Unutzer et al. 2000; Loeppke et al. 2009). Conversely, happiness is related to mental and physical health, better social relationships, helping others at work, and higher economic well-being (e.g. Lyubomirsky et al. 2005).

Physical health and healthy behaviors

The WBI assesses individual’s perceptions and understanding of their own health including but not limited to overall health, chronic condition diagnoses, health problems that get in the way of daily functioning, experience of pain, and feeling rested. The growing emphasis on well-being and physical health is based, in part, on a more inclusive definition of health. The World Health Organization defines health as: “a state of complete physical, mental and social well-being and not merely the absence of infirmity” (World Health Organization 1946). The relationship between physical health and well-being is well-documented. Self reports of physical health problems are related to actual health, health care costs, quality of life, and lost productivity due to absenteeism and presenteeism (e.g. Achat et al. 2000; Murrell et al. 2003; Ostir et al. 2000; Loeppke et al. 2009). Across five continents, a clear and consistent positive correlation between psychological illness and persistent pain has been found (Gureje et al. 1998) and healthy behaviors are related to decreased risks for chronic diseases, disability, mortality and lower health care costs (e.g. Stampfer et al. 2000; Schultz and Edington 2007).

Work environment

Occupational stress has a negative impact on individuals, organizations, and the larger economy (Hoel et al. 2001). The WBI “work environment” domain includes a subjective assessment of individual’s job satisfaction, ability to use one’s strengths at work, relationship with one’s supervisor, and openness and trust within the work environment. The work setting has a substantial impact on well-being at work and outside of work, which in turn has consequences for the individual and organization (Danna and Griffin 1999). Research indicates that a positive work environment predicts greater happiness, less stress, better social relationships and greater productivity (e.g. Spector 1997; Gilboa et al. 2008).

Basic access

The WBI assesses basic access to needs in three areas of subjective assessment: financial resources, community quality and health care access. These areas encompass impressions of community quality, access to necessities such as clean water, medicine, places to exercise, and money for food, shelter and health care. Although basic access logically relates to socioeconomic status, basic access represents more proximal factors that affect ones well-being. For example, the strong positive relationship between monetary wealth and subjective well-being weakens when comparing high-income nations/individuals (Howell and Howell 2008). To explain this relationship, “need theory” posits that income and assets have their strongest influence on well-being when they are able to satisfy the most basic of physiological needs (Diener & Biswas-Diener, 2002). Therefore, the WBI focuses on access to financial, community and health resources required to meet basic needs rather than assessing income or socioeconomic status, which may be less directly related to well-being.

The WBI survey items are included in a comprehensive survey called the Healthways’ Well-being Assessment that currently serves to inform organizational health and wellness initiatives. Although the WBI is a validated measure for assessing well-being at the population level based on internal consistency, reliability, criterion-related validity and construct validity, it was not validated as a measure of individual well-being as this was not the purpose of the survey (Gallup, Inc 2009) Therefore, the content and scoring used in the WBI is not necessarily well suited to explain well-being variance between individuals. Therefore, its use is limited for exploring well-being variance within a population, conducting outcomes research, or to develop normative comparisons. Overall, an individual-level assessment of well-being will advance efforts to develop and evaluate interventions that can be tailored to guide and motivate individuals to make changes that enhance their well-being.

The current study

The goal of the research was to develop an assessment of Individual level Well-being Scores (IWBS) that would meet a series of criteria. First, it should be highly correlated to the WBI, to ensure that they both capture the same overall well-being construct. Second, the individual-level measure should be limited to items also found in the WBI to allow for comparison of the scores of individuals or groups of individuals to community data. Approximately 1000 WBI surveys have been conducted nightly since 2008, offering a wealth of data for the more detailed exploration of well-being differences at both the community and individual level. An individual scoring method opens up new avenues for well-being research. Third, the structure of the measure should be comparable to the WBI in that it contains six similar domains. Lastly, it should demonstrate adequate construct validity, demonstrating theoretically appropriate psychometric structure and relationships to measures of physical health, emotional health, health risk behaviors and to overall life-evaluation.

Methods

Participants

Participants were recruited via the Internet through a survey sampling company that has a national pool of 1,500,000 potential participants. A total of 4,136 participants from 39 states were recruited for a clinical trial. For the purpose of this research, all participant baseline responses were used. The mean age of participants was 48.29 (SD = 13.36). Roughly 58.9% of the sample was female. The sample was predominantly white, non-Hispanic (78.0%), 11.0% African-American, and 3.3% Hispanic. Only 52.2% (n = 2157) were currently employed, and 52.5% (n = 391) were married while 16.1% (n = 120) were single, never married. Participants who completed the cross sectional survey were entered into a sweepstakes. All human subjects’ procedures were approved by the Pro-Change Institutional Review Board.

Measures

Demographics

Age, gender, race and ethnicity, marital and work status were assessed.

Health risk intervention assessment

The Health Risk Intervention is a self-report assessment of risk status and stage of change for nine health risk behaviors: smoking, physical activity, healthy diet (low-fat and calorie control), fruit and vegetable consumption, effective stress management, depression prevention, alcohol consumption, adherence with anti-hypertensive and cholesterol medication. Stage of Change assesses an individual’s readiness to change a problem behavior typically using one or two questions. (Prochaska et al. 2008).

WBI

The group-level WBI consists of 53 self-report assessment items representing six reliable and valid domains of subjective well-being: Life Evaluation, Physical Health, Emotional Health, Work Environment, Healthy Behaviors, and Basic Access (Gallup, Inc. 2009). While individual items are not uniformly scored, the calculated overall score ranges from 0–100 and is comprised of the average of all individual domains. Life Evaluation is based on combined responses from the Cantril Self-Anchoring Scale (Cantril, 1965).

Overall health

For validity purposes, participants were asked to rate their overall health based on a five-point Likert scale ranging from 1 (poor) to 5 (excellent).

Patient Health Questionnaire-2 (PHQ-2)

The PHQ-2 is a 2-item self-report measure that assesses the frequency of depressed mood and anhedonia over the past 2 weeks. It asks “Over the last 2 weeks, how often have you been bothered by any of the following problems?” Participants are asked to rate the items “little interest or pleasure in doing things” and “feeling down, depressed, or hopeless”. with response options of 0 (not at all), 1 (several days), 2 (more than half the days), and 3 (nearly every day). A cut-off score of 3 has been identified as an optimal cut point for screening purposes by the developers of this measure and it has been shown to have good psychometric properties (Kroenke et al. 2003).

Statistical analysis

The sample was randomly split in half for exploratory and confirmatory analyses. Exploratory analyses were conducted on half the sample (n = 2036) using principal component analyses (PCA) with varimax rotation. The Life Evaluation Scale, which is based on combined responses from the Cantril Self-Anchoring Scale (Cantril 1965) was not included in the Principle Components Analysis for this study as it is an established measure.

Factor structure decisions were guided by the minimum average partial procedure (Velicer 1976; Velicer et al. 2000) and parallel analysis (Horn 1965). The decision regarding how many factors to retain was based on factor loadings and the alignment of factors with the WBI sub-domains. All items loading less than .40 or loading on more than one factor were discarded before running a second PCA using the same procedures on the remaining items. Coefficient alphas and maximizing retainment of the original WBI items also were considered in the decision regarding the final items to be retained. Confirmatory factor analysis (CFA) was conducted on the second half of the sample (n = 2100) using structural equation modeling to validate the measurement model found by the PCA. For each construct, five indices of fit, the likelihood ratio chi-square statistics, the goodness of fit index (GFI), the comparative fit index (CFI), the root mean square error of approximation (RMSEA), and the average absolute standardized residual statistic (AASR) were calculated and compared to determine the model that best fit the data. Traditionally, values of .80–.89 on the GFI and CFI indicate adequate to marginal fit, while values of .90 and above indicate good to excellent fit (Knight et al., 1994). For the RMSEA, 0.01, 0.05, and 0.08 indicate excellent, good, and mediocre fit respectively (MacCallum et al. 1996). For the AASR, values below .06 indicate excellent fit (Tabachnick and Fidell 2007). Multivariate Analysis of Variance (MANOVA), and follow-up tests investigated the relationships among the IWBS domains and the categorical variables (i.e. Overall Health, Life Evaluation categories, and number of behavior risks). Multiple regression analysis examined the relationships between IWBS domains and depression as measured by the PHQ-2.

Results

Initial iterations on the measurement model removed 12 items due to factor loadings (low or complex) or negative impact on scale reliability. Optimal results in the exploratory sample were achieved for a seven-factor solution, accounting for 52.0% of the variance. Factor loadings ranged from .37 to .83 (average loading = .67). All domains displayed adequate reliability, ranging from .42 to .79. CFA on the second half of the sample confirmed the seven-factor exploratory model, X²(303, N = 957) = 967.11, CFI = .87, GFI = .93, RMSEA = .05, AASR = .03. Factor loadings ranged from .26 to .79 (average loading = .57) with alphas from .41 to .79 (See Figure 1).

Figure 1

Measurement model for the IWBS.

Results for this seven factor solution were then compared to the structure of the six WBI domains, as stated in the third goal of this research. The Life Evaluation domain, as measured by the Cantril Self-Anchoring Scale (Cantril 1965) was added back into the overall IWBS model. This resulted in an eight factor solution (Life Evaluation, Emotional Health, Physical Health, Work Environment, Healthy Behaviors, and three different constructs related to basic access. In order to meet the third goal of this research, alignment of survey structure with the WBI (Gallup, Inc 2009) and based high correlations between the 3 basic access constructs, these three factors were combined into a single Basic Access domain for the remainder of the analysis in this research. The final IWBS domains were then identified as Life Evaluation, Emotional Health, Physical Health, Work Environment, Healthy Behaviors, and Basic Access.

Domain means, standard deviations, Cronbach’s alphas, and intercorrelations of IWBS domains are presented in Table 1. Cronbach’s alphas ranged from .41 to .79, indicating acceptable to good reliability. IWBS domain intercorrelations ranged from .07 to .78, with an average of .24. Correlations between the domains and the overall IWBS score ranged from .46 to .75. Correlations between the IWBS and WBI domains are presented in Table 2 and indicate that between 62% and 90% shared variance accounted for between the two domains (Cohen, 1988). Based on Cohen (1988), these would be considered large effect sizes. These results provide evidence of convergent validity for the IWBS and WBI.

Table 1 Descriptive statistics, coefficient alphas, and intercorrelations for IWBS domains

Table 2 Intercorrelations for IWBS and WBI domains

A MANOVA indicated a significant overall relationship between IWBS domains and self-ratings of Overall Health, F (24, 6539) = 31.69, p < .001, η² = .09. Follow-up Analysis of Variance (ANOVA) tests showed significant relationships between each IWBS domain and Overall Health (Table 3). Post-hoc comparisons revealed that those reporting ‘poor’ health scored consistently lower on all well-being domains than those reporting ‘good’, ‘very good’ and ‘excellent’ health. Compared to those reporting ‘fair’ health, those with ‘poor’ health scored lower on Life Evaluation, Physical Health, Emotional Health, and Basic Access domains. Those with ‘fair’ health reported lower well-being for all domains compared to those with ‘very good’ and ‘excellent’ health. In addition, those with ‘fair’ health scored higher on Life Evaluation, Emotional Health, Physical Health, Work Environment, and Basic Access domains when compared to those with ‘good’ health. Those reporting ‘good’ health scored lower on Life Evaluation, Emotional Health, Physical Health, and Healthy Behaviors domains than those with ‘very good’ and excellent’ health. Work Environment domain scores were lower for those reporting ‘good’ versus ‘excellent’ health. Basic Access scores were lower for those with ‘good’ versus ‘very good’ health. Life Evaluation domain scores were lower for those with ‘very good’ health compared to those with ‘excellent’ health.

Table 3 IWBS domains and overall health

The results of a MANOVA indicated a significant overall relationship between the 5 domains of IWBS and Life Evaluation F (10, 3754) = 55.44, p < .001, η² = .13. Follow-up ANOVAs showed significant relationships between each IWBS domain and Life Evaluation (Table 4). Post-hoc Tukey tests revealed that those classified as ‘thriving’ scored higher on all well-being domains compared to those classified as ‘suffering’ and ‘struggling’. Those ‘struggling’ reported higher well-being on Emotional Health, Physical Health, Work Environment, and Basic Access domains compared to those ’suffering’.

Table 4 IWBS domains and life evaluation

The results of a MANOVA found a significant overall relationship between IWBS and Behavior Risk categories, F (12, 3752) = 54.56, p < .001, η² = .15. Follow-up ANOVAs showed significant relationships between each IWBS domain and Behavior Risk category (Table 5). Post hoc comparisons revealed that Life Evaluation, Emotional Health, Healthy Behaviors, and Basic Access domains were higher for those with 0–2 risks compared to those with 3–4 risks and 5 or more risks. Life Evaluation, Emotional Health, Healthy Behaviors, and Basic Access domains were also higher for those with 3–4 risks compared to those with 5 or more risks. Physical Health was lower among those with 5 or more risks compared to those with 0–2 risks, while Work Environment was lower for those with 5 or more risks compared to those with 0–2 risks and 3–4 risks.

Table 5 IWBS domains and behavior risks

A stepwise mutilple regression analysis examined the relationship between the IWBS domains and depression as measured by the PHQ-2. Results indicate that depression scores were significantly associated with Emotional Health, Life Evaluation, Physical Health, and Basic Access (Table 6). The model was able to account for 48% of the variance in depression scores, F (4, 1879) = 435.43, p < .001.

Table 6 Multiple regressions of IWBS domains on PHQ-2

Discussion

The results of this research provided initial support for using the IWBS to assess well-being at the individual level. The IWBS met each of the criteria that we established for measurement development. First, the overall well-being score for the IWBS was highly correlated with the overall WBI score (r = .89) indicating that 79% of the variance in either scale can be accounted for or predicted by the variance in the other (Cohen, 1988). For the specific domains of well-being, the correlations between the IWBS and WBI were also high (range .94-.79). However, even Cohen cautions about using his levels of effects sizes as a frame of reference (Cohen, 1988). These results provide evidence of convergent validity for the IWBS and WBI.

As a second goal, we aimed to develop the IWBS using only items from the WBI to enable comparisons across measures. With a matched set of items using a new scoring technique, the IWBS is a subset of the WBI items which will allow the scores of samples of individuals to be compared with national data scored using the same method. In addition, future research can include the IWBS as an outcome measure for interventions by measuring pre- and post-test scores. The application of the scoring method is already creating insights about the relationship between well-being and healthcare cost and hospital visits, (Harrison, 2012) and the use of the measure to evaluate workplace wellness programs (Merrill, 2011). Furthermore, a recent randomized trial found that online and telephonic health coaching programs achieved significant improvements in well-being after 6 months. (Prochaska, 2012) This new research demonstrates the broad applicability of and need for an individual scoring method. In addition, Given the increased focus on health promotion and wellness among employers and in recent legislation (Compilation of Patient Protection and Affordable Care Act, as Amended Through May 1, 2010, 111th Congress, 2nd Session 2010), it is important that an individual measure of well-being impart a better understanding of how such initiatives benefit individuals within a population.

The third goal of this research was to ensure the IWBS and WBI have comparable factor structures and psychometric properties. Results indicated a five factor solution (physical health, emotional health, healthy behaviors, work environment and basic access) plus a sixth domain, life evaluation, based on the well-established Cantril scale (Cantril 1965). All fit indices indicated “good” to “excellent” model fit, with the exception of CFI of .87 which fell in the adequate/marginal range.

Lastly, the IWBS demonstrated favorable initial estimates of reliability as well as internal and external validity. The low correlations between IWBS domains indicate small to medium amount of shared variance (range 0.49% to 28%), indicating showed that while there were a series of significant relationships, each scale appears to be assessing a distinct construct. Only one correlation, between Life Evaluation and Emotional Health, showed a large effect size. In addition, the higher correlations with the overall IWBS score, which translate into between 21% and 56% shared variance, indicate that they assessing a similar higher construct based on Cohen’s definition of medium to large correlational effect sizes (Cohen, 1988). Overall, the Emotional Health scale had the largest correlations with the other domains, ranging from .53 with Life Evaluation to .19 with Healthy Behaviors. The Healthy Behaviors scale had the smallest correlations, ranging from .19 for Emotional Health and Life Evaluation to .10 for Work Environment and .07 for Physical Health.

External validity was evidenced by relationships between WBI domain scores and established measures of overall health, health-risk behaviors, and depression/anhedonia. Those who reported good overall health consistently reported higher well-being across the multiple domains of the IWBS. Those who reported numerous health-risk behaviors (such as smoking and poor stress management) and depression/anhedonia had lower well-being scores across domains. Future research should also examine the external vailidty of the IWBS domains across demographics such as ethnicity, marital status and gender.

Limitations

This research is limited by the cross-sectional design and will need to be followed by longitudinal research that assesses the predictive validity of the IWBS. Another limitation was the use of a sample of convenience and the use of self reported measures for determining external validity.

Conclusions

This research has established a new method for measuring well-being at the individual level. The IWBS is a valid measure that explains meaningful variance in well-being across individuals, and has the advantage of providing a new, more inclusive outcome measure for health and wellness programs and allowing use of the rich set of community WBI data to evaluate the well-being of individuals within and between regions or communities.

Author’s contributions

KE, JOP, JMP, CC and JEP conceived of the study. KE oversaw the implementation of the study, interactions with study participants and all data collection. PC conducted all statistical analysis with assistance from JJ. JOP, KE and PC drafted the manuscript. ER provided reviews and updates to the manuscript. All authors participated in the review of data analysis and approved the final manuscript.