4727.0.55.002 - Australian Aboriginal and Torres Strait Islander Health Survey: Users' Guide, 2012-13  
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 27/11/2013  First Issue
   Page tools: Print Print Page Print all pages in this productPrint All  
Contents >> Nutrition >> Under-reporting

This document was added or updated on 20/03/2015.

UNDER-REPORTING IN THE NATIONAL ABORIGINAL AND TORRES STRAIT ISLANDER NUTRITION SURVEY

There are a range of possible sources of error related to survey data. Of particular importance to nutrition surveys is a widely observed tendency for people to underestimate their food intakes1. This is called under-reporting and can include:

    • actual changes in foods eaten because people know they will be participating in the survey
    • misrepresentation (deliberate, unconscious or accidental), e.g. to make their diets appear more ‘healthy’ or be quicker to report.
Minimising under-reporting in dietary surveys

The Automated Multiple-Pass Method (AMPM) used in the 2012-13 National Aboriginal and Torres Strait Islander Nutrition and Physical Activity Survey (NATSINPAS) developed by the United States Department of Agriculture (USDA) and enhanced over several decades uses several methods to assist respondents to recall their food and beverage intake accurately and completely in order to reduce the amount of under-reporting in nutrition surveys. Even so, an evaluation of the performance of this instrument against an independent measure of intake (i.e. energy expenditure measured by doubly-labelled water) in 524 volunteers from the Washington DC area showed that although the AMPM accurately reported energy intakes for normal weight subjects, under-reporting remained an issue for overweight and obese subjects2.

Analysis of the results of the 2012-13 NATSINPAS and 2011-12 National Nutrition and Physical Activity Survey (NNPAS), suggests that, like other nutrition surveys, there has been some under-reporting of food intake by participants in these surveys. It appears that there is a greater propensity to under-report amongst the Aboriginal and Torres Strait Islander population for a variety of reasons including the difference in age and Body Mass Index (BMI) structures and this should be considered in any comparisons. (For more information on the under-reporting for the 2011-12 NNPAS see, Australian Health Survey Users Guide.)

ANALYSING UNDER-REPORTING

Identifying Under-reporters

A common approach used to identify under-reporters in surveys is to compare each person's reported energy intake (EI) with their basal metabolic rate (BMR) (predicted by their age, sex and weight) and assess whether the energy intake reported is plausible. BMR represents the amount of energy expended at rest over a 24-hour period by an individual (see Nutrient Intakes for method of calculation). The EI:BMR ratio provides an indication of whether the reported energy intake for one day is consistent with the energy intake required for a person to live a normal (not bed-bound) lifestyle. The habitual energy expenditure by an individual will exceed their BMR, mainly as a result of physical activity. It is therefore expected that habitual energy intake will be greater than BMR. A lower than expected EI:BMR value may indicate dieting, unusually low consumption or under-reporting of food consumption during the 24-hour reference period.3

BMR is commonly estimated for populations using the Schofield equations which predict energy requirements based on age, sex and body weight. However, investigations of the validity of these equations suggest that the Schofield equations may significantly over-estimate energy requirements in a range of populations 4.

In the absence of specific equations or techniques for estimating the Aboriginal or Torres Strait Islander populations and despite the limitations, the following analysis uses the Schofield equations. Brief analysis using the Oxford equations has also been undertaken and a summary is included at the end of this paper.

Low Energy Reporters

Goldberg et al. developed the use of the EI:BMR ratio as a method of establishing cut-off limits for determining those adults whose reported energy intakes were incompatible with the amount of energy required for a normally active but sedentary population (not sick, disabled or frail elderly). This is equivalent to a physical activity level or EI:BMR ratio of 1.55. Using this value, Goldberg et al. established a cut-off limit of 0.9 for EI:BMR for a plausible intake, which is the lower 95% confidence limit for a single day of data for a single individual, allowing for day-to-day variation in energy intakes, and errors in calculation of EI:BMR.3 In this analysis, results are for respondents aged 10 years and over only, as the Goldberg cut-off is not useful for growing children5.

Applying these methods shows a higher proportion of implausible intakes or low energy reporters (LERs) amongst Aboriginal and Torres Strait Islander people collected in the NATSINPAS than collected of the general population in the NNPAS.

Low Energy Reporters in 2011-2012 NNPAS and 2012-2013 NATSINPAS as classified by the Goldberg Cut-off of 0.9: Comparison by Age and Sex (a)

Males
Females

Age
NATSINPAS
%
NNPAS
%
Percentage points difference
NATSINPAS
%
NNPAS
%
Percentage points difference

10-18
23
16
7
26
19
7
19-30
29
16
13*
34
26
8
31-50
29
20
9*
36
23
13*
51 and over
36
21
15*
32
23
9*
All 10 and over
29
19
10*
33
23
10*

(a) Respondents for whom a measured weight or height were not available, children under 10 years of age, and pregnant women (where pregnancy status was known) were excluded from all analyses based on EI:BMR For the NNPAS breastfeeding women were also excluded. All proportions are given as a weighted proportion of respondents for whom EI:BMR could be calculated.
(b) * Represents fields where there is a statistically significantly different between NATSINPAS and NNPAS.

As in past studies, including the 2011-12 NNPAS, a higher proportion of NATSINPAS females than males were identified as low energy reporters (LERs). Also similar to previous studies, the proportion of LERs increased with increasing BMI for both males and females. For males, the proportion of LERs was smallest for those aged 10-18 (23%) and largest for older males 51 years and over (36%). For females, the age pattern varied with the youngest age group (10-18 years) having the lowest proportion of LERs, followed by females aged 51 years and over and 19-30 years.

When compared to the 2011-12 NNPAS, the 2012-13 NATSINPAS appears to have a higher proportion of LERs for both males and females, with one in three NATSINPAS females (33%) over the age of 10 identified as an LER (compared to 23% for NNPAS females) and one in three NATSINPAS males (29%) over the age of 10 identified as an LER (compared to 19% for NNPAS males). For both males and females, in all age groups over 30, there were a higher proportion of LERs in the NATSINPAS population than the NNPAS population.

Low Energy Reporters in 2011-2012 NNPAS and 2012-2013 NATSINPAS as classified by the Goldberg Cut-off of 0.9: Comparison by BMI and Sex (a)

Males
Females

BMI
NATSINPAS
%
NNPAS
%
Percentage points difference
NATSINPAS
%
NNPAS
%
Percentage points difference

Normal
17
10
7*
23
16
7
Overweight
28
18
10*
31
23
8*
Obese
41
34
7
44
37
7

(a) Respondents for whom a measured weight or height were not available, children under 10 years of age, and pregnant women (where pregnancy status was known) were excluded from all analyses based on EI:BMR. All proportions are given as a weighted proportion of respondents for whom EI:BMR could be calculated.
(b) * represents fields where there is a statistically significant difference between NATSINPAS and NNPAS
    Obese people were more likely to be LERs than either normal or overweight people with around two in five obese people being LERs for both males and females in 2012-13. This rate is similar to the NNPAS population for both males and females. However, a higher proportion of normal and overweight NATSINPAS males were LERs than males in the NNPAS population (17% compared to 10% for normal weight males and 28% compared to 18% for overweight males). The same is true for overweight NATSINPAS females, where 31% were LERs compared to 23% in the NNPAS population.

    2012-13 NATSINPAS energy intakes compared with the 2011-12 NNPAS

    A comparison of energy intakes between the two surveys is another way of analysing the comparative amount of under-reporting in the two surveys.

    The following graphs show how the whole distributions of reported energy intakes are relatively consistent between the two surveys. For example, the percentage of reported energy intakes of less than 8,000kJ was relatively constant for females at 63% for both surveys. At the same time, while 64% of NATSINPAS males reported energy intakes of less than 10,000kJ, this percentage was slightly lower (60%) for NNPAS.





    The table below shows reported mean energy intakes do not differ significantly between NATSINPAS and NNPAS for most age and sex groups.

    Mean Energy Intake in 2011-12 NNPAS and 2012-2013 NATSINPAS: Comparison by Sex and Age

    Males
    Females

    Age
    NATSINPAS
    NNPAS
    Difference
    NATSINPAS
    NNPAS
    Difference

    Mean energy intake (kJ)
    %
    Mean energy intake (kJ)
    %

    10-13
    8,698
    9,391
    -692
    -8
    8,236
    7,928
    308
    4
    14-18
    9,669
    10,186
    -517
    -5
    7,498
    8,126
    -628
    -8
    19-30
    10,620
    11,004
    -384
    -4
    7,699
    7,761
    -62
    -1
    31-50
    10,159
    10,220
    -60
    -1
    6,939
    7,481
    -542
    -8*
    51 and over
    7,852
    9,044
    -1,191
    -15*
    6,694
    7,066
    -372
    -6
    All 10 and over
    9,664
    9,934
    -270
    -3
    7,328
    7,451
    -123
    -2

    Footnote: *represents fields where there is a statistically significant difference between NATSINPAS and NNPAS

    There was no difference in mean energy intakes between NATSINPAS males and NNPAS males, with the exception of males aged 51 years and over where NATSINPAS males reported lower mean energy intakes than NNPAS males (7852 kJ compared with 9,044 kJ). For females mean energy intakes were similar for NATSINPAS and NNPAS females. For females aged 31-50 years, however, mean energy intakes for NATSINPAS females were lower than those for NNPAS females (6,939 kJ compared with 7,481 kJ).

    EI:BMR Ratio in 2011-12 NNPAS and 2012-2013 NATSINPAS: Comparison by Sex and Age (a)

    Males
    Females

    Age
    NATSINPAS
    NNPAS
    Difference
    NATSINPAS
    NNPAS
    Difference

    Mean energy intake (kJ)
    %
    Mean energy intake (kJ)
    %

    10-18
    1.31
    1.43
    -0.12
    -9
    1.38
    1.37
    0.01
    1
    19-30
    1.28
    1.41
    -0.13
    -10
    1.18
    1.31
    -0.13
    -11*
    31-50
    1.31
    1.32
    -0.01
    -1
    1.13
    1.29
    -0.16
    -14*
    51 and over
    1.09
    1.30
    -0.21
    -19*
    1.17
    1.28
    -0.11
    -9*
    All 10 and over
    1.27
    1.35
    -0.08
    -6*
    1.21
    1.30
    -0.09
    -7*

    (a) Respondents for whom a measured weight or height were not available, children under 10 years of age, and pregnant women (where pregnancy was known) were excluded from all analyses based on EI:BMR.
    Footnote: *represents fields where there is a statistically significant difference between NNPAS and NATSINPAS
      For females, the differences in EI:BMR ratio between the NATSINPAS and NNPAS populations indicate the level of under-reporting was higher for NATSINPAS females in all age groups except those aged 10 to 18 years. For all males aged 10 years and over the EI:BMR ratio is also lower when comparing NATSINPAS and NNPAS males, however the only age group for males where the EI:BMR ratio was different between NATSINPAS and NNPAS was for those aged over 51 years. This suggests that overall, assuming that there are no differences in physical activity between the NATSINPAS and NNPAS populations, the level of under-reporting was higher for NATSINPAS males and females.

      Factors associated with low energy reporting

      The most consistent characteristics associated with low energy reporting in past studies are that females under-report more than males, and obese/overweight respondents report less food/energy than those of normal weight status, when they in fact consume more food/energy1.

      The following tables show mean energy intakes and mean EI:BMR ratios by BMI. Earlier in the paper it was shown that, when comparing to the NNPAS population, the proportion of LERs was higher for NATSINPAS males who were normal and overweight and females who were overweight. Below, mean energy intakes are lower for NATSINPAS females who are overweight compared to NNPAS females; however they are not different for the remaining groups or for males.

      Mean Energy Intake in 2011-12 NNPAS and 2012-2013 NATSINPAS: Comparison by Sex and Age

      Males
      Females

      BMI
      NATSINPAS
      NNPAS
      Difference
      NATSINPAS
      NNPAS
      Difference

      Mean energy intake (kJ)
      %
      Mean energy intake (kJ)
      %

      Normal
      10,106
      10,595
      -489
      -5
      7,523
      7,849
      -326
      -4
      Overweight
      10,199
      9,888
      311
      3
      7,014
      7,538
      -524
      -8*
      Obese
      9,241
      9,450
      -209
      -2
      7,289
      7,148
      141
      2

      Footnote: *represents fields where there is a statistically significant difference between NATSINPAS and NNPAS

      EI:BMR Ratio in 2011-12 NNPAS and 2012-2013 NATSINPAS: Comparison by Sex and Age (a)

      Males
      Females

      BMI
      NATSINPAS
      NNPAS
      Difference
      NATSINPAS
      NNPAS
      Difference

      Mean energy intake (kJ)
      %
      Mean energy intake (kJ)
      %

      Normal
      1.46
      1.54
      -0.08
      -6
      1.35
      1.42
      -0.07
      -5
      Overweight
      1.31
      1.3
      0.01
      1
      1.15
      1.27
      -0.12
      -10*
      Obese
      1.01
      1.12
      -0.11
      -11*
      1.03
      1.07
      -0.04
      -4

      (a) Respondents for whom a measured weight or height were not available, children under 10 years of age, and pregnant women (where pregnancy was known) were excluded from all analyses based on EI:BMR.
      Footnote: *represents fields where there is a statistically significant difference between NNPAS and NATSINPAS

      When considering the EI:BMR ratio, obese NATSINPAS males and overweight NATSINPAS females both had lower EI:BMR ratios than obese NNPAS males and overweight NNPAS females respectively. However for the majority of BMI categories the EI:BMR ratio was the same between NATSINPAS and NNPAS males and females.

      How much energy is potentially missing?

      It is difficult from the available data to accurately estimate how much energy might be missing from the intakes reported by respondents in 2012-13 NATSINPAS. One approach to estimate the effect of under-reporting in nutrition surveys has been to look at the difference between overall mean energy intakes, and energy intakes after removal of LERs (i.e. the difference between all intakes and intakes for plausible reporters only). This approach assumes that removal of LERs is sufficient to remove under-reporting bias from the sample. In the table below this method shows increases in mean energy intakes of 20% for males and 22% for females in 2012-13 when the LERs are removed.

      Comparison of Mean Energy Intakes (kJ) for All Respondents (a), and after removal of Low Energy Reporters (b)

      Males
      Females

      All respondents
      Plausible respondents
      Difference
      All respondents
      Plausible respondents
      Difference

      Energy (kJ)
      %
      Energy (kJ)
      %

      NATSINPAS
      9,664
      11,602
      1,939
      20
      7,328
      8,906
      1,578
      22
      NNPAS
      9,934
      11,078
      1,145
      12
      7,497
      8,598
      1,101
      15

      (a) Energy intakes presented are for all respondents aged 10 years and over. All energy intakes include the contribution from fibre so will not match previously published figures for the 1995 NNS.
      (b) Those respondents for whom an EI:BMR ratio was not calculated were also removed, so figures given are specifically for plausible energy reporters only. Respondents for whom a measured weight or height were not available, children under 10 years of age, and pregnant women (where pregnancy status was known) were excluded from all analyses based on EI:BMR.


      However, the Goldberg cut-off, when used to identify low energy reporters based on a single day of intake and without overall energy expenditure from physical activity, has been estimated to find only about half of all actual under-reporters6. That is, after making the conservative assumption that the person is sedentary, and allowing for day-to-day variation in intakes and errors in calculating BMR, the cut-off only classifies someone as an under-reporter when their reported intake is so low that it is implausible (with 95% confidence) for their age, sex and weight7. Therefore, removal of LERs from the dataset does not remove all under-reporting bias8.

      Given that the mean EI:BMR ratio for plausible energy reporters for males (1.52) and for females (1.48) in the NATSINPAS are both below the conservative minimum energy requirement of 1.55 for a normally active but sedentary population7, the removal of LERs does seem likely to underestimate the potential bias. Another method of estimating the bias, the mean amount of energy required for each individual to achieve an EI:BMR ratio of 1.55, is presented in the table below. This table highlights the difference in the energy intakes that are required to reach a plausible intake for NATSINPAS males and females.

      On average, an increase in mean energy intake of 24% for males and 31% for females would be required to achieve an EI:BMR ratio of 1.55 and greater increases are required for overweight and obese people than those of normal weight.

      Estimation of Under-reporting Bias for Energy: Mean Energy to Achieve an EI:BMR of 1.55 (a) (b)

      Males
      Females

      Mean Energy IntakeMean Energy Deficit (kJ) to Achieve an EI:BMR of 1.55Deficit as a % of reported energyMean Energy IntakeMean Energy Deficit (kJ) to Achieve an EI:BMR of 1.55Deficit as a % of reported energy

      NATSINPAS
      9,664
      2,491
      24
      7,328
      2,299
      31
      NNPAS
      9,934
      1,672
      17
      7,497
      1,573
      21

      (a) Energy intakes presented are for all respondents aged 10 years and over.
      (b) Respondents for whom a measured weight or height were not available, children under 10 years of age, and pregnant women (where pregnancy status was known) were excluded from all analyses based on EI:BMR.


      The graph below shows that respondents in all BMI categories, on average, reported less than the conservative minimum energy requirement with the largest deficits for obese males and females.


      (a) Respondents for whom a measured weight or height were not available, children under 10 years of age were excluded from all analyses based on EI:BMR.
      (b) Line at EI:BMR = 1.55 indicates minimum average energy requirement for a normally active but sedentary population (not sick, disabled or frail elderly).


      For all age groups, males and females, on average, reported less than the conservative minimum energy requirement.


      (a) Respondents for whom a measured weight or height were not available, children under 10 years of age were excluded from all analyses based on EI:BMR.
      (b) Line at EI:BMR = 1.55 indicates minimum average energy requirement for a normally active but sedentary population (not sick, disabled or frail elderly).


      Additional considerations associated with understanding NATSINPAS under-reporting

      In addition to the analysis above, food security and remoteness were considered as other possible factors associated with under-reporting.

      Running out of food may be a factor which could influence the pattern of energy reporting, as those who ran out of food may genuinely have had low consumption. Over one in five Aboriginal and Torres Strait Islander people (22%) were living in households which reported running out of food in the last 12 months and being unable to afford to buy more including 7% who were in households where someone went without food. (This compares with 4% of people in the 2011-12 NNPAS who were in households which ran out of food and 1.5% who went without.)

      Although this may account for some of the under-reporting, food security does not account for the full extent of under-reporting in the NATSINPAS, as only 35% of those who ran out of food were also LERs. While over one in five (23%) of LERs were in households which ran out of food in 2012-13, the same proportion of people (22%) who had plausible intakes also reported being in households which ran out of food.

      Remoteness was also investigated, however there was no difference between remote and non-remote in relation to under-reporting.

      Summary of analysis using the Oxford equations

      A brief analysis was undertaken using the Oxford Equations in place of the Schofield equations to estimate BMR. As with the analysis earlier in the paper, the Goldberg cut-off of 0.9 was applied (which is equivalent to a physical activity level or EI:BMR ratio of 1.55).

      The analysis showed that with the Oxford equations, less people in both the NATSINPAS and NNPAS populations were considered to be low energy reporters than when the Schofield equations were used. Accordingly, energy required to achieve an EI:BMR ratio of 1.55 was smaller for both surveys. The table below highlights the difference in the energy intakes that are required to reach a plausible intake for males and females when using the Schofield and Oxford equations to calculate BMR.

      Estimation of Under-reporting Bias for Energy: Mean Energy to Achieve an EI:BMR of 1.55 Schofield Equations and Oxford equations (a) (b)

      Schofield EquationsOxford Equations

      Mean Energy IntakeMean Energy Deficit (kJ) to Achieve an EI:BMR of 1.55Deficit as a % of reported energyMean Energy Deficit (kJ) to Achieve an EI:BMR of 1.55Deficit as a % of reported energy

      MalesNATSINPAS
      9,664
      2,491
      24
      2,247
      23
      NNPAS
      9,934
      1,672
      17
      1,490
      15
      FemalesNATSINPAS
      7,328
      2,299
      31
      2,032
      28
      NNPAS
      7,497
      1,573
      21
      1,330
      18

      (a) Energy intakes presented are for all respondents aged 10 years and over.
      (b) Respondents for whom a measured weight or height were not available, children under 10 years of age, and pregnant women (where pregnancy status was known) were excluded from all analyses based on EI:BMR.


      However, as with the Schofield equations, analysis using the Oxford equations also showed that there were a higher proportion of LERs and that more energy was ‘missing’ from the NATSINPAS population when compared to the NNPAS population.

      SUMMARY

      Key findings of this analysis include:
        • It is likely that under-reporting is present in both surveys.
        • There appears to be a higher rate of under-reporting in the NATSINPAS than in the NNPAS for both males and females.
        • Under-reporting appears to be more prevalent amongst females than males.
        • Under-reporting appears to increase with age.
        • Under-reporting appear to increase as BMI increases.
        • In order for each member of the population to achieve an Energy Intake to Basal Metabolic Rate Ratio (EI:BMR) of 1.55 which is the ratio expected for a normally active but sedentary population, an increase in mean energy intake of 24% for Aboriginal and Torres Strait Islander males and 31% for Aboriginal and Torres Strait Islander females is required and greater increases are required for overweight and obese people than those of normal weight. This compares with 17% for males and 21% for females in NNPAS.
        • Given the association of under-reporting with overweight/obesity and consciousness of socially acceptable/desirable dietary patterns, under-reporting is unlikely to affect all foods and nutrients equally.

      (For more information on the under-reporting for the 2011-12 NNPAS see, Australian Health Survey Users Guide.)

      Taken together, this analysis shows that, as for other nutrition collections, under-reporting should be kept in mind when considering the results of the NATSINPAS collection, particularly in comparison with results from the NNPAS. It must be noted that there is still further work that could be conducted in this area, particularly in relation to the under-reporting of particular food groups.

      ENDNOTES

      1 Macdiarmid J and Blundell J 1998, ‘Assessing dietary intake: Who, what and why of under-reporting’, Nutrition Research Reviews, 11, pp 231-253. doi:10.1079/NRR19980017, Available from <http://www.ncbi.nlm.nih.gov/pubmed/19094249>.
      2 Moshfegh AJ, Rhodes DG, Baer DJ, et al 2008, 'The US Department of Agriculture Automated Multiple-Pass Method reduces bias in the collection of energy intakes', The American journal of clinical nutrition, 88(2), 324-332, Last accessed 05/05/2014, <http://ajcn.nutrition.org/content/88/2/324.full.pdf+html>.
      3 Goldberg GR, Black AE, & Jebb SA et al 1991, ‘Critical evaluation of energy intake data using fundamental principles of energy physiology: 1 Derivation of cut-off limits to identify under-reporting’, European Journal of Clinical Nutrition, vol 45, pp.569-581.
      4 Henry, CJK 2005,'Basal metabolic rate studies in humans: measurement and development of new equations', Public Health Nutrition, 8(7A), pp 1133–1152
      5 Gibson RS 2005, Chapter 5: 'Measurement errors in dietary assessment', Principles of Nutritional Assessment Second Edition, Oxford University Press, p.168.
      6 Black AE 2000, ‘The sensitivity and specificity of the Goldberg cut-off for EI:BMR for identifying diet reports of poor validity’, European Journal of Clinical Nutrition, vol. 54, pp395-404.
      7 Black AE 2000, ‘Critical evaluation of energy intake using the Goldberg cut-off for energy intake: basal metabolic rate. A practical guide to its calculation, use and limitations’, International Journal of Obesity, vol. 24, pp1119-1130.
      8 Gibson RS 2005, Principles of Nutritional Assessment Second Edition, Oxford University Press, p.121.



      Previous PageNext Page