Measuring wellbeing – lessons from the Boer War

‘There are three kinds of lies: lies, damn lies and statistics’. The saying’s used when numbers have been used to bolster weak arguments, or when expressing scepticism about numbers more generally.

Sometimes statistics can be robust and help to tell a compelling story. But it’s also good to be reminded that at other times, numbers should be approached with caution. That was one of the lessons from a great presentation I attended last week by Geoffrey Brooke and Lydia Cheung on the relationship between height and wellbeing in Boer War soldiers at the AAHANZBS¹ conference hosted at AUT.

Brooke and Cheung’s study reports on the height of New Zealand soldiers who served in the Second Boer War (1899-1902) and then in WWI (1914-1918). They were interested to find out whether the height of New Zealand soldiers increased or decreased between the two periods, which would give an indication of changes in living standards.

What are anthropometric measures and why are they useful?

Adult height is what is known as an anthropometric measure, and includes other measures like weight, infant mortality and life expectancy. Height however is widely used as evidence of standard of living in historical studies because “heights reflect the outcome of numerous inputs, including maternal health, nutrition, work intensity, and exposure to disease. Adult height also predicts health outcomes, as being taller is associated with higher life expectancy and lower rates of disease” (p. 5).

Anthropometric measures have a number of advantages over income-based measures of the standard of living, including that they are more reliable or more readily available than data on wages or incomes in historical studies. Importantly, they also measure outcomes directly, while measures like GDP and real wages only measure inputs.

Brooke and Cheung point out that while debates continue on the relative importance of using income as a measure for wellbeing (compared to other measures such as spending on public health and health care), overall they reinforce the argument that outputs are a better measure of the standard of living than inputs. 

Exercising caution

There is however a problem with using heights, especially the heights of soldiers, to measure living standards. It’s that young soldiers are more likely to say that they’re older than they are in order to enlist. Brooke and Cheung find that dishonest reporting of age is “prevalent… [with] more than 35% of those who reported their age as either 19 or 20 [being] younger than that age” (p. 18).

For example, in the figure below, you can clearly see a skew whereby younger soldiers over-report their age, especially for those under 20. Older soldiers go the other way and are more likely to under-report their age (although there are considerably more soldiers below 30 than over 30). The consequence of this deception is that it makes soldiers as a cohort seem shorter than they actually are – as a 17 year old male is generally shorter than a 19 or 20 year old male.

Brooke and Cheung get around this problem using an ingenious matching solution. First, they identify all 10 contingents of soldiers that participated in the Boer War. Then they match their enlistment records with records in the New Zealand Births, Deaths, and Marriages Online (BDM) database. By doing this, they manage to identify the actual ages, rather than reported ages, of 53% of the soldiers in their sample.

After this matching process they find that, unlike what the reported age data indicated – that soldiers were getting shorter (indicative of a lowering of living standards) – there was actually no decline in height when soldiers’ true ages were used.

Cross-country comparison

The next step in the analysis was to look at other countries, and to extend the research to include WWI enlistments in order to look at living standards over time. The figure below shows striking differences between heights for Australian and Canadian soldiers, but not so much for New Zealand soldiers.

Brooke and Cheung explain that this result is down to differences in the selection criteria for recruits, as well as approaches to data analysis (using actual means for Boer War heights and truncated regressions for WWI heights). In summary, they explain that if one was to 

accept the hypothesis that the differences in selection criteria between the Boer War and WWI were binding, and that young soldiers lied about their age in sufficient numbers to push mean heights downwards, then the evidence in figure 4 is consistent with average male heights in New Zealand rising in the last decades of the nineteenth century, including for those born in the 1890s (p. 23)

Take-outs

I’ve got a couple of take-outs from this research:

• First, that output measures of wellbeing are important, and likely to tell a fuller story than just using inputs. And that this is just as apt for historical analysis as how we think about measuring wellbeing today.
• Second, that it’s important to be careful with data! It’s always worth delving deeply into data sources, and to beware of measurement errors, in order to understand the story behind the numbers.

Note
1. Academic Association of Historians in Australian and New Zealand Business Schools. I was attending and presenting, along with Guanyu Zheng, New jobs, old jobs: The evolution of work in New Zealand’s cities and towns. See presentation slides.

Photo
Marlborough members of the NZ 4th contingent, known as the Rough Riders, pose ahead of their March 1900 deployment on the SS Monowai for the Boer War in South Africa. Marlborough Historical Society Collection.