Data reliability and econometric method¶
How reliable are official statistics (farm, strike, homeland, national accounts) and which estimation methods are appropriate?
Works engaging the debate: 26
Positions¶
- Official statistics flawed (farm census, strike data, homeland accounts)
- Methodological disputes (gravity outliers, HP filter vs SVAR, productivity indexes)
Evidence base¶
The debate is argued overwhelmingly from official South African series, read critically against one another. The Agricultural Census (Department of Statistics, covering only White, Coloured and Asian holders) is the most-used source. It underpins (Hendrie (1976a)) and Hendrie (1976b), which audit Census Nos. 39-46 for 1960-74. It is used by Simkins (1984a) and Simkins (1984b) for 1923-69 farm population, employment and wages at magisterial-district level. Knight (1976b) draws on it for 1953/4-1972/3 farm employment. Conradie et al. (2009) builds Tornqvist-Theil TFP indices from the 1952-2002 censuses for 31 Western Cape districts and nine regions. Liebenberg (2012b) rebuilds 1910/11-2010/11 output and input series from census reports, the Abstract of Agricultural Statistics, the Handbook of Agricultural Statistics 1904-1950, Union Statistics for Fifty Years and unpublished Directorate ledger files. The Population Census (1921-1980) is the second pillar. Simkins interpolates six census observations of the African farm population, and Knight (1976b) dissects census coding instructions for 1946-70. Savage (1986) relies on Simkins's corrected census distribution, and Hoffman and Todd (2000) uses 1991 census densities across 348 districts. Smaller-scale benchmarks come from farm surveys: Neethling's 109 Free State maize farms in 1927-28 (Neethling (1930)) and de Klerk's 61 Western Transvaal maize farms in 1980-81 (De Klerk (1984b)). Both are explicitly compared with census and Department of Agriculture figures to gauge sample bias.
Outside agriculture, the evidence divides by sub-question. Strike reliability rests on the official government strike series (strikes, workers involved, days lost, from 1917) and on newspaper-index counts. Beittel (1995a) and Beittel (1995b) use the WLG database of 2,306 Times (London) mentions and 487 New York Times mentions. A hand-coded file of 2,678 strikes from Work In Progress for 1972-91 (Descriptive statistics on the strikes... (n.d.)) is checked against ILO aggregates and GDELT. Homeland national accounts rest on BEPA, Department of Statistics and BENSO estimates. Abedian (1982) rebuilds Transkei's 1980/81 GDP from a 1981 business survey, Auditor-General accounts and district agricultural officers' returns. McGrath and Simkins (1985) audits Simkins's homeland income simulations against BMR household surveys (Venda 1978, KaNgwane 1980, Pietermaritzburg 1976) and BENSO 1982. Macro-econometric work uses SARB series: quarterly real GDP for 1960Q2-2006Q4 in Du Plessis et al. (2008a), flow-of-funds stocks from 1970 to 2003 in Aron and Muellbauer (2006), and land prices, CPI and capital flows for 1955-2003 in Obi (2006). The trade work (Evenett (2002a)) (Evenett (2002b)) (Evenett (2002c)) uses World Bank Trade and Production Database bilateral imports for 1978-99. Its sample is 8 industrialised importers and 31 middle-income exporters (248 pairs), with Haveman great-circle distances. Fedderke and Szalontai (2003) uses grouped Census of Manufacturing output distributions for 24 three-digit sectors, 1972-96. Kirsten et al. (2007) computes NRAs from domestic and border price series (SAGIS, FAOSTAT, IMF, Abstract).
Contested evidence¶
On farm statistics, the core dispute is between the two censuses. Simkins (1984a) and Simkins (1984b) show Agricultural Census farm-population counts falling far below Population Census estimates after 1955. They attribute the gap to unrecorded de facto African occupation of unoccupied farms, which is inferred from the discrepancy itself and supported only by the 1959 Commission and police evidence. They argue that the 1960 census did not undercount regular workers. Knight (1976b) finds the Agricultural Census, the 1969 SAAU survey and the 1970 Population Census mutually inconsistent, and concludes that inter-census comparisons of the economically active are worthless because definitions changed. Hendrie (1976a) shows casual employment swinging from 592,000 to 770,000 to 328,000 to 779,000, and the 1970/71 census maize marketing figure running 22% above reported production. Hendrie (1976b) treats the single August count and farmer-valued wages in kind as unrepresentative but offers no alternative data. De Klerk (1984b) finds survey permanent wages (R1,777 a year) above Department of Agriculture estimates and reads this as upward sample bias. It nonetheless uses 1978 census wage bills to put domestic workers at about R24 a month.
On productivity, identical commodity price-quantity data yield sharply different results depending on the index used. Liebenberg (2012b) obtains output growth of 2.75% a year with a Fisher index against 2.90% with the spliced-Laspeyres Thirtle index. Its MFP growth for 1945-2010 is 0.86% against 1.84%. It also argues that official machinery capital formation was up to 52.5% too high in 1981 and that fertilizer spending was understated by 7-44%. Williams (1996) independently re-computes the Thirtle index and argues that its rapid growth reflects a drought-year base. Conradie et al. (2009) works with census-based district indices but publishes no weights or series against which to test measurement error. Kirsten et al. (2007) reports NRAs more volatile than OECD estimates because negative values are retained.
On strikes, (Beittel (1995a)) and Beittel (1995b) find that the Shorter-Tilly rule applied to official strike numbers produces 27 wave-years, many of them false positives. Official days-lost and Times-based WLG indicators agree on 1922, 1947, 1973-74, 1980 and 1985-87. They diverge on the 1950s stay-aways, which official statistics exclude, and on 1919-20 and 1941-46, which the Times underreported. Descriptive statistics on the strikes... (n.d.) finds WIP and ILO counts agreeing on the 1973 surge but not on the 1981 jump.
Homeland accounts are contested at the level of basic inputs. Abedian (1982) finds migrant earnings of R402m against R574m, a 43% gap that destabilises Transkei GNP, and judges BEPA, Department of Statistics and BENSO estimates conceptually inconsistent. McGrath and Simkins (1985) puts homeland agricultural employment at 767,200 rather than Simkins's 413,600, and the two disagree on whether a chi-squared test against the BMR distribution was validly applied.
Methodologically, Evenett's EC sanctions effect falls from about a 50% cut on the full sample to about 27% once 10% tails are trimmed. US sanctions then emerge as the largest effect, cutting exports by about a third. The trimming rule is ad hoc. Du Plessis et al. (2008a) validates its SVAR-identified 30% fall in potential GDP only by visual matching to SARB and Du Plessis turning points. Fedderke and Szalontai (2003) finds the Gini rising in 22 of 24 sectors against only 7 on the Rosenbluth index.
Underused and missing sources¶
Several sources that could settle these disputes are rarely used. Farm-level and district-level census microdata, which (Liebenberg (2012b)) says are no longer accessible, would allow direct estimation of black and homeland production. They would also allow Simkins's undercount thesis and Hendrie's erratic casual-employment series to be tested.
Other sources are never used at all. These include Department of Manpower and Chamber of Mines strike records, police and employer archives, and the South African press for coding strike events independently of a secondary-literature chronology. Product-level COMTRADE data could separate sanctioned from unsanctioned goods, and the gravity work also lacks the World Bank trade database and FAOSTAT. Deeds Registry transaction data by buyer and province are missing for land prices. Representative homeland household income surveys for 1960 and 1970 are absent. Remote-sensed or measured degradation indicators would allow validation of the perception scores in Hoffman and Todd (2000).
Strength of key claims¶
| Basis | Share of key claims |
|---|---|
| Systematic data | 27% |
| Partial data | 56% |
| Illustrative or anecdotal | 13% |
| Asserted | 4% |
Comparative figures for all debates are on the Literature overview.
Best-evidenced works¶
Liebenberg (2012b), Simkins (1984b), Knight (1976b), Abedian (1982), Evenett (2002a), Kirsten et al. (2007)
Most-used primary sources¶
Holdings are matched by machine against the archive's file manifest; a dash means no match was found, not that the project certainly lacks the source.
Works and their positions¶
| Work | Position |
|---|---|
| Abedian (1982) | Abedian rejects the 'Black' versus 'non-Black' split in the official estimates as having no theoretical or practical justification, calls BENSO's figures inconsistent, and finds the Transkei government's revisions overstated. He builds his own estimates, with a 4-5% stated margin of error. |
| Abedian (1982) | He holds that the SNA is unsuited to LDCs but flexible enough to be adapted, and that a modified SNA remains the only workable framework. |
| Aron and Muellbauer (2006) | Argues South Africa's financial depth makes wealth effects relevant despite deep inequality, so market-value household balance sheets are needed; the authors take the Reserve Bank to be the right body to publish them. |
| Beittel (1995a) | Argues that the government statistics are poorly documented, incomplete before 1917, exclude the nominally independent homelands, and omit political stay-aways. The WLG press indicator is a better guide to disruptive turning points, despite British-press bias. |
| Beittel (1995a) | The strike-count method yields 27 wave years with many false positives. The days-lost method (Bordogna-Provasi) fits the stylized facts fairly well and is the best comparator for the WLG indicator. |
| Beittel (1995b) | Argues that strike counts give false positives. Working days lost is a fair proxy but tracks magnitude only. The WLG mentions indicator is the better guide to disruptive, turning-point unrest. |
| Beittel (1995b) | Sceptical: the series is poorly documented, incomplete before 1917, excludes the nominally independent homelands from the 1970s, and probably understates 1986 unrest. |
| Conradie et al. (2009) | Argues that aggregate national indices mask regional heterogeneity, citing Olmstead and Rhode's regional critique of Hayami-Ruttan induced innovation, and favours disaggregated district and regional indices with local policy advice rather than one-size-fits-all policy. |
| De Klerk (1984b) | Adopts the MLL, adjusted to exclude housing, transport, taxes and the like, and argues that average monthly income is misleading. Poverty should be assessed seasonally by comparing the regular cash-plus-food flow with the MLL. |
| Du Plessis et al. (2008a) | Argues for an SVAR long-run restriction approach that jointly yields potential output and a business-cycle measure, rather than the mostly HP-filter and production-function approaches used in earlier South African work. |
| Evenett (2002a) | Argues that outliers inflate estimated distance elasticities and sanction dummies, and that the elasticity falls by about half once outliers are excluded. |
| Evenett (2002b) | Argues that outliers inflate estimated distance elasticities and sanctions dummies. Trimming reduces the distance elasticity by about half. |
| Evenett (2002c) | Argues outliers inflate estimated distance elasticities and dummy coefficients. Trimming reduces the distance elasticity by about half, partly addressing the anomaly raised by Grossman (1998). |
| Fedderke and Szalontai (2003) | Uses both because the Herfindahl cannot be computed from the historical census data. Notes the Gini shows clearly rising concentration while the Rosenbluth is more mixed and may understate concentration because it is sensitive to small firms. |
| Hendrie (1976a) | Sceptical. Casual employment figures fluctuate implausibly, likely because farmers misread the 'casual labourer' category. Production returns conflict with marketing data. Employment and earnings cannot be independently checked. |
| Hendrie (1976b) | Argues against the Department of Statistics' reliance on an annual police-assisted census. Favours a less frequent census plus quarterly stratified sample surveys. |
| Hendrie (1976b) | Holds that census employment and earnings figures are too flawed to support firm conclusions on farm labour use or wage rates. |
| Hoffman and Todd (2000) | Defends the use of expert perceptions as reasonably balanced and partly corroborated by other studies. Concedes that no comprehensive reliability analysis was done and that the data are subjective. |
| Kirsten et al. (2007) | Allows negative NRAs and splits traded and non-traded fruit, where the OECD imposes zero. This yields greater volatility, and the authors caution that the estimates are volatile and indicative. |
| Knight (1976b) | Contends that census unemployment figures and the Economic Development Programme's methods are misleading, because of definitional changes and undercounting. Only wide ranges of estimates are defensible. |
| Liebenberg (2012a) | Argues earlier hybrid-Laspeyres 'Thirtle' indexes suffer aggregation bias and data errors, overstating output, MFP growth and the productivity share of output growth; the Fisher-based estimates are lower and track policy phases better. |
| Liebenberg (2012b) | Argues official series (Abstract of Agricultural Statistics) are inconsistent, exclude homeland production, double-count milk, omit horticulture before 1958 and understate fertilizer use; the earlier Thirtle et al. (1993) hybrid index overstates MFP growth. |
| Liebenberg (2012b) | Finds MFP growth of 0.86% a year 1945-2010 with the Fisher index against 1.84% with the Thirtle index; growth is much lower than earlier work and shows a slight contraction 1980-94. |
| McGrath and Simkins (1985) | McGrath says the urban MLL exaggerates poverty by ignoring rural-urban living cost differences. Simkins says a uniform convention is defensible because the rural MLL leaves non-market production unvalued. |
| McGrath and Simkins (1985) | McGrath says the simulation is untestable and not robust. Simkins says the chi-squared test was misapplied and uninformative for a synthetic first approximation, and that sensitivity analysis is complicated in a stochastic model. |
| Neethling (1930) | Prefers the survey method over complete cost accounts, since cost accounts represent only the most successful, specially trained farmers. Argues the 'law of averages' makes the survey representative. |
| Obi (2006) | Reviews both sides and uses level data with policy dummies, treating all series as non-stationary and differencing them. |
| Savage (1986) | Treats the figures as flawed orders of magnitude, noting census undercounts and official manipulation of figures, but uses them to show patterns. |
| Simkins (1984a) | Acknowledges that critics say farmer valuations are too high while others say many goods and services go uncounted; he uses farmer valuations and excludes the value of land and housing. |
| Simkins (1984b) | Holds that population is substantially undercounted after 1955 while employment is measured fairly well. Uses farmers' in-kind valuations, noting others claim they are too high or omit goods and services. |
| Williams (1996) | Disputes the Thirtle et al. and World Bank reading of productivity growth after 1983, noting that the 'rapid growth' uses the 1983/84 drought nadir as base, so weather explains it more simply than policy. |