Skip to content

Data Discovery: A rough guide to microdata in Brazil, China, India and South Africa

Literature › Works

Iain Stewart (2007). Data Discovery: A rough guide to microdata in Brazil, China, India and South Africa. ESRC International Data Forum scoping studies summary. Economic and Social Research Council (ESRC), Swindon.
Type report
Scope Comparative and methods
Method synthesis review
Period studied 1872–2007
Regions Brazil, China, India, South Africa
Themes methods data sources, comparative theory
Citation key stewart2007

Argument

This ESRC booklet summarises four scoping studies of microdata in Brazil, China, India and South Africa. It argues that sharing microdata with bona fide researchers improves research and policy, and that cross-border access is hampered by language, the lack of a central archive, confidentiality worries, and legal and cultural barriers. It argues through country overviews and tables of datasets, not through any historical analysis. The South African section lists over 40 accessible datasets, held mainly by Stats SA, the HSRC, SADA and DataFirst, and notes that administrative microdata are generally not released.

Evidence and method

Four commissioned scoping reports, drawing on expert knowledge and interviews (57 people for South Africa, 59 for China), plus tables of datasets and contact details. The South African datasets are mostly post-1990s surveys (census 2001, October Household Survey, Labour Force Survey, KwaZulu-Natal Income Dynamics Study, etc.).

Findings

  • South Africa has a large number of easily accessible datasets, mostly held by a few organisations. (p. 10)

    There are over 40 key South African datasets that are easily accessible.

  • Stats SA's main microdata sources are the census and household surveys, and most of what is publicly downloadable is macrodata. (p. 10)

    The main microdata sources are the Population Census and household surveys.

  • Individual-level administrative data in South Africa are generally unavailable to outside researchers because of confidentiality, so access must be negotiated case by case. (p. 10)

    The data at an individual level are not routinely made available for researchers outside the government because of concerns about confidentiality.

  • The report recommends a single South African data repository, modelled on the UK Data Archive, to reduce duplication. (p. 10)

    It would be extremely useful for researchers if all data were held in one repository, which would also reduce the duplication of effort

  • The labour-market datasets that South Africa makes accessible include the October Household Survey and the Labour Force Survey; the listed series are post-apartheid. (p. 11)

    Labour Force Survey Contact Stats SA User Information Services by email: info@statssa.gov.za

  • The report defines microdata and macrodata. (p. 3)

    Microdata are data about individual objects such as persons, companies, events and transactions. Macrodata are data aggregated to a country or regional level.

  • The report names language, the lack of a central repository, confidentiality and legal/cultural factors as obstacles to international data sharing. (p. 12)

    The four scoping studies highlighted some of the obstacles and suggested ways forward.