Another key source of data is the Quality of Life Survey that was carried out in September and October 2002.
Data breakdowns are provided in the following order in this report where possible and appropriate:
- Total New Zealand trends
- Differences between the Eight Cities Total (combined total for the eight cities) and Rest of New Zealand ( New Zealand total minus the eight largest cities)
- Individual Eight Cities trends (data are presented in the following geographical order: North Shore , Waitakere, Auckland , Manukau, Hamilton , Wellington , Christchurch and Dunedin )
Trend data have been included in the report where possible, with many indicators incorporating time series data back to 1991. Where possible and/or appropriate, indicators are broken down by ethnicity, age and sex. Identification of key sub-groups within the cities is important for understanding quality of life issues. These can be obscured when analysing total aggregate data.
The following 10 selection criteria were used to assess the applicability of indicators:
- Relevant (to social, environmental and economic outcomes in New Zealand 's largest cities)
- Measurable (quantifiable, with data existing to measure it)
- Cost effective (obtainable at a reasonable cost in terms of time and financial resourcing)
- Valid (providing a true reflection or measure of the issue; scientifically credible or otherwise defensible)
- Comparable (able to be standardised or accurately compared with similar indicators)
- Understandable (easily understandable, able to be presented in a simple and appealing way to target audiences)
- Responsive (responsive to changing conditions)
- Time related (repeatable, showing trends over time)
- Disagggregation (able to be disaggregated or broken down by demographic and other characteristics)
- Leading/lagging (providing leading indicators to give early warning or predictors of change; providing lagging indicators to show the effects or outcomes)
While these criteria were applied to all potential indicators highlighted for inclusion in the project, significant data issues dictated the makeup of the final key indicator set. There were trade-offs between criteria. For example, comparable data sets on some social issues were not available from secondary data sources so we had to undertake a joint Quality of Life Survey, which had significant cost implications for the project.
The production of this report once again highlighted significant data management issues, many of which could not be resolved during production of this report. These data issues have impacted on the types of indicators selected for inclusion in this report, resulting in the omission of some indicators, the use of proxy measures for other indicators and in the need for caution in interpreting and analysing others. Despite data challenges, some issues were deemed so important to the analysis of quality of life in larger cities that an indicator commentary was provided where it was not possible to include a concrete measure. However, this compromise impedes the monitoring of trends over time and thus the ability to determine whether some things are getting better or worse in larger cities. Indicators where commentary rather than measure analyses are provided include: soil quality, biodiversity, natural and cultural heritage preservation, immunisation, diabetes and smoking.
|On a positive note, cost of accessing data, which proved to be a considerable challenge for our 2001 report, was not a major issue for the 2003 report preparation. This is largely a result of recent changes in data charging policies by Statistics New Zealand.
- While considerable amounts of information and data exist across many areas and issues, there are still significant challenges around the availability, and therefore usefulness, of some data sets.
- The main problem is the presence of data gaps, with relevant information on some issues being unavailable at sub-national levels and at times not even available at a national level. Examples of issues where key information gaps occur at either the national or sub-national level that impacted on our reporting: soil quality, biodiversity, debt and savings levels, diabetes and obesity, mental health outcomes, physical disabilities, immunisation, housing quality, poverty lines, truancy, police response times and adult literacy.
- Limited data are available in some areas where disaggregation by ethnicity would be desirable. This is in part related to survey sampling issues. Survey samples in some of New Zealand 's official surveys are too small to allow ethnicity breakdowns at a city or regional level. Ethnicity breakdowns of sub-national population estimates are not currently produced by Statistics New Zealand, which means ethnicity analyses of data collected outside of the five yearly Census periods are difficult and at times impossible to undertake. Reliable ethnicity data are needed as our populations become more ethnically diverse. For example, more information is needed on economic activity of Maori, on ethnicity of elected representatives on local councils and on social outcomes for various ethnic groups.
- Another data availability issue encountered by the project team was a lack of robust time series data on some issues. Where data did exist, it was often in the form of a one-off survey, which didn't allow for ongoing monitoring. Where adequate time series data did not exist, some indicators had to be excluded from the report, or alternative measures were sought. For example, we were unable to get robust breakdowns on time use around voluntary and unpaid work. The New Zealand Time Use Survey was a one-off sample survey that did not provide sub-national data and changing definitions of unpaid and voluntary work across Census periods made comparisons over time impossible. An obesity measure was excluded because no sub-national breakdowns were available. Additional gaps in time series data exist for issues such as sense of belonging, trust in others and in public institutions, tolerance of diversity, social support networks and social isolation. The joint Quality of Life Survey, undertaken for the first time in 2002, may address some of these gaps at a city level over time.
- A lack of data on quality of life dynamics prevented us from providing measures that tell us about drivers and outcomes of changes in circumstances over time. Most current indicator measures instead provide data for snapshots in time. For example, what drives the movement of people into and out of poverty situations? The collection of time series data and indicators of quality of life dynamics can be costly, but are a necessary component of monitoring complex interrelationships in quality of life issues.
- The date at which some data sets are made available publicly also proved to be an issue for our report, particularly in the health chapter where much of the data available to us was at least three years old. Data age can be critical where phenomena change dramatically over short time periods. For example, the most recently available suicide statistics we could secure were for 1999. Yet there have been significant reductions in suicide deaths, particularly in younger people, in the intervening three and a half years.
- Inconsistencies in data collection made it difficult for us to report indicator results across different locations. Different methods and/or measures are being used at times to collect data. This is particularly an issue in environmental monitoring by local authorities. For example, there is currently no set standard for measuring and reporting beach water quality data and a similar situation occurs with open space data where different types of open space are recorded in various local authority tallies. Limited coordination of environmental monitoring between councils has exacerbated this problem. Lack of data comparability across local authorities is also an issue for building consents data.
- Adding to consistency challenges, changing definitions and measurement conventions have created difficulties for the monitoring of long term trends. For example, changes in Census ethnicity classifications mean that 2001 data cannot be compared with 1996 data. Further, ethnicity classifications are not consistent across data collection agencies.
- Inconsistencies in geographical boundaries used to report data have also caused difficulties for our team. A mismatch between central and local government boundaries was particularly an issue with data from the Police and some data from the Ministry of Social Development.
It is important to bear in mind some general cautions about data when reading the analytical interpretations included in this report.
- The above data challenges have shaped the selection and analysis of indicators for this report. Where no suitable indicator exists for an issue, we have either excluded the indicator, used proxy measures or included a commentary.
- The population bases of each of the eight largest cities vary considerably. Caution must be exercised when interpreting and comparing percentages across the cities as the base numbers may be quite different. To aid comparability, percentages are calculated as a proportion of the population in the city rather than as a percentage of all cities.
- People and groups at the extremes may not always be reflected in official statistics, such as income data. This is especially the case for marginalized people, who often remain uncounted or undercounted. Accurate data monitoring of the experiences of these groups is critical for successful policy and programme intervention.
- Indicator data presented in this report are disaggregated by ethnicity, age, sex and geographic location to help identify important sub-groups for analysis. However, our report does not identify variations within these sub-groups. Differences within groups can at times be greater than differences between groups. Further, it is important to bear in mind that there is often overlap in the life circumstances of those in different sub-groups.
- Ethnicity is a self-perceived status and people can belong to more than one ethnic category. The current report uses non-prioritised Census ethnicity data where people are coded to all ethnic groups that they say they belong to. Proportions may therefore total to greater than 100% in any given category. Further, the wording of the 2001 Census ethnicity question is not consistent with the 1996 question and therefore time series comparisons cannot be undertaken between 1996 and 2001.
- Sometimes the numbers of people experiencing a situation are too small to allow for meaningful or detailed interpretations of results. Caution must be exercised in drawing conclusions based on small numbers. For example, suicide is a relatively rare event, with few people dying and with the actual numbers of deaths varying from year to year: "Any interpretation of trends requires an examination of rates over several years. Data on the rates of suicide for geographical regions and cities may be of little value for reporting comparisons because of the low numbers, and hence highly variable suicide rates. For example, where populations are small, the rate of suicide can be greatly inflated by one or two deaths" (Ministry of Social Development 2001 Social Report Wellington : 116). Where the numbers quoted in the report for some indicators are fairly small, rates or percentages may not have been calculated.
- Most of the indicators presented in this report come from official data sources. Generally they include only reported incidences of a phenomenon, rather than the actual number of cases, which may in fact be higher. This is especially relevant to sensitive issues such as levels of crime and child abuse and neglect, where under-reporting is a recognised issue.
- Aside from the 2002 Quality of Life Survey, the data presented in this report have not been subjected to any form of significance testing. While the purpose of the report was not to produce city league tables, the lack of significance testing could be a limitation when comparing results across cities. For example, comparisons between cities and over different time periods may or may not be statistically significant even where large differences are found.
A joint Quality of Life Survey was carried out for the first time in September and October 2002 by the eight cities in the Quality of Life Indicators Project. The telephone survey, undertaken by AC Nielsen, was designed as a practical tool to collect data for indicators where the team could not get information from secondary or official sources. The issues explored in the survey were mainly perception based and focused on big picture city outcomes. As the survey was designed to meet information gaps in the current report, it did not include a comprehensive coverage of quality of life issues.
The survey sample was 4000 individuals (aged 18 years and over) from randomly selected households, with 500 interviews conducted per city. To ensure the final sample was representative of the populations of the cities, quotas were placed on ethnicity, sex, age and city wards. This method ensures that 'hard to find' sub-groups are represented in surveys.
The survey project was completed successfully, with few problems. Quotas proved difficult to fill in some cities, especially for Maori and Pacific Island respondents. This slowed the completion of the survey and resulted in low numbers of respondents in some categories. Conclusions couldn't be drawn where the numbers of respondents in a category were too low. The length of the questionnaire introduced an element of fatigue for some respondents. To reduce biases associated with respondent fatigue, the 20-minute interview included randomised question ordering. A sample of 500 respondents per city limited the ability to disaggregate results for various demographic groups and to provide sub-city breakdowns. This was particularly the case for larger cities such as Auckland and Christchurch .
As well as filling gaps in the current report, survey results are currently being used by individual cities for input into annual plan reporting and for informing the formulation of outcome statements for Long Term Council Community Plans under the Local Government Act 2002.
This survey is unique in New Zealand in that it is the first time cities have joined together to measure quality of life issues among their populations. As such, the survey results make a significant contribution to outcome research in New Zealand . The survey may be developed by the Quality of Life Indicators Project Team and the Ministry of Social Development into a more comprehensive quality of life survey, possibly with input from other government agencies. The new survey may be expanded to include other New Zealand cities and may include larger samples to allow for meaningful city level breakdowns for larger urban areas. The survey will be ongoing to allow for continued outcome monitoring.
The AC Nielsen Survey Report is available in pdf format 1.2MB pdf »
The following key recommendations are based on experience of gathering data for this report.
- Enhanced social statistics reporting and monitoring is needed:
- A comprehensive, ongoing New Zealand Quality of Life Survey is required that builds on the work undertaken in the Eight Cities Quality of Life Survey and on the relationship established between the Quality of Life Indicators Project Team and the Ministry of Social Development's Social Report Team. The involvement of a number of government agencies in this initiative will be explored, along with raising the sample size, both at an individual city level (to allow for greater disaggregation) and in widening a survey to include a national sample. A sub-team from the eight cities and the Ministry of Social Development is currently exploring an option for a first joint New Zealand Quality of Life Survey in 2004. It is recognised that Statistics New Zealand's Social Statistics Programme will also look at addressing social data needs.
- More information is required on how people view their lives and their situations (ie. subjective perceptions) to complement the plethora of quantitative measures that are often presented in monitoring reports. These data sets will become more important to local authorities because of Local Government Act 2002 outcome monitoring requirements. The proposed New Zealand Quality of Life Survey would contribute significantly to advancement of perception based monitoring. Likewise, the development of measures of quality of life dynamics are needed to allow us to monitor the drivers and outcomes of changes in circumstances over time. Accurate data monitoring of the experiences of marginalized groups must be seen as an important component of New Zealand 's social statistics programme.
- Alignment is required of official statistics and measures reporting to Long Term Council Community Plan (LTCCP) outcomes that are common across larger local authorities. This will require a commitment from central government agencies to assisting local authorities to gather reliable and timely data that will help in the monitoring of their LTCCPs. This may require the addition of new data sets and monitoring tools to official statistics measurement regimens. Greater sub-national reporting and data availability will be imperative to this requirement.
- Coordination of environmental indicators monitoring across local authorities and central government is needed urgently. Representatives from local authorities will meet shortly to begin development of a comprehensive, comparable set of environmental measurement standards for the eight cities (that might also have application to all local authorities) in association with the Ministry for the Environment and Statistics New Zealand .
NEW ZEALAND CENSUS OF POPULATION AND DWELLINGS
The Census of Population and Dwellings is the primary source of information on the size, composition, distribution, economic activities and state of well-being of the population. Census data are used for analysing trends, planning public services and allocating public funds in the areas of health, housing, transport, education, income and law and order. Census figures are also the basis of population estimates and projections, which are critical to many planning and policy-related activities, such as allocation of health funds.
Data Provider: Statistics New Zealand
Time Series: five yearly (1991, 1996, 2001)
Sample Size: Census of New Zealand population
Data Breakdowns: Geographical: CAU, city, region, total New Zealand ; Demographic: age, ethnicity, sex
Limitations and Issues:
- Limitations associated with self-completed questionnaires (eg. literacy, motivation to complete, etc).
- Traditional problems such as undercount and non-response. In general, there was a higher level of non-response to many questions in the 2001 Census than in the 1996 Census. This impacts on data quality.
- Statistics New Zealand 's Rounding Procedures - Statistics New Zealand utilised a confidentiality assurance technique of randomly rounding census statistics to base three. This enables the greatest possible amount of census data to be published, without compromising the privacy of individual responses. Under the random rounding process, all table cell values including row and column totals, are rounded as follows:
- zero counts and counts which are already multiples of three are left unchanged;
- other counts are rounded to one of the nearest multiples of three.
All rounding, including separate rounding of total and sub-totals, is carried out on the recorded results. The effect of this rounding on the accuracy of census statistics for practically any proposed use is insignificant. Furthermore, on occasion, figures or percentages have been rounded off to the nearest unit or decimal point. This may result in a total disagreeing slightly with the total of the individual items as shown in tables.
- The 2001 Census recorded non-prioritised ethnicity data, while the 1996 Census collected prioritised data. It is not possible to compare ethnicity results across the two Census periods for this reason.
- Income data may be unreliable as it is based on what respondents can normally recall or can readily retrieve from their financial records. It is also based on what respondents are prepared to divulge about their financial circumstances. To overcome collection difficulties total personal income is collected as an income range rather than an actual dollar income.
Chapters using this Data Set: The People, Health, Housing, etc
POPULATION ESTIMATES AND PROJECTIONS
To provide population data between Census dates, Statistics New Zealand estimates the population, using the most recent Census data as a base. The estimated resident population is updated regularly for population changes due to births, deaths and net migration (arrivals less departures) of residents. The estimate gives the best measure of the population that usually lives in an area, for a limited range of variables (age, sex, ethnicity) and limited geographic areas.
Population projections provide an indication of the total size or composition of the population at a future date based on certain assumptions about future fertility, mortality and migration. The projected resident population has as a base the estimated resident population at a given date.
Data Provider: Statistics New Zealand
Time Series: Geographic: national, sub-national; Demographic: age, ethnicity
Sample Size: Full coverage output
Data Breakdowns: Geographical: city, region, total New Zealand ; Demographic: age, ethnicity, sex
Limitations and Issues:
- Post-censal population estimates and projections are not directly comparable with Census counts. Census counts give a snapshot of the population at that time but make no allowance for New Zealand residents temporarily overseas or for net Census undercount.
- Demographic projections are based on assumptions made about future fertility, mortality, net migration, inter-ethnic mobility, labour force participation and householder rate patterns of the population. Although the assumptions are carefully formulated to represent future trends, they are subject to uncertainty. Therefore, the projections should be used as guidelines rather than exact forecasts. They provide an indication of the overall trend but do not attempt to project specific annual variation. The projections do not take into account non-demographic factors (eg. war, catastrophes, major government and business decisions) that may invalidate the projections. Demographic trends are monitored regularly, and when it is necessary the projections are revised to reflect new trends and to maintain their relevance and usefulness.
Chapters using this Data Set: The People, etc
HOUSEHOLD ECONOMIC SURVEY
The Household Economic Survey (HES) collects information on household income and expenditure as well as a wide range of demographic information on individuals and households.
Data Provider: Statistics New Zealand
Time Series: The HES moved to a three yearly cycle beginning in the 2000/2001 year. 1997/1998 was the last year that the survey was run annually. No survey was carried out in the intervening years.
Sample Size: A sample of approximately 3,000 eligible responding households is achieved each year. The sample comprises 752 primary sampling units (PSUs), with an average 4 eligible responding households obtained per PSU. The sample is divided equally between the four quarters: 188 PSUs are in the sample each quarter. There is no overlap of PSUs between consecutive quarters.
Data Breakdowns: region, Total New Zealand
Limitations and Issues:
- The HES is revised on an annual basis because of the need to collect additional (or different) information for the maintenance of the weighting base of the Consumers Price index or for changes to income related information requirements. This can result in changes to questions and definitions used in the survey and in the deletion or addition of some expenditure items, which at times compromises the comparability across survey time periods.
- Care is required in making comparisons of expenditure with income from the HES, as the method of surveying income and expenditure does not provide for consistency at an individual respondent level. Respondents in the lower and higher income bands tend to be under-represented in the survey.
- The relative sampling error on annual estimates of total expenditure from the HES is approximately 4 percent.
- It is not possible to compare current survey results with those from 1997/1998, as the HES changed from an annual survey to a three yearly survey and now reports using a June year rather than a March year.
- Expenditure on some household items, such as tobacco or alcohol and meals away from home, tends to be understated. Reported income may also be understated.
Chapters using this Data Set: Housing, Economic Standard of Living, Economic Development (?)
HOUSEHOLD LABOUR FORCE SURVEY
This survey produces a range of statistics relating to the employed, the unemployed and those not in the labour force who comprise New Zealand's working-age population.
Data Provider: Statistics New Zealand
Time Series: Quarterly (199\\\\\)
Sample Size: 15,000 households and approximately 30,000 individuals in the usually resident population aged 15 years and over.
Data Breakdowns: Geographical: region, total New Zealand; Demographic: age, ethnicity, sex
Limitations and Issues:
- High sampling errors associated with small estimates. This makes many of the smaller estimates unreliable or unusable.
- Young males tend to be under-represented because their high mobility makes them difficult to get hold of.
- The survey does not measure the quality of people's jobs (for example, whether they work in casual jobs, how much they are paid, whether they get sick leave, etc).
- There are issues with some of the definitions used in the survey . For example, to be counted as employed, a respondent only has to work for one hour or more in a week or could work unpaid in a family business. To be classed as unemployed, a respondent must be available to start a job and be actively seeking work.
Chapters using this Data Set: Economic Development
BUILDING CONSENTS ISSUED DATA BASE
The Building Consents Issued Data Base summarises information on the value and number of all Building Consents issued for more than $4,999 within New Zealand in the reference month. Data are collected by the Territorial Authorities who issue the building consents.
Data Provider: Statistics New Zealand
Time Series: Monthly
Sample Size: All building consents issued by local authorities over the value of $4999
Data Breakdowns: City, Total New Zealand
Limitations and Issues:
- The data base includes building consents issued over a value of $4999 which will result in a small under count of alterations and additions to both residential and non residential buildings under this value.
- Building Consents data includes G.S.T. whereas the Quarterly Building Activity Survey (QBAS) excludes GST..
- When a small number of consents have been issued for a region and they have a high value, the average dwelling value for that region is overstated.
- Manual data entry may cause errors at different stages of the information provision process. These errors can occur at the Territorial Authority or Statistics New Zealand levels. Inadequate details on the consent form cause difficulty in assigning consent value to specific a building type.
Chapters using this Data Set: Housing, Economic Development