Commonwealth Arrangements Information Series

Commonwealth administrative data is a valuable public asset and is increasingly being recognised as a strategic resource to support research, discussion and decision making. The integration of Commonwealth datasets at the unit record level to create new and more comprehensive datasets has also been recognised as having the potential to greatly enhance the evidence base for research and evaluation. However, consistent governance and institutional arrangements for the integration of data are needed to protect the identity and privacy of individuals and organisations providing the data, while still maximising the value and usefulness of the data.

To create a safe and effective environment for data integration, the Portfolio Secretaries (now the Secretaries Board) endorsed a set of High Level Principles for Data Integration Involving Commonwealth Data for Statistical and Research Purposes, as well as Governance and Institutional Arrangements (the Commonwealth arrangements) to support these principles.

Implementation of the Commonwealth Arrangements

Initial implementation of the Commonwealth arrangements commenced on 1 July 2014. The implementation priority in the first six months of operation is to establish the arrangements for external statistical and research projects involving Commonwealth data. External projects are defined as those projects where the data user (researcher) is external to the the Commonwealth (e.g. from a State/Territory government agency or an academic research organisation).

Projects are in scope of the Commonwealth arrangements for data integration if they:

1. are approved and undertaken for statistical and research purposes only. Integration for non-statistical purposes, such as delivery of services to particular individuals, compliance monitoring, incident investigation or regulatory purposes, are out of scope, as these activities have different processes and legislative requirements governing them; and
2. involve integration at the unit record level (records for a person or business), or at the micro level (e.g. information on small geographic areas); and
3. the data custodian for at least one dataset is a Commonwealth agency; and
4. the data user is external to the Commonwealth (e.g. for State/Territory government or academic research), as mentioned above.

The initial implementation of the Commonwealth arrangements will be for selected external projects. Once approved, details of the selected projects will be available on the public register of data integration projects. During the initial implementation, the integrating authority for the selected projects are to be Commonwealth agencies. If the project is assessed as a high risk after using the Risk Framework the integrating authority must be accredited.

Agencies are encouraged to use the Commonwealth arrangements for all external projects during this initial implementation period. This will help in minimising the general risk presented during the execution of a data integration project. Other data integration projects that satisfy the first three criteria, but are not external projects may also utilise the Commonwealth arrangements for data integration, if this is the preference of data custodians. Data users wishing to apply for access to Commonwealth data for a data integration project should discuss their proposal with the Commonwealth data custodian(s).

Please be advised that the scope of the Commonwealth arrangements is subject to change at the discretion of the Cross Portfolio Data Integration Oversight Board.

Help for conducting data integration projects

As part of the implementation of the Commonwealth arrangements, a guide has been released to assist stakeholders when conducting statistical data integration projects involving Commonwealth data. The guide provides practical advice on how to comply with the Commonwealth arrangements, including guidance on the planning, management and approval processes for stakeholders of data integration projects.

The arrangements for data integration involving Commonwealth data for statistical and research purposes (the Commonwealth arrangements), specify three stakeholder roles in a data integration project that uses Commonwealth data (the project). These are the data user, the data custodian and the integrating authority.

Role and responsibilities

The data user (or researcher) is the person or entity who accesses the integrated dataset (at a unit record level) to conduct research. For example, the data user can be an academic working in a research institution, or an employee in a State/Territory government agency.

The data custodianis the entity responsible for the source data and any activities relating to it, such as its collection, management, protection and access approval. Often their responsibilities are set out in relevant legislation.

The integrating authority is the single organisation ultimately responsible for the sound conduct of the statistical data integration project. It is responsible for the implementation of the data integration project and the management of the integrated dataset throughout its entire life cycle, ensuring full compliance with commitments made to data custodians, and in line with the Commonwealth arrangements. The integrating authority is also responsible for providing data users with safe and secure access to the integrated data in line with the requirements of data custodians.

The process for conducting a project

There are many ways a data integration project can be conducted. On the reverse side of this information sheet a process map outlines the stages a project will generally go through to align with the Commonwealth arrangements, although there may be variations depending on specific project requirements. Most of the stages should already align with existing project approval procedures, with only some additional steps incorporated to reflect the requirements of the Commonwealth arrangements.

A data custodian, in the context of the arrangements for data integration involving Commonwealth data for statistical and research purposes (Commonwealth arrangements), is the entity responsible for the source data and any activities relating to it, such as its collection, management, protection and access approval.

Often the responsibilities of a data custodian are set out in relevant legislation which may extend to the provision of data for statistical and/or research purposes.

This Information Sheet provides a summary of the considerations data custodians need to take into account at each stage of a data integration project, where it is in scope of the Commonwealth arrangements. A checklist for data custodians is also provided on the back of this sheet.

Deciding if data can be released

Each data custodian is responsible for deciding whether to approve the release of their data for a data integration project (a project). When a project is first proposed and access to data is requested, each data custodian must consider if they are authorised to provide the data under governing legislation or with data provider’s consent.

If there is authorisation to release the data, the data custodian may then also consider if the release of their data for the project will provide a public benefit that outweighs the privacy imposition and risks to confidentiality involved.

Data custodian roles within the Commonwealth arrangements

• Role 1 - Maximise the value of data holdings
• Role 2 - Assess project risk
• Role 3 - Comply with policy and legislation
• Role 4 - Ensure safe storage of unit record data
• Role 5 - Safely transmit unit record data
• Role 6 - Enter project agreements

Assessing the risk of the project

It is the responsibility of data custodian(s) to assess the risk of a project. The risks to be considered are:

1. a breach resulting in an unauthorised disclosure of personal or business information; and
2. a reduction in public trust of the Australian government and its institutions.

While the final decision and agreement on the risk rating (high, medium or low) of the project remain with the data custodian(s), input may be sought from integrating authorities (the agency that manages, and is ultimately accountable for, the sound conduct of the project, (see Sheet 5).

Appointing an integrating authority

Following the completion of the risk assessment, the data custodian(s) is then responsible for appointing an integrating authority. If the project risk is rated high, then the project must be managed by an accredited Integrating Authority (refer to Information Sheets 5 and 6).

The data custodian(s) should also ensure the integrating authority nominated to manage the project is authorised to receive the data, either through legislation or data providers’ consent.

Finalising project agreements

Once the details of the project have been decided, project agreements are signed between the data custodian(s) and the integrating authority. These agreements provide a mechanism for the data custodian(s) to exercise their accountability for the security and confidentiality of their data once it is provided to the integrating authority for the project.

Agreements should detail any conditions specified by the data custodian(s) relating to data security obligations, privacy and confidentiality requirements, data access provisions and potential sanctions which may apply to misuse of the data.

Extracting and providing data

Data custodian(s) are responsible for ensuring the safe extraction and transmission of their data (as specified in the project agreements) to the integrating authority. This transmission of data should be consistent with Australian Privacy Principles (APPs) and the Australian Government Protective Security Policy Framework.

A checklist for data custodians

This is a list of considerations for data custodians who have been approached for their data to be included in a data integration project for statistical or research purposes.

In Principle approval

✔ Is your project in scope of the Commonwealth arrangements?

✔ Does the public benefit outweigh the privacy imposition of the project?

✔ Do you have authorisation to release the data for the purpose of the project?

✔Is the purpose consistent with departmental policies and purposes?

✔ Have you completed the risk assessment in conjunction with other data custodians?

✔ Are you satisfied that this project does not present an unacceptably high risk to public trust in the Australian Government and its institutions?

✔ Do you require an accredited Integrating Authority to manage the project?

✔ Do you require any further assessments to be undertaken prior to signing project agreements (e.g. ethics committee approval or privacy impact assessment)?

✔ Have you specified any special conditions to be met as part of your project approval?

✔ Do you give in principle approval for this project to proceed to the next stage?

Final approval

✔ Are you satisfied with the arrangements provided by the integrating authority for security of the data (e.g. data transfer, access, use, storage and destruction or retention)?

✔ Have you considered how confidentiality and privacy will be protected?

• How will the separation principle be applied (if applicable)?
• How will data be de-identified and/or confidentialised?
• What conditions will data users be expected to comply with (e.g. signing confidentiality undertakings, review of research results before publication)?
• What are the consequences if there is a misuse of data or breach of privacy?

✔ Have all necessary authorisations been received (e.g. Ethics committee approval, privacy impact assessment, departmental authorisation, consent of the data provider, Public Interest Determination)?

✔ Have you entered into a project agreement with the integrating authority (and all other data custodians)?

Project delivery

✔ Has data been extracted and delivered to the integrating authority according to project agreements?

✔ Have you provided metadata and information about the quality of the data to assist the integrating authority and data users understand the source data?

An essential pillar of establishing a safe and effective environment for data integration involving Commonwealth data is the nomination of an organisation to manage each statistical data integration project from start to finish.

This role is known as the integrating authority. The integrating authority is responsible for the management of the data integration project on behalf of the data custodians and minimising privacy concerns associated with the use of the data once it is received from data custodians and after integration. It is also responsible for facilitating the use of the integrated data within the constraints of privacy requirements and legislation.

This Information Sheet provides a summary of the roles and responsibilities of an integrating authority, including a checklist of considerations on the back of this sheet.

Nominating an integrating authority

When a data integration project involving Commonwealth data for statistical and research purposes (a project) is proposed, it is considered by the data custodians and a risk assessment is conducted (see Sheet 4). Once this is complete, the data custodians appoint an integrating authority to manage and conduct the project from start to finish.

The integrating authority must be a secure and trusted institution (an individual cannot be an integrating authority) and in a position to comply with the requirements of the Privacy Act 1988 or the equivalent State/Territory legislation (in regards to information about individuals) and secrecy provisions generally (in regards to information with respect to the affairs of any third party, corporate or individual). In addition, if the project is rated as high risk, the integrating authority appointed must be accredited (see Sheet 6).

When nominating an integrating authority, the data custodians should also ensure they are authorised to release the identifiable data, either by legislation or consent, to the integrating authority appointed (see Sheet 4)

Integrating authority roles within the Commonwealth arrangements

• Role 1 - Enter into project and data access agreements
• Role 2 - Implement safe and effective arrangements for data integration projects
• Role 3 - Manage datasets for the duration of data integration projects
• Role 4 - Provide transparency in operation

Finalising agreements

Once appointed, the integrating authority is responsible for reviewing and finalising the arrangements for the project and preparing project agreements with data custodians and data access agreements with data users.

The purpose of project agreements is to help ensure that integrated datasets are managed and used in accordance with data custodian requirements, protecting privacy and ensuring that data is not likely to enable the identification (or re-identification) of individuals and businesses. Project agreements set out the terms and conditions that accompany the final project approval.

Once the project is approved the integrating authority is responsible for registering the project on the public register of data integration projects.

Delivering a project

The integrating authority has an important role in managing the increased risk of identification that exists when two or more datasets are integrated. Generally an integrating authority will be chosen because they have the skills and expertise to conduct a data integration project safely and securely.

However, it is possible for an integrating authority to outsource or involve consortium/partnership arrangements to complete a project. For example, it might use another agency to create linkage keys, or it may use infrastructure provided by another agency to support dissemination or access to the integrated dataset by data users.

Providing access to integrated data

Part of the integrating authority's role in managing integrated datasets is to provide data users with secure access to the integrated data.

Data access agreements between the integrating authority and data users should set out the conditions and arrangements for accessing and producing output from the integrated data.

The conditions and nature of access may vary from project to project, according to the requirements of data custodians.

A checklist for integrating authorities

This is a list of considerations for integrating authorities who have been approached to undertake the end to end management of a data integration project for statistical or research purposes.

If the project is assessed as high risk, the integrating authority must be accredited.

In Principle approval

✔ Have you assessed the technical feasibility of the project and ensured that you can deliver on all aspects of the project? (e.g. linking methods, data security and provision of secure access to the integrated data, timeframes, data quality, risk management, etc.)

Final Project Details

✔ Have you discussed and agreed the arrangements for security of the data (e.g. data transfer, access, use, storage and destruction or retention) with all data custodians?

✔ Have you discussed and agreed with data custodians how confidentiality and privacy will be protected?

• How will the separation principle be applied (if applicable)?
• How will the data be de-identified and/or confidentialised?
• Do the data custodians have any special conditions to be met for users of the data (e.g. signing confidentiality undertakings, review of research results before publication)?
• Are the consequences, if there is a misuse of data or breach of privacy, understood for both the integrating authority and the data user?

✔ Will there be a fee for service or cost recovery charge to the data user and has a quote been prepared and accepted?

✔ Do you need to assist the data users to seek Ethics Committee approval for the project?

✔ Is a privacy impact assessment required?

✔ Do you intend to outsource or work in partnership with other organisations to complete components of the project (such as creation of linkage keys) and have the data custodians been advised?

✔ Have you entered into a project agreement with all of the data custodians to formalise the arrangements for the project?

✔ Have you entered into a data access agreement with the data user to set out the conditions and arrangements for accessing the integrated data for research and analysis?

✔ Have you registered the project on the public register of data integration projects and submitted the risk assessment to the Oversight Board?

Project delivery

✔ Have you conducted the following steps in accordance with the project agreements?

• Data prepared, linked, merged and quality checked using agreed protocols (e.g. application of the separation principle);
• Integrated data de-identified and confidentialised;
• Secure access to the integrated data provided to data users;
• Ensure that the integrated data is stored securely if it is to be retained and a review process is in place or that it is destroyed at project completion; and
• Completed an evaluation of the project and updated the project findings on the NSS website.

The appointment of an integrating authority for each statistical data integration project is an essential pillar in establishing a safe and effective environment for data integration involving Commonwealth data. An integrating authority is the single organisation ultimately responsible for the sound conduct of the statistical data integration project.

For each data integration project involving Commonwealth data for statistical and research purposes (a project), a risk assessment should be conducted by the data custodian(s) to assess whether a project should proceed and its level of systemic risk (high/medium/low)(Endnote 1).

If a project is assessed as posing a high risk, the integrating authority must be accredited. This information sheet provides an overview of the accreditation process.

Accreditation

Accreditation of an integrating authority is the recognition by the Cross Portfolio Data Integration Oversight Board (the Oversight Board) that the organisation has the requisite expertise, skills and knowledge, infrastructure and secure environment to undertake data integration projects, particularly those considered to be a high risk rating.

It is important to note, the accreditation scheme is an administrative arrangement which does not override legislation. All legal obligations (e.g. with regard to the Privacy Act 1988 or privacy and secrecy provisions in agency-specific legislation) must be met.

Accreditation criteria

The eight criteria integrating authorities must meet to gain accreditation are:

1. ability to ensure secure data management;
2. demonstrated ability to ensure that information that is likely to enable identification of individuals or organisations is not disclosed to external users;
3. availability of appropriate skills;
4. appropriate technical capability;
5. lack of conflict of interest;
6. culture and values that ensure protection of confidential information and support the use of data as a strategic resource;
7. transparency of operation; and
8. appropriate governance and administrative framework.

The steps involved in gaining accreditation

The process for accreditation of integrating authorities involves:

1. Self-assessment. Integrating authorities apply for accreditation by preparing a self-assessment report explaining how they meet the criteria for accreditation and evidenced by supporting documentation. The self-assessment should be succinct and avoid qualitative statements that cannot be directly verified in the supporting documentation provided. The assessment should be signed off by the head of the agency.
2. Audit. An independent third party audits the integrating authority’s self-assessment to verify the statements of the claims made in the self-assessment. This verification is made on the basis of the documentary evidence. This audit is paid for by the integrating authority applying to become accredited.
3. Decision. The Oversight Board will make the final decision on interim accreditation, based on the self-assessment and results of the audit.
4. Publication of list of accredited agencies. The Secretariat publishes a list of accredited integrating authorities, together with a summarised version of the integrating authority’s application and a summary of the audit report

Who can apply for accreditation?

The interim accreditation arrangements will be tested on Commonwealth government agencies first. While this does not preclude State/Territory government agencies applying for accreditation against the interim arrangements (provided that they meet all the requirements), it will not be possible for any State/Territory government agencies to be accredited in the short term, as this would not allow time for sufficient testing and evaluation of the arrangements with Commonwealth agencies.

The system is not yet mature enough to ensure that adequate safeguards apply to private firms. State/Territory government agencies and private firms can continue to apply for access to Commonwealth data under existing arrangements. The Oversight Board will only consider applications for accreditation, against the interim accreditation arrangements, for those agencies covered by privacy legislation (either the Privacy Act 1988 or State/Territory equivalent).

Will the accreditation process be reviewed?

The accreditation process is currently interim. Once a sufficient number of organisations have successfully gone through the interim accreditation process, a review will be conducted, and any necessary changes will be made to the process. The accreditation process will then become final.

Data users (researchers), are those who propose a project and are involved in accessing and investigating the integrated dataset, at the unit record level, for research.

The arrangements for data integration, involving Commonwealth data, for statistical and research purposes (the Commonwealth arrangements), provide data users with a clear framework in which to do this.

This Information Sheet provides a summary of the roles and responsibilities of data users within the Commonwealth arrangements.

Initiating a data integration project

To initiate a data integration project involving Commonwealth data for statistical or research purposes (a project), data users should prepare a project proposal for consideration by the data custodian(s) (see Sheet 4). The project proposal can be developed in consultation with the data custodian(s) and/or, where appropriate, an integrating authority (the agency that manages, and is ultimately accountable for, the sound conduct of the project, (see Sheet 5).

When proposing a project, it is important to take into account the public benefit (e.g. social, economic and/or environmental benefit) that can be derived. A project should only occur where the public good outweighs the privacy imposition and risks to confidentiality. Under the Commonwealth arrangements, data users may work with the nominated integrating authority and the data custodian(s) to finalise the details of the project.

Data user roles within the Commonwealth arrangements

• Role 1 - Develop research proposals that produce significant overall public benefit
• Role 2 - Collaborate with the integrating authority and data custodian(s) to get projects approved
• Role 3 - Enter into agreements with integrating authorities to ensure the safe management and use of datasets
• Role 4 - Access integrated datasets through secure arrangements in order to conduct analysis for statistical and research purposes
• Role 5 - Liaise with data custodians on the valid uses of integrated datasets

Access to integrated data - preserving privacy and confidentiality

To ensure the risks of potential disclosure of identifiable information are minimised, the integrating authority has responsibility to provide data users with secure access to the integrated dataset, for example, via an on-site or remote access data laboratory.

To access the integrated dataset, the data user will enter into an agreement with the integrating authority. This agreement will set out the requirements for accessing the integrated dataset and producing outputs. The conditions and nature of access may vary from project to project, according to the requirements of data custodians.

Generally, the integrating authority will provide the data user with access to integrated data once it has been de-identified and/or confidentialised, in line with the requirements of data custodians.

It is recommended that data users only be given access to identified or potentially identifiable data where: legislation allows; it is necessary to achieve the approved purpose of the projects; and the data custodians agree.

A checklist for data users

This is a list of considerations for data users who are planning to undertake a data integration project involving Commonwealth data for statistical or research purposes.

Project proposal

✔ Is your project in scope of the Commonwealth arrangements for data integration and does it offer public benefit that will outweigh the imposition to privacy?

✔ Have you consulted with data custodians to clarify details for the project (particularly in relation to the data items required for the project)?

✔ Do you have sufficient funding to meet any fee for service or cost recovery charges?

✔ Have you prepared and submitted a project proposal for in principle approval by data custodians, including all relevant details (e.g. listing of responsible officers and researchers, summary project description and objectives, expected outputs, data access arrangements, the required datasets and data items and possible time frames)?

Final approval

✔ Do you understand your obligations regarding the protection of confidentiality and privacy and the consequences if there is a misuse of data or a breach of privacy?

✔ Have you entered into a data access agreement with the integrating authority to set out the conditions and arrangements for accessing the integrated data for research and analysis, including any special conditions required by the data custodian for this project?

✔ Have all researchers who will access the data signed confidentiality undertakings required by the integrating authority or the data custodian?

✔ Have you obtained Ethics Committee approval for the project (if required)?

Project delivery

✔ Have all researchers completed any required training?

✔ Has the integrated data been received from the integrating authority and verified for completeness?

✔ Have you given the data custodians the opportunity to review the research results prior to publication (if required)?

✔ Have you destroyed or returned the integrated data to the integrating authority following publication of the project findings, in accordance with the agreement?

✔ Do you have any documented feedback to provide to the integrating authority (e.g. procedural or data quality issues)?