Open Data Presentation 2013 by Pia Waugh

1. {Open|Big|Linked} Data
Enabling better policy, services and cost efficiency in Government
Version 0.5
Pia Waugh
Director of Coordination and Gov 2.0
Technology and Procurement Division
Office of the Australian Government CTO
1

2. Benefits to Government in Opening Data
Cuts red tape
• Improves efficiencies in sharing data across government and with public
• Proactive automated publishing rather than manual retrospective approach
Improves Government Operations
• Enables collaboration and consistency across government and with public
• Improves policy analysis, development, implementation and reporting
• Improves service delivery by enabling thematic personalised approach to info
including mobile services that leverage cloud hosted data and automated APIs
• Improves data quality through enabling verifiable public contributions
Innovation
• Enables innovation and new opportunities in government, industry and research
• Enables greater capacity for public to contribute meaningfully to public policy
2

3. The Government Data Landscape (latest version online)
3

4. The APS Policy Landscape
Others:
• Publishing Public Sector Information & National Standards Framework
• Open Public Sector Information: From Principles to Practice Report
• Declaration of Open Government
• Gov 2.0 Taskforce Report
• Statement of IP Principles for Government (CC-BY)
• Ahead of the Game
• Digital Transition Policy (Archives) & Accessibility Policy
• Emerging Open Research Policies
• Open Government Partnership (TBD)
4

5. State and Territory Policies
5

6. Policies Components
APS:
• Permissive copyright – CC-BY as the default
• Open by default
• Support reuse and innovation
• More public engagement
• Better use of data for government policy and service development
States/Territories add:
• Procurement – open by design
• Reporting – dashboards
• Departmental strategies
6

7. APS ICT Strategy
 Declaration of open govt
 Online engagement
 Online interaction
 APS use of social media
 Connected service delivery  Open public sector info
 ICT investment framework
 Skills and capability
APS ICT Strategy
2012-2015
 Operational efficiency ($1.8B)
 Improved agency capability
 Whole-of-govt approaches
 Coordinated procurement
 Benchmarking
promoting better government through
the innovative and strategic use of ICT
Using ICT to increase public sector
and national productivity by:
 Enabling better service delivery
 Improving the efficiency of
government operations
 Supporting open engagement to
better inform decisions
addressing today’s challenges…
Public
expectations
Productivity
performance
Technology
advances
Agencies
pressures
Influenced by innovative digital
private sector services,
broadband availability,
smartphone take-up, social
media and blogs
Global economic impacts,
increasing global competition,
reduced resource demand,
demographic changes,
environmental constraints
Mobile, broadband, cloud
computing, virtualisation, big
data analytics and other
emerging technologies
Meeting outcomes and
expectations, funding
challenges, efficiency dividends,
organisational change, and
achieving ICT delivery targets
Australian Government Information Management Office
7

8. Big Data Strategy
VISION
PRINCIPLES
ACTIONS
 Enhanced services
 Data is a national asset
 Develop better practice guidance
 New services and business
partnership opportunities
 Privacy by design
 Identify and report on barriers to
adoption of big data analytics
 Improved policy
development
 Data integrity and the
transparency of process
 Protection of privacy
 Skills, resources and
capabilities will be shared
 Leveraging Government’s
investment in ICT
 Collaboration with industry
and academia
 Enhancing open data
 Enhance skills and experience in
big data analysis – initiation and
support of pilot projects
 Develop guide to responsible
data analytics
 Develop guidance to enable
agencies to create information
asset registers
 Monitor technical advances
Australian Government Information Management Office
8

9. Privacy and confidentiality
• Custom API approach that confidentialises on the fly (eg, ABS)
• Deidentification of data – to appropriate level
• Aggregation
• Leveraging existing processes for researcher (unit level) access rather
than conflating open data discussions
• Privacy Commissioner as point of reference and support
• Avoiding common identifiers across multiple datasets
9

10. Open by Design
Building proactive publishing into:
• Systems
• Processes
• Procurement
• Planning
• Records management
Leveraging open data through:
• Public APIs
• Analysis tools and datavis
• Internal processes looking for external sources
10

11. Data Portals
11

12. Other Data Projects
• Spatial
• Geosciences
• Research
• Sensor
• Realtime (eg Transport)
• Census/Statistics
• Cultural
• Data about government
• International:
Aid/Extractive Industries
12

13. Loads of Tools Available
• Publishing tools – CKAN, Socrata, bespoke
• Automation – FME, Kettle
• Data visualisation – Tableau, SuperDataHub,
SpatialKey
• Analysis – R, domain specialist software, Palantir
• API development
• Application development
• Linked data tools
• Metadata tools
13

14. New and Old Skills Required
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Publishing
Automation
Metadata/linked data
API development
Plumbing
Data visualisation
Analysis and statistics
Policy development
Public consultation
Online skills
14

15. Some Challenges
• Legislative
• Culture
• Systems
• Reactive vs proactive
• Metadata/semantic context
• Too much data
• Real time vs historic
• Definitions and common references
• Limited skills and over specialisation
15

16. data.gov.au
Free, cloud based, highly scalable platform for hosting government data.
Staged approach
1. Publishing (2013)
Improving the functionality and ease of
publishing for agencies with training and
documentation
2.
Value realisation (Early 2014)
Providing useful front end tools for data.gov.au
including data visualisation and analysis tools
3.
Data quality (Late 2014)
Looking at ways to provide agencies the ability
to accept iterative data improvements in a
verifiable way
Features
• Federated search making data
and data services easier to find
• Manual and automated publishing
options
• API access to government data
• Easy to publish, download and
interact with data online
• Basic data visualisation capability
16

17. All the pieces are in place,
we need people to put the puzzle together
17

18. {Open|Big|Linked}
Data
Without data, you only have open, big & linked.
What {public|private} data do you need?
Questions?
18