Context

* The eGovernment and Digital Economy Policy pre-election commitment as well as the National Commission of Audit clearly prioritise greater sharing and publishing of government data for improved economic benefits, as well as for improved policy, services and efficiences across government.
* There are significant opportunities to better leveraging data across Government and jurisdictions in Australia.

Not this Jon Snow!

Mapping the 1854 London Cholera Outbreak
Dr. John Snow is regarded as one of the founding fathers of modern epidemiology. As London suffered a series of cholera outbreaks during the mid-19th century, Snow theorized that cholera reproduced in the human body and was spread through contaminated water. This contradicted the prevailing theory that diseasses were spread by "miasma" in the air.

London's water supply system consisted of shallow public wells where people could pump their own water to carry home, and about a dozen water utilities that drew water from the Thames to supply a jumble of water lines to more upscale houses. London's sewage system was even more ad hoc: privies emptied into cesspools or cellars more often than directly into sewer pipes. So the pervasive stench of animal and human feces combined with rotting garbage made the miasma theory of disease seem very plausible. Disease was more prevalent in lower-class neighborhoods because they stank more, and because the supposed moral depravity of poor people weakened their constitutions and made them more vulnerable to disease.

The September 1854 cholera outbreak was centered in the Soho district, close to Snow's house. Snow mapped the 13 public wells and all the known cholera deaths around Soho, and noted the spatial clustering of cases around one particular water pump on the southwest corner of the intersection of Broad (now Broadwick) Street and Cambridge (now Lexington) Street. He examined water samples from various wells under a microscope, and confirmed the presence of an unknown bacterium in the Broad Street samples. Despite strong scepticism from the local authorities, he had the pump handle removed from the Broad Street pump and the outbreak quickly subsided.

Snow subsequently published a map of the epidemic to support his theory. A detail fom this map is shown below. The complete map shows the locations of the 13 public wells in the area, and the 578 cholera deaths mapped by home address, marked as black bars stacked perpendicular to the streets.

Some anomalies are worth noting. Although the large workhouse just north of Broad Street housed over 500 paupers, it suffered very few cholera deaths because it had its own well (not shown on the map). Likewise, The workers at the brewery one block east of the Broad Street pump could drink all the beer they wanted; the fermentation killed the cholera bacteria, and none of the brewery workers contracted cholera. Many of the deaths further away from the Broad Street pump were people who walked to work or market on the Broad Street and drank from that well. The water from the Broad Street well reportedly tasted better than water from most of the neighboring wells, particularly the smelly water from the Carnaby Street/Little Marlborough Street well a few blocks to the northeast.

http://www.udel.edu/johnmack/frec682/cholera/

https://public.resource.org/8_principles.html

Open Government Data Principles

Government data shall be considered open if it is made public in a way that complies with the principles below:

1. Complete
All public data is made available. Public data is data that is not subject to valid privacy, security or privilege limitations.
2. Primary
Data is as collected at the source, with the highest possible level of granularity, not in aggregate or modified forms.
3. Timely
Data is made available as quickly as necessary to preserve the value of the data.
4. Accessible
Data is available to the widest range of users for the widest range of purposes.
5. Machine processable
Data is reasonably structured to allow automated processing.
6. Non-discriminatory
Data is available to anyone, with no requirement of registration.
7. Non-proprietary
Data is available in a format over which no entity has exclusive control.
8. License-free
Data is not subject to any copyright, patent, trademark or trade secret regulation. Reasonable privacy, security and privilege restrictions may be allowed.
Compliance must be reviewable.

The presentation describes the evolution of data.gov.au from its beginning in 2009 as an activity as part of the Gov 2.0 Task Force work until the present.

The beta version of data.gov.au was called data.australia.gov.au. For our data.australia.gov.au beta website built during the Gov 2 Taskforce, we focussed on publishing the data and a working minimum of metadata, and encouraging the use of Creative Commons BY licences (which as of October 2010 were included in the Statement of Intellectual Property Principles for Australian Government Agencies as the default option for licensing public sector information).

Over the next three years, we continued to provide the platform, moving from beta to operational and slowly increasing the data offered. Within Finance, we were unable to give particular priority to data curation and growth relied on agency efforts rather than any additional work done centrally.

In early 2013, Finance was able to devote additional effort to data.gov.au. We began a refresh period for the site. We discovered that a third of the 1500 claimed data sets were dead links and a third were not so much data sets as files of various sorts. We reduced the real number of data sets to around 500.

We also re-platformed the site to use the CKAN platform. The global not-for-profit agency Open Knowledge Foundation is encouraging data publishers to consider using the Comprehensive Knowledge Archive Network (CKAN) platform. CKAN provides a sleek design interface that can be branded with any business or government’s livery to create an easy-to-use access point into a company, city, state, or nation’s open data catalog. Most recently, the US Government has upgraded its data catalog at data.gov to make use of the new CKAN 2.0 platform.

The site was thus able to offer the automated creation of APIs to enhance the data offering.

New government's policy commitments:

Request that AGIMO consult a range of private sector and community voices to

identify value-adding public data sets that are not currently on data.gov.au. Where

appropriate, work with agencies to expedite such access.

Centrally driven additional publication of more data. Seven fold increase to date. Also other resources. Linking with other jurisdictions.

Data request site launched.
Voting system in place.
Note desire for availability of geospatial data.

data.gov.au and spatial

* data.gov.au is the authoritative point of discovery for all Commonwealth data.
* Finance is working closely with the Department of Communications to improve data policy, guidance, support and tools for agencies across the Commonwealth.
* A new data.gov.au was launched almost a year ago on best practice open data platform, CKAN.
* data.gov.au functionality has been and continues to be extended to meet the needs of data publishers and data users.
* We are not focused on citizens, we are focused on data users, and believe making programatic access to data creates opportunities for data users to create new analysis and services for a non-technical audience.
* On advice from the Office of Spatial Policy (now the Department of Communications Data Policy Branch), we added a geoserver to the data.gov.au stack, so government agencies can freely and easily publish their spatial data on data.gov.au as web services if they don't have the capacity to host their own.
* data.gov.au is integrated with FIND so that:
a) all Commonwealth spatial data is discoverable through data.gov.au; and
b) all spatially enabled data hosted or linked on data.gov.au is discoverable through FIND and viewable on the National Map.
* We have finalised an update of the data.gov.au metadata to require meaningful spatial metadata, and to encourage and support agencies to improve geocoding within datasets. This is being implemented in the coming weeks.
* We are also working informally with State, Territory and Local Governments to share best practise, metadata standards, documentation and code to better enable reuse of data across jurisdictions.

Australia is working in partnership with The GovLab at New York University to conduct the Open Data 500 Australia. This is the first study of Australian companies and NGOs that use open government data to generate new business, develop new products and services, and create social value.

http://www.opendata500.com/au/

Questions from the floor