HOW do you crash a supercomputer? Try crunching the numbers from 150 terabytes of data water simulation models, reading inputs and writing outputs simultaneously.
The recently-completed Bioregional Assessments Programme is a prime example of the value of big data. It explored the risks to water resources, and the assets that rely on them, posed by an estimated $2.6 billion of proposed coal mines and coal seam gas developments.
Six-years and 200 experts in the making, it covered regions across 860,000 km² in central and eastern Australia.
It included running complex simulation models – frequently too much input/output for even the world’s most powerful supercomputers.
One of the challenges of the program was how to make supercomputers more flexible to cope with the motherlode of data being processed.
The Australian Government Department of the Environment and Energy commissioned CSIRO, the Bureau of Meteorology and Geoscience Australia, to undertake the BA program in 2012 to provide the best science available to inform decisions around major coal and coal seam gas (CSG) developments, and provide a common, objective information base for governments, landowners, communities, businesses and investors.
Six years on, the invention and implementation of new scientific methods through the $62 million BA technical program has led to the world’s most comprehensive biophysical assessment of the cumulative impacts of resource extraction, and the communication of findings in forms that regulators, communities and industry can easily understand and use.
“The science moved the goal posts,” says Mr Peter Baker, the BA program’s Technical Assurance Reference Group chair and Principal Science Advisor of the Australian Department of the Environment and Energy. “It is new; it is innovative. We were all working together to get the best outcomes that we could.”
It has changed the way we do collaborative science.
From day one it involved consultation across communities and governments to establish an understanding of the water-dependent assets most valued in each region. Assets identified included surface and groundwater management areas including bores and water rights providing economic benefits; environmental assets, such as wetlands and springs, and the landscapes reliant upon them; and sociocultural assets such as important cultural and social sites and habitats of species with cultural significance.
“What the community thought was important was a key part of the establishment of the BA program,” says Mr Baker.
The issue of water resources in relation to CSG and large coal mining developments was elevated to a matter of national environmental significance in 2013 through the introduction of ‘water trigger’ provisions under the Commonwealth’s Environment Protection and Biodiversity Conservation Act 1999 (EPBC Act).
This means that CSG and large coal mining developments that are likely to have a significant impact on water resources now require Federal assessment and approval. Historically, state regulators assessed individual proposals independent of other existing or proposed mines, creating an environmental risk if cumulative impacts are not properly accounted for.
The BA program developed an impact assessment framework and comprehensive regional information base which can be used to inform and evaluate site-specific development proposals.
CSIRO’s BA project leader Dr David Post says the complex assessment process included regional context setting, building development pathways, undertaking hazard analyses, and modelling the movement of water through the landscape. To understand the potential impacts on the ecology, it meant creating landscape classes to categorise different parts of the environment, determining key water-dependent assets, and understanding causal pathways of connectivity within those environments.
This involved developing geological models to understand the barriers and conduits for water movement under the earth’s surface, and incorporating groundwater and surface water models to investigate changes in surface and groundwater, and then investigating how the ecology might respond in the face of changes to water regimes.
This meant lots and lots of data. “We repeatedly broke some of the world’s biggest computers,” says groundwater modeller Dr Russell Crosbie, about the challenge of making super computers more flexible for the BA’s big data needs.
“The biggest problem we had was that we could only use about 200 nodes on machines that have 10,000s nodes due to our requirements for reading and writing of input and output files.
“Supercomputers are designed for making huge numbers of computations very quickly. Our use case required fast computations as well as the ability to read input files of many GBs and then write output files that were even bigger. The throughput of the link between the compute nodes and the storage only allowed us to run 200 models at once when the compute resources available could have run thousands of models.”
The program has left a significant legacy of tools and information, including:
The results of the BA program are being used by the Independent Expert Scientific Committee on Coal Seam Gas and Large Coal Mining Development, the statutory committee of leading scientists that independently advises government regulators on the impacts that CSG and coal mining may have on Australia’s water resources.
Results are also being used by a range of Commonwealth and State regulators, industry and communities. As a science-based regulator, the Australian Government Department of the Environment and Energy relies on science such as this for decision making.
A 2017 Jacobs Group (Australia) independent evaluation described the BA program as “the most inter-locked and connected program (the collaborators) been involved in”.
The program was awarded the CSIRO’s Chairman’s Medal for Science Excellence, recognising teams that have made significant scientific or technical advances that create value through innovation that delivers positive impact for Australia.
CSIRO Land and Water Director Jane Coram says the program’s inception was a courageous step on part of government.
“It was a precedent-setting step that challenged us all scientifically, and incorporated high-level communications, project management and stakeholder engagement,” she says.
“It required an enormous amount of interaction between scientific disciplines, and between science organisations. It required relationship building across and within institutions; it required, at times, difficult conversations because it challenged us. But we will reap the benefits from it for a long time to come.”
Former BA Technical Programme Director Bronwyn Ray, from the Bureau of Meteorology, says the program was “a fantastic example of what happens when you build trust between disciplines. The gaps in between is where the innovation happens, and how we solve problems”.
A key achievement was the program’s delivery of open access to data, models, workflows and reports, through a curated information platform and a map-based interface to query the scientific results, she said.
As well as creating new benchmarks in transparency, the lasting legacy of science, data and systems developed through the program helped change the way we think about doing science in the Commonwealth, according to Geoscience Australia’s Dr Stuart Minchin.
“That’s enormous credit to the collaboration that happened across the team,” Dr Minchin says.
The Australian Government has commissioned a follow-up four year, $30m Geological and Bioregional Assessment (GBA) program. This builds on the BA approach to examine the potential impacts of shale and tight gas extraction, informing future development decisions.