Communicating trust, enabling criticism
•Als PPTX, PDF herunterladen•
1 gefällt mir•559 views
Talk given at the Research Without Borders panel on Communicating Computational Research at Columbia University on 4th April 2013.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie Communicating trust, enabling criticism
Ähnlich wie Communicating trust, enabling criticism (20)
Mehr von Neil Chue Hong
Mehr von Neil Chue Hong (12)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Communicating trust, enabling criticism
- 1. www.software.ac.uk Communicating trust, enabling criticism Communicating Computational Research 4 April 2013, Columbia University, New York Neil Chue Hong (@npch) ORCID: 0000-0002-8876-7606 N.ChueHong@software.ac.ukSustainability Institute Software Where indicated slides licensed under
- 2. The Software Sustainability Institute www.software.ac.uk A national facility for cultivating world- class research through software • Better software enables better research • Software reaches boundaries in its development cycle that prevent improvement, growth and adoption • Providing the expertise and services needed to negotiate to the next stage • Developing the policy and tools to support the community developing and using research software Supported by EPSRC Software Sustainability Institute Grant EP/H043160/1
- 3. “Episodis Critics” Photo by Camilay el Arte CC-BY-NC-SA www.software.ac.uk www.software.ac.uk If we cannot be critical, we cannot be researchers Software Sustainability Institute
- 4. www.software.ac.uk www.software.ac.uk But systems are complex “Paint by Particle – aerosol movement” Photo by NASA Goddard Photo and Video Software Sustainability Institute CC-BY
- 5. Function … and software is always growing www.software.ac.uk Library / Suite / Package Algorithm Program … Software Sustainability Institute
- 6. Water Swap Reaction Coordinate www.software.ac.uk A water-swap reaction coordinate for the calculation of absolute protein-ligand binding free energies Woods CJ, Malaisree M, Hannongbua S, Mulholland AJ J. Chem. Phys. (2011) vol. 134, pp. 054114 Software Sustainability Institute http://dx.doi.org/10.1063/1.3519057
- 7. How can I trust this code? www.software.ac.uk lam = Symbol("lambda") S_sym = Symbol("S") S_scl = S_sym - 4*(S_sym-1)*(lam-0.5)**2 Openness ligand_bound_nrg_sym = Symbol("E_{ligand:bound}") ligand_bound_nrg = ligand_intraclj.components().total() + ligand_intraff.components().total() + bound_ligand_mobile.components().total(0) + Review bound_ligand_fixed_nrg swap_bound_nrg_sym = Symbol("E_{swap:bound}") swap_bound_nrg = ((S_scl) * swap_interclj.components().coulomb()) + Tests swap_interclj.components().lj() + bound_swap_fixed_nrg bound_swap_mobile.components().total(0) + bound_nrg_sym = Symbol("E_{bound}") Automation bound_nrg = bound_bound_nrg_sym + ((1-lam) * ligand_bound_nrg_sym) + (lam * swap_bound_nrg_sym) free_nrg_sym = Symbol("E_{free}") free_nrg = free_free_nrg_sym + (lam * ligand_free_nrg_sym) + ((1-lam) * swap_free_nrg_sym) total_nrg_sym = system.totalComponent() total_nrg = bound_nrg_sym + free_nrg_sym system.setComponent(total_nrg_sym, total_nrg) Software Sustainability Institute
- 8. “Social Media Information Overload” Photo by Joe Intersection Consulting CC-BY-NC Increase www.software.ac.uk the amount of information on the software (without drowning in it) increase the trust Software Sustainability Institute
- 9. Academic Software Should Not Suck www.software.ac.uk Courtesy of C. Titus Brown Software Sustainability Institute http://ivory.idyll.org/blog/ladder-of-academic-software-notsuck.html
- 10. 5 Stars of Research Software www.software.ac.uk • Community There is a community infrastructure C • Open R O Software has permissive license • Defined Accurate metadata for the software E D • Extensible Usable, modifiable for my purpose c.f. • Runnable 5 Stars of Linked Data (Berners-Lee) I can access and run software 5 Stars of Online Journals (Shotton) “Golden Star” Originally bySsolbergj CC-BY Software Sustainability Institute
- 11. “Scared” Photo by Capture Queen CC-BY www.software.ac.uk www.software.ac.uk Isn’t sharing scary? Software Sustainability Institute
- 12. There is only one thing in life www.software.ac.uk www.software.ac.uk worse than being talked about, and that is not being talked about. – Oscar Wilde Software Sustainability Institute
- 13. Journal of Open Research Software www.software.ac.uk http://openresearchsoftware.metajnl.com Software Sustainability Institute
- 14. “Lady and the Tramp” Photo by Joe Shlabotnik CC-BY-NC-SA www.software.ac.uk www.software.ac.uk The stigma of sharing will disappear Software Sustainability Institute
- 15. Communication www.software.ac.uk isn’t just questions + answers http://bit.ly/datasharingpanda … it’s about sharing knowledge Software Sustainability Institute
- 16. SSI Organisation www.software.ac.uk • Community Engagement (Lead: Shoaib Sufi) Fellowship Programme Events and Roadshows • Consultancy (Lead: Steve Crouch) Open Call for Projects / Funded Collaborations Software Evaluation • Policy and Publicity (Lead: Simon Hettrick) Guides and Case Studies Best Practice and Policy • Training (Lead: Mike Jackson) Software Carpentry (500+ students/year) Software Surgeries • Collaboration between universities of Edinburgh, Manchester, Oxford and Southampton. 9.5 FTEs for 5 yrs supplemented by additional project funding. Software Sustainability Institute
Hinweis der Redaktion
- In this talk I’ll be considering where I think software needs to get to so that we can be “Doing Research Properly In A Digital World”.In particular, I’ll look at the increasing complexity of systems and what this means for understanding, and assessing whether the new generation of data sharers might change the way in which we collaborate.
- The Software Sustainability Institute can help with: software reviews and refactoring, collaborations to develop your project, guidance and best practice on software development, project management, community building, publicity and more…Drawing on pool of specialists to drive the continued improvement and impact of research software developed by and for researchersProviding services for research software users and developersDeveloping research community interactions and capacityPromoting research software best practice and capability
- Criticism is a key concept in research. Intellectual debate drives research forward.I propose that just as we must be able to peruse, criticise and build on traditional research results, the same must be true of those created digitally.This does not necessarily mean that researchers must make their work accessible to everyone – there is a mutual relationship that must be built such that the critic can claim to be qualified to understand and comment on the subject in the same way that the researcher must make their results open to scrutiny.“Episodis Critics”Photo by Camilay el ArteCC-BY-NC-SA
- We are increasingly seeing the coupling of models, each a complex simulation in their own right.Sensors and instruments are providing ever increasing amounts of data to be analysed but it is about extracting knowledge, and specialists are increasingly necessary.Noone can grasp the entirety of the system, which is a cause for concern. Where is your gedanken now?“Paint by Particle – aerosol movement”Photo by NASA Goddard Photo and VideoCC-BYWinds disperse vast quantities of dust (red), sea salt (blue), sulphate (white) and black and organic carbon (green) around the world.
- Code is complex and often treated like a black box. It has helped us to improve the efficiency and accuracy of our research, but it also means that there are plenty of black boxes.
- Simulation of tamiflu bound to a drug-resistant mutant of influenza neuraminidase. The red parts of the protein prefer to be bound to the drug than water, while the blue parts prefer the water. At the start of the movie there are two blue residues at the top, with a red residue between them. The blue residue on the left is the mutant residue that confers drug resistance to the virus.
- How much must we take on trust to allow knowledge to scale?The very fact that I can show you this piece of code is a huge start – that it is being reviewed is another.In this case the simulations are completely reproducible as they are based on a set random number seed and are thus deterministic. However, changes in the software may change the order in which random numbers are drawn, so can only guarantee that the simulation is reproducible for this version of the software.
- A common issue is the question of “how much is enough” and whose job is it to provide it“Social Media Information Overload”Photo by Joe Intersection ConsultingCC-BY-NC
- This question of minimal requiremements ducks the issue of who is to enforce them.
- Star graphic modified under CC-BY fromSsolbergjC.f.5 Stars of Linked Data (Berners-Lee):Available w/ open license, machine-readable, non-proprietary format, open standards, linked to provide context 5 Stars of Online Journals (Shotton):Peer Review, Open Access, Enriched Content, Available Datasets, Machine-readable metadataWhat about community?
- We are afraid of what might happen if we choose to share too widely.An oft misused example is that of Geoffrey Chang, whose internal data analysis software was wrong and he was forced to retract work in Science. But he has continued to publish and been awarded a EUREKA grant, "for exceptionally innovative research projects that could have an extraordinarily significant impact on many areas of science" and, importantly, he had *not* shared his software – perhaps if he’d shared earlier the mistake could have been corrected. After all, his science method was fundamentally sound.“Scared”Photo by Capture QueenCC-BY
- The best thing that a young researcher can do.Recent studies in The Atlantic suggest that getting an academic career is no easier now than 10 years ago. Getting a name for yourself is important, and sharing knowledge (and criticism) improves your profile (see work from HeatherPiwowar). Indeed it cannot be long before academic credit is gained for reviewing others work – signed reviews are the first step. Sharing is required by fundersAs well as enabling critique, discussion, repeatibility, reproducibility,reusabilityIt’s about reuse for improved efficiency and productivity
- A metajournal which encourages the publication of information that encourages the reuse of software.A way of using the current tools and practices to make software betterrecognised.
- The social stigma of sharing data is very different in the next generation. Can we ensure it doesn’t get driven out of them by the generation above?“Lady and the Tramp”Photo by Joe ShlabotnikCC-BY-NC-SA
- Some further resources:http://www.software.ac.uk/blog/2012-11-09-craftsperson-and-scholarhttp://software.ac.uk/blog/2012-08-16-what-research-software-community-and-why-should-you-carehttp://www.software.ac.uk/blog/2011-05-02-publish-or-be-damned-alternative-impact-manifesto-research-softwarehttp://www.software.ac.uk/software-evaluation-guidehttp://www.software.ac.uk/software-carpentryhttp://www.software.ac.uk/resources/guideshttp://www.software.ac.uk/training
- Data Sharing and Management Snafu in 3 Short Actsby Karen Hanson, Alisa Surkis and Karen Yacobucci of NYU Health Sciences LibrariesLicensed under CC-BY