In October 2023, the United States National Institutes of Health (NIH) released a Request for Information (RFI) asking for input on best practices for sharing NIH supported research software. NIH specifically requested responses to the following:
- Comment on the current NIH Best Practices for Sharing Research Software.
- Describe how, when, and where you share your research software. What, if any, resources for best practices do you rely upon to make your shared software open and reusable?
- What existing standards or criteria do you use to evaluate the openness, FAIRness, quality, and/or security of the software you share or reuse?
- Describe the collaborative settings in which you develop and share research software. Name communities or organizations, if any, you participate in that are actively promoting or developing software sharing best practices.
- What factors influence your decision to share or reuse your research software (or not)? What technical, policy, financial, institutional, and/or social barriers to sharing or reuse of research software have you encountered?
- Comment on your ability to reuse open-source research software developed by others. Describe factors used to determine whether to reuse existing research software or develop anew.
- How can NIH support research software communities of practice to better aid development of best practices for sharing and reuse of high-quality research software?
- Comment on any other topic which may be relevant for NIH to consider in enhancing the sharing of research software.
Responses were due by February 1, 2024. The SciCodes Consortium, which represents over 35 research software registries, submitted a response to this RFI that offered general and specific comments and suggested improvements to the NIH software guidelines based on the diverse experience and expertise of our members. Our general comments are below; our specific feedback is in the full response to the RFI and is available on Zenodo.
We believe the current FAQ-style NIH Best Practices for Sharing Research Software can be improved and made easier to follow and implement. While a FAQ is useful as discoverable, supplemental information for those with specific question(s), a prioritized checklist with clear and actionable step-by-step instructions for researchers (e.g., the machine learning reproducibility checklist) might be of greater use, with each best practice turned into a declarative statement. For example, the first best practice could be stated as “Make your research software open,” with additional guidance and concrete examples as to how this can be done.
Further, the Best Practices do not cover all the requirements of the FAIR Principles for Research Software (https://doi.org/10.15497/RDA00068). A detailed analysis of the current NIH best practices and FAIR4RS principles is available at https://fair-biors.org/docs/crosswalk.
The FAIR Biomedical Research Software (FAIR-BioRS) Guidelines aims to provide a clear and accessible list of specific best practices with concrete examples (https://doi.org/10.1038/s41597-023-02463-x, https://fair-biors.org). We suggest using these as a baseline for the NIH Best Practices for Sharing Research Software, recognizing that the NIH Best Practices may cover a broader range of topics than the FAIR-BioRS guidelines.