An Overview Of the Peer Review Process#

pyOpenSci open peer review steps#

There are several components to the pyOpenSci peer review process. Below, we overview the entire process from start to finish.

Step 0. optional : Author submits pre-submission inquiry#

A presubmission inquiry is useful if you are unsure whether your package is in scope. To submit a pre-submission inquiry, open up an issue using the presubmission template in our pyopensci/software-review repository. During this time an editor in chief will review for scope and perform a basic check for package infrastructure.

  • Estimated time: ~1-2 weeks

Below, are the basic checks that your package should have prior to being submitted for peer review. These are the checks that an editor in chief and/or editor will look at when evaluating your package for review.

## Editor in Chief checks

Hi there! Thank you for submitting your package for pyOpenSci
review. Below are the basic checks that your package needs to pass
to begin our review. If some of these are missing, we will ask you
to work on them before the review process begins.

Please check our [Python packaging guide](https://www.pyopensci.org/python-package-guide) for more information on the elements
below.

- [ ] **Installation** The package can be installed from a community repository such as PyPI (preferred), and/or a community channel on conda (e.g. conda-forge, bioconda).
  - [ ] The package imports properly into a standard Python environment `import package`.
- [ ] **Fit** The package meets criteria for [fit](https://www.pyopensci.org/software-peer-review/about/package-scope.html#what-types-of-packages-does-pyopensci-review) and [overlap](https://www.pyopensci.org/software-peer-review/about/package-scope.html#package-overlap).
- [ ] **Documentation** The package has sufficient online documentation to allow us to evaluate package function and scope *without installing the package*. This includes:
  - [ ] User-facing documentation that overviews how to install and start using the package.
  - [ ] Short tutorials that help a user understand how to use the package and what it can do for them.
  - [ ] API documentation (documentation for your code's functions, classes, methods and attributes): this includes clearly written docstrings with variables defined using a standard docstring format.
- [ ] Core GitHub repository Files
  - [ ] **README** The package has a `README.md` file with clear explanation of what the package does, instructions on how to install it, and a link to development instructions.
  - [ ] **Contributing File** The package has a `CONTRIBUTING.md` file that details how to install and contribute to the package.
  - [ ] **Code of Conduct** The package has a `CODE_OF_CONDUCT.md` file.
  - [ ] **License** The package has an [OSI approved license](https://opensource.org/licenses).
NOTE: We prefer that you have development instructions in your documentation too.
- [ ] **Issue Submission Documentation** All of the information is filled out in the `YAML` header of the issue (located at the top of the issue template).
- [ ] **Automated tests** Package has a testing suite and is tested via a Continuous Integration service.
- [ ] **Repository** The repository link resolves correctly.
- [ ] **Package overlap** The package doesn't entirely overlap with the functionality of other packages that have already been submitted to pyOpenSci.
- [ ] **Archive** (JOSS only, may be post-review): The repository DOI resolves correctly.
- [ ] **Version** (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?

---
- [ ] [Initial onboarding survey was filled out ](https://forms.gle/F9mou7S3jhe8DMJ16)
We appreciate each maintainer of the package filling out this survey individually. :raised_hands:
Thank you authors in advance for setting aside five to ten minutes to do this. It truly helps our organization. :raised_hands:
---

*******

## Editor comments

1. Author submits their package for review#

To do this, you open an issue using the software submission template in our pyopensci/software-review repository.

2. Editor in chief reviews package submission#

The editor in chief will review your submission at this point for both package scope and minimal infrastructure criteria (listed above).

  • TIME ~2 weeks (or longer if editor requests changes that take the author longer to implement)

3. Editor finds reviewers for package#

At this point if your package has the minimal infrastructure requirements and is in scope, the editor in chief will assign an editor to review your package. That editor will then identify suitable reviewers.

Time: ~2-3 weeks

4. Peer review of submitted Python Package begins#

Once we have an editor and 2 reviewers on board, review begins. Reviewers have 3 weeks to return a review. To do this, they will use our reviewer template in the reviewer guide and paste that, filled out, into the issue.

TIME: ~3 weeks

5. Author responds to reviews#

At this point the authors should respond to the review. We prefer that authors respond within 2 weeks of the submitted review. We also understand that it may take longer to actually implement the changes requested in the review. However, we kindly request that authors respond to reviews to acknowledge that they have seen them.

The reviewers are encouraged to open pull requests and issues to help the maintainers update their package.

Often there is some back and forth between reviewers and maintainers at this step.

There is no specified duration for this period. Rather as long as all parties are responsive within 2 weeks, the review shall continue until the author has completed work to address the reviews.

6. Package acceptance#

Once the maintainers have completed updating the package, the assigned editor will ask the reviewers if they are happy with changes made. At this point the editor performs one last check on the package and accepts it if that is appropriate.

Now, there are a few final cleanup activities including:

  • Adding the pyOpenSci peer-reviewed badge to the package README file.

  • Creating a new release on GitHub from the reviewed version.

  • Adding package authors and the package to the pyOpenSci website.

The package is now accepted into the pyOpenSci ecosystem!

JOSS submission#

JOSS refers to the Journal of Open Source Software. If the maintainer wishes, and their package is within JOSS’ scope, they can now be fast tracked through the JOSS review process (another review is not required for this step).

Peer review guides#

If you want to learn more about each step listed above, we suggest that you read through the peer-review guides below that are tailored to each role in the peer review process:

Author / Maintainer Guide

Learn everything that you need to know about the peer review for package maintainers who submit a package to pyOpenSci for peer review.

Peer Review Guide for Python Open Source Authors / Authors
Editor Guide

Learn about the process that editors follow in the pyOpenSci peer review process.

pyOpenSci Software Review Editor Guide
Reviewer Guide

Click here to read more about the process that reviewers take when reviewing a Python package for pyOpenSci.

Guide for Reviewers
✨ Editor in chief Guide

The editor in chief is a rotating position within pyOpenSci held by members of the pyOpenSci editorial board. Learn more about the processes involved with being an editor in chief for pyOpenSci.

Editor in Chief Guide