pyOpenSci open peer review steps
There are several components to the pyOpenSci peer review process. Below, we overview the entire process
from start to finish.
Step 0. optional : Author submits pre-submission inquiry
A presubmission inquiry is useful if you are unsure whether your package
is in scope. To submit a pre-submission inquiry, open up an issue using the presubmission template in our pyopensci/software-review repository. During this time an editor in chief will review for scope and perform
a basic check for package infrastructure.
Below, are the basic checks that your package should have prior to being
submitted for peer review. These are the checks that an editor in chief and/or
editor will look at when evaluating your package for review.
## Editor in Chief checks
Hi there! Thank you for submitting your package for pyOpenSci
review. Below are the basic checks that your package needs to pass
to begin our review. If some of these are missing, we will ask you
to work on them before the review process begins.
Please check our [Python packaging guide](https://www.pyopensci.org/python-package-guide) for more information on the elements
below.
- [ ] **Installation** The package can be installed from a community repository such as PyPI (preferred), and/or a community channel on conda (e.g. conda-forge, bioconda).
- [ ] The package imports properly into a standard Python environment `import package`.
- [ ] **Fit** The package meets criteria for [fit](https://www.pyopensci.org/software-peer-review/about/package-scope.html#what-types-of-packages-does-pyopensci-review) and [overlap](https://www.pyopensci.org/software-peer-review/about/package-scope.html#package-overlap).
- [ ] **Documentation** The package has sufficient online documentation to allow us to evaluate package function and scope *without installing the package*. This includes:
- [ ] User-facing documentation that overviews how to install and start using the package.
- [ ] Short tutorials that help a user understand how to use the package and what it can do for them.
- [ ] API documentation (documentation for your code's functions, classes, methods and attributes): this includes clearly written docstrings with variables defined using a standard docstring format.
- [ ] Core GitHub repository Files
- [ ] **README** The package has a `README.md` file with clear explanation of what the package does, instructions on how to install it, and a link to development instructions.
- [ ] **Contributing File** The package has a `CONTRIBUTING.md` file that details how to install and contribute to the package.
- [ ] **Code of Conduct** The package has a `CODE_OF_CONDUCT.md` file.
- [ ] **License** The package has an [OSI approved license](https://opensource.org/licenses).
NOTE: We prefer that you have development instructions in your documentation too.
- [ ] **Issue Submission Documentation** All of the information is filled out in the `YAML` header of the issue (located at the top of the issue template).
- [ ] **Automated tests** Package has a testing suite and is tested via a Continuous Integration service.
- [ ] **Repository** The repository link resolves correctly.
- [ ] **Package overlap** The package doesn't entirely overlap with the functionality of other packages that have already been submitted to pyOpenSci.
- [ ] **Archive** (JOSS only, may be post-review): The repository DOI resolves correctly.
- [ ] **Version** (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?
---
- [ ] [Initial onboarding survey was filled out ](https://forms.gle/F9mou7S3jhe8DMJ16)
We appreciate each maintainer of the package filling out this survey individually. :raised_hands:
Thank you authors in advance for setting aside five to ten minutes to do this. It truly helps our organization. :raised_hands:
---
*******
## Editor comments
2. Editor in chief reviews package submission
The editor in chief will review your submission at this point for both package scope and minimal infrastructure criteria
(listed above).
3. Editor finds reviewers for package
At this point if your package has the minimal infrastructure
requirements and is in scope, the editor in chief will assign an editor
to review your package. That editor will then identify
suitable reviewers.
Time: ~2-3 weeks
4. Peer review of submitted Python Package begins
Once we have an editor and 2 reviewers on board, review begins. Reviewers have 3 weeks to return a review. To do this,
they will use our reviewer template in the reviewer guide and paste that, filled out, into the issue.
TIME: ~3 weeks
5. Author responds to reviews
At this point the authors should respond to the review. We prefer that authors
respond within 2 weeks of the submitted review. We also understand that it may
take longer to actually implement the changes requested in the review. However, we
kindly request that authors respond to reviews to acknowledge
that they have seen them.
The reviewers are encouraged to open pull requests and issues to help the
maintainers update their package.
Often there is some back and forth between reviewers and maintainers at this step.
There is no specified duration for this period. Rather as long as all
parties are responsive within 2 weeks, the review shall continue until the author has completed work to address the reviews.
6. Package acceptance
Once the maintainers have completed updating the package, the assigned editor
will ask the reviewers if they are happy with changes made. At this point the
editor performs one last check on the package and accepts it if that is appropriate.
Now, there are a few final cleanup activities including:
Adding the pyOpenSci peer-reviewed badge to the package README file.
Creating a new release on GitHub from the reviewed version.
Adding package authors and the package to the pyOpenSci website.
The package is now accepted into the pyOpenSci ecosystem!
JOSS submission
JOSS refers to the Journal of Open Source Software. If the maintainer wishes, and their package is within JOSS’ scope, they can now
be fast tracked through the JOSS review process (another review is not required
for this step).