Editor in Chief Guide#
Useful pyOpenSci resources
Here are some helpful resources to support you in your Editor in Chief role:
Python Package Guide: A community-developed guide offering an overview of the Python packaging ecosystem, plus tutorials for building and publishing a package.
pyOpenSci metrics dashboard: View the current state of peer reviews and track review activity over time.
pyOpenSci Python package template: A Copier-powered template for creating a new Python package that follows pyOpenSci best practices.
Editor in Chief (EiC) role & responsibilities#
The Editor in Chief (EiC) is a rotating position, typically selected from the current editorial team. It may also be held by a member of the broader pyOpenSci community, who is familiar with our organization and software peer review.
The EiC term lasts 3 to 6 months, or another duration agreed upon with the Peer Review Lead (currently the Executive Director). If the EiC needs to step down early, we request at least two weeks’ notice to allow time to identify and onboard a replacement.
The Editor in Chief fulfills the roles discussed below:
1. Manage incoming reviews#
Watch all issues in the software-review repo
Tip
Enable GitHub notifications and join the #feed-software-review Slack channel
to stay up to date on new submissions.
Perform the initial editor checks on new submissions using the editor checklist.
Flag scope or overlap concerns. Raise questions by tagging editors in
#software-reviewor, if needed, in#private-editorial-team.Assign submissions to handling editors based on:
subject matter fit
editor availability
The EiC may assign a package to themselves if they have capacity and want to lead the review.
2. Manage pre-submission inquiries#
Respond to pre-submission inquiries and assess package scope or structure.
Handle or delegate review referrals from the Journal of Open Source Software (JOSS) or other partners.
3. Monitor review progress#
Check the status of active reviews monthly. Use the peer review dashboard to identify stalled reviews or submissions needing editors or reviewers.
The dashboard highlights:
Packages still seeking reviewers
Packages needing an editor
Open pre-submission inquiries
Stale issues with no recent comments
If a review has been inactive for 3–4 weeks, message the editor in Slack to
check in. You or the Peer Review Lead can help move things forward. If needed,
discuss in the #private-editorial-team channel.
Tip
Need help unblocking a review? Contact the acting Peer Review Lead.
4. Monitor package maintenance#
Once or twice during your term, review our list of accepted packages and flag any that appear unmaintained. If needed, follow the archiving process.
5. Support EiC transition#
As your term nears its end, discuss with the peer-review lead the process of identifying your successor. Set a calendar reminder 2–3 weeks in advance to start the transition process.
Note
See this section for guidance on responding to out-of-scope pre-submissions.
Note
If the EiC has limited time to handle pre-review checks for a package, a conflict of interest, or lacks relevant expertise, they may ask another editor to perform initial checks on a package at any time.
Editor in Chief pre-review submission checklist#
The EiC is also responsible for determining the scope of packages submitted via a pre-submission inquiry.
The steps include:
Determine if the package is in scope for review: Have a look at the pre-submission inquiry. Use our project scope page to determine whether that package is in scope for pyOpenSci review.
If the package is in scope, add the
submission-requestedlabel to the pre-submission inquiry.Let the author know that
pre-reviewchecks are next and will happen in the full submission issue. At this stage, it’s a good idea to let the author know that the package will undergo pre-review checks upon submission for review. You can share the pre-review checks template with the author at this time if you wish
Note
It is OK to perform pre-review checks in the pre-review submission if you wish. However, for documentation purposes, you will need to perform them in the full submission issue, as that is what JOSS and other partners look at. Be sure that you aren’t duplicating your time!
Editor in Chief peer review checklist#
When a new package is submitted for review, the Editor in Chief should:
1. ✔️ Tag the issue with the 0/pre-review-checks label in GitHub#
3. ✔️ Add the pre-review checks to the issue#
When you add pre-review checks to the issue, remove the New Submission! label from the issue.
This lets people know that someone is working on the issue and moving it forward.
Important
Post your complete pre-review/editor checks in one single comment, rather than in multiple comments. This keeps the review easy to follow and lets the author address everything at once.
Before posting:
Review the editor-in-chief checklist.
Estimate how long the checks might take and share that with the author.
Perform all checks locally.
Post the completed checklist and any feedback in a single GitHub comment.
Copy the template below and add it to the issue.
These checks confirm that the package meets the minimum standards for review. In many cases, going through this process improves the package itself.
The EiC may ask another editor to complete the checks if:
They are overwhelmed with new submissions.
They are temporarily unavailable.
Another editor has relevant domain or technical expertise.
Editor-in-chief checklist#
Copy the template below and use it in the review issue.
## Editor in Chief checks
Hi there! Thank you for submitting your package for pyOpenSci
review. Below are the basic checks that your package needs to pass
to begin our review. If some of these are missing, we will ask you
to work on them before the review process begins.
Please check our [Python packaging guide](https://www.pyopensci.org/python-package-guide) for more information on the elements
below.
- [ ] **Installation** The package can be installed from a community repository such as PyPI (preferred), and/or a community channel on conda (e.g., conda-forge, bioconda).
- [ ] The package imports properly into a standard Python environment `import package`.
- [ ] **Fit** The package meets criteria for [fit](https://www.pyopensci.org/software-peer-review/about/package-scope.html#what-types-of-packages-does-pyopensci-review) and [overlap](https://www.pyopensci.org/software-peer-review/about/package-scope.html#package-overlap).
- [ ] **Documentation** The package has sufficient online documentation to allow us to evaluate the package's function and scope *without installing the package*. This includes:
- [ ] User-facing documentation that overviews how to install and start using the package.
- [ ] Short quickstart tutorials that help a user understand how to use the package and what it can do for them.
- [ ] API documentation (documentation for your code's functions, classes, methods, and attributes): this includes clearly written docstrings with variables defined using a standard docstring format.
- [ ] Core GitHub repository Files
- [ ] **README** The package has a `README.md` file with a clear explanation of what the package does, instructions on how to install it, and a link to development instructions.
- [ ] **Contributing File** The package has a `CONTRIBUTING.md` file that details how to install and contribute to the package.
- [ ] **Code of Conduct** The package has a `CODE_OF_CONDUCT.md` file.
- [ ] **License** The package has an [OSI approved license](https://opensource.org/licenses).
NOTE: We prefer that you have development instructions in your documentation too.
- [ ] **Issue Submission Documentation** All of the information is filled out in the `YAML` header of the issue (located at the top of the issue template).
- [ ] **Automated tests** Package has a testing suite and is tested via a Continuous Integration service.
- [ ] **Repository** The repository link resolves correctly.
- [ ] **Package overlap** The package doesn't entirely overlap with the functionality of other packages that have already been submitted to pyOpenSci.
- [ ] **Archive** (JOSS only, may be post-review): The repository DOI resolves correctly.
- [ ] **Version** (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?
**Optional:** Let projects know that it's a great idea for projects to have a .github repository for the project organization where they can host a commonly used LICENSE, Code of Conduct, and even a YAML file with label definitions. These items will then be automatically applied to every repository in the organization to ensure consistency (but can be customized within repos too). The [SunPy project](https://github.com/sunpy/.github/) has a great example of this.
---
- [ ] [Initial onboarding survey was filled out ](https://forms.gle/F9mou7S3jhe8DMJ16)
We appreciate each maintainer of the package filling out this survey individually. :raised_hands:
Thank you, authors, in advance for setting aside five to ten minutes to do this. It truly helps our organization. :raised_hands:
---
*******
## Editor comments
4. ✔️ Ensure that the package onboarding survey is complete#
Use this spreadsheet to check whether the author—and ideally all maintainers—have completed the pre-review survey. You can search using their GitHub handle.
Thank the author for completing the survey. While some sections are optional, we do need contact information to follow up about package maintenance. The survey also helps us track and improve our peer review process.
5. ✔️ Assign an editor and add the 1/editor-assigned label to the GitHub issue#
Once the package is determined to be in scope, the pre-review surveys are complete, and pre-review checks are done, it’s time to assign an editor.
Use the editorial dashboard to check which editors are currently available and what domains those editors are familiar with.
Once you’ve selected an editor:
Assign the issue to that editor in GitHub This step helps us keep track of who is running which reviews. Please do not forget to do this!
Update the editor section of the YAML at the top of the issue with their GitHub username
Add the
1/editor-assignedlabel to the issue
Note
If there isn’t an available editor with relevant expertise for a review, work with the Peer Review Lead to recruit a guest editor or onboard someone new.
Follow the onboarding guide, and complete the full onboarding process to ensure they have:
GitHub access
Slack access
Everything needed to manage the review
Share this document with any new editor to help them get started.
See the editor guide for more on an editor’s responsibilities. Once the editor is assigned, your work on the review is complete, and they will now begin identifying reviewers.
Diversity in the editorial & reviewer team is important
Inclusion is core to the pyOpenSci mission. It’s important to have an editorial team comprised of an editor + 2 reviewers from different backgrounds and identities to enrich the review process.
As you select an editor (and guide that editor in finding reviewers), seek diversity in the team supporting package review. Specifically, reviewers and the editor should have a mix of gender identities whenever possible. pyOpenSci also supports mentoring new reviewers and editors if needed!
6. ✔️ Update the YAML header#
After assigning the editor to the issue and adding the 1/editor-assigned label:
Update the YAML at the top of the issue with the editor’s GitHub username
Add any final comments to the bottom of your pre-review checklist comment
Important
Be sure to update both the GitHub assignment and the YAML header with the editor’s name. Our peer review dashboard depends on both!
Handling submissions that are incomplete or vague#
Sometimes a submission lacks enough information to assess whether it’s in scope. For example:
Online documentation is sparse
The README is minimal or unclear
There’s no documentation structure
The YAML template at the top of the issue is incomplete
In these cases, ask the author for more information. Even if the package is ultimately out of scope, this will still help improve its documentation.
You can adapt the message below when requesting more details:
Hello <username>, and thank you for your submission!
We are reviewing whether your package fits within pyOpenSci’s scope and
could use a bit more information.
Please expand your `README` to give more context. After reading it, someone
without domain expertise should understand the package’s goals, use cases,
and core functionality.
<optional>
If your package overlaps with others, please explain in the README and in
this issue how your package is "best in class." You can refer to our
[guidance on package overlap](https://www.pyopensci.org/software-peer-review/about/package-scope.html#package-overlap).
</optional>
Addressing challenges in the review process#
During the review, you may encounter challenges that require resolution.
For technical issues such as repository access, GitHub integration, or platform problems, post in the
#pyos-infrastructureSlack channel where our technical team can assist you.For behavior concerns including unprofessional communication, harassment, or guideline violations, use our pyOpenSci Code of Conduct as guidance. Follow the Code of Conduct reporting procedures and notify the Peer Review Lead of the issue.
For conflicts of interest involving financial, professional, or personal conflicts between reviewers and authors, you can post in the
private-editorial-channelSlack channel or contact the Peer Review Lead directly if you prefer a more confidential approach.For all other questions related to the review process, you can contact the Peer Review Lead for guidance and support.
Keeping the review process moving forward#
The pyOpenSci peer review metrics dashboard can help you keep tabs on review status.
Keeping track of reviews that need editors#
The editors needed table and reviewers needed table lists reviews that are still searching for editors and reviewers.
How to handle stalled reviews#
This table contains a list of all open reviews (and pre-review submissions).. You can sort it by last comment date to understand better what reviews have been “quiet” and might need a nudge to move forward.
If you notice that a review hasn’t been active in over 1+ months, do the following:
Leave a note on the review to see what the state of the review is.
If no one responds to the comment within ~2 weeks, consider DM’ing the editor in the pyOpenSci Slack to see if they are still able to move the review forward.
If a DM to the editor doesn’t work, consider sending an email next. Oftentimes, people lose track of GitHub notifications, but they respond to emails.
If all of the above fail and there is no movement on the review, post in the
#private-editorial-teamchannel in our Slack, and the peer review lead can help you move things forward. In some cases, you may need to find a new editor who has more bandwidth to complete the review. In other cases, the editor will step back in and get things moving again.
If you need to find a new editor, the editors available for a review lists editors that are not currently running a review.
Tip
If the editor is struggling with reviewers not responding, see the editor guide for more information on how to manage this
Responding to out-of-scope submissions#
If a package is determined to be out of scope, thank the author for their submission, briefly explain why it is not in scope, and suggest other venues for review if relevant.
Keep the conversation respectful and transparent. If the author has questions, continue the discussion in the GitHub issue.
Stepping down from the EiC role#
As your term ends, help onboard the next Editor in Chief.
Set a calendar reminder 2–3 weeks before your term ends and begin to communicate with the incoming editor-in-chief about the transition.
Craft a summary of where peer review is, and where the incoming Editor in Chief will need to pick up.
Let the Peer Review Lead know that a transition is coming soon so they can support as needed.
If you can, try to be available after your term has ended to answer questions and support a smooth transition for the new Editor in Chief
Important
If you are leading any active reviews as EiC, you must hand them off to the incoming Editor in Chief before stepping down. This ensures continuity and prevents reviews from stalling.
The new EiC assumes full responsibility for all active reviews and pre-submission inquiries previously managed by the outgoing EiC.
EiC Transition Checklist#
You can use the checklist below to manage the tasks.
### EiC Transition Checklist
- [ ] Set calendar reminder 3 weeks before term end
- [ ] Contact peer review lead about successor
- [ ] Document active reviews and their status
- [ ] Hand off any reviews you're personally handling
- [ ] Write reflection summary
- [ ] Schedule handoff meeting with incoming EiC