SPEC Purpose and Process

Description#

The SPEC process is designed to identify areas of shared concern between projects in the scientific Python ecosystem and to produce collaboratively written, community adopted guidelines for addressing these. Such guidelines are known as SPECs: Scientific Python Ecosystem Coordination documents.

Specifically, the purpose of the SPEC process is

to help unify the ecosystem for users and developers;
to provide guidance to projects on technical issues or processes;
to document standard APIs, development tools, and community practices; and
to foster ecosystem-wide discussions of common problems and to develop shared solutions.

Projects in the ecosystem have an existing, diverse set of proposal processes and development constraints. SPECs complement these: they are a mechanism to encourage shared practices and improve uniformity of experience across projects. SPECs may, for example, capture established practices so that new projects can learn from them; or they may propose a new practice that the authors believe will benefit the ecosystem as a whole.

Projects decide for themselves whether to adopt any given SPEC—often, this would be through team consensus. A SPEC may not be a good fit for every single project, and thus there is no expectation that all SPECs must be adopted by all projects. That said, SPECs serve their purpose through being adopted by several projects—and their authority stems from the extent to which they are.

Participants in the SPEC process must adhere to our Code of Conduct.

Key Terms#

Scientific Python Ecosystem: The ecosystem is a loose federation of community developed Python projects widely used in scientific research that interact well with one another and that follow similar norms of development, documentation, testing, and so forth.
SPEC Core Projects: The Core Projects are a small subset of the ecosystem consisting of mature, community developed projects that are (a) depended upon by most of the other projects and (b) responsible for reviewing, discussing, implementing, and endorsing SPEC documents.
SPEC Steering Committee: The Steering Committee leads the SPEC project and manages the SPEC process including moderating the SPECs discussion forum, accepting SPEC documents, and maintaining the SPEC process documents.
SPEC Process: The SPEC process is outlined in this document and the associated SPEC Steering Committee and SPEC Core Projects documents. This process is managed and overseen by the Steering Committee, and functions in collaboration with the Core Projects, community members, and projects across the ecosystem.
SPEC Document: A SPEC document provides operational guidelines for projects and helps coordinate the ecosystem to provide a more unified experience for users and developers. These documents are developed collaboratively with the Core Projects and other interested ecosystem projects and community members.

Format#

SPECs are UTF-8 encoded text files using Markdown format and stored in the SPEC repository. The SPEC documents are converted to HTML by code in the scientific-python.org repository using Hugo and deployed to https://scientific-python.org/specs/. Each SPEC has a corresponding discussion with the same title, where anyone can comment, ask questions, or vote on existing comments.

Implementation#

The Steering Committee manages the SPEC process and will provide guidance to contributors throughout the process. In this section, we provide an overview of the main decision points in the SPEC process and provide guidance for how to get started with a new SPEC proposal.

Decision Points#

A SPEC passes through three decision points over the course of its development and implementation: Accept, Endorse, and Adopt.

graph LR

START[ ]--> |Propose| A[Accept]
A--> |Draft| B[Endorse]
B--> |Recommend| C[Adopt]

click A callback "Steering Committee Action"
click B callback "Core Project Action"
click C callback "Ecosystem Action"

style START fill:#FFFFFF, stroke:#FFFFFF;

The accept decision is made by the Steering Committee and means the proposed SPEC is accepted as a draft and added to the main branch of the SPEC repository. Proposed SPECs are accepted once (a) the draft is written to clearly explain the area of common concern and a general approach to a shared solution and (b) there are contributors (from at least two Core Projects) interested in working on the new SPEC and in championing it to their projects as well as the larger community. Additional details may be found in Steering Committee documentation.

The endorse decision is made by the Core Projects. The Core Projects and interested community members revise the accepted SPEC in a collabortive and iterative process focused on ensuring the SPEC implementation plan that is broadly applicable and likely to be widely adopted. The intent is that most SPECs will have several authors from numerous projects including several Core Projects. A SPEC is recommended for wide-spread adoption once it is endorsed by two (or more) Core Projects. Once a SPEC is recommended, substantive changes require the approval of all endorsing Core Projects. Additional details may be found in Core Project documentation.

The adopt decision is made by individual projects according to their own decision-making processes. Any project in the ecosystem is welcome to adopt a SPEC at any point. However, it may make sense to wait until a SPEC is endorsed by several Core Projects. This ensures that the SPEC has been vetted and deemed stable enough for widespread adoption. Once a SPEC is endorsed by several Core Projects it may still evolve, but the barrier for modifying the SPEC will increase substantially (since all endorsing projects would need to agree to changes). Projects that adopt a SPEC early should engage in the collaborative process leading to the SPEC being endorsed by the Core Projects. Each SPEC describes what adopting it means in its Ecosystem Adoption section.

New SPEC Proposals#

A good SPEC proposal focuses on a single key recommendation or idea for coordinating projects in the scientific Python ecosystem. Before proposing a SPEC, we highly recommended that you first vet the idea by doing one or more of the following:

discuss the idea with at least one project in the ecosystem,
discuss the idea with at least one other member of the ecosystem, or
create a minimal, proof of concept prototype.

Before a proposed SPEC can be accepted, the idea must be discussed on the discussion forum under the SPECS/Ideas topic. Thereafter a new SPEC document must be submitted as a pull request to the SPEC repository.

Use the quickstart.py script to create the new SPEC document. Located at the top-level of the SPEC repository, the script will ask you a few questions¹ and then create a new file appropriately named with a basic template for you to complete (e.g., spec-0000/index.md). Once the SPEC is in reasonable shape, file a pull request against the SPEC repository. The Steering Committee then considers the SPEC as presented in the pull request and will provide additional guidance.

Notes#

When asked to enter the SPEC number, choose the next available number that has not yet been used. Before the SPEC is merged, the Steering Committee may ask you to change the SPEC number so that it doesn’t conflict with another pull request. If so, just rename the file as appropriate and update the SPEC number in the title field of the SPEC header.

The script currently only supports adding one author. If you need to add additional authors, just edit the text file.

Additional files associated with a SPEC document may be kept in the directory containing the SPEC. For example, files associated with spec-0000/index.md are in spec-0000/.

Leave the draft field set to true and the endorsed-by field empty. ↩︎