QA Code Checklist
Quality assurance checklist
This checklist is meant as a guide to support quality assurance of code. It is split into 3 standards of bronze, silver and gold. Good practice is to achieve the highest level possible, appropriate to the analysis.
The standards are not mutually exclusive, it would be expected to meet bronze standards as a minimum and achieve as many of the higher standards as appropriate.
The coding standards are independent to the ‘tiering’ of the piece of work. Always try to achieve the highest appropriate coding standards to your work.
Bronze Standard
Good coding practices
☐ Names used in the code are informative and concise.
☐ Names used in the code are explicit, rather than implicit.
☐ Code logic is clear and avoids unnecessary complexity.
• http://adv-r.had.co.nz/Style.html
• [https://peps.python.org/pep-0008/ ] (https://peps.python.org/pep-0008/)
• https://docs.telemetry.mozilla.org/concepts/sql_style
Code documentation
☐ Comments are used to describe why code is written in a particular way, rather than describing what the code is doing.
☐ Comments are kept up to date, so they do not confuse the reader.
☐ Code is not commented out to adjust which lines of code run.
Project documentation
☐ A README file details the purpose of the project, basic installation instructions, and examples of usage.
☐ Where appropriate, guidance for prospective contributors is available including a code of conduct.
☐ If the code’s users are not familiar with the code, desk instructions are provided to guide lead users through example use cases.
☐ Copyright and licenses are specified for both documentation and code.
☐ Assumptions in the analysis and their quality are documented next to the code that implements them. These are also made available to users.
Version control
☐ Code is version controlled using Git.
☐ Code is committed regularly, preferably when a discrete unit of work has been completed.
☐ Code is open-sourced. Any sensitive data are omitted or replaced with dummy data.
Configuration
☐ Credentials and other secrets are not written in code but are configured as environment variables or input separately.
☐ Configuration is stored in a dedicated configuration file, separate to the code.
☐ Required passwords, secrets and tokens are documented, but are stored outside of version control.
• https://cran.r-project.org/web/packages/httr/vignettes/secrets.html
Peer review
☐ Users are encouraged to participate in peer review.
Dependency management
☐ There are as few dependencies as possible.
☐ Dependencies are documented.
Project management
☐ The roles and responsibilities of team members are clearly defined.
☐ New issues or tasks are guided by users’ needs and stories.
Silver Standard
Modular code
☐ Main analysis scripts import and run high level functions from the package.
☐ Repetition in the code is minimalised. For example, by moving reusable code into functions or classes.
Functional programming
☐ Repetition in the code is minimalised. For example, by moving reusable code into functions or classes.
☐ Individual pieces of logic are written as functions. Classes are used if more appropriate.
☐ Low level functions and classes carry out one specific task. As such, there is only one reason to change each function.
☐ All functions and classes are documented to describe what they do, what inputs they take and what they return.
☐ Objects and functions are open for extension but closed for modification; functionality can be extended without modifying the source code.
Project Structure
☐ Code is grouped in themed files (modules) and is packaged for easier use.
☐ A clear, standard directory structure is used to separate input data, outputs, code and documentation.
☐ Input data are stored safely and are treated as read-only.
Testing
☐ The whole process is tested from start to finish using one or more realistic end-to-end tests.
• https://r-pkgs.org/testing-basics.html
Code Style
☐ Code follows a standard style
• [https://www.python.org/dev/peps/pep-0008/](https://www.python.org/dev/peps/pep-0008/)
• [https://google.github.io/styleguide/Rguide.html](https://google.github.io/styleguide/Rguide.html)
• https://style.tidyverse.org/
• https://docs.telemetry.mozilla.org/concepts/sql_style
Documentation
☐ Python code is documented using docstrings. R code is documented using roxygen2 comments.
• https://www.python.org/dev/peps/pep-0257
• https://cran.r-project.org/web/packages/roxygen2/vignettes/roxygen2.html
☐ Human-readable (preferably HTML) documentation is generated automatically from code documentation.
☐ Documentation is hosted for easy access. GitHub Pages https://pages.github.com/ and Read the Docs https://readthedocs.org/ provide a free service for hosting documentation publicly.
☐ Instructions for how to cite the project are given.
Version Control
☐ An appropriate branching strategy is defined and used throughout development.
☐ Committing standards are followed such as appropriate commit summary and message supplied.
☐ Commits are tagged at significant stages. This is used to indicate the state of code for specific releases or model versions.
☐ Input data are versioned. All changes to the data result in new versions being created, or changes are recorded as new records
• https://en.wikipedia.org/wiki/Slowly_changing_dimension
Publishing
☐ Published outputs meet accessibility regulations
• https://analysisfunction.civilservice.gov.uk/area_of_work/accessibility/
Project management
☐ Outputs from your analysis are disposable and are regularly deleted and regenerated while analysis develops. Your analysis code is able to reproduce them at any time.
☐ Misuse or failure in the code produces informative error messages.
☐ Quality assurance standards and processes for the project are defined. These are based around the quality assurance of code for analysis and research guidance document:
• https://best-practice-and-impact.github.io/qa-of-code-guidance/intro.html
Testing
☐ Core functionality is unit tested as code. See pytest for Python and testthat for R.
☐ Code based tests are run regularly.
☐ Bug fixes include implementing new unit tests to ensure that the same bug does not reoccur.
☐ Informal tests are recorded near to the code.
☐ Test code is clean and readable. Tests make use of fixtures and parametrisation to reduce repetition.
Dependency management
☐ Required libraries and packages are documented, including their versions.
☐ Package dependencies are managed using an environment manager such as virtualenv for Python or renv for R.
Gold Standard
Processes & Project management
☐ Main analysis scripts import and run high level functions from the package.
☐ Subclasses retain the functionality of their parent class while adding new functionality. Parent class objects can be replaced with instances of the subclass and still work as expected.
☐ The roles and responsibilities of team members are clearly defined.
☐ An issue tracker (e.g GitHub Project, Trello or Jira) is used to record development tasks.
☐ New issues or tasks are guided by users’ needs and stories.
☐ Issues templates are used to ensure proper logging of the title, description, labels and comments.
☐ Acceptance criteria are noted for issues and tasks. Fulfilment of acceptance criteria is recorded.
Communications
☐ Releases of the project used for reports, publications, or other outputs are versioned using a standard pattern such as semantic versioning.
☐ A summary of changes to functionality are documented in a changelog following releases. The changelog is available to users.
☐ Stakeholder or user acceptance sign-offs are recorded near to the code.
Package production
☐ Example usage of packages and underlying functionality is documented for developers and users.
☐ Design certificates confirm that the design is compliant with requirements.
☐ If appropriate, the software is fully specified.
CI/CD
☐ Continuous integration is applied through tools such as GitHub Actions, to ensure that each change is integrated into the workflow smoothly.
Testing
☐ Test are automatically run and recorded using continuous integration or git hooks.
☐ Formal user acceptance testing is conducted and recorded.
☐ Integration tests ensure that multiple units of code work together as expected.
Configuration
☐ If appropriate, multiple configuration files are used depending on system/local/user.
☐ Configuration files are version controlled separately to the analysis code, so that they can be updated independently.
☐ The configuration used to generate particular outputs, releases and publications is recorded.
☐ Example configuration file templates are provided alongside the code, but do not include real data.
Data management
☐ All input data is documented in a data register, including where they come from and their importance to the analysis.
☐ Non-sensitive data are made available to users. If data are sensitive, dummy data is made available so that the code can be run by others.
☐ Data quality is monitored, as per the government data quality framework.
☐ Fields within input and output datasets are documented in a data dictionary.
☐ Large or complex data are stored in a database.
☐ Data are documented in an information asset register.
Dependency management & deployment
☐ Working operating system environments are documented.
☐ Example configuration files are provided.
☐ Where appropriate, code runs independent of operating system (e.g. suitable management of file paths).
☐ Dependencies are managed separately for users, developers, and testers.
☐ Docker containers or virtual machine builds are available for the code execution environment and these are version controlled.
Logging
☐ Code configuration is recorded when the code is run.
☐ Pipeline route is recorded if decisions are made in code.
Quality Assurance Log
Issues Found and need to be addressed
| Criteria | Notes | Mitigation | Date |
|---|---|---|---|
QA Conducted and met criteria for bronze / silver / gold RAP process
| Reviewed by | Date | RAP Level | Notes |
|---|---|---|---|