Docs / Developer Test Suite
GitHub

This page is a developer-focused guide to OpenTestability testing strategy, execution, and debugging.

1. Test suite goals

2. Test directory layout

tests/
|-- conftest.py
|-- fixtures/
|-- unit/
|   |-- core/
|   |-- yosys/
|   `-- utils/
|-- integration/
|   `-- yosys_mode/
`-- system/

Typical intent:

3. Marker map

Marker Purpose Typical command
smoke fastest confidence checks pytest tests/ -m smoke -v
unit isolated module logic pytest tests/ -m unit -v
integration cross-module behavior pytest tests/ -m integration -v
system end-to-end flow validation pytest tests/ -m system -v
yosys yosys mode specific tests pytest tests/ -m yosys -v

4. Running tests locally

Linux/macOS workflow (on Windows, use WSL; or nix develop for a one-command shell):

cd <path-to>/OpenTestability
source venv/bin/activate

# All tests
pytest tests/ -v

# Fast loop during development
pytest tests/ -m smoke -v
pytest tests/ -m unit -v

# Yosys-mode validation
pytest tests/ -m yosys -v

Useful targeted runs:

pytest tests/unit/utils/test_logging_config.py -v
pytest tests/unit/utils/test_session_manager.py -v
pytest tests/integration/test_verbose_logging.py -v
pytest tests/system/test_s27.py -v
  1. add or update unit tests close to changed logic
  2. add at least one integration test for the affected flow
  3. run marker subsets first (smoke, unit)
  4. run full affected marker class (integration or system) before merge
  5. run the full integration suite when touching shared code

6. Fixtures and shared setup

7. Reports and artifacts during testing

Depending on test type and verbosity:

When writing tests, prefer temporary paths/fixtures to avoid polluting shared output directories.

8. Common failure patterns

9. Developer PR checklist