Docs / Developer Test Suite
GitHub

This page is a developer-focused guide to OpenTestability testing strategy, execution, and debugging.

1. Test suite goals

2. Test directory layout

tests/
|-- conftest.py
|-- fixtures/
|-- unit/
|   |-- core/
|   |-- yosys/
|   `-- utils/
|-- integration/
|   |-- genus_mode/
|   `-- yosys_mode/
`-- system/

Typical intent:

3. Marker map

Marker Purpose Typical command
smoke fastest confidence checks pytest tests/ -m smoke -v
unit isolated module logic pytest tests/ -m unit -v
integration cross-module behavior pytest tests/ -m integration -v
system end-to-end flow validation pytest tests/ -m system -v
genus genus mode specific tests pytest tests/ -m genus -v
yosys yosys mode specific tests pytest tests/ -m yosys -v

4. Running tests locally

WSL/Linux workflow:

cd /mnt/c/Users/Potato/Desktop/OpenTestability
source venv/bin/activate

# All tests
pytest tests/ -v

# Fast loop during development
pytest tests/ -m smoke -v
pytest tests/ -m unit -v

# Mode-specific validation
pytest tests/ -m genus -v
pytest tests/ -m yosys -v

Useful targeted runs:

pytest tests/unit/utils/test_logging_config.py -v
pytest tests/unit/utils/test_session_manager.py -v
pytest tests/integration/test_verbose_logging.py -v
pytest tests/system/test_s27.py -v
  1. add or update unit tests close to changed logic
  2. add at least one integration test for the affected flow (--genus or --yosys)
  3. run marker subsets first (smoke, unit)
  4. run full affected marker class (integration or system) before merge
  5. confirm no mode-specific regressions by running both genus and yosys markers when touching shared code

6. Fixtures and shared setup

7. Reports and artifacts during testing

Depending on test type and verbosity:

When writing tests, prefer temporary paths/fixtures to avoid polluting shared output directories.

8. Common failure patterns

9. Developer PR checklist