Commit e0dd3948 authored by Benoit Orihuela's avatar Benoit Orihuela
Browse files

doc: align the documentation-process instructions to the statistics-only model

parent e70e4008
Loading
Loading
Loading
Loading
+29 −35
Original line number Diff line number Diff line
@@ -16,7 +16,6 @@ Thank you for contributing! This document explains the conventions and rules all
  - [Keywords](#keywords)
  - [Test data files](#test-data-files)
- [Updating the documentation](#updating-the-documentation)
- [Running the unit tests](#running-the-unit-tests)
- [Self-review checklist](#self-review-checklist)

---
@@ -51,7 +50,7 @@ This project uses **[Conventional Commits](https://www.conventionalcommits.org/)
| `chore`    | Tooling, configuration, dependencies, CI/CD changes         |
| `refactor` | Internal restructuring with no behaviour change             |
| `docs`     | Documentation-only changes                                  |
| `test`     | Changes to the unit test suite under `doc/tests/`           |
| `test`     | Changes to the documentation generation tooling under `doc/`|

The scope (in parentheses) is optional but encouraged when the change is limited to a specific API group or
subsystem, e.g. `feat(subscription)`, `fix(temporal)`, `chore(robocop)`.
@@ -187,30 +186,39 @@ python3 scripts/find_unused_test_data.py

## Updating the documentation

The test suite ships with an auto-generated documentation system. Changes that affect the documentation must be
accompanied by the appropriate updates.
The test suite ships with an auto-generated documentation system that produces the deliverable listing every
Test Case of the Test Suite. The documentation is regenerated when a new version of the deliverable is needed:

**When a new Test Case is created:**

1. Declare it in `doc/tests/test_{group}.py`.
2. Run the documentation generator for the new TC:
   ```bash
   python doc/generateDocumentationData.py {new_tc_id}
   ```
3. Copy the generated file to the correct group folder:
```bash
   cp doc/results/{new_tc_id}.json doc/files/{group}/{subgroup}/
cd doc ; python statisticsDocumentationData.py
```
4. Update `doc/analysis/initial_setup.py` if a new setup keyword was introduced.
5. Update `doc/analysis/requests.py` if a new request parameter or endpoint was added.

**When a new permutation is added to an existing Test Case:**
This walks every `.robot` file and reports how many Test Cases failed to be parsed. The documentation is
considered healthy when it reports **`0` failures**. Run it after any change that could affect generation — and
between deliveries — to confirm the documentation still generates successfully and that no drift was introduced.

> **Note:** the per-Test-Case JSON files are **no longer versioned**. There is no manual "copy the generated
> JSON into `doc/files/`" step anymore: the generator writes to `doc/results/` (git-ignored), which is
> overwritten on each run. As a result, legitimate changes to a Test Case (adding/removing a tag, rewording its
> documentation, …) require no committed JSON update and never fail documentation generation.

Run the documentation generator for the affected TC and copy the output, as above.
For generation to keep succeeding, keep the following declarations in sync with the `.robot` files:

**When a new directory of Test Cases is created:**
- **New assertion keyword** — declare it in `doc/analysis/checks.py` (`self.checks` and `self.args`) with a
  method that returns its documentation string.
- **New endpoint keyword** — declare it in `doc/analysis/requests.py` (`self.op` and `self.description`) with
  its documentation method.
- **New request/query parameter** on an existing endpoint — add it to that endpoint's `expected_parameters` and
  its `match` cases in `doc/analysis/requests.py`.
- **New setup/teardown keyword** — declare it in `doc/analysis/initial_setup.py` (or reuse an existing name).
- **New directory of Test Cases** — declare it in `doc/analysis/generaterobotdata.py` along with its acronym.

Declare it in `doc/analysis/generaterobotdata.py` along with its acronym.
When `statisticsDocumentationData.py` reports a failure, its message points at the cause — most often a keyword
used in a `.robot` file but not declared above, a setup keyword name that does not match `initial_setup.py`, or a
Test Case whose name does not match its file name.

To inspect the generated documentation of a single Test Case, run `python doc/generateDocumentationData.py
{tc_id}`; it writes `doc/results/{tc_id}.json` (git-ignored) for you to review.

### Generate documentation for support keywords

@@ -225,19 +233,6 @@ And, to generate documentation for the Test Cases:

---

## Running the unit tests

Before submitting, verify that the documentation unit tests pass:

```bash
python -m unittest discover -s ./doc/tests -t ./doc
```

These tests detect undeclared Test Cases and inconsistencies between the test suite files and the documentation
metadata. A failing unit test indicates either a missing declaration or a stale JSON documentation file.

---

## Maintenance scripts

### Find unused test data files
@@ -279,7 +274,6 @@ Before opening a merge request, confirm the following:
- [ ] The pre-commit hook ran without errors (or reformatted files were re-staged).
- [ ] New Test Cases follow the naming convention (`XXX_YY_NN`) and carry the required tags.
- [ ] New or modified keywords are declared in the relevant `doc/analysis/` files.
- [ ] Documentation data has been regenerated for any new or modified Test Case.
- [ ] New test data files have been placed in the correct `data/` subdirectory.
- [ ] No unused test data files were introduced (`find_unused_test_data.py` returns no new entries).
- [ ] `python -m unittest discover -s ./doc/tests -t ./doc` passes locally.
- [ ] Documentation still generates without failures (`cd doc ; python statisticsDocumentationData.py` reports `0` failures).
+1 −1
Original line number Diff line number Diff line
@@ -11,7 +11,7 @@ Test Teardown Delete Created Entities


*** Test Cases ***
060_03_02 Purge All Entities In Local Mode
060_03_03 Purge All Entities In Local Mode
    [Documentation]    Check that purge entities deletes all entities in local mode
    [Tags]    e-purge    5_6_21    6_4_3_3    since_v1.9.1

+1 −1
Original line number Diff line number Diff line
@@ -11,7 +11,7 @@ Test Teardown Delete Created Entities


*** Test Cases ***
060_03_01 Purge Entities Matching Type
060_03_04 Purge Entities Matching Type
    [Documentation]    Check that purge entities deletes matching a type and an id
    [Tags]    e-purge    5_6_21    6_4_3_3    since_v1.9.1

+5 −5
Original line number Diff line number Diff line
@@ -92,8 +92,8 @@ class Checks:
                Checks.wait_for_notification,
            'Wait for no notification':
                Checks.wait_for_no_notification,
            'Wait for notification and validate it':
                Checks.wait_for_notification_and_validate_it,
            'Wait for CSource notification and validate it':
                Checks.wait_for_csource_notification_and_validate_it,
            'Should be Equal':
                Checks.should_be_equal,
            'Dictionary Should Contain Key':
@@ -307,7 +307,7 @@ class Checks:
                'params': ['timeout'],
                'position': []
            },
            'Wait for notification and validate it': {
            'Wait for CSource notification and validate it': {
                'params': ['expected_subscription_id', 'expected_context_source_registration_ids',
                           'expected_trigger_reason', 'expected_notification_data_entities',
                           'timeout'],
@@ -453,7 +453,7 @@ class Checks:
        return result

    @staticmethod
    def wait_for_notification_and_validate_it(kwargs: list) -> str:
    def wait_for_csource_notification_and_validate_it(kwargs: list) -> str:
        expected_parameters = ['expected_subscription_id', 'expected_context_source_registration_ids',
                               'expected_trigger_reason', 'expected_notification_data_entities',
                               'timeout']
@@ -465,7 +465,7 @@ class Checks:
            kwargs['timeout'] = '5'

        result = [x for x in kwargs if x not in expected_parameters]
        response = "Waiting for Notification and validate it"
        response = "Waiting for CSource Notification and validate it"
        for key, value in kwargs.items():
            match key:
                case 'expected_subscription_id':
+3 −3
Original line number Diff line number Diff line
@@ -291,7 +291,7 @@ class ParseRobotFile:
            # then we need to check the 'Should be Equal' sentences
            pattern = \
                (r'(Wait for no notification)|'
                 r'(Wait for notification and validate it)|'
                 r'(Wait for CSource notification and validate it)|'
                 r'(Wait for notification)([ ]{4}(.*))?')

            param = re.findall(pattern=pattern, string=content, flags=re.MULTILINE)
@@ -299,8 +299,8 @@ class ParseRobotFile:
                data = tuple(element for element in param[i] if element != '')

                match data[0]:
                    case 'Wait for notification and validate it':
                        new_list.append(f'Wait for notification and validate it')
                    case 'Wait for CSource notification and validate it':
                        new_list.append(f'Wait for CSource notification and validate it')
                    case 'Wait for no notification':
                        new_list.append(f'Wait for no notification')
                    case 'Wait for notification':
Loading