Loading testsuite-doubts.md +448 −0 Original line number Diff line number Diff line Loading @@ -93,6 +93,11 @@ when no stub matches, returning a default 404 immediately would let broker-side error handling exercise (and would be obvious in test logs). **Status:** PARTIALLY RESOLVED in MR !284 — the specific trailing-`/` slice of `__is_satisfy` (stub registered with trailing `/` never matching a request with trailing `/`) is fixed there. The broader substring-matching / odd-control-flow concerns remain. --- ## 3. Mock server started AFTER CSR registered Loading Loading @@ -659,6 +664,15 @@ Robotframework-httpctrl supports `Set Stub Reply` with regex match in newer versions — adopting that across the DistOps suite would flip ~30 tests without touching the broker. **Status:** PARTIALLY RESOLVED. The trailing-`/` slice — stub URL ending in `/` not matching a request also ending in `/` — is fixed in MR !284 (`__is_satisfy` now `rstrip("/")`-s both sides). Broker-side `?pick=type[,scope],<attrs>` is emitted per § 6.3.4 + § 6.3.6 (swBroker commit 700d07f); HttpCtrl's query-string matching needs further work to interpret it (substring matching mishandles the canonical order). The wider regex/prefix proposal still stands for the URL-params slice. ## 23. DistOps — `Get Stub Count 1 (integer) != 1 (string)` (~12 tests) Loading @@ -674,6 +688,9 @@ strict and fails. **Fix wanted:** use `Should Be Equal As Numbers` / `Should Be Equal As Integers`, or `Should Be True ${stub_count} == 1`. **Status:** RESOLVED — fixed in MR !285 (25 test files; `Should Be Equal` → `Should Be Equal As Integers`). ## 24. DistOps — `No keyword with name 'Get Request Url Params' found` Loading Loading @@ -1110,6 +1127,12 @@ request to `/a/b/attrs` (missing trailing slash) or (c) switch tests that need precise URL matching to `Wait For Request` + manual reply instead of the stub mechanism. **Status:** Slice #1 (trailing-`/` mismatch) RESOLVED in MR !284 — `__is_satisfy` now strips trailing `/` from both sides of every comparison. Slices #2 (`?sysAttrs=true` etc.) and #3 (`?options=…`, `?deleteAll=…`) remain — those need HttpCtrl URL-params handling or test-side stub registration to accept the canonical broker form. ## 41. `020_14_01/02` — default temporal page size is not in the spec Loading Loading @@ -2087,6 +2110,52 @@ str/bytes" failure rather than a mysterious half-response and broker-side "transport failure". ## 69. HttpCtrl vendored 0.3.1 — `Set Stub Reply` double-encodes a string body **Hit:** DistOps tests that call `Set Stub Reply <method> <url> <status> <body>` with a body produced by `Convert JSON To String` (a Python `str` already holding serialised JSON) — including `D011_02_exc_01` and structurally similar tests across the suite. The broker forwards correctly and the stub matches, but the wire body it receives is `'"[{\"id\":\"X\"}]"'` — a JSON string wrapping the intended array. The broker's response parser sees a `String` where it expects an `Array`, the merge can't proceed, and the entity-level assertion (`$.speed`, etc.) fails. **Why:** in `libraries/robotframework-httpctrl/src/HttpCtrl/__init__.py`, `set_stub_reply` used to wrap every body in `json.dumps()`: ```python response = Response(int(status), None, json.dumps(body), None, None) ``` That was presumably added to fix the **opposite** problem (`Set Stub Reply` receiving a Python `dict` and sending it raw — the `D010_01_aux` / MR !273 case; see #68's "Independent HttpCtrl bug" note). The fork over-corrected: dict callers were served correctly, but callers passing an already-serialised JSON string got it double-encoded. **Impact / broker:** none — the broker is correct; the wire body was mangled upstream of it. **Status:** RESOLVED — auto-serialise only non-`str`/`bytes` bodies: ```python if not isinstance(body, (str, bytes)) and body is not None: body = json.dumps(body) response = Response(int(status), None, body, None, None) ``` This handles both callers: dicts/lists get auto-serialised (the original intent of the fork's patch), str/bytes pass through unchanged. Landed on develop via the housekeeping bundle **!278** and, redundantly, the standalone **!282** (the fix had been duplicated across both branches — !278 merged first, so !282 reduced to the explanatory comment + docstring). The `D010_01_aux` test-side work-around (`Evaluate json.dumps($entity_body)`, landed via !273) stays harmless — a string `json.dumps`'d to itself is unchanged — so !273 needs no follow-up. Verified: `D011_02_exc_01` passes and the DistOps subset PASS count rose ~9 vs the pre-fix baseline. ## 70. `D003_01_exc` — Append-Attrs fragment sent as `application/json` with no Link header, so attribute names don't expand to the CSR's **Hit:** `D003_01_exc Append Entity Attribute` fails the Loading Loading @@ -2505,6 +2574,9 @@ The spec could be clearer that distributed-operation forwards must translate any legacy `attrs=` from a 1.6-era CSR into `pick=` so that mock test stubs written against the modern spec match the forwarded URL. **Status:** documented (no fix yet) — the test still needs to flip `attrs=speed` → `pick=speed` in `D011_02_exc.robot`. ## 80. `046_22_08` — `deleteAll=${true}` serializes as `deleteAll=True` (capital T) **Files:** Loading Loading @@ -2591,6 +2663,14 @@ delete cycle is consistent. But: The second form matches the convention already used elsewhere in the suite (`deleteAll=${EMPTY}` etc.). **Status — 80:** RESOLVED — `deleteAll=${true}` part fixed in MR !283 (same MR that swept all `local=${True}` URL params); the broker now gets a lowercase `true` and the test progresses past the parseBool 400. **Status — 80b:** documented (no fix yet) — the `${entity_id_suffix}` default-arg bug in `Create Subscription And Entity` is still on the queue. Low priority since the tests "work" by accident. ## 82. `D010_01_exc` / `D002_02_exc` / `D003_02_exc` / `D004_01_exc` — distop tests surfaced as test-side after § 9.3.3 enforcement The triage of four distop "later error" failures (from the `lowercase Loading Loading @@ -2667,3 +2747,371 @@ doesn't match the URL id. `$.id` with the actual `${entity_id}` before PATCHing. Or omit the `id` from the fragment entirely (PATCH attrs doesn't need it). **Status — 82a/82b/82c/82d:** documented (no test-side fix yet). The broker-side enforcement that surfaced each item is upstream: swBroker commits a0384f3 (§ 9.3.3 exclusive structural validation) + 700d07f (forward `pick=type[,scope]` with id-reassemble) + ldHooks Link-header @context propagation. Each item still needs its own test-suite-side fix per the recipe in the entry above. ## 83. HttpCtrl matcher accidentally couples `?options=…` / `?sysAttrs=true` etc. to GET-only sub-branches **Library:** `libraries/robotframework-httpctrl/src/HttpCtrl/http_stub.py`, `HttpStubContainer.__is_satisfy`. **Hit:** Even after the trailing-`/` fix (MR !284), a forward whose URL carries the canonical NGSI-LD URL params still misses its stub on **non-GET** methods. The matcher has two branches inside `if '?' in criteria.url:`: - GET branch (lines ~84–126): walks both the stub and request query strings, validates the stub's parameters appear in the request, and accepts the match. - Non-GET branch (lines ~128–144): only verifies `stub.criteria.url == criteria_url_components[0]` — the stub URL WITHOUT a `?` against the request URL minus its `?…` tail. If the stub registers any query at all, the non-GET branch won't take it. This affects all write-op forwards that the broker (correctly per § 10.2.4 / § 10.2.7 / § 10.3.4) emits with a query string: `?options=noOverwrite`, `?options=update`, `?deleteAll=true`, `?sysAttrs=true`, … **Concrete hits in the swBroker ETSI run:** - `D003_02_red` — `POST /attrs/?options=noOverwrite` - `D006_02_inc` — `DELETE /attrs/speed?deleteAll=true` - `D013_01_inc` — `POST /entityOperations/upsert?options=replace` - `D013_02_exc` / `D013_02_inc` — `?options=update` In each case the broker forwards a spec-correct URL and HttpCtrl's non-GET branch falls through without matching. Mock either times out or returns its default, broker reports `207` instead of `204` (or `502` instead of `200`). **Spec:** TS 104-175 § 10.2.4.4, § 10.2.7.4, § 10.3.4.4 require these params to be propagated end-to-end on distop forwards. **Our call:** broker uninvolved — emits canonical URL. **Fix wanted:** mirror the GET-branch parameter-walk in the non-GET branch, OR adopt the wider regex/glob proposal already noted in #22 / #40 (slice 2-3). ## 84. New ETSI failures cluster — invalid-fixture / context / HttpCtrl AttributeError **Background.** After MRs !283 (lowercase URL bool params), !284 (HttpCtrl trailing-`/` matcher), !285 (`Should Be Equal` → `As Integers`) all landed and the swBroker side already enforces § 9.3.3 + forward-pick `type[,scope]` (commits a0384f3 + 700d07f), the remaining "Unclear / needs triage" cluster from the swBroker ETSI run reduced to six tests with three distinct test-side root causes (broker is correct in all six): **84a) `D013_01_exc` / `D014_02_exc` — fixture invalid for exclusive (§ 9.3.3 surface, new tests)** Same pattern as #82b but on different tests. Setup builds an exclusive CSR from `csourceRegistrations/context-source-registration-vehicle-redirection-ops.jsonld` — a type-only `{"entities":[{"type":"Vehicle"}]}` selector with no attribute names. Per § 9.3.3 an exclusive registration shall define BOTH a specific entity id AND propertyNames / relationshipNames; a spec-strict broker rejects it with `400 BadRequestData`. Test expects `201` and the setup `Check Response Status Code 201` fails (`201 != 400`). **Fix:** give the test its own fixture with id + propertyNames matching what the test exercises (same recipe as #82b). **84b) `D013_01_red` / `D005_01_exc` — CSR creation conflict with local entity (`201 != 409`)** Setup creates the entity locally with `local=true`, then registers a redirect / exclusive CSR for the same entity id. Per § 5.9.2 the broker checks for overlap between the new exclusive/redirect CSR and any locally-held data: if the local entity already carries the attrs the CSR claims, the broker returns `409 AlreadyExists`. Test expects `201`. The original test design relied on the pre-§ 9.3.3 broker behaviour where CSRs that only specified `type: Vehicle` and no attribute set didn't trigger the conflict check (nothing to compare). After § 9.3.3 the CSR must carry propertyNames, and as soon as those propertyNames overlap the local entity's attrs the conflict check fires. **Fix:** seed the local entity with attrs that do NOT overlap the CSR's claim, OR create the CSR before the entity, OR drop the `local=true` on entity creation so the broker's distop dispatcher takes ownership. **84c) `D014_02_inc` — HttpCtrl `'NoneType' object has no attribute 'get_url'`** When the broker's forward to the CSR's mock takes a path HttpCtrl's `__is_satisfy` doesn't match, the handler falls through to `ResponseStorage().pop()` — but for this test the storage is empty and pop returns `None`. The next `response.get_url()` raises `AttributeError`. This is the "blocking fallback" side-issue already called out in #2 surfacing as a None-deref rather than a hang. Likely cause: same query-string mismatch as #83 (`?options=update` on update-batch forwards). Once #83 is fixed this should clear. **Fix wanted:** test-side stubs aligned with what the broker emits once #83 lands. **84d) `D005_01_inc` / `D005_01_red` / `D003_02_red` — expanded IRI in response (#70 pattern on different tests)** Body assertion `Dictionary does not contain key 'speed'` because the response returns `https://ngsi-ld-test-suite/context#speed` (the test-suite-context expansion). Same root cause as #70: the test sends `Content-Type: application/json` to the broker without a Link header, so the broker uses core context; CSR was registered with test-suite-compound context; the two namespaces don't reconcile in the response. **Fix:** add the test-suite-context Link header on the test's main call (or move to `application/ld+json` with a root `@context`). **84e) `D018_02_02` — `204 != 207`** Not triaged in detail yet — left for a follow-up run. **84f) `D016_01_red` — test forgets to stub the GET for the post-merge query** The test merges two entities via a redirect CSR (stubs for `POST /broker{1,2}/.../entityOperations/merge` are registered). Both merge forwards return 204 successfully. The verification step then calls `Query Entities entity_types=Vehicle ...` — but the test never stubs `GET /broker{1,2}/.../entities?type=Vehicle&id=…`. With redirect mode the broker holds no local data, so it forwards the query to both CSes; neither stub matches; both forwards time out; the broker merges zero results and returns `[]`. The assertion `Should Contain ${response.json()} speed` fails with `'[]' does not contain 'speed'`. **Fix:** add stubs for the `GET /broker{1,2}/ngsi-ld/v1/entities?…` that return a merged-entity payload containing `speed`. Mirror what `D016_01_inc` does (which stubs both legs). **84g) `D001_01_inc` — joins #72 (leftover Vehicle entities in current state)** Full-suite failure mode `Lengths are different: 1 != 5`. The `Query Entities entity_types=Vehicle local=true` returns 5 entities — one created by this test plus 4 leftovers from earlier DistOps tests in the full run (mirrors the cluster already documented in #72: `D001_02_inc / D001_03_03_inc / D011_02_inc`). Passes in isolation. Same fix as #72. **84h) `D007_01_exc` — Robot keyword choice doesn't tolerate the spec-mandated absence of speed** After PUT-Replace + an exclusive CSR claiming `speed`, the local entity correctly drops the claimed attr per § 4.3.6.3 — the local copy carries `{id, type, isParked2}` and no `speed`. The test then does ```robot ${body}= Get From Dictionary ${response.json()} speed Should Not Contain ${body} speed ``` `Get From Dictionary` raises when the key is absent, so the test fails on line 40 with `Dictionary does not contain key 'speed'` — even though that's exactly the broker behaviour the test is meant to verify. **Fix:** use `Dictionary Should Not Contain Key ${response.json()} speed` instead of fetching the value and substring-checking it. The extracted-then-substring-checked pattern only makes sense if you expect the key to be present and want to verify it equals something — not the negative-presence assertion this test is trying to do. ## 85. `D010_01_aux` — stub body passed as a Python dict; HttpCtrl doesn't auto-serialise **Hit:** `D010_01_aux` (auxiliary RetrieveEntity gap-fill merge) fails with `No value found for path $.isParked` — the broker's response contains only the local entity, missing the gap-fill attribute the auxiliary CSR should provide. **Why:** the test does ```robotframework ${entity_body}= Load Entity ${entity_payload_filename} ${entity_id} Set Stub Reply GET /ngsi-ld/v1/entities/${entity_id} 200 ${entity_body} ``` `Load Entity` returns a Python **dict**, and HttpCtrl's response writer (`http_handler.py::__send_response`) only encodes the body when it is already a `str`. A dict falls through: `isinstance(body, str)` is false → no encoding; `len(dict)` returns the number of KEYS → wrong `Content-Length`; `wfile.write(dict)` raises (silently swallowed) → empty body on the wire. The mock sends headers only, the broker has no auxiliary body to merge, the test fails on the missing gap-fill attribute. **Impact / broker:** none. A correct aux-merge produces the spec-correct § 9.3.2 result (local wins for shared attrs, CSR fills gaps) when verified end-to-end against a real upstream returning valid JSON. **Status:** RESOLVED via !273 — the test now serialises the body first (`Evaluate json.dumps($entity_body) json`) so the wire payload is non-empty. This is the dict-body counterpart of [[#69]]: once the vendored fork's conditional auto-serialise landed (via !278/!282, see #69), this test-side `json.dumps` became belt-and-suspenders — a string re-`json.dumps`'d to itself is unchanged — so it stays harmless. The same dict-body pattern may affect other `D010_*` retrieve tests; the framework-side guard in #69 is the cleaner long-term home. ## 86. `D011_02_exc_01` — CSR response without a Link header → broker can't reconcile attribute IRIs **Hit:** Even with the upgraded HttpCtrl matcher (!277) and the spec-canonical `pick=` stub URL (see #79), `D011_02_exc_01` still fails with `No value found for path $.speed`. The mock is matched, the broker forwards correctly and receives the speed entity body — but the merged response contains only the local-broker attributes (`brandName`, `isParked`); the CSR-side `speed` is silently dropped. **Root cause (likely):** the HttpCtrl mock can set Content-Type + body via `Set Stub Reply` but cannot set a Link header. So the mock's response carries no `Link: <…>; rel="http://www.w3.org/ns/json-ld#context"`. The broker, receiving a JSON-only response with no Link, falls back to the **default core context** when expanding the CSR's short attribute names. The local entity was stored with the test-suite context, so its `speed` expanded to one IRI; the CSR-side `speed` expands to `…/default-context/speed`. Different IRIs → the merge keeps them as different attributes; the response `@context` only knows the local one, so the CSR-side speed is lost on compaction. **Two-sided issue:** - **Test-side:** `Set Stub Reply` cannot add response headers, so it cannot simulate a real CS that echoes back the request's `Link`. The right HttpCtrl-fork fix is `Set Stub Reply` with optional headers. - **Broker-side:** when a CSR responds with no Link header, the broker should fall back to the registered CSR's `@context` (§ 5.2.12) or the original request's `@context` — not the default core. § 9.3 is permissive here; arguably a spec gap too. **Status:** OPEN. !277 switched the stub URL to spec-canonical `pick=speed` so the mock is at least reached when paired with the matcher fix; the merge bug itself needs broker-side review. Continuation of #79 (same test, next layer); related to the header-less-stub limitation in [[#83]]. ## 87. `020_17 / 020_18 / 020_19 / 020_20` setup — pair the temporal POST with a current-state POST **Status:** PARTIALLY RESOLVED via !279 (setup side, fixing [[#9]]); a broker-side deletedAt-tombstone bug remains, see below. **Hit + spec recap:** see [[#9]] for the full analysis. Briefly: each test's setup did only `POST /temporal/entities`, then the body's `DELETE /entities/{id}/attrs/{name}` (a current-state operation) returned 404 because the entity had no current-state representation; no `deletedAt` tombstone was written, so the temporal-evolution assertion failed with `Item root[<attr>] removed from dictionary`. **Fix (via !279):** each `Create Temporal Entity` setup now also POSTs a matching current-state entity via `Create Entity`, using two new minimal fixtures: `vehicle-different-attribute-types.jsonld` (fuelLevel/isParkedIn/name, 020_17/18) and `vehicle-with-scope.jsonld` (scope + fuelLevel, 020_19/20). The following `POST /temporal/entities` may return 201 *or* 204 per § 11.2.2.5 (append to existing), so the status check is relaxed accordingly. **Result:** 020_17_01 (Property) and 020_18_01 (Property + temporalValues) now pass. The four non-Property cases (020_17_02/03, 020_18_02/03, 020_19_01, 020_20_01) still fail on a *different*, broker-side root cause: **Broker bug surfaced (separate work):** the `deletedAt` tombstone collapses all attribute kinds to `Property` with `urn:ngsi-ld:null`. For a deleted **Relationship**, the temporal-store tombstone should preserve `type: Relationship` and substitute `object: urn:ngsi-ld:null` — not switch the kind to Property. Same for **LanguageProperty** (`languageMap`) and **Scope**. The broker emits a uniform `{ "type": "Property", "value": "urn:ngsi-ld:null", "deletedAt": … }` for every kind, so the deepdiff fails with `… removed` + `type changed from "<kind>" to "Property"`. The broker fix belongs in its own MR — out of scope for the test-side change. ## 88. DistOps write-op status-code mismatches — 30 tests across 7 patterns, mostly test-side design issues A categorisation pass on the 30 DistOps tests whose `HTTP status code comparison failed`. Triaged into seven patterns; for most the broker is correct and the tests need a redesign by the original author. None of this is implementation-blocking: the broker behaviour is the spec-compliant or spec-defensible reading in every case. Overlaps the clusters in #82 / [[#84]]. ### 88a) 201 ≠ 409 (8 tests) — setup creates a CSR that conflicts with locally-stored data Affected: `D001_02_exc`, `D002_02_exc`, `D005_01_exc`, `D009_01_exc`, `D013_01_red`, `D006_01_red`, `D006_02_red`, `D009_01_red`. Each setup does `Create Entity … local=true` then `Create Context Source Registration …` (exclusive or redirect) for the same entity. Both CSR shapes conflict with a locally-stored copy and the broker correctly returns **409**: - **Exclusive** (4): an exclusive CSR must also specify `propertyNames` / `relationshipNames`; claiming a whole entity exclusively is not permitted. The redirection-ops fixture has neither, so it is malformed for exclusive use anyway. - **Redirect** (4): a redirect CSR cannot coexist with a local copy of the same entity — writes would go to the CS, reads to local. 409 by design. **Impact / broker:** none — conflict detection at CSR-creation time is implemented and tested. **Fix wanted:** the tests need a redesign by their author — the docstrings don't match the setup as written. Likely the intent was to register the CSR first and exercise the distop path in the test body, not to pre-create a conflicting local entity. The four `exc` cases also need `propertyNames` in the CSR fixture. ### 88b) 204 ≠ 404 (5 tests) — redirect-mode forwards return CS's 404 Affected: `D003_01_red`, `D004_01_red`, `D014_01_red`, `D014_02_red`, `D015_01_red`. Test expects 204; broker returns 404. Likely the stub matches a different URL shape than the broker forwards (see the matcher work in !277), so the mock returns its default 404 — or the same setup-conflict pattern as 88a. **Status:** needs per-test triage. ### 88c) 207 ≠ 204 (3 tests) and 204 ≠ 207 (2 tests) — inclusive-mode forward failure status Affected: `D002_02_01_inc`, `D002_02_02_inc`, `D004_01_inc`, `D018_02_02`, `D003_01_inc`. Open spec ambiguity (already logged in spec-doubts-2 #90): when a write forwarded to an *inclusive* CSR fails, should the broker return 207 (partial success) or a clean 204/201? § 10.2.1.4 and § 9.3.2 imply different answers; current behaviour varies per op. **Resolution:** wait for spec clarification. ### 88d) 200 ≠ 400 (3 tests) — query rejected as invalid Affected: `D011_02_red_02`, `D011_03_inc_02`, `D011_04_inc_02`. Broker considers the query malformed (400); test expects 200. **Status:** needs per-test triage of whether the strictness is justified. ### 88e) 201 ≠ 400 (3 tests) — batch op rejected as invalid Affected: `D012_01_red`, `D013_02_red`, `D015_01_inc`. Same as 88d for batch ops; probably related to 88a (CSR-side conflict) since these are redirect/inclusive. **Status:** needs per-test triage. ### 88f) 200 ≠ 404 (2 tests) — retrieve says not-found Affected: `D010_03_inc_02`, `D010_03_inc_03`. Inclusive-mode retrieve where the broker reports 404 but the test expects the entity (presumably via the mocked CSR). Likely a stub-URL mismatch or the 88a conflict pattern. **Status:** needs per-test triage. ### 88g) Singletons (4 tests) - `D010_02_red` (404 ≠ 200) — retrieve returns 200 where test expects 404. - `D018_01` (508 ≠ 204) — loop-detection test expects 508; broker returns 204. - `D003_02_red` (207 ≠ 404) — append-attr test expects 207; broker returns 404. - `D004_01_exc` (204 ≠ 400) — exclusive create-entity-with-CSR; broker rejects. Each likely maps to one of the patterns above; needs individual inspection. ### Summary Of the 30 mismatches: **8 (88a)** confirmed test bug (setup violates CSR-creation conflict rules; tests need redesign); **5 (88c)** spec ambiguity already in spec-doubts-2 #90, not actionable until the spec resolves; **17 (88b/d/e/f/g)** need per-test triage, structurally suggestive of the same setup-shape issues plus stub-URL gaps but unconfirmed. For all 30 the broker's behaviour is at minimum spec-defensible; no broker-side change is recommended without a spec/conformance discussion. Loading
testsuite-doubts.md +448 −0 Original line number Diff line number Diff line Loading @@ -93,6 +93,11 @@ when no stub matches, returning a default 404 immediately would let broker-side error handling exercise (and would be obvious in test logs). **Status:** PARTIALLY RESOLVED in MR !284 — the specific trailing-`/` slice of `__is_satisfy` (stub registered with trailing `/` never matching a request with trailing `/`) is fixed there. The broader substring-matching / odd-control-flow concerns remain. --- ## 3. Mock server started AFTER CSR registered Loading Loading @@ -659,6 +664,15 @@ Robotframework-httpctrl supports `Set Stub Reply` with regex match in newer versions — adopting that across the DistOps suite would flip ~30 tests without touching the broker. **Status:** PARTIALLY RESOLVED. The trailing-`/` slice — stub URL ending in `/` not matching a request also ending in `/` — is fixed in MR !284 (`__is_satisfy` now `rstrip("/")`-s both sides). Broker-side `?pick=type[,scope],<attrs>` is emitted per § 6.3.4 + § 6.3.6 (swBroker commit 700d07f); HttpCtrl's query-string matching needs further work to interpret it (substring matching mishandles the canonical order). The wider regex/prefix proposal still stands for the URL-params slice. ## 23. DistOps — `Get Stub Count 1 (integer) != 1 (string)` (~12 tests) Loading @@ -674,6 +688,9 @@ strict and fails. **Fix wanted:** use `Should Be Equal As Numbers` / `Should Be Equal As Integers`, or `Should Be True ${stub_count} == 1`. **Status:** RESOLVED — fixed in MR !285 (25 test files; `Should Be Equal` → `Should Be Equal As Integers`). ## 24. DistOps — `No keyword with name 'Get Request Url Params' found` Loading Loading @@ -1110,6 +1127,12 @@ request to `/a/b/attrs` (missing trailing slash) or (c) switch tests that need precise URL matching to `Wait For Request` + manual reply instead of the stub mechanism. **Status:** Slice #1 (trailing-`/` mismatch) RESOLVED in MR !284 — `__is_satisfy` now strips trailing `/` from both sides of every comparison. Slices #2 (`?sysAttrs=true` etc.) and #3 (`?options=…`, `?deleteAll=…`) remain — those need HttpCtrl URL-params handling or test-side stub registration to accept the canonical broker form. ## 41. `020_14_01/02` — default temporal page size is not in the spec Loading Loading @@ -2087,6 +2110,52 @@ str/bytes" failure rather than a mysterious half-response and broker-side "transport failure". ## 69. HttpCtrl vendored 0.3.1 — `Set Stub Reply` double-encodes a string body **Hit:** DistOps tests that call `Set Stub Reply <method> <url> <status> <body>` with a body produced by `Convert JSON To String` (a Python `str` already holding serialised JSON) — including `D011_02_exc_01` and structurally similar tests across the suite. The broker forwards correctly and the stub matches, but the wire body it receives is `'"[{\"id\":\"X\"}]"'` — a JSON string wrapping the intended array. The broker's response parser sees a `String` where it expects an `Array`, the merge can't proceed, and the entity-level assertion (`$.speed`, etc.) fails. **Why:** in `libraries/robotframework-httpctrl/src/HttpCtrl/__init__.py`, `set_stub_reply` used to wrap every body in `json.dumps()`: ```python response = Response(int(status), None, json.dumps(body), None, None) ``` That was presumably added to fix the **opposite** problem (`Set Stub Reply` receiving a Python `dict` and sending it raw — the `D010_01_aux` / MR !273 case; see #68's "Independent HttpCtrl bug" note). The fork over-corrected: dict callers were served correctly, but callers passing an already-serialised JSON string got it double-encoded. **Impact / broker:** none — the broker is correct; the wire body was mangled upstream of it. **Status:** RESOLVED — auto-serialise only non-`str`/`bytes` bodies: ```python if not isinstance(body, (str, bytes)) and body is not None: body = json.dumps(body) response = Response(int(status), None, body, None, None) ``` This handles both callers: dicts/lists get auto-serialised (the original intent of the fork's patch), str/bytes pass through unchanged. Landed on develop via the housekeeping bundle **!278** and, redundantly, the standalone **!282** (the fix had been duplicated across both branches — !278 merged first, so !282 reduced to the explanatory comment + docstring). The `D010_01_aux` test-side work-around (`Evaluate json.dumps($entity_body)`, landed via !273) stays harmless — a string `json.dumps`'d to itself is unchanged — so !273 needs no follow-up. Verified: `D011_02_exc_01` passes and the DistOps subset PASS count rose ~9 vs the pre-fix baseline. ## 70. `D003_01_exc` — Append-Attrs fragment sent as `application/json` with no Link header, so attribute names don't expand to the CSR's **Hit:** `D003_01_exc Append Entity Attribute` fails the Loading Loading @@ -2505,6 +2574,9 @@ The spec could be clearer that distributed-operation forwards must translate any legacy `attrs=` from a 1.6-era CSR into `pick=` so that mock test stubs written against the modern spec match the forwarded URL. **Status:** documented (no fix yet) — the test still needs to flip `attrs=speed` → `pick=speed` in `D011_02_exc.robot`. ## 80. `046_22_08` — `deleteAll=${true}` serializes as `deleteAll=True` (capital T) **Files:** Loading Loading @@ -2591,6 +2663,14 @@ delete cycle is consistent. But: The second form matches the convention already used elsewhere in the suite (`deleteAll=${EMPTY}` etc.). **Status — 80:** RESOLVED — `deleteAll=${true}` part fixed in MR !283 (same MR that swept all `local=${True}` URL params); the broker now gets a lowercase `true` and the test progresses past the parseBool 400. **Status — 80b:** documented (no fix yet) — the `${entity_id_suffix}` default-arg bug in `Create Subscription And Entity` is still on the queue. Low priority since the tests "work" by accident. ## 82. `D010_01_exc` / `D002_02_exc` / `D003_02_exc` / `D004_01_exc` — distop tests surfaced as test-side after § 9.3.3 enforcement The triage of four distop "later error" failures (from the `lowercase Loading Loading @@ -2667,3 +2747,371 @@ doesn't match the URL id. `$.id` with the actual `${entity_id}` before PATCHing. Or omit the `id` from the fragment entirely (PATCH attrs doesn't need it). **Status — 82a/82b/82c/82d:** documented (no test-side fix yet). The broker-side enforcement that surfaced each item is upstream: swBroker commits a0384f3 (§ 9.3.3 exclusive structural validation) + 700d07f (forward `pick=type[,scope]` with id-reassemble) + ldHooks Link-header @context propagation. Each item still needs its own test-suite-side fix per the recipe in the entry above. ## 83. HttpCtrl matcher accidentally couples `?options=…` / `?sysAttrs=true` etc. to GET-only sub-branches **Library:** `libraries/robotframework-httpctrl/src/HttpCtrl/http_stub.py`, `HttpStubContainer.__is_satisfy`. **Hit:** Even after the trailing-`/` fix (MR !284), a forward whose URL carries the canonical NGSI-LD URL params still misses its stub on **non-GET** methods. The matcher has two branches inside `if '?' in criteria.url:`: - GET branch (lines ~84–126): walks both the stub and request query strings, validates the stub's parameters appear in the request, and accepts the match. - Non-GET branch (lines ~128–144): only verifies `stub.criteria.url == criteria_url_components[0]` — the stub URL WITHOUT a `?` against the request URL minus its `?…` tail. If the stub registers any query at all, the non-GET branch won't take it. This affects all write-op forwards that the broker (correctly per § 10.2.4 / § 10.2.7 / § 10.3.4) emits with a query string: `?options=noOverwrite`, `?options=update`, `?deleteAll=true`, `?sysAttrs=true`, … **Concrete hits in the swBroker ETSI run:** - `D003_02_red` — `POST /attrs/?options=noOverwrite` - `D006_02_inc` — `DELETE /attrs/speed?deleteAll=true` - `D013_01_inc` — `POST /entityOperations/upsert?options=replace` - `D013_02_exc` / `D013_02_inc` — `?options=update` In each case the broker forwards a spec-correct URL and HttpCtrl's non-GET branch falls through without matching. Mock either times out or returns its default, broker reports `207` instead of `204` (or `502` instead of `200`). **Spec:** TS 104-175 § 10.2.4.4, § 10.2.7.4, § 10.3.4.4 require these params to be propagated end-to-end on distop forwards. **Our call:** broker uninvolved — emits canonical URL. **Fix wanted:** mirror the GET-branch parameter-walk in the non-GET branch, OR adopt the wider regex/glob proposal already noted in #22 / #40 (slice 2-3). ## 84. New ETSI failures cluster — invalid-fixture / context / HttpCtrl AttributeError **Background.** After MRs !283 (lowercase URL bool params), !284 (HttpCtrl trailing-`/` matcher), !285 (`Should Be Equal` → `As Integers`) all landed and the swBroker side already enforces § 9.3.3 + forward-pick `type[,scope]` (commits a0384f3 + 700d07f), the remaining "Unclear / needs triage" cluster from the swBroker ETSI run reduced to six tests with three distinct test-side root causes (broker is correct in all six): **84a) `D013_01_exc` / `D014_02_exc` — fixture invalid for exclusive (§ 9.3.3 surface, new tests)** Same pattern as #82b but on different tests. Setup builds an exclusive CSR from `csourceRegistrations/context-source-registration-vehicle-redirection-ops.jsonld` — a type-only `{"entities":[{"type":"Vehicle"}]}` selector with no attribute names. Per § 9.3.3 an exclusive registration shall define BOTH a specific entity id AND propertyNames / relationshipNames; a spec-strict broker rejects it with `400 BadRequestData`. Test expects `201` and the setup `Check Response Status Code 201` fails (`201 != 400`). **Fix:** give the test its own fixture with id + propertyNames matching what the test exercises (same recipe as #82b). **84b) `D013_01_red` / `D005_01_exc` — CSR creation conflict with local entity (`201 != 409`)** Setup creates the entity locally with `local=true`, then registers a redirect / exclusive CSR for the same entity id. Per § 5.9.2 the broker checks for overlap between the new exclusive/redirect CSR and any locally-held data: if the local entity already carries the attrs the CSR claims, the broker returns `409 AlreadyExists`. Test expects `201`. The original test design relied on the pre-§ 9.3.3 broker behaviour where CSRs that only specified `type: Vehicle` and no attribute set didn't trigger the conflict check (nothing to compare). After § 9.3.3 the CSR must carry propertyNames, and as soon as those propertyNames overlap the local entity's attrs the conflict check fires. **Fix:** seed the local entity with attrs that do NOT overlap the CSR's claim, OR create the CSR before the entity, OR drop the `local=true` on entity creation so the broker's distop dispatcher takes ownership. **84c) `D014_02_inc` — HttpCtrl `'NoneType' object has no attribute 'get_url'`** When the broker's forward to the CSR's mock takes a path HttpCtrl's `__is_satisfy` doesn't match, the handler falls through to `ResponseStorage().pop()` — but for this test the storage is empty and pop returns `None`. The next `response.get_url()` raises `AttributeError`. This is the "blocking fallback" side-issue already called out in #2 surfacing as a None-deref rather than a hang. Likely cause: same query-string mismatch as #83 (`?options=update` on update-batch forwards). Once #83 is fixed this should clear. **Fix wanted:** test-side stubs aligned with what the broker emits once #83 lands. **84d) `D005_01_inc` / `D005_01_red` / `D003_02_red` — expanded IRI in response (#70 pattern on different tests)** Body assertion `Dictionary does not contain key 'speed'` because the response returns `https://ngsi-ld-test-suite/context#speed` (the test-suite-context expansion). Same root cause as #70: the test sends `Content-Type: application/json` to the broker without a Link header, so the broker uses core context; CSR was registered with test-suite-compound context; the two namespaces don't reconcile in the response. **Fix:** add the test-suite-context Link header on the test's main call (or move to `application/ld+json` with a root `@context`). **84e) `D018_02_02` — `204 != 207`** Not triaged in detail yet — left for a follow-up run. **84f) `D016_01_red` — test forgets to stub the GET for the post-merge query** The test merges two entities via a redirect CSR (stubs for `POST /broker{1,2}/.../entityOperations/merge` are registered). Both merge forwards return 204 successfully. The verification step then calls `Query Entities entity_types=Vehicle ...` — but the test never stubs `GET /broker{1,2}/.../entities?type=Vehicle&id=…`. With redirect mode the broker holds no local data, so it forwards the query to both CSes; neither stub matches; both forwards time out; the broker merges zero results and returns `[]`. The assertion `Should Contain ${response.json()} speed` fails with `'[]' does not contain 'speed'`. **Fix:** add stubs for the `GET /broker{1,2}/ngsi-ld/v1/entities?…` that return a merged-entity payload containing `speed`. Mirror what `D016_01_inc` does (which stubs both legs). **84g) `D001_01_inc` — joins #72 (leftover Vehicle entities in current state)** Full-suite failure mode `Lengths are different: 1 != 5`. The `Query Entities entity_types=Vehicle local=true` returns 5 entities — one created by this test plus 4 leftovers from earlier DistOps tests in the full run (mirrors the cluster already documented in #72: `D001_02_inc / D001_03_03_inc / D011_02_inc`). Passes in isolation. Same fix as #72. **84h) `D007_01_exc` — Robot keyword choice doesn't tolerate the spec-mandated absence of speed** After PUT-Replace + an exclusive CSR claiming `speed`, the local entity correctly drops the claimed attr per § 4.3.6.3 — the local copy carries `{id, type, isParked2}` and no `speed`. The test then does ```robot ${body}= Get From Dictionary ${response.json()} speed Should Not Contain ${body} speed ``` `Get From Dictionary` raises when the key is absent, so the test fails on line 40 with `Dictionary does not contain key 'speed'` — even though that's exactly the broker behaviour the test is meant to verify. **Fix:** use `Dictionary Should Not Contain Key ${response.json()} speed` instead of fetching the value and substring-checking it. The extracted-then-substring-checked pattern only makes sense if you expect the key to be present and want to verify it equals something — not the negative-presence assertion this test is trying to do. ## 85. `D010_01_aux` — stub body passed as a Python dict; HttpCtrl doesn't auto-serialise **Hit:** `D010_01_aux` (auxiliary RetrieveEntity gap-fill merge) fails with `No value found for path $.isParked` — the broker's response contains only the local entity, missing the gap-fill attribute the auxiliary CSR should provide. **Why:** the test does ```robotframework ${entity_body}= Load Entity ${entity_payload_filename} ${entity_id} Set Stub Reply GET /ngsi-ld/v1/entities/${entity_id} 200 ${entity_body} ``` `Load Entity` returns a Python **dict**, and HttpCtrl's response writer (`http_handler.py::__send_response`) only encodes the body when it is already a `str`. A dict falls through: `isinstance(body, str)` is false → no encoding; `len(dict)` returns the number of KEYS → wrong `Content-Length`; `wfile.write(dict)` raises (silently swallowed) → empty body on the wire. The mock sends headers only, the broker has no auxiliary body to merge, the test fails on the missing gap-fill attribute. **Impact / broker:** none. A correct aux-merge produces the spec-correct § 9.3.2 result (local wins for shared attrs, CSR fills gaps) when verified end-to-end against a real upstream returning valid JSON. **Status:** RESOLVED via !273 — the test now serialises the body first (`Evaluate json.dumps($entity_body) json`) so the wire payload is non-empty. This is the dict-body counterpart of [[#69]]: once the vendored fork's conditional auto-serialise landed (via !278/!282, see #69), this test-side `json.dumps` became belt-and-suspenders — a string re-`json.dumps`'d to itself is unchanged — so it stays harmless. The same dict-body pattern may affect other `D010_*` retrieve tests; the framework-side guard in #69 is the cleaner long-term home. ## 86. `D011_02_exc_01` — CSR response without a Link header → broker can't reconcile attribute IRIs **Hit:** Even with the upgraded HttpCtrl matcher (!277) and the spec-canonical `pick=` stub URL (see #79), `D011_02_exc_01` still fails with `No value found for path $.speed`. The mock is matched, the broker forwards correctly and receives the speed entity body — but the merged response contains only the local-broker attributes (`brandName`, `isParked`); the CSR-side `speed` is silently dropped. **Root cause (likely):** the HttpCtrl mock can set Content-Type + body via `Set Stub Reply` but cannot set a Link header. So the mock's response carries no `Link: <…>; rel="http://www.w3.org/ns/json-ld#context"`. The broker, receiving a JSON-only response with no Link, falls back to the **default core context** when expanding the CSR's short attribute names. The local entity was stored with the test-suite context, so its `speed` expanded to one IRI; the CSR-side `speed` expands to `…/default-context/speed`. Different IRIs → the merge keeps them as different attributes; the response `@context` only knows the local one, so the CSR-side speed is lost on compaction. **Two-sided issue:** - **Test-side:** `Set Stub Reply` cannot add response headers, so it cannot simulate a real CS that echoes back the request's `Link`. The right HttpCtrl-fork fix is `Set Stub Reply` with optional headers. - **Broker-side:** when a CSR responds with no Link header, the broker should fall back to the registered CSR's `@context` (§ 5.2.12) or the original request's `@context` — not the default core. § 9.3 is permissive here; arguably a spec gap too. **Status:** OPEN. !277 switched the stub URL to spec-canonical `pick=speed` so the mock is at least reached when paired with the matcher fix; the merge bug itself needs broker-side review. Continuation of #79 (same test, next layer); related to the header-less-stub limitation in [[#83]]. ## 87. `020_17 / 020_18 / 020_19 / 020_20` setup — pair the temporal POST with a current-state POST **Status:** PARTIALLY RESOLVED via !279 (setup side, fixing [[#9]]); a broker-side deletedAt-tombstone bug remains, see below. **Hit + spec recap:** see [[#9]] for the full analysis. Briefly: each test's setup did only `POST /temporal/entities`, then the body's `DELETE /entities/{id}/attrs/{name}` (a current-state operation) returned 404 because the entity had no current-state representation; no `deletedAt` tombstone was written, so the temporal-evolution assertion failed with `Item root[<attr>] removed from dictionary`. **Fix (via !279):** each `Create Temporal Entity` setup now also POSTs a matching current-state entity via `Create Entity`, using two new minimal fixtures: `vehicle-different-attribute-types.jsonld` (fuelLevel/isParkedIn/name, 020_17/18) and `vehicle-with-scope.jsonld` (scope + fuelLevel, 020_19/20). The following `POST /temporal/entities` may return 201 *or* 204 per § 11.2.2.5 (append to existing), so the status check is relaxed accordingly. **Result:** 020_17_01 (Property) and 020_18_01 (Property + temporalValues) now pass. The four non-Property cases (020_17_02/03, 020_18_02/03, 020_19_01, 020_20_01) still fail on a *different*, broker-side root cause: **Broker bug surfaced (separate work):** the `deletedAt` tombstone collapses all attribute kinds to `Property` with `urn:ngsi-ld:null`. For a deleted **Relationship**, the temporal-store tombstone should preserve `type: Relationship` and substitute `object: urn:ngsi-ld:null` — not switch the kind to Property. Same for **LanguageProperty** (`languageMap`) and **Scope**. The broker emits a uniform `{ "type": "Property", "value": "urn:ngsi-ld:null", "deletedAt": … }` for every kind, so the deepdiff fails with `… removed` + `type changed from "<kind>" to "Property"`. The broker fix belongs in its own MR — out of scope for the test-side change. ## 88. DistOps write-op status-code mismatches — 30 tests across 7 patterns, mostly test-side design issues A categorisation pass on the 30 DistOps tests whose `HTTP status code comparison failed`. Triaged into seven patterns; for most the broker is correct and the tests need a redesign by the original author. None of this is implementation-blocking: the broker behaviour is the spec-compliant or spec-defensible reading in every case. Overlaps the clusters in #82 / [[#84]]. ### 88a) 201 ≠ 409 (8 tests) — setup creates a CSR that conflicts with locally-stored data Affected: `D001_02_exc`, `D002_02_exc`, `D005_01_exc`, `D009_01_exc`, `D013_01_red`, `D006_01_red`, `D006_02_red`, `D009_01_red`. Each setup does `Create Entity … local=true` then `Create Context Source Registration …` (exclusive or redirect) for the same entity. Both CSR shapes conflict with a locally-stored copy and the broker correctly returns **409**: - **Exclusive** (4): an exclusive CSR must also specify `propertyNames` / `relationshipNames`; claiming a whole entity exclusively is not permitted. The redirection-ops fixture has neither, so it is malformed for exclusive use anyway. - **Redirect** (4): a redirect CSR cannot coexist with a local copy of the same entity — writes would go to the CS, reads to local. 409 by design. **Impact / broker:** none — conflict detection at CSR-creation time is implemented and tested. **Fix wanted:** the tests need a redesign by their author — the docstrings don't match the setup as written. Likely the intent was to register the CSR first and exercise the distop path in the test body, not to pre-create a conflicting local entity. The four `exc` cases also need `propertyNames` in the CSR fixture. ### 88b) 204 ≠ 404 (5 tests) — redirect-mode forwards return CS's 404 Affected: `D003_01_red`, `D004_01_red`, `D014_01_red`, `D014_02_red`, `D015_01_red`. Test expects 204; broker returns 404. Likely the stub matches a different URL shape than the broker forwards (see the matcher work in !277), so the mock returns its default 404 — or the same setup-conflict pattern as 88a. **Status:** needs per-test triage. ### 88c) 207 ≠ 204 (3 tests) and 204 ≠ 207 (2 tests) — inclusive-mode forward failure status Affected: `D002_02_01_inc`, `D002_02_02_inc`, `D004_01_inc`, `D018_02_02`, `D003_01_inc`. Open spec ambiguity (already logged in spec-doubts-2 #90): when a write forwarded to an *inclusive* CSR fails, should the broker return 207 (partial success) or a clean 204/201? § 10.2.1.4 and § 9.3.2 imply different answers; current behaviour varies per op. **Resolution:** wait for spec clarification. ### 88d) 200 ≠ 400 (3 tests) — query rejected as invalid Affected: `D011_02_red_02`, `D011_03_inc_02`, `D011_04_inc_02`. Broker considers the query malformed (400); test expects 200. **Status:** needs per-test triage of whether the strictness is justified. ### 88e) 201 ≠ 400 (3 tests) — batch op rejected as invalid Affected: `D012_01_red`, `D013_02_red`, `D015_01_inc`. Same as 88d for batch ops; probably related to 88a (CSR-side conflict) since these are redirect/inclusive. **Status:** needs per-test triage. ### 88f) 200 ≠ 404 (2 tests) — retrieve says not-found Affected: `D010_03_inc_02`, `D010_03_inc_03`. Inclusive-mode retrieve where the broker reports 404 but the test expects the entity (presumably via the mocked CSR). Likely a stub-URL mismatch or the 88a conflict pattern. **Status:** needs per-test triage. ### 88g) Singletons (4 tests) - `D010_02_red` (404 ≠ 200) — retrieve returns 200 where test expects 404. - `D018_01` (508 ≠ 204) — loop-detection test expects 508; broker returns 204. - `D003_02_red` (207 ≠ 404) — append-attr test expects 207; broker returns 404. - `D004_01_exc` (204 ≠ 400) — exclusive create-entity-with-CSR; broker rejects. Each likely maps to one of the patterns above; needs individual inspection. ### Summary Of the 30 mismatches: **8 (88a)** confirmed test bug (setup violates CSR-creation conflict rules; tests need redesign); **5 (88c)** spec ambiguity already in spec-doubts-2 #90, not actionable until the spec resolves; **17 (88b/d/e/f/g)** need per-test triage, structurally suggestive of the same setup-shape issues plus stub-URL gaps but unconfirmed. For all 30 the broker's behaviour is at minimum spec-defensible; no broker-side change is recommended without a spec/conformance discussion.
