Commit 15c402ce authored by kzangeli's avatar kzangeli
Browse files

doubts: relocate #85-#88 from code/doubts MRs (273/277/279/281) into the doubts MR

Per the "doubts.md changes live in one dedicated MR" policy, fold the doubt
entries that were riding inside other MRs into this one:

- #85 D010_01_aux dict-body stub (RESOLVED via !273; dict-body counterpart of #69)
- #86 D011_02_exc_01 CSR-without-Link-header context mismatch (OPEN; !277 workaround; continues #79)
- #87 020_17..020_20 temporal-setup pairing (PARTIALLY RESOLVED via !279; broker tombstone-kind bug still open)
- #88 DistOps write-op status-code triage, 30 tests / 7 patterns (folded from !281)

!273/!277/!279 were rebased to drop their doubts.md hunks; !281 (doubts-only)
is superseded by #88 here and can be closed. Doc-only.
parent 63751476
Loading
Loading
Loading
Loading
+191 −0
Original line number Diff line number Diff line
@@ -2924,3 +2924,194 @@ expect the key to be present and want to verify it equals
something — not the negative-presence assertion this test is
trying to do.


## 85. `D010_01_aux` — stub body passed as a Python dict; HttpCtrl doesn't auto-serialise

**Hit:** `D010_01_aux` (auxiliary RetrieveEntity gap-fill merge) fails with
`No value found for path $.isParked` — the broker's response contains only
the local entity, missing the gap-fill attribute the auxiliary CSR should
provide.

**Why:** the test does

```robotframework
${entity_body}=    Load Entity    ${entity_payload_filename}    ${entity_id}
Set Stub Reply    GET    /ngsi-ld/v1/entities/${entity_id}    200    ${entity_body}
```

`Load Entity` returns a Python **dict**, and HttpCtrl's response writer
(`http_handler.py::__send_response`) only encodes the body when it is already
a `str`. A dict falls through: `isinstance(body, str)` is false → no encoding;
`len(dict)` returns the number of KEYS → wrong `Content-Length`;
`wfile.write(dict)` raises (silently swallowed) → empty body on the wire. The
mock sends headers only, the broker has no auxiliary body to merge, the test
fails on the missing gap-fill attribute.

**Impact / broker:** none. A correct aux-merge produces the spec-correct
§ 9.3.2 result (local wins for shared attrs, CSR fills gaps) when verified
end-to-end against a real upstream returning valid JSON.

**Status:** RESOLVED via !273 — the test now serialises the body first
(`Evaluate    json.dumps($entity_body)    json`) so the wire payload is
non-empty. This is the dict-body counterpart of [[#69]]: once the vendored
fork's conditional auto-serialise landed (via !278/!282, see #69), this
test-side `json.dumps` became belt-and-suspenders — a string re-`json.dumps`'d
to itself is unchanged — so it stays harmless. The same dict-body pattern may
affect other `D010_*` retrieve tests; the framework-side guard in #69 is the
cleaner long-term home.


## 86. `D011_02_exc_01` — CSR response without a Link header → broker can't reconcile attribute IRIs

**Hit:** Even with the upgraded HttpCtrl matcher (!277) and the spec-canonical
`pick=` stub URL (see #79), `D011_02_exc_01` still fails with `No value found
for path $.speed`. The mock is matched, the broker forwards correctly and
receives the speed entity body — but the merged response contains only the
local-broker attributes (`brandName`, `isParked`); the CSR-side `speed` is
silently dropped.

**Root cause (likely):** the HttpCtrl mock can set Content-Type + body via
`Set Stub Reply` but cannot set a Link header. So the mock's response carries
no `Link: <…>; rel="http://www.w3.org/ns/json-ld#context"`. The broker,
receiving a JSON-only response with no Link, falls back to the **default core
context** when expanding the CSR's short attribute names. The local entity was
stored with the test-suite context, so its `speed` expanded to one IRI; the
CSR-side `speed` expands to `…/default-context/speed`. Different IRIs → the
merge keeps them as different attributes; the response `@context` only knows
the local one, so the CSR-side speed is lost on compaction.

**Two-sided issue:**

- **Test-side:** `Set Stub Reply` cannot add response headers, so it cannot
  simulate a real CS that echoes back the request's `Link`. The right
  HttpCtrl-fork fix is `Set Stub Reply` with optional headers.
- **Broker-side:** when a CSR responds with no Link header, the broker should
  fall back to the registered CSR's `@context` (§ 5.2.12) or the original
  request's `@context` — not the default core. § 9.3 is permissive here;
  arguably a spec gap too.

**Status:** OPEN. !277 switched the stub URL to spec-canonical `pick=speed` so
the mock is at least reached when paired with the matcher fix; the merge bug
itself needs broker-side review. Continuation of #79 (same test, next layer);
related to the header-less-stub limitation in [[#83]].


## 87. `020_17 / 020_18 / 020_19 / 020_20` setup — pair the temporal POST with a current-state POST

**Status:** PARTIALLY RESOLVED via !279 (setup side, fixing [[#9]]); a
broker-side deletedAt-tombstone bug remains, see below.

**Hit + spec recap:** see [[#9]] for the full analysis. Briefly: each test's
setup did only `POST /temporal/entities`, then the body's `DELETE
/entities/{id}/attrs/{name}` (a current-state operation) returned 404 because
the entity had no current-state representation; no `deletedAt` tombstone was
written, so the temporal-evolution assertion failed with `Item root[<attr>]
removed from dictionary`.

**Fix (via !279):** each `Create Temporal Entity` setup now also POSTs a
matching current-state entity via `Create Entity`, using two new minimal
fixtures: `vehicle-different-attribute-types.jsonld` (fuelLevel/isParkedIn/name,
020_17/18) and `vehicle-with-scope.jsonld` (scope + fuelLevel, 020_19/20). The
following `POST /temporal/entities` may return 201 *or* 204 per § 11.2.2.5
(append to existing), so the status check is relaxed accordingly.

**Result:** 020_17_01 (Property) and 020_18_01 (Property + temporalValues) now
pass. The four non-Property cases (020_17_02/03, 020_18_02/03, 020_19_01,
020_20_01) still fail on a *different*, broker-side root cause:

**Broker bug surfaced (separate work):** the `deletedAt` tombstone collapses
all attribute kinds to `Property` with `urn:ngsi-ld:null`. For a deleted
**Relationship**, the temporal-store tombstone should preserve
`type: Relationship` and substitute `object: urn:ngsi-ld:null` — not switch
the kind to Property. Same for **LanguageProperty** (`languageMap`) and
**Scope**. The broker emits a uniform `{ "type": "Property", "value":
"urn:ngsi-ld:null", "deletedAt": … }` for every kind, so the deepdiff fails
with `… removed` + `type changed from "<kind>" to "Property"`. The broker fix
belongs in its own MR — out of scope for the test-side change.


## 88. DistOps write-op status-code mismatches — 30 tests across 7 patterns, mostly test-side design issues

A categorisation pass on the 30 DistOps tests whose `HTTP status code
comparison failed`. Triaged into seven patterns; for most the broker is
correct and the tests need a redesign by the original author. None of this is
implementation-blocking: the broker behaviour is the spec-compliant or
spec-defensible reading in every case. Overlaps the clusters in #82 / [[#84]].

### 88a) 201 ≠ 409 (8 tests) — setup creates a CSR that conflicts with locally-stored data

Affected: `D001_02_exc`, `D002_02_exc`, `D005_01_exc`, `D009_01_exc`,
`D013_01_red`, `D006_01_red`, `D006_02_red`, `D009_01_red`. Each setup does
`Create Entity … local=true` then `Create Context Source Registration …`
(exclusive or redirect) for the same entity. Both CSR shapes conflict with a
locally-stored copy and the broker correctly returns **409**:

- **Exclusive** (4): an exclusive CSR must also specify `propertyNames` /
  `relationshipNames`; claiming a whole entity exclusively is not permitted.
  The redirection-ops fixture has neither, so it is malformed for exclusive
  use anyway.
- **Redirect** (4): a redirect CSR cannot coexist with a local copy of the
  same entity — writes would go to the CS, reads to local. 409 by design.

**Impact / broker:** none — conflict detection at CSR-creation time is
implemented and tested. **Fix wanted:** the tests need a redesign by their
author — the docstrings don't match the setup as written. Likely the intent
was to register the CSR first and exercise the distop path in the test body,
not to pre-create a conflicting local entity. The four `exc` cases also need
`propertyNames` in the CSR fixture.

### 88b) 204 ≠ 404 (5 tests) — redirect-mode forwards return CS's 404

Affected: `D003_01_red`, `D004_01_red`, `D014_01_red`, `D014_02_red`,
`D015_01_red`. Test expects 204; broker returns 404. Likely the stub matches a
different URL shape than the broker forwards (see the matcher work in !277), so
the mock returns its default 404 — or the same setup-conflict pattern as 88a.
**Status:** needs per-test triage.

### 88c) 207 ≠ 204 (3 tests) and 204 ≠ 207 (2 tests) — inclusive-mode forward failure status

Affected: `D002_02_01_inc`, `D002_02_02_inc`, `D004_01_inc`, `D018_02_02`,
`D003_01_inc`. Open spec ambiguity (already logged in spec-doubts-2 #90): when
a write forwarded to an *inclusive* CSR fails, should the broker return 207
(partial success) or a clean 204/201? § 10.2.1.4 and § 9.3.2 imply different
answers; current behaviour varies per op. **Resolution:** wait for spec
clarification.

### 88d) 200 ≠ 400 (3 tests) — query rejected as invalid

Affected: `D011_02_red_02`, `D011_03_inc_02`, `D011_04_inc_02`. Broker
considers the query malformed (400); test expects 200. **Status:** needs
per-test triage of whether the strictness is justified.

### 88e) 201 ≠ 400 (3 tests) — batch op rejected as invalid

Affected: `D012_01_red`, `D013_02_red`, `D015_01_inc`. Same as 88d for batch
ops; probably related to 88a (CSR-side conflict) since these are
redirect/inclusive. **Status:** needs per-test triage.

### 88f) 200 ≠ 404 (2 tests) — retrieve says not-found

Affected: `D010_03_inc_02`, `D010_03_inc_03`. Inclusive-mode retrieve where
the broker reports 404 but the test expects the entity (presumably via the
mocked CSR). Likely a stub-URL mismatch or the 88a conflict pattern.
**Status:** needs per-test triage.

### 88g) Singletons (4 tests)

- `D010_02_red` (404 ≠ 200) — retrieve returns 200 where test expects 404.
- `D018_01` (508 ≠ 204) — loop-detection test expects 508; broker returns 204.
- `D003_02_red` (207 ≠ 404) — append-attr test expects 207; broker returns 404.
- `D004_01_exc` (204 ≠ 400) — exclusive create-entity-with-CSR; broker rejects.

Each likely maps to one of the patterns above; needs individual inspection.

### Summary

Of the 30 mismatches: **8 (88a)** confirmed test bug (setup violates
CSR-creation conflict rules; tests need redesign); **5 (88c)** spec ambiguity
already in spec-doubts-2 #90, not actionable until the spec resolves;
**17 (88b/d/e/f/g)** need per-test triage, structurally suggestive of the same
setup-shape issues plus stub-URL gaps but unconfirmed. For all 30 the broker's
behaviour is at minimum spec-defensible; no broker-side change is recommended
without a spec/conformance discussion.