Logo etsi

ETSI's Bug Tracker
Main | My View | View Issues | Change Log | Roadmap | Monitor project
  
Notice: information submitted on the ETSI issue Tracker may be incorporated in ETSI publication(s) and therefore subject to the ETSI IPR policy.

View Issue Details [ Jump to Notes ] [ Issue History ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0006009Part 01: TTCN-3 Core LanguageClarificationpublic19-01-2012 17:2321-12-2012 10:48
ReporterBogdan Stanca-Kaposta 
Assigned ToIna Schieferdecker 
PrioritynormalSeverityminorReproducibilityhave not tried
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target Versionv4.5.1 (published 2013-04)Fixed in Versionv4.5.1 (published 2013-04) 
Summary0006009: Charstring Pattern Reference Parts should not be allowed
DescriptionThe two templates from section B.1.5.2 Reference expression / EXAMPLE 2 should not be accepted since only a part of a reference is given.

pattern "{Reg"

is not a valid pattern expression (see BNF 113.PatternElement) since the brace is neither closed nor escaped

Solution: either has to be either escaped or pattern keyword should be removed

template charstring RegExp4 := pattern "\{Reg"
  or
template charstring RegExp4 := "{Reg"

so the templates

template charstring RegExp4 := pattern "{Reg";
template charstring RegExp5 := pattern "Exp1}";
template charstring RegExp6 := pattern "{RegExp4}{RegExp5}";
// matches the string "{RegExp1}" only (i.e. shall not be handled as a reference expression
// after insertion)

have to be changed to:

template charstring RegExp4 := pattern "\{Reg";
template charstring RegExp5 := pattern "Exp1\}";
template charstring RegExp6 := pattern "{RegExp4}{RegExp5}";
// matches the string "{RegExp1}" only (i.e. shall not be handled as a reference expression
// after insertion)


109.CharStringMatch ::= PatternKeyword PatternParticle {"&" PatternParticle}
110.PatternParticle ::= Pattern | ReferencedValue
111.PatternKeyword ::= "pattern" 112.Pattern ::= """ {PatternElement} """
113.PatternElement ::= ("\" ("?" | "*" | "\" | "[" | "]" | "{" | "}" |
"""
"w" )) |
| "|" | "(" | ")" | "#" | "+" | "d" |
| "t" | "n" | "r" | "s" | "b"
("?" | "*" | "\" | "|" | "+"
) | ("[" ["^"] [{PatternChar ["-" PatternChar]}]
"]") |
("{" ReferencedValue "}") |
("\" "N" "{" (ReferencedValue | Type) "}") |
(""" """) |
("(" PatternElement ")") |
("#" (Num |
("(" Num "," [Num] ")") | ("(" "," Num ")")
)) |
PatternChar
114.PatternChar ::= Char | PatternQuadruple
Additional Informationsee also mantis issue 0005835
TagsNo tags attached.
Clause Reference(s)B.1.5.2, A.1.6.1.3
Source (company - Author)Bogdan Stanca-Kaposta (Testingtech) STF433
Attached Filesdoc file icon CR6009_resolution_v1.doc [^] (104,448 bytes) 11-07-2012 17:54
doc file icon CR6009_resolution_v2.doc [^] (87,552 bytes) 12-07-2012 11:08

- Relationships
related to 0006169closedIna Schieferdecker The pattern char grammar has been mixed up 
related to 0006014closedIna Schieferdecker Simplifications request for escaping chars inside the string pattern 

-  Notes
(0010747)
Gyorgy Rethy (reporter)
10-07-2012 12:28
edited on: 11-07-2012 17:17

 STF discussion 2012-07-10:
Check if there is an ambiguity in the current text and examples.
(0010852)
Gyorgy Rethy (reporter)
11-07-2012 17:57

 The comment is not valid as lonely curly brackets do not have metacharacter meaning (table B.1 clearly shows that only {reference} has a metacharacter meaning).

This clarification and some examples for this has been added to the proposed text in CR6009_resolution_v1.doc. Pls. review it.
(0010869)
Bogdan Stanca-Kaposta (reporter)
12-07-2012 11:18

 proposed change base on v1 which complies also with CR 6014

Treating curly brackets with two meaning inside the same context is error prone

Consider a long string which contains a single opening { and afterwards some normal alphanumeric characters. If the { has no special meaning, then it would be natural for someone else to add } (intended as } literal) at a later place.

Suddenly, the pair of brackets { and } gets the reference meaning.

This rule will therefore make such code unmaintainable.
(0010881)
Gyorgy Rethy (reporter)
12-07-2012 12:27
edited on: 12-07-2012 12:32

 A few years ago I would agree. But now, actually, adding this restriction would cause existing code containing single { without \ to fail. Also, if adding a } to an existing list of characters randomly, it is of very little probability that the new "reference" would match an existing charstring or univ. charstring value or template. It is much more probable, that this simply would cause a semantic checking error. In the rare cases when the new "reference" imitates an existing definition, it will cause a fail or inconc verdict that the testers do debug during functional testing anyway.

So, I stick to leave the standard backward compatible as to my experience the users are much more sensitive to backward compatibility than to correct a new error; even a simplest change takes huge time: write a TR, checking out the code from the version handling system, make the correction, checking the code in, possibly making a new product release if this is in a SW library, administer the TR - all together takes at least half a day, while the technical change is just a few minutes. Also, "old" existing code is normally not touched, just executed as regression test cases. Users expect from a new version of the tool to compile and execute their existing millions of lines of code seemlessly, without changing anything. For this reason even the smallest non-backward compatible change is a very sensitive issue, it shall give a significant value for the user.
(0010884)
Bogdan Stanca-Kaposta (reporter)
12-07-2012 13:30
edited on: 12-07-2012 14:57

 I would propose to escalate this to the TTCN-3 users list.

This feature probably has never been used (since not all tools - if any - implement this rule), and I think that the standard changes should improve the life of the other 99.9% that use this special feature of the charstring pattern.
(0010887)
Gyorgy Rethy (reporter)
12-07-2012 14:34
edited on: 12-07-2012 18:52

 Where are you taking those strange figures from? I do not know myself how many TTCN-3 users are there as tool vendors do not publish their figures (but I would be interested about your source myself). I know only that our tool Titan do implements patterns as specified today and we have millions of lines of TTCN-3 code and about 3600 active licenses in the company. I really hope that this is just 0,1 % of all TTCN-3 users on the world :-)... but at the same time I have some doubt ...

So, as explained earlier, for Ericsson only the backward-compatible proposal in CR6009_resolution_v1.doc is acceptable.
(0010889)
Jacob Wieland - Spirent (reporter)
12-07-2012 15:49

 Actually, we do not believe that anyone would use this feature willingly as it is simply confusing and will thus yield bad code. We can of course speak only for our users (and extrapolate from that) which use a tool where we explicitly decided not to implement this strange feature and in the last 10 years, no one has complained about missing it.

From my experience with regexp languages in general (and other languages), it is not normal that a part of an expression changes its semantics dependent on another part of the expression which can come MUCH MUCH MUCH later.

Most importantly, the rule about the single { or } also violates the normal compositionality rule for regular expressions, namely that you can compose two valid regular expressions by concatenation and always get a valid regular expression as the result.
(0010892)
Jacob Wieland - Spirent (reporter)
12-07-2012 16:03

 Also, pragmatically, no one forces Ericsson to REMOVE the existing feature from their tool, thereby maintaining the possibility for their users to keep their code untouched. As there are also other non-standard-compliant features in existence in that tool, I don't see the relevance to keep the feature in the standard in its current form which can only cause more problems than it solves.

Thus, only for publishing new globally standardized testsuites, this can become an issue and there it is relevant to tag the testsuite with the TTCN-3 edition anyway.
(0010896)
Jens Grabowski (manager)
12-07-2012 16:36

 Even though I am technically on your side, a non backwards compatible change requires an agreement from MTS. After our last non backwards compatible change that we introduced accidentally and which caused problems, I doubt that we will get the agreement.

One way of working towards a removal in the future would be to move the problematic parts into the Annex G "Depreciated language features". I don't know if it can be done here and if this is acceptable for Ericsson.
(0010905)
Gyorgy Rethy (reporter)
12-07-2012 18:34

 Jacob, your comment is totally misleading. You are talking about composing references from different patterns that I also think would be a bad code design.

But the feature discussed here is to use a single curly bracket without escaping it (a.k.a escaping is optional). And as this feature has been in the language for a long time and a huge amount of code may use this feature (i.e. matching a single curly bracket as a literal character, without escaping), we do not agree to change the language in a non-backward compatible way without resolving an error and without any major benefit. We do not agree to move this feature to be a deprecated one.
(0010906)
Gyorgy Rethy (reporter)
12-07-2012 18:49
edited on: 12-07-2012 19:48

 Secondly, this change would cause other technical problems. E.g.

const charstring user:= "me";
const charstring passw:= "a{bc";
template charstring temp := pattern "*{user};{passw}*";
...
p.send(me);
p.send(passw);
p.receive(temp);

this code is working fine today, isn't it? By making escaping for single curly brackets mandatory, this code would become invalid and the use case could not be solved at all, as no one knowns when and at which position passw is containing a curly bracket (pls. note, if there is a pair of curly brackets in passw, the code is still working as references are resolved only once, i.e. the reference {passw} is resolved but then the content of passw is processed literally, i.e. if passw contained "{abc}", temp will match the string containing "me;{abc}").
(0010908)
Jacob Wieland - Spirent (reporter)
13-07-2012 09:56
edited on: 13-07-2012 10:22

 No, referencing a charstring *value* (not a pattern) from a pattern automatically quotes all characters in the charstring value which would be special characters in a charstring pattern (to interpret the value as its literal value). Only in the regexp function is a charstring value reinterpreted as a regular expression.

So, even if the feature would be deprecated, the code you provided would still be as valid. The resulting pattern would be: pattern "*me;a\{bc*" which has your desired semantics.

Therefore, I still think that making the feature of single unquoted { or } deprecated is a good compromise which will not break any code, but allow tools to warn users about (or prevent them from) such potentially harmful code. It will not change any semantics whatsoever, but will only deprecate some syntax which is now valid.

The problem of missing compositionality is especially harmful considering the regexp function which can (unfortunately) also take charstring values and reinterpret them as regular expressions. Here, it is even more probable that someone will simply concatenate two or more charstring values which for them are all valid regular expressions (to be arbitrarily received from somewhere) and then give them to the regexp function only to find out at runtime that the resulting regular expression was not valid.
(0011273)
Tomas Urban (developer)
14-12-2012 11:33

 This issue is a part of several related problems concerning the pattern template rules. The STF 446 has decided to solve these issues by opting for such rule that efficiently excludes any possibility of errors caused by invalid pattern format. Tool vendors can still include their own tool-specific format checks, e.g. warning users when certain characters do not form a valid metacharacters.

Please take a look at resolution a of CR 6014. The STF 446 identified a potential problem mentioned in the discussion concerning automatic escaping of referenced strings inserted into patterns and added such a functionality to the draft.
(0011295)
Bogdan Stanca-Kaposta (reporter)
19-12-2012 15:28

 Maybe I missed some of the comments/discussions Threads on this bug. Would it be possible to comment how the {\referenceName} may solve the problem raised in this bug.

Thank you!

STF451
(0011304)
Gyorgy Rethy (reporter)
21-12-2012 10:17

 I think that this issue has been handled in 6014 (main text part) and 6169 (BNF), hence this one can be closed with resolution fixed and by a note saying that the technical solution is contained in the above CRs.
(0011306)
Ina Schieferdecker (reporter)
21-12-2012 10:48

 Technical solution in 6169 and 6014 - hence closed.

- Issue History
Date Modified Username Field Change
19-01-2012 17:23 Bogdan Stanca-Kaposta New Issue
19-01-2012 17:23 Bogdan Stanca-Kaposta Clause Reference(s) => B.1.5.2, A.1.6.1.3
19-01-2012 17:23 Bogdan Stanca-Kaposta Source (company - Author) => Bogdan Stanca-Kaposta (Testingtech) STF433
09-07-2012 16:57 Ina Schieferdecker Project TTCN-3 Change Requests => Part 01: TTCN-3 Core Language
10-07-2012 12:28 Gyorgy Rethy Note Added: 0010747
10-07-2012 12:28 Gyorgy Rethy Assigned To => Gyorgy Rethy
10-07-2012 12:28 Gyorgy Rethy Status new => assigned
10-07-2012 12:28 Gyorgy Rethy Target Version => Edition 4.5.1
11-07-2012 17:17 Gyorgy Rethy Note Edited: 0010747
11-07-2012 17:54 Gyorgy Rethy File Added: CR6009_resolution_v1.doc
11-07-2012 17:57 Gyorgy Rethy Note Added: 0010852
11-07-2012 17:57 Gyorgy Rethy Assigned To Gyorgy Rethy => Jens Grabowski
12-07-2012 11:08 Bogdan Stanca-Kaposta File Added: CR6009_resolution_v2.doc
12-07-2012 11:18 Bogdan Stanca-Kaposta Note Added: 0010869
12-07-2012 12:27 Gyorgy Rethy Note Added: 0010881
12-07-2012 12:32 Gyorgy Rethy Note Edited: 0010881
12-07-2012 13:30 Bogdan Stanca-Kaposta Note Added: 0010884
12-07-2012 14:34 Gyorgy Rethy Note Added: 0010887
12-07-2012 14:35 Gyorgy Rethy Note Edited: 0010887
12-07-2012 14:57 Bogdan Stanca-Kaposta Note Edited: 0010884
12-07-2012 15:49 Jacob Wieland - Spirent Note Added: 0010889
12-07-2012 16:03 Jacob Wieland - Spirent Note Added: 0010892
12-07-2012 16:36 Jens Grabowski Note Added: 0010896
12-07-2012 18:34 Gyorgy Rethy Note Added: 0010905
12-07-2012 18:49 Gyorgy Rethy Note Added: 0010906
12-07-2012 18:51 Gyorgy Rethy Note Edited: 0010887
12-07-2012 18:52 Gyorgy Rethy Note Edited: 0010887
12-07-2012 18:52 Gyorgy Rethy Note Edited: 0010887
12-07-2012 18:57 Gyorgy Rethy Note Edited: 0010906
12-07-2012 19:00 Gyorgy Rethy Note Edited: 0010906
12-07-2012 19:48 Gyorgy Rethy Note Edited: 0010906
13-07-2012 09:56 Jacob Wieland - Spirent Note Added: 0010908
13-07-2012 10:22 Jacob Wieland - Spirent Note Edited: 0010908
12-12-2012 09:20 Gyorgy Rethy Assigned To Jens Grabowski => Tomas Urban
12-12-2012 10:49 Ina Schieferdecker Relationship added related to 0006169
14-12-2012 11:16 Tomas Urban Relationship added related to 0006014
14-12-2012 11:33 Tomas Urban Note Added: 0011273
14-12-2012 11:33 Tomas Urban Assigned To Tomas Urban => Gyorgy Rethy
19-12-2012 15:28 Bogdan Stanca-Kaposta Note Added: 0011295
21-12-2012 10:17 Gyorgy Rethy Note Added: 0011304
21-12-2012 10:19 Gyorgy Rethy Assigned To Gyorgy Rethy => Ina Schieferdecker
21-12-2012 10:48 Ina Schieferdecker Note Added: 0011306
21-12-2012 10:48 Ina Schieferdecker Status assigned => closed
21-12-2012 10:48 Ina Schieferdecker Resolution open => fixed
21-12-2012 10:48 Ina Schieferdecker Fixed in Version => Edition 4.5.1

MantisBT 1.2.14 [^]
Copyright © 2000 - 2024 MantisBT Team
Powered by Mantis Bugtracker