Austin Group Defect Tracker

Aardvark Mark IV

Main | My View | View Issues | Change Log | Docs
  

Viewing Issue Simple Details [ Jump to Notes ] [ Issue History ] [ Print ]
ID Category Severity Type Date Submitted Last Update
0000944 [1003.1(2013)/Issue7+TC1] Shell and Utilities Comment Enhancement Request 2015-05-04 21:36 2019-06-10 08:54
Reporter stephane View Status public  
Assigned To
Priority normal Resolution Accepted As Marked  
Status Closed  
Name Stephane Chazelas
Organization
User Reference
Section sed editing commands
Page Number 3182
Line Number 106416
Interp Status Approved
Final Accepted Text Note: 0002720
Summary 0000944: sed: relax requirement for } to be preceded by newline
Description In the sed spec, we read:


     [2addr] {editing command
     editing command
     ...
     }
             Execute a list of sed editing commands only when the pattern
             space is selected. The list of sed editing commands shall be
             surrounded by braces and separated by <newline> characters, and
             conform to the following rules. The braces can be preceded or
             followed by <blank> characters. The editing commands can be
             preceded by <blank> characters, but shall not be followed by
             <blank> characters. The <right-brace> shall be preceded by a
             <newline> and can be preceded or followed by <blank> characters.


And further down:


     Historically, the sed ! and } editing commands did not permit multiple
     commands on a single line using a <semicolon> as a command delimiter.
     Implementations are permitted, but not required, to support this
     extension.


(that one being weirdly worded especially considering than neither ! nor } are clearly listed as editing commands).

I've never come across a system where:

sed '/foo/!{/bar/{=;=;=; };}'


was not accepted. As just tested on a PDP11 emulator, the sed in Unix version 7 (where sed was first introduced) allowed it so I'd be curious to know what "historical" implementation did not allow them. Those kind of syntax are widely used. The sed FAQ has some examples with ";}" in them.

I suspect the confusion may come from the fact that the last command in a command group is often a branching one (:, t, b) where semicolon is valid in the label (as per POSIX, not as per several sed implementations), but that's a limitation on ; after those commands, not ; before }.

This part is also confusing:


     Command verbs other than {, a, b, c, i, r, t, w, :, and # can be
     followed by a <semicolon>, optional <blank> characters, and another
     command verb. However, when the s command verb is used with the w flag,
     following it with another command in this manner produces undefined
     results.


as it's unclear what is meant by "command verbs" in the case of }. The only sensible explanation is a whole command group ({ to the matching }). As in {...;} cannot be followed by ;, but then again all implementations I tested including Unix V7 allow:

sed '{=;};{=;}'

Desired Action Change the specification so that ; be allowed to separate commands inside a command group (though with the limitations on b, t, :, r, w and s with w flag), and that } may be preceded by ;.
Tags tc2-2008
Attached Files

- Relationships
related to 0000262Closedajosey 1003.1(2008)/Issue 7 sed with multiple -e options 
related to 0000269Closedajosey 1003.1(2004)/Issue 6 sed addr in braces 
related to 0000961Applied 1003.1(2013)/Issue7+TC1 permit <semicolon> to terminate {...} command 

-  Notes
(0002648)
geoffclare (manager)
2015-05-05 09:14

 Regarding this statement in the description:

> I've never come across a system where:
>
> sed '/foo/!{/bar/{=;=;=; };}'
>
> was not accepted.

That command is not accepted on HP-UX or Solaris (/usr/xpg4/bin/sed and
/bin/sed). They report that there are too many '{'s. There may be
more systems that don't accept it - I only tried those two systems
and Linux (GNU sed).
(0002649)
Michael Wilson (reporter)
2015-05-05 09:48

 On AIX the behaviour is also to report too many '{'.

Minimally, the command is accepted by adding 1 newline before the second '}', thus

sed '/foo/!{/bar/{=;=;=; };^J}'
(0002650)
stephane (reporter)
2015-05-05 10:14

 Sorry, my bad, I had messed up my testing (I was testing sed '/a/{/b/=;=;};}' instead of sed '/a/{/b/{=;=;};}' on Solaris/v7 which is OK (missing {, but the text after } is ignored)), and it now confirms my recollection that } could not be <strong>followed</strong> by a semicolon (the part about the { command verb in the spec).

(note that I did not request that to be changed in the "Desired action", though IMO, the text covering that should be clarified)

The rest still holds so far though: separating commands with semicolon within a group and have } preceded by a semicolon instead of newline is portable (including to the initial implementation on V7) and is common practice.
(0002719)
eblake (manager)
2015-06-18 16:04

 Do we even have an example of ! followed by semicolon choking, where the same editing command without ! before the function does not? That is, if sed -n '/foo/p; /bar/p' works (as longhand for sed -n /\(foo\|bar\)/p), where would sed '/foo/!p; /bar/p' fail?

Conversely, the standard says one or more ! is allowed, but GNU sed doesn't like it:

$ sed '!!p'
sed: -e expression #1, char 2: multiple `!'s
(0002720)
geoffclare (manager)
2015-06-18 16:21
edited on: 2015-06-18 16:40

 Interpretation response
------------------------
The standard states the requirements for the use of <semicolon> to terminate sed editing commands, and conforming applications must conform to this. However, concerns have been raised about this which are being referred to the sponsor.

Rationale:
-------------
The use of <semicolon> within {...} is common practice and should be specified in the standard. The standard developers are not aware of any implementation that does not already support it. A related issue is that some implementations do not support multiple '!' characters before a function. Since it is not portable in practice, it seems unlikely that any applications would be affected by removing the requirement from the standard.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
On page 3182 line 106404, change:
Command verbs other than {, a, b, c, i, r, t, w, :, and # can be followed by a <semicolon>, optional <blank> characters, and another command verb. However, when the s command verb is used with the w flag, following it with another command in this manner produces undefined results.

to:
Editing commands other than {...}, a, b, c, i, r, t, w, :, and # can be followed by a <semicolon>, optional <blank> characters, and another editing command. However, when an s editing command is used with the w flag, following it with another command in this manner produces undefined results.

On page 3182 line 106408, change:
A function can be preceded by one or more ’!’ characters,

to:
A function can be preceded by a ’!’ character,

On page 3182 line 106416, change:
Execute a list of sed editing commands only when the pattern space is selected. The list of sed editing commands shall be surrounded by braces and separated by <newline> characters, and conform to the following rules. The braces can be preceded or followed by <blank> characters. The editing commands can be preceded by <blank> characters, but shall not be followed by <blank> characters. The <right-brace> shall be preceded by a <newline> and can be preceded or followed by <blank> characters.

to:
Execute a list of sed editing commands only when the pattern space is selected. The list of sed editing commands shall be surrounded by braces. The braces can be preceded or followed by <blank> characters. The <right-brace> shall be preceded by a <newline> or <semicolon> (before any optional <blank> characters preceding the <right-brace>).

Each command in the list of commands shall be terminated by a <newline> character, or by a <semicolon> character if permitted when the command is used outside the braces. The editing commands can be preceded by <blank> characters, but shall not be followed by <blank> characters.



On page 3187 line 106647, delete:
Historically, the sed ! and } editing commands did not permit multiple commands on a single line using a <semicolon> as a command delimiter. Implementations are permitted, but not required, to support this extension.
(0002723)
ajosey (manager)
2015-06-19 07:07

 Interpretation Proposed: June 19 2015
(0002819)
ajosey (manager)
2015-09-07 11:33

 Interpretation approved: 7 Sep 2015

- Issue History
Date Modified Username Field Change
2015-05-04 21:36 stephane New Issue
2015-05-04 21:36 stephane Name => Stephane Chazelas
2015-05-04 21:36 stephane Section => sed editing commands
2015-05-04 21:36 stephane Page Number => http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html#tag_20_116_13_03 [^]
2015-05-04 21:36 stephane Line Number => http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html#tag_20_116_13_03 [^]
2015-05-05 09:14 geoffclare Note Added: 0002648
2015-05-05 09:48 Michael Wilson Note Added: 0002649
2015-05-05 10:14 stephane Note Added: 0002650
2015-06-18 16:04 eblake Note Added: 0002719
2015-06-18 16:15 eblake Relationship added related to 0000262
2015-06-18 16:21 geoffclare Note Added: 0002720
2015-06-18 16:22 rhansen Relationship added related to 0000961
2015-06-18 16:23 geoffclare Page Number http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html#tag_20_116_13_03 [^] => 3182
2015-06-18 16:23 geoffclare Line Number http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html#tag_20_116_13_03 [^] => 106416
2015-06-18 16:23 geoffclare Interp Status => ---
2015-06-18 16:23 geoffclare Final Accepted Text => Note: 0002720
2015-06-18 16:23 geoffclare Status New => Resolved
2015-06-18 16:23 geoffclare Resolution Open => Accepted As Marked
2015-06-18 16:23 geoffclare Tag Attached: issue8
2015-06-18 16:29 geoffclare Tag Detached: issue8
2015-06-18 16:30 geoffclare Tag Attached: tc2-2008
2015-06-18 16:36 eblake Relationship added related to 0000269
2015-06-18 16:40 geoffclare Note Edited: 0002720
2015-06-18 16:42 geoffclare Interp Status --- => Pending
2015-06-18 16:42 geoffclare Status Resolved => Interpretation Required
2015-06-19 07:07 ajosey Interp Status Pending => Proposed
2015-06-19 07:07 ajosey Note Added: 0002723
2015-09-07 11:33 ajosey Interp Status Proposed => Approved
2015-09-07 11:33 ajosey Note Added: 0002819
2019-06-10 08:54 agadmin Status Interpretation Required => Closed

Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker