View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
00009441003.1(2013)/Issue7+TC1Shell and Utilitiespublic2015-05-04 21:362019-06-10 08:54
Reporterstephane Assigned To 
PrioritynormalSeverityCommentTypeEnhancement Request
Status ClosedResolutionAccepted As Marked 
NameStephane Chazelas
Organization
User Reference
Sectionsed editing commands
Page Number3182
Line Number106416
Interp StatusApproved
Final Accepted Text0000944:0002720
Summary0000944: sed: relax requirement for } to be preceded by newline
DescriptionIn the sed spec, we read:


     [2addr] {editing command
     editing command
     ...
     }
             Execute a list of sed editing commands only when the pattern
             space is selected. The list of sed editing commands shall be
             surrounded by braces and separated by <newline> characters, and
             conform to the following rules. The braces can be preceded or
             followed by <blank> characters. The editing commands can be
             preceded by <blank> characters, but shall not be followed by
             <blank> characters. The <right-brace> shall be preceded by a
             <newline> and can be preceded or followed by <blank> characters.


And further down:


     Historically, the sed ! and } editing commands did not permit multiple
     commands on a single line using a <semicolon> as a command delimiter.
     Implementations are permitted, but not required, to support this
     extension.


(that one being weirdly worded especially considering than neither ! nor } are clearly listed as editing commands).

I've never come across a system where:

sed '/foo/!{/bar/{=;=;=; };}'


was not accepted. As just tested on a PDP11 emulator, the sed in Unix version 7 (where sed was first introduced) allowed it so I'd be curious to know what "historical" implementation did not allow them. Those kind of syntax are widely used. The sed FAQ has some examples with ";}" in them.

I suspect the confusion may come from the fact that the last command in a command group is often a branching one (:, t, b) where semicolon is valid in the label (as per POSIX, not as per several sed implementations), but that's a limitation on ; after those commands, not ; before }.

This part is also confusing:


     Command verbs other than {, a, b, c, i, r, t, w, :, and # can be
     followed by a <semicolon>, optional <blank> characters, and another
     command verb. However, when the s command verb is used with the w flag,
     following it with another command in this manner produces undefined
     results.


as it's unclear what is meant by "command verbs" in the case of }. The only sensible explanation is a whole command group ({ to the matching }). As in {...;} cannot be followed by ;, but then again all implementations I tested including Unix V7 allow:

sed '{=;};{=;}'

Desired ActionChange the specification so that ; be allowed to separate commands inside a command group (though with the limitations on b, t, :, r, w and s with w flag), and that } may be preceded by ;.
Tagstc2-2008

Relationships

related to 0000262 Closedajosey 1003.1(2008)/Issue 7 sed with multiple -e options 
related to 0000269 Closedajosey 1003.1(2004)/Issue 6 sed addr in braces 
related to 0000961 Closed 1003.1(2013)/Issue7+TC1 permit <semicolon> to terminate {...} command 

Activities

geoffclare

2015-05-05 09:14

manager   bugnote:0002648

Regarding this statement in the description:

> I've never come across a system where:
>
> sed '/foo/!{/bar/{=;=;=; };}'
>
> was not accepted.

That command is not accepted on HP-UX or Solaris (/usr/xpg4/bin/sed and
/bin/sed). They report that there are too many '{'s. There may be
more systems that don't accept it - I only tried those two systems
and Linux (GNU sed).

user235

2015-05-05 09:48

  bugnote:0002649

On AIX the behaviour is also to report too many '{'.

Minimally, the command is accepted by adding 1 newline before the second '}', thus

sed '/foo/!{/bar/{=;=;=; };^J}'

stephane

2015-05-05 10:14

reporter   bugnote:0002650

Sorry, my bad, I had messed up my testing (I was testing sed '/a/{/b/=;=;};}' instead of sed '/a/{/b/{=;=;};}' on Solaris/v7 which is OK (missing {, but the text after } is ignored)), and it now confirms my recollection that } could not be followed by a semicolon (the part about the { command verb in the spec).

(note that I did not request that to be changed in the "Desired action", though IMO, the text covering that should be clarified)

The rest still holds so far though: separating commands with semicolon within a group and have } preceded by a semicolon instead of newline is portable (including to the initial implementation on V7) and is common practice.

eblake

2015-06-18 16:04

manager   bugnote:0002719

Do we even have an example of ! followed by semicolon choking, where the same editing command without ! before the function does not? That is, if sed -n '/foo/p; /bar/p' works (as longhand for sed -n /\(foo\|bar\)/p), where would sed '/foo/!p; /bar/p' fail?

Conversely, the standard says one or more ! is allowed, but GNU sed doesn't like it:

$ sed '!!p'
sed: -e expression #1, char 2: multiple `!'s

geoffclare

2015-06-18 16:21

manager   bugnote:0002720

Last edited: 2015-06-18 16:40

 Interpretation response
------------------------
The standard states the requirements for the use of <semicolon> to terminate sed editing commands, and conforming applications must conform to this. However, concerns have been raised about this which are being referred to the sponsor.

Rationale:
-------------
The use of <semicolon> within {...} is common practice and should be specified in the standard. The standard developers are not aware of any implementation that does not already support it. A related issue is that some implementations do not support multiple '!' characters before a function. Since it is not portable in practice, it seems unlikely that any applications would be affected by removing the requirement from the standard.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
On page 3182 line 106404, change:
Command verbs other than {, a, b, c, i, r, t, w, :, and # can be followed by a <semicolon>, optional <blank> characters, and another command verb. However, when the s command verb is used with the w flag, following it with another command in this manner produces undefined results.

to:
Editing commands other than {...}, a, b, c, i, r, t, w, :, and # can be followed by a <semicolon>, optional <blank> characters, and another editing command. However, when an s editing command is used with the w flag, following it with another command in this manner produces undefined results.

On page 3182 line 106408, change:
A function can be preceded by one or more ’!’ characters,

to:
A function can be preceded by a ’!’ character,

On page 3182 line 106416, change:
Execute a list of sed editing commands only when the pattern space is selected. The list of sed editing commands shall be surrounded by braces and separated by <newline> characters, and conform to the following rules. The braces can be preceded or followed by <blank> characters. The editing commands can be preceded by <blank> characters, but shall not be followed by <blank> characters. The <right-brace> shall be preceded by a <newline> and can be preceded or followed by <blank> characters.

to:
Execute a list of sed editing commands only when the pattern space is selected. The list of sed editing commands shall be surrounded by braces. The braces can be preceded or followed by <blank> characters. The <right-brace> shall be preceded by a <newline> or <semicolon> (before any optional <blank> characters preceding the <right-brace>).

Each command in the list of commands shall be terminated by a <newline> character, or by a <semicolon> character if permitted when the command is used outside the braces. The editing commands can be preceded by <blank> characters, but shall not be followed by <blank> characters.



On page 3187 line 106647, delete:
Historically, the sed ! and } editing commands did not permit multiple commands on a single line using a <semicolon> as a command delimiter. Implementations are permitted, but not required, to support this extension.

ajosey

2015-06-19 07:07

manager   bugnote:0002723

Interpretation Proposed: June 19 2015

ajosey

2015-09-07 11:33

manager   bugnote:0002819

Interpretation approved: 7 Sep 2015

Issue History

Date Modified Username Field Change
2015-05-04 21:36 stephane New Issue
2015-05-04 21:36 stephane Name => Stephane Chazelas
2015-05-04 21:36 stephane Section => sed editing commands
2015-05-04 21:36 stephane Page Number => http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html#tag_20_116_13_03
2015-05-04 21:36 stephane Line Number => http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html#tag_20_116_13_03
2015-05-05 09:14 geoffclare Note Added: 0002648
2015-05-05 09:48 user235 Note Added: 0002649
2015-05-05 10:14 stephane Note Added: 0002650
2015-06-18 16:04 eblake Note Added: 0002719
2015-06-18 16:15 eblake Relationship added related to 0000262
2015-06-18 16:21 geoffclare Note Added: 0002720
2015-06-18 16:22 rhansen Relationship added related to 0000961
2015-06-18 16:23 geoffclare Page Number http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html#tag_20_116_13_03 => 3182
2015-06-18 16:23 geoffclare Line Number http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html#tag_20_116_13_03 => 106416
2015-06-18 16:23 geoffclare Interp Status => ---
2015-06-18 16:23 geoffclare Final Accepted Text => 0000944:0002720
2015-06-18 16:23 geoffclare Status New => Resolved
2015-06-18 16:23 geoffclare Resolution Open => Accepted As Marked
2015-06-18 16:23 geoffclare Tag Attached: issue8
2015-06-18 16:29 geoffclare Tag Detached: issue8
2015-06-18 16:30 geoffclare Tag Attached: tc2-2008
2015-06-18 16:36 eblake Relationship added related to 0000269
2015-06-18 16:40 geoffclare Note Edited: 0002720
2015-06-18 16:42 geoffclare Interp Status --- => Pending
2015-06-18 16:42 geoffclare Status Resolved => Interpretation Required
2015-06-19 07:07 ajosey Interp Status Pending => Proposed
2015-06-19 07:07 ajosey Note Added: 0002723
2015-09-07 11:33 ajosey Interp Status Proposed => Approved
2015-09-07 11:33 ajosey Note Added: 0002819
2019-06-10 08:54 agadmin Status Interpretation Required => Closed