Austin Group Defect Tracker

Aardvark Mark IV

Main | My View | View Issues | Change Log | Docs
  

Viewing Issue Simple Details [ Jump to Notes ] [ Issue History ] [ Print ]
ID Category Severity Type Date Submitted Last Update
0001786 [Issue 8 drafts] Shell and Utilities Objection Clarification Requested 2023-11-02 15:13 2023-12-07 15:07
Reporter eblake View Status public  
Assigned To
Priority normal Resolution Accepted As Marked  
Status Applied   Product Version Draft 3
Name Eric Blake
Organization Red Hat
User Reference ebb.ed
Section XCU ed
Page Number 2801
Line Number 92899
Final Accepted Text Note: 0006577
Summary 0001786: ed behavior on non-existing filename
Description This bug was created as fallout from investigating 0000251, investigating whether ed can create pathnames containing a newline.

It turns out that the standard is silent on what happens when the e and f commands are used with a filename (whether explicit or implicit) that does not yet exist. Existing implementations differ on whether e can create a file:

GNU ed:

$ rm -f hi
$ echo q | ed hi
hi: No such file or directory
$ echo $?
0
$ echo f | ed hi
hi: No such file or directory
hi
$ echo $?
0
$ ls hi
ls: cannot access 'hi': No such file or directory
$ echo w | ed hi
hi: No such file or directory
0
$ ls hi
hi


whereas on FreeBSD 13:

# rm -f hi
# echo q | ed hi
hi: No such file or directory
# echo $?
2
# echo f | ed hi
hi: No such file or directory
# echo $?
2
# echo w | ed hi
hi: No such file or directory
# ls hi
ls: hi: No such file or directory


However, all implementations I tested appear to allow f and w allow a filename argument that does not yet exist, at which point w creates that file (which matches the requirement for w at line 93110):



$ rm -f hi
$ printf 'f hi\nw\n' | ed
hi
0
$ ls hi
hi

Back in the context of 0000251, where we stated that the file command line argument to ed is treated the same as an explicit argument to the e command, other than the possibility of including a newline in the name, whether or not you can use ed to create a filename with a newline then depends on the behavior of whether a non-existing pathname is permitted with the e command. However, the proposed wording here is intended to apply as-is regardless of whether we also include the changes in 0000251.
Desired Action In XCU ed EXTENDED DESCRIPTION, page 2801 line 92901, in the Edit Command text, after the sentence:
If no pathname is given, the currently remembered pathname, if any, shall be used (see the f command).
insert another sentence
If the pathname names a file that does not exist, it is unspecified whether this is treated as an error, or whether a warning is emitted in place of a byte count and the resulting buffer is left empty.


On page 2802 line 92918, in the Filename Command text, change:
If file is given, the f command shall change the currently remembered pathname to file;
to:
If file is given, the f command shall change the currently remembered pathname to file, whether or not file names an existing file;
Tags applied_after_i8d3, issue8
Attached Files

- Relationships
related to 0000251Appliedajosey 1003.1(2008)/Issue 7 Forbid newline, or even bytes 1 through 31 (inclusive), in filenames 

-  Notes
(0006567)
geoffclare (manager)
2023-11-13 17:31

 On page 2797 line 92714 section ed ASYNCHRONOUS EVENTS, change:
If the buffer is not empty and has changed since the last write, the
to:
If the buffer is not empty and the buffer change flag is set (see the EXTENDED DESCRIPTION section), the


On page 2797 line 92725 section ed STDERR, change:
The standard error shall be used only for diagnostic messages.
to:
The standard error shall be used for diagnostic messages and may be used for warning messages.


On page 2797 line 92730 section ed EXTENDED DESCRIPTION, after:
The ed utility shall operate on a copy of the file it is editing; changes made to the copy shall have no effect on the file until a w (write) command is given. The copy of the text is called the buffer.
add these sentences:
The ed utility shall keep track of whether the buffer has been modified. This shall be maintained as if via an internal flag which is set by any command that modifies the buffer and is cleared by an e or E command that reloads (or empties) the buffer, or a w command that writes the entire buffer. A command that makes changes to the buffer in such a way that its contents are the same after the command (for example s/a/a/) shall be considered to have modified the buffer, unless explicitly stated otherwise. In the remainder of the description, this flag is referred to as the buffer change flag.


On page 2800 line 92839 section ed EXTENDED DESCRIPTION, change:
If changes have been made in the buffer since the last w command that wrote the entire buffer
to:
If the buffer change flag is set


On page 2800 line 92845, change:
If the e or q command is repeated with no intervening command, it shall take effect.
to:
If another e or q command is then attempted with no intervening command that sets the buffer change flag, it shall take effect.


On page 2801 line 92881, in the Append Command text, add a sentence:
If <text> is empty (that is, the terminating '.' immediately follows the 'a'), the buffer change flag shall not be set.


On page 2801 line 92901, in the Edit Command text, after the sentence:
If no pathname is given, the currently remembered pathname, if any, shall be used (see the f command).
insert another sentence:
If the pathname names a file that does not exist and the buffer change flag is clear, it is unspecified whether this is treated as an error, or whether the resulting buffer is emptied and a warning is written to standard error instead of writing the byte count to standard out.


On page 2801 line 92909, in the Edit Command text, change:
If the buffer has changed since the last time the entire buffer was written
to:
If the buffer change flag is set


On page 2802 line 92914, in the Edit Without Checking Command text, change:
shall not check to see whether any changes have been made to the buffer since the last w command.
to:
shall not check to see whether the buffer change flag is set.


On page 2802 line 92918, in the Filename Command text, change:
If file is given, the f command shall change the currently remembered pathname to file;
to:
If file is given, the f command shall change the currently remembered pathname to file, whether or not file names an existing file;


On page 2803 line 92977, in the Insert Command text, add a sentence:
If <text> is empty (that is, the terminating '.' immediately follows the 'i'), the buffer change flag shall not be set.


On page 2805 Read Command, add a new paragraph at the end (after line 93044):
If the number of bytes read is 0 it is unspecified if the buffer change flag is set.


On page 2805 line 93027, in the Quit Command text, change:
If the buffer has changed since the last time the entire buffer was written
to:
If the buffer change flag is set


On page 2805 line 93031, in the Quit Without Checking Command text, change:
without checking whether changes have been made in the buffer since the last w command
to:
without checking whether the buffer change flag is set


On page 2807 line 93120, in the Write Command text, change:
This usage of the write command with '!' shall not be considered as a ``last w command that wrote the entire buffer'', as described previously; thus, this alone ...
to:
This usage of the w command with '!' shall not clear the buffer change flag; thus, this alone ...
(0006570)
Antonio Diaz (reporter)
2023-11-14 16:00

 About the change proposed in note 6567 for page 2800 line 92845:

"If another e or q command is then attempted with no intervening command that sets the buffer change flag, it shall take effect."

Given that the buffer change flag is already set, it is not easy for ed to tell whether an intervening command would have set the buffer change flag. For example, 'a\n.\n' would not set the buffer change flag, but should make e or q repeat the warning.

Therefore I propose a wording like the following:

"If another e or q command is then attempted with no intervening command that can set the buffer change flag, it shall take effect."
(0006571)
eblake (manager)
2023-11-16 16:40
edited on: 2023-11-16 16:55

 Based on Note: 0006570, we did some more investigation into existing implementations. It appears that most implementations clear the buffer change flag on an 'e' or 'q' that warned about a modified buffer, as in the following behavior:
$ ed
a
text
.
q
?
a
.
q
$

where GNU ed was the outlier.

The following modifications to the changes suggested at lines 92730 and 92845 in Note: 0006567 would capture that:

On page 2797 line 92730 section ed EXTENDED DESCRIPTION, after:
    
The ed utility shall operate on a copy of the file it is editing; changes made to the copy shall have no effect on the file until a w (write) command is given. The copy of the text is called the buffer.

add these sentences:
    
The ed utility shall keep track of whether the buffer has been modified. This shall be maintained as if via an internal flag which is set by any command that modifies the buffer and is cleared by a w command that writes the entire buffer, or by an e, E, or q command. A command that makes changes to the buffer in such a way that its contents are the same after the command (for example s/a/a/) shall be considered to have modified the buffer, unless explicitly stated otherwise. In the remainder of the description, this flag is referred to as the buffer change flag.



On page 2800 line 92844, change:
    
... and shall continue in command mode with the current line number unchanged. If the e or q command is repeated with no intervening command, it shall take effect.

to
    
... and shall continue in command mode with the buffer change flag cleared and the current line number unchanged. If another e or q command is then attempted with no intervening command that sets the buffer change flag, it shall take effect.


For now, we've reopened the bug to allow more feedback
(0006572)
Antonio Diaz (reporter)
2023-11-16 17:55

 Clearing the buffer change flag on an 'e' or 'q' that warns about a modified buffer has the undesirable side effect that ed will not save the modified buffer to ed.hup if SIGHUP is received. I very much would prefer not to implement such unsafe behavior in GNU ed.
(0006574)
geoffclare (manager)
2023-11-20 09:33

 Re Note: 0006572 That is what happens on Solaris, but I see that it doesn't on macOS and I agree it is undesirable. I think we should allow both behaviours in Issue 8 but recommend the GNU and macOS one (either in rationale or by using "should").
(0006577)
geoffclare (manager)
2023-11-20 17:21

 On page 2797 line 92714 section ed ASYNCHRONOUS EVENTS, change:
If the buffer is not empty and has changed since the last write, the
to:
If the buffer is not empty and the buffer change flag is currently set to either changed or changed-and-warned (see the EXTENDED DESCRIPTION section), the


On page 2797 line 92725 section ed STDERR, change:
The standard error shall be used only for diagnostic messages.
to:
The standard error shall be used for diagnostic messages and may be used for warning messages.


On page 2797 line 92730 section ed EXTENDED DESCRIPTION, after:
The ed utility shall operate on a copy of the file it is editing; changes made to the copy shall have no effect on the file until a w (write) command is given. The copy of the text is called the buffer.
add these sentences:
The ed utility shall keep track of whether the buffer has been modified. This shall be maintained as if via a tri-state internal flag with the state values unchanged, changed, and changed-and-warned, which is:

  • Initially set to unchanged

  • Set to changed by any command that modifies the buffer

  • Set to unchanged by an e or E command that reloads (or empties) the buffer, or a w command that writes the entire buffer

  • Set to either changed-and-warned or unchanged by an e or q command that warns an attempt was made to destroy the editor buffer

A command that makes changes to the buffer in such a way that its contents are the same after the command (for example s/a/a/) shall be considered to have modified the buffer, unless explicitly stated otherwise. In the remainder of the description, this flag is referred to as the buffer change flag.


On page 2800 line 92839 section ed EXTENDED DESCRIPTION, change:
If changes have been made in the buffer since the last w command that wrote the entire buffer
to:
If the buffer change flag is currently set to changed


On page 2800 line 92844, change:
... and shall continue in command mode with the current line number unchanged. If the e or q command is repeated with no intervening command, it shall take effect.
to:
... and shall continue in command mode with the buffer change flag set to either changed-and-warned or unchanged and the current line number unchanged. If another e or q command is then attempted with no intervening command that sets the buffer change flag to changed, it shall take effect.


On page 2801 line 92881, in the Append Command text, add a sentence:
If <text> is empty (that is, the terminating '.' immediately follows the 'a'), the buffer change flag shall not be altered.


On page 2801 line 92901, in the Edit Command text, after the sentence:
If no pathname is given, the currently remembered pathname, if any, shall be used (see the f command).
insert another sentence:
If the pathname names a file that does not exist and the buffer change flag is currently set to unchanged, it is unspecified whether this is treated as an error, or whether the resulting buffer is emptied and a warning is written to standard error instead of writing the byte count to standard out.


On page 2801 line 92909, in the Edit Command text, change:
If the buffer has changed since the last time the entire buffer was written
to:
If the buffer change flag is currently set to changed


On page 2802 line 92914, in the Edit Without Checking Command text, change:
shall not check to see whether any changes have been made to the buffer since the last w command.
to:
shall not check the current state of the buffer change flag.


On page 2802 line 92918, in the Filename Command text, change:
If file is given, the f command shall change the currently remembered pathname to file;
to:
If file is given, the f command shall change the currently remembered pathname to file, whether or not file names an existing file;


On page 2803 line 92977, in the Insert Command text, add a sentence:
If <text> is empty (that is, the terminating '.' immediately follows the 'i'), the buffer change flag shall not be altered.


On page 2805 Read Command, add a new paragraph at the end (after line 93044):
If the number of bytes read is 0 it is unspecified whether the buffer change flag is set to changed or left unaltered.


On page 2805 line 93027, in the Quit Command text, change:
If the buffer has changed since the last time the entire buffer was written
to:
If the buffer change flag is currently set to changed


On page 2805 line 93031, in the Quit Without Checking Command text, change:
without checking whether changes have been made in the buffer since the last w command
to:
without checking the current state of the buffer change flag


On page 2807 line 93120, in the Write Command text, change:
This usage of the write command with '!' shall not be considered as a ``last w command that wrote the entire buffer'', as described previously; thus, this alone ...
to:
This usage of the w command with '!' shall not alter the buffer change flag; thus, this alone ...


After page 2811 line 93292 section ed RATIONALE, add a paragraph:
Implementations are encouraged to set the buffer change flag to changed-and-warned when an e or q command warns that an attempt was made to destroy the editor buffer. Some existing implementations set it to unchanged, but this has the undesirable side-effect that a SIGHUP received after the warning is given does not write the buffer to ed.hup.


On page 2811 line 93294, change FUTURE DIRECTIONS from "None" to:
A future version of this standard may require that the buffer change flag is set to changed-and-warned when an e or q command warns that an attempt was made to destroy the editor buffer.

- Issue History
Date Modified Username Field Change
2023-11-02 15:13 eblake New Issue
2023-11-02 15:13 eblake Name => Eric Blake
2023-11-02 15:13 eblake Organization => Red Hat
2023-11-02 15:13 eblake User Reference => ebb.ed
2023-11-02 15:13 eblake Section => XCU ed
2023-11-02 15:13 eblake Page Number => 2801
2023-11-02 15:13 eblake Line Number => 92899
2023-11-02 15:16 eblake Relationship added related to 0000251
2023-11-02 15:18 eblake Description Updated
2023-11-02 15:18 eblake Desired Action Updated
2023-11-13 17:31 geoffclare Note Added: 0006567
2023-11-13 17:32 geoffclare Final Accepted Text => Note: 0006567
2023-11-13 17:32 geoffclare Status New => Resolved
2023-11-13 17:32 geoffclare Resolution Open => Accepted As Marked
2023-11-13 17:32 geoffclare Tag Attached: issue8
2023-11-14 16:00 Antonio Diaz Note Added: 0006570
2023-11-16 16:37 eblake Final Accepted Text Note: 0006567 =>
2023-11-16 16:37 eblake Resolution Accepted As Marked => Reopened
2023-11-16 16:40 eblake Note Added: 0006571
2023-11-16 16:44 eblake Note Edited: 0006571
2023-11-16 16:55 eblake Note Edited: 0006571
2023-11-16 17:55 Antonio Diaz Note Added: 0006572
2023-11-20 09:33 geoffclare Note Added: 0006574
2023-11-20 17:21 geoffclare Note Added: 0006577
2023-11-20 17:22 geoffclare Final Accepted Text => Note: 0006577
2023-11-20 17:22 geoffclare Resolution Reopened => Accepted As Marked
2023-12-07 15:07 geoffclare Status Resolved => Applied
2023-12-07 15:07 geoffclare Tag Attached: applied_after_i8d3

Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker