View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0001913 | 1003.1(2024)/Issue8 | Shell and Utilities | public | 2025-03-12 03:33 | 2025-09-25 11:34 |
Reporter | calestyo | Assigned To | |||
Priority | normal | Severity | Editorial | Type | Enhancement Request |
Status | New | Resolution | Open | ||
Name | Christoph Anton Mitterer | ||||
Organization | |||||
User Reference | Shell & Utilities | ||||
Section | 2.7.5, 2.7.6 | ||||
Page Number | 2497 | ||||
Line Number | 81097-81118 | ||||
Interp Status | |||||
Final Accepted Text | |||||
Summary | 0001913: clarify/define the meaning of n<&n and m>&m redirections | ||||
Description | Hey. This originated from a thread that started at https://collaboration.opengroup.org/operational/mailarch.php?soph=N&action=show&archive=austin-group-l&num=38114&limit=100&offset=100&sid= In short: The motivation was whether it's possible to portably differentiate whether a non-zero exit status originated from a utility or failed redirection on that. The idea was that, with the clarifications from #1879 and assuming that there is at least on exit status which a utility is known not to use, a construct like the following: ( command exec <some redirections> || exit 125; utility ) where the subshell is merely to clean up any redirections, can be used to differentiate between a redirection error (which above would be indicated by 125) or a non-zero exit status from the utility. For file descriptors less than or equal to 2, this should already be guaranteed to work portably, as POSIX demands those to be passed on to the utility. For FDs greater than two, this is no longer guaranteed, though. An idea was brought up by Harald van Dijk, that n<&n and m>&m could be used e.g.: ( command exec <some redirections> || exit 125; utility n<&n... m>&m... ) in order to "manually" pass on on any such FDs. Questions remained: a) Does that behaviour even follow from POSIX (in an obvious way where it's really clear that shells should behave like this, not just some wobbly way) b) Does it even solve the original problem, or could e.g. such a n<&n respectively m>&m fail itself (not e.g. because a file doesn't exist, but because of something like resource exhaustion, etc.) When the original thread was continued a bit later at: https://collaboration.opengroup.org/operational/mailarch.php?soph=N&action=show&archive=austin-group-l&num=38279&limit=100&offset=0&sid= it apparently turned out that n<&n and m>&m redirections are not defined because POSIX uses the wordings: "The redirection operator: [n]<&word shall duplicate one input file descriptor from another" respectively "The redirection operator: [n]>&word shall duplicate one output file descriptor from another" i.e. one from another, implying the two must not be the same, as Geoff Clare pointed out. | ||||
Desired Action | 1. It shall be clarified whether or not the n<&n and m>&m forms are covered by the current wording of the standard. (One could perhaps also interpret the "one from another" as the "one" being the FD from the utility's PoV, and the "another" being the FD of the same number from the shell's PoV.) 2. If possible - i.e. no conflicting behaviour of shell implementations - it would be nice if the definition could be change in such a way, that it actually allows portably for the goals described above. I have no strong opinion one how this definition should look like, one suggestion on the list was that implementations that the n<&n and m>&m forms shalll be no-ops for implementations which *do* pass on FDs > 2, and for those that don't, it shall have the meaning of explicitly passing the given FD on. It would be even nicer, if the standard mention in a sentence that this is explicitly meant to be usable for the purpose of differentiating between utility and redirection errors. Thanks, Chris. | ||||
Tags | No tags attached. |
|
Desired Action:It would be even nicer, if the standard mention in a sentence that this is explicitly meant to be usable for the purpose of differentiating between utility and redirection errors.The standard does not exist for your personal benefit, so I do not think it should bless your highly specific use case. However, it would make sense to briefly mention the closing-high-FDs behavior as motivation. Something along the lines of Geoff's wording in the thread would be more than sufficient: So I would support updating the standard to require that n<&n and n>&n are always a no-op if fd n is open, except that if the shell normally closes fds > 2, that were opened with exec, when it executes a non-built-in utility, then applying n<&n or n>&n to such commands causes fd n to remain open. |
|
The standard does not exist for your personal benefit, so I do not think it should bless your highly specific use case. I don't think I was asking to change it to "my personal benefit" (actually my own use case doesn't need FDs > 2, so I'm already happy with that.).
What I at least would want to avoid is that people might ever come across this issue or the corresponding mailing list thread and assume that this is now the way to portably differentiate between utility non-zero exit status and redirection error, when wouldn't be really the case. If you don't think it is, fine for me,... we can still make the change here, but should also mention that this cannot be expected to portably allow the above. If you think it is and if you think that "briefly mentioning the closing-high-FDs behavior as motivation" is enough to also make sure that this is understood by any shell implementer, then I'm all good. My idea to mention the deeper purpose was merely for the case that would be needed. |
|
In the Mar 13, 2025 teleconference the following wording was agreed, but the bug is being left open for now for feedback. On page 2497 line 81097-81118 replace sections 2.7.5 and 2.7.6 with: 2.7.5 Duplicating a File Descriptor (Added after the meeting) A change to merge XRAT C.2.7.5 and C.2.7.6 will also be needed, plus a change to lines 135455-135459 in C.2.9.3. |
|
The only two things that came to my mind: 1) Over-pedantic and probably not really needed to take care of: This text is now for both <& and >& ... and the wording uses "or standard input or standard output" as well as "or if n is not specified and word evaluates to 0 or 1",... so what about things like <1 or >0 ? 2) As far as I understand the wording about the n = word case, wouldn't that also mean that e.g.: exec 8<some file; exec 8<8; utility would now be expected to keep FD 8 open when invoking utility, and not just a construct like: exec 8<some file; utility 8<8 ? |
|
0001913:0007117:so what about things like <1 or >0 ?The former is covered by section 2.7.1 and redirects stdin from a file literally named "1"; the latter is covered by section 2.7.2 and redirects stdout to a file literally named "0". This report isn't about those forms of redirection. As far as I understand the wording about the n = word case, wouldn't that also mean that e.g.:I don't see how, assuming you actually mean exec 8<somefile; exec 8<&8; utilityThe proposed wording says (emphasis mine): if the shell would have closed the file descriptor because it was opened using exec and has a value greater than 2, when the redirection is being performed in a command that will execute a non-built-in utility, the file descriptor shall instead remain open when the utility is executedYour second exec command isn't executing a non-built-in utility. |
|
0001913:0007112:What I at least would want to avoid is that people might ever come across this issue or the corresponding mailing list thread and assume that this is now the way to portably differentiate between utility non-zero exit status and redirection error, when wouldn't be really the case.Why not? Is it because the redirections can't be absolutely guaranteed to succeed, as Geoff mentioned on the mailing list? If it is a no-op then it can't fail. So the only possible failure case would be in the "remain open" requirement. In practice this will involve calling fcntl() to clear the FD_CLOEXEC flag, which could indeed fail because of something like resource exhaustion, but I don't see that it increases the likelihood of internal shell failure significantly. Any command execution can fail within the shell because of resource exhaustion (e.g. fork() failure) before it gets as far as doing the exec.https://www.mail-archive.com/austin-group-l@opengroup.org/msg13640.html If you don't think it is, fine for me,... we can still make the change here, but should also mention that this cannot be expected to portably allow the above.I don't think the standard should mention your use case at all. If you think it is and if you think that "briefly mentioning the closing-high-FDs behavior as motivation" is enough to also make sure that this is understood by any shell implementer, then I'm all good.Alright. The readers I had in mind were application developers who might otherwise assume that n<&n serves no purpose whatsoever. |
|
> This text is now for both <& and >& ... and the wording uses "or standard input or standard output" as well as "or if n is not specified and word evaluates to 0 or 1", The proposed text is careful to use "respectively" where necessary. > so what about things like <1 or >0 ? I assume this was meant to be "like <&1 or >&0", in which case the answer is "a redirection error may result" (assuming that 0 is open readonly and 1 is open writeonly). |
|
Suggested Interpretation response ------------------------ The standard is unclear on this issue, and no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor. Rationale: ------------- As the submitter points out, the wording in the standard "one input file descriptor from another" implies that they are two different file descriptors, and so the behavior of n<&n is currently unspecified (implicitly). Likewise for m>&m. There is an example in XRAT C.2.9.3 Lists (under "Asynchronous AND-OR Lists") which shows that the intention is for <&0 (and therefore also 0<&0) to be a no-op, but since this is non-normative it does not affect the requirements of the standard. Notes to the Editor (not part of this interpretation): ------------------------------------------------------- On page 2497 line 81097-81118 replace sections 2.7.5 and 2.7.6 with: 2.7.5 Duplicating a File Descriptorand renumber 2.7.7 to 2.7.6.The redirection operators:[n]<&wordand:[n]>&wordshall duplicate one input file descriptor or output file descriptor, respectively, from another, or shall close one. On page 2506 line 81501 section 2.9.3.1 Asynchronous AND-OR Lists, change: If, and only if, job control is disabled, the standard input for the subshell in which an asynchronous AND-OR list is executed shall initially be assigned to an open file description that behaves as if /dev/null had been opened for reading only. This initial assignment shall be overridden by any explicit redirection of standard input within the AND-OR list.to: If, and only if, job control is disabled, the standard input for the subshell in which an asynchronous AND-OR list is executed shall be assigned to an open file description that behaves as if /dev/null had been opened for reading only, except that: On page 3893 line 135146 section C.2.7.5, change the section heading: Duplicating an Input File Descriptorto: Duplicating a File Descriptor After page 3893 line 135152 section C.2.7.5, add a paragraph: If word and n evaluate to the same open file descriptor, the operation is a no-op except in shells which set the close-on-exec flag for file descriptors greater than 2 opened using [xref to exec]. In these shells, a redirection of this form can be used to clear the close-on-exec flag so that the file descriptor will remain open when executing a non-built-in utility. For example:exec 3<infile 4>outfile utility 3<&3 4>&4One use for this feature, together with command and exec, is to differentiate between utility and redirection errors. For example:( command exec 3<infile 4>outfile || exit 128; utility 3<&3 4>&4 ) case $? in 0) # success ;; 128) # redirection error ... ;; *) # utility error ... ;; esac(This assumes utility is known not to use 128 as an exit status and that the shell does not detect an internal error such as resource exhaustion.) On page 3893 line 135155-135158, delete section C.2.7.6 Duplicating an Output File Descriptor On page 3894 line 135159, change section number C.2.7.7 to C.2.7.6 On page 3901 line 135455 section C.2.9.3 Lists, change: Since the connection of the input to the equivalent of /dev/null is considered to occur before redirections, the following script would produce no output:to:exec < /etc/passwd cat <&0 & wait The assignment of standard input from an open file description that behaves like /dev/null is not overridden by an explicit <tt><&0</tt> redirection because this redirection does not perform a duplication and thus has no effect on where standard input comes from. This was the original Korn Shell behavior but was not clearly required by versions of this standard earlier than Issue 8 TC1, although in all those versions there was rationale stating that the following script would produce no output:exec < /etc/passwd cat <&0 & wait |
Date Modified | Username | Field | Change |
---|---|---|---|
2025-03-12 03:33 | calestyo | New Issue | |
2025-03-12 07:00 | larryv | Note Added: 0007111 | |
2025-03-13 02:41 | calestyo | Note Added: 0007112 | |
2025-03-13 16:12 | geoffclare | Note Added: 0007115 | |
2025-03-13 17:48 | calestyo | Note Added: 0007117 | |
2025-03-13 20:20 | larryv | Note Added: 0007119 | |
2025-03-13 20:43 | larryv | Note Added: 0007120 | |
2025-03-14 09:44 | geoffclare | Note Edited: 0007115 | |
2025-03-18 12:30 | geoffclare | Note Added: 0007125 | |
2025-03-20 14:49 | geoffclare | Note Edited: 0007115 | |
2025-09-25 11:34 | geoffclare | Note Added: 0007277 |