View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0001913 | 1003.1(2024)/Issue8 | Shell and Utilities | public | 2025-03-12 03:33 | 2025-03-20 14:49 |
Reporter | calestyo | Assigned To | |||
Priority | normal | Severity | Editorial | Type | Enhancement Request |
Status | New | Resolution | Open | ||
Name | Christoph Anton Mitterer | ||||
Organization | |||||
User Reference | Shell & Utilities | ||||
Section | 2.7.5, 2.7.6 | ||||
Page Number | 2497 | ||||
Line Number | 81097-81118 | ||||
Interp Status | |||||
Final Accepted Text | |||||
Summary | 0001913: clarify/define the meaning of n<&n and m>&m redirections | ||||
Description | Hey. This originated from a thread that started at https://collaboration.opengroup.org/operational/mailarch.php?soph=N&action=show&archive=austin-group-l&num=38114&limit=100&offset=100&sid= In short: The motivation was whether it's possible to portably differentiate whether a non-zero exit status originated from a utility or failed redirection on that. The idea was that, with the clarifications from #1879 and assuming that there is at least on exit status which a utility is known not to use, a construct like the following: ( command exec <some redirections> || exit 125; utility ) where the subshell is merely to clean up any redirections, can be used to differentiate between a redirection error (which above would be indicated by 125) or a non-zero exit status from the utility. For file descriptors less than or equal to 2, this should already be guaranteed to work portably, as POSIX demands those to be passed on to the utility. For FDs greater than two, this is no longer guaranteed, though. An idea was brought up by Harald van Dijk, that n<&n and m>&m could be used e.g.: ( command exec <some redirections> || exit 125; utility n<&n... m>&m... ) in order to "manually" pass on on any such FDs. Questions remained: a) Does that behaviour even follow from POSIX (in an obvious way where it's really clear that shells should behave like this, not just some wobbly way) b) Does it even solve the original problem, or could e.g. such a n<&n respectively m>&m fail itself (not e.g. because a file doesn't exist, but because of something like resource exhaustion, etc.) When the original thread was continued a bit later at: https://collaboration.opengroup.org/operational/mailarch.php?soph=N&action=show&archive=austin-group-l&num=38279&limit=100&offset=0&sid= it apparently turned out that n<&n and m>&m redirections are not defined because POSIX uses the wordings: "The redirection operator: [n]<&word shall duplicate one input file descriptor from another" respectively "The redirection operator: [n]>&word shall duplicate one output file descriptor from another" i.e. one from another, implying the two must not be the same, as Geoff Clare pointed out. | ||||
Desired Action | 1. It shall be clarified whether or not the n<&n and m>&m forms are covered by the current wording of the standard. (One could perhaps also interpret the "one from another" as the "one" being the FD from the utility's PoV, and the "another" being the FD of the same number from the shell's PoV.) 2. If possible - i.e. no conflicting behaviour of shell implementations - it would be nice if the definition could be change in such a way, that it actually allows portably for the goals described above. I have no strong opinion one how this definition should look like, one suggestion on the list was that implementations that the n<&n and m>&m forms shalll be no-ops for implementations which *do* pass on FDs > 2, and for those that don't, it shall have the meaning of explicitly passing the given FD on. It would be even nicer, if the standard mention in a sentence that this is explicitly meant to be usable for the purpose of differentiating between utility and redirection errors. Thanks, Chris. | ||||
Tags | No tags attached. |
|
Desired Action:It would be even nicer, if the standard mention in a sentence that this is explicitly meant to be usable for the purpose of differentiating between utility and redirection errors.The standard does not exist for your personal benefit, so I do not think it should bless your highly specific use case. However, it would make sense to briefly mention the closing-high-FDs behavior as motivation. Something along the lines of Geoff's wording in the thread would be more than sufficient: So I would support updating the standard to require that n<&n and n>&n are always a no-op if fd n is open, except that if the shell normally closes fds > 2, that were opened with exec, when it executes a non-built-in utility, then applying n<&n or n>&n to such commands causes fd n to remain open. |
|
The standard does not exist for your personal benefit, so I do not think it should bless your highly specific use case. I don't think I was asking to change it to "my personal benefit" (actually my own use case doesn't need FDs > 2, so I'm already happy with that.).
What I at least would want to avoid is that people might ever come across this issue or the corresponding mailing list thread and assume that this is now the way to portably differentiate between utility non-zero exit status and redirection error, when wouldn't be really the case. If you don't think it is, fine for me,... we can still make the change here, but should also mention that this cannot be expected to portably allow the above. If you think it is and if you think that "briefly mentioning the closing-high-FDs behavior as motivation" is enough to also make sure that this is understood by any shell implementer, then I'm all good. My idea to mention the deeper purpose was merely for the case that would be needed. |
|
In the Mar 13, 2025 teleconference the following wording was agreed, but the bug is being left open for now for feedback. On page 2497 line 81097-81118 replace sections 2.7.5 and 2.7.6 with: 2.7.5 Duplicating a File Descriptor (Added after the meeting) A change to merge XRAT C.2.7.5 and C.2.7.6 will also be needed, plus a change to lines 135455-135459 in C.2.9.3. |
|
The only two things that came to my mind: 1) Over-pedantic and probably not really needed to take care of: This text is now for both <& and >& ... and the wording uses "or standard input or standard output" as well as "or if n is not specified and word evaluates to 0 or 1",... so what about things like <1 or >0 ? 2) As far as I understand the wording about the n = word case, wouldn't that also mean that e.g.: exec 8<some file; exec 8<8; utility would now be expected to keep FD 8 open when invoking utility, and not just a construct like: exec 8<some file; utility 8<8 ? |
|
0001913:0007117:so what about things like <1 or >0 ?The former is covered by section 2.7.1 and redirects stdin from a file literally named "1"; the latter is covered by section 2.7.2 and redirects stdout to a file literally named "0". This report isn't about those forms of redirection. As far as I understand the wording about the n = word case, wouldn't that also mean that e.g.:I don't see how, assuming you actually mean exec 8<somefile; exec 8<&8; utilityThe proposed wording says (emphasis mine): if the shell would have closed the file descriptor because it was opened using exec and has a value greater than 2, when the redirection is being performed in a command that will execute a non-built-in utility, the file descriptor shall instead remain open when the utility is executedYour second exec command isn't executing a non-built-in utility. |
|
0001913:0007112:What I at least would want to avoid is that people might ever come across this issue or the corresponding mailing list thread and assume that this is now the way to portably differentiate between utility non-zero exit status and redirection error, when wouldn't be really the case.Why not? Is it because the redirections can't be absolutely guaranteed to succeed, as Geoff mentioned on the mailing list? If it is a no-op then it can't fail. So the only possible failure case would be in the "remain open" requirement. In practice this will involve calling fcntl() to clear the FD_CLOEXEC flag, which could indeed fail because of something like resource exhaustion, but I don't see that it increases the likelihood of internal shell failure significantly. Any command execution can fail within the shell because of resource exhaustion (e.g. fork() failure) before it gets as far as doing the exec.https://www.mail-archive.com/austin-group-l@opengroup.org/msg13640.html If you don't think it is, fine for me,... we can still make the change here, but should also mention that this cannot be expected to portably allow the above.I don't think the standard should mention your use case at all. If you think it is and if you think that "briefly mentioning the closing-high-FDs behavior as motivation" is enough to also make sure that this is understood by any shell implementer, then I'm all good.Alright. The readers I had in mind were application developers who might otherwise assume that n<&n serves no purpose whatsoever. |
|
> This text is now for both <& and >& ... and the wording uses "or standard input or standard output" as well as "or if n is not specified and word evaluates to 0 or 1", The proposed text is careful to use "respectively" where necessary. > so what about things like <1 or >0 ? I assume this was meant to be "like <&1 or >&0", in which case the answer is "a redirection error may result" (assuming that 0 is open readonly and 1 is open writeonly). |
Date Modified | Username | Field | Change |
---|---|---|---|
2025-03-12 03:33 | calestyo | New Issue | |
2025-03-12 07:00 | larryv | Note Added: 0007111 | |
2025-03-13 02:41 | calestyo | Note Added: 0007112 | |
2025-03-13 16:12 | geoffclare | Note Added: 0007115 | |
2025-03-13 17:48 | calestyo | Note Added: 0007117 | |
2025-03-13 20:20 | larryv | Note Added: 0007119 | |
2025-03-13 20:43 | larryv | Note Added: 0007120 | |
2025-03-14 09:44 | geoffclare | Note Edited: 0007115 | |
2025-03-18 12:30 | geoffclare | Note Added: 0007125 | |
2025-03-20 14:49 | geoffclare | Note Edited: 0007115 |