 Relationships |
|||||||||||||||||||
|
|||||||||||||||||||
| Anonymous | Login | 2024-04-16 19:37 UTC |
| Main | My View | View Issues | Change Log | Docs |
| Viewing Issue Simple Details [ Jump to Notes ] | [ Issue History ] [ Print ] | ||||||
| ID | Category | Severity | Type | Date Submitted | Last Update | ||
| 0000947 | [1003.1(2008)/Issue 7] Shell and Utilities | Objection | Enhancement Request | 2015-05-06 11:51 | 2019-10-21 09:32 | ||
| Reporter | joerg | View Status | public | ||||
| Assigned To | ajosey | ||||||
| Priority | normal | Resolution | Accepted As Marked | ||||
| Status | Applied | ||||||
| Name | Jörg Schilling | ||||||
| Organization | |||||||
| User Reference | |||||||
| Section | Shell command language | ||||||
| Page Number | 2324 | ||||||
| Line Number | 73758 | ||||||
| Interp Status | Approved | ||||||
| Final Accepted Text | See Note: 0003054. | ||||||
| Summary | 0000947: Shell should not have $? == 0 for exit(256) | ||||||
| Description |
Since SVr4 (1989) we have waitid() and therefore, the full 32 bit value if the exit() parameter is available but the shell still does not make use of this feature now available sinve 25 years. We should require the shell to use waitid() and si.si_status to retrieve the exit code from a child process. It may make sense to introduce a new macro "$status" that carries the full exit status of a child. |
||||||
| Desired Action |
Change line 73738 from: ? Expands to the decimal exit status of the most recent pipeline (see Section 2.9.2). to: ? Expands to the decimal exit status of the most recent pipeline (see Section 2.9.2). If the lower 8 bits of the exit() function have all been zero, it expands to 1. |
||||||
| Tags | tc3-2008 | ||||||
| Attached Files | |||||||
|
|
|||||||
|
Relationships |
|||||||||||||||||||
|
|||||||||||||||||||
|
Notes |
|
|
(0002657) stephane (reporter) 2015-05-07 09:29 |
Note that it depends on bug http://austingroupbugs.net/view.php?id=594 [^] awk's system() should probably be updated as well (at the moment, it's not clear what it returns. As worded it suggests it should return the same thing as system(3), but I don't think any implementation does, it doesn't say what awk's close() should return either (some return what pclose() returns, some a shell-like ($?) exit status) It may also be worth pointing out in the description of exit/_exit that applications should avoid using exit codes above 255 as they won't be available to the shell (and awk's system()/close()) and that system(3), pclose(3), wait(3), waitpid(3) will only get the lower 8 bits of it. I would also find worth mentioning that values 128 to 255 are used by most shell implementations to report the death-by-signal of its last child (the only exception I know (and that may be why POSIX doesn't clearly specify it) being recent versions of ksh93 that kill themselves with the same signal resulting in nasty side effects like spurious logs and extra core files and IMO is plain wrong, see also http://unix.stackexchange.com/a/99134). [^] So those values should be avoided except to convey the same meaning. Special values like the shell's (and other's) 127 for command-not-found may even be mentioned there. Also, at the moment, the only cases I've seen of applications calling exit() with a value greater than 255 is exit(-1)/return -1 (often not intentionally). I wouldn't be surprised if there are a number of shell scripts out there that do: cmd ret=$? if [ "$ret" -eq 255 ]; then echo "something horribly wrong happened to cmd" exit 1 fi And the proposed change would break those scripts. Maybe using 255 for any exit code above 255 instead of 1 would be better even if it overlaps with reporting the death of a child by signal 127. |
|
(0002658) joerg (reporter) 2015-05-07 09:47 edited on: 2015-06-18 09:37 |
My proposal was to introduce a migration path that tries to fix problems with the historic implementations. To upgrade to something that is usable for today, it would be a good idea to have additional macros: $status ----> si_status $code ----> si_code $pid ----> si_pid si_code is from sigonfo.h: #define CLD_EXITED 1 /* child has exited */ #define CLD_KILLED 2 /* child was killed */ #define CLD_DUMPED 3 /* child has coredumped */ #define CLD_TRAPPED 4 /* traced child has stopped */ #define CLD_STOPPED 5 /* child has stopped on signal */ #define CLD_CONTINUED 6 /* stopped child has continued */ If a child dies from a signal, $code would be == 2 (CLD_KILLED) and $status would hold the signal number, For a shell, we just would need to define special values to flag "file not found" and "permission denied" from exec(). In order to make $code useful in a shell, we would also need to standardize the exact values for CLD_* #defines. Given the fact that waitid() was introduced by Svr4, I recommend to use the above values taken from Solaris. It may also be a good idea to define a method to signal the error situations: file found but not executable that currently results in $? == 126 and executable not found that currently results in $? == 127 The new system could also be a solution for the current underspecified situation where POSIX requires just a value > 128 in $? when a command was terminated by a signal, that results in 128 + signal number on a Bourne Shell and the closed source POSIX Korn Shell that was derived from ksh88 (0x80 + signo) but in 0x100 + signo for ksh93. The new system would help because $code in such a case would be 2 (killed by signal) and $status would contain the signal number. Note: bash and zsh also deliver 0x80 + signo. |
|
(0002728) joerg (reporter) 2015-06-22 14:58 edited on: 2015-07-02 08:45 |
After working on a related implementation in the Bourne Shell, I have the following proposal: .sh.code The numerical reason waitid(2) returned for the child status change. It matches the CLD_* defini- tions from signal.h. Note that the numbers are usu- ally in the range 1..6 but this is not guaranteed. Use ${.sh.codename} for portability. .sh.codename The reason waitid(2) returned for the child status change as text that is generated by stripping off CLD_ from the related definitions from signal.h. Possible values are: EXITED The program had a normal termination and the exit(2) code is in ${.sh.status}. KILLED The program was killed by a signal, the signal number is in ${.sh.status} the signal name is in ${.sh.termsig}. DUMPED The program was killed by a signal, similar to KILLED above, but the program in addition created a core dump. TRAPPED A traced child has trapped. STOPPED The program was stopped by a signal, the signal number is in ${.sh.status} the signal name is in ${.sh.termsig}. CONTINUED A stopped child was continued. .sh.pid The process number of the process that caused the current waitid(2) status. .sh.signame The name of the causing signal. If the status is related to a set of waitid(2) return values, this is CHLD or CLD, depending on the os. When a trap com- mand is executed, ${.sh.signame} holds the signal that caused the trap. .sh.signo The signal number related to ${.sh.signame}. .sh.status The decimal value returned by the last synchronously executed command. The value is unaltered and con- tains the full int from the exit(2) call in the child in case the shell is run on a modern os. .sh.termsig The signal name related to the numerical ${.sh.status} value. The translation to signal names takes place regardless of whether the child was ter- minated by a signal or terminated normally. Note that trying to use the ${.sh.xxx} parameters on older shells will cause the older shells to exit with a bad sub- stitution message unless the shell is an interactive shell. .sh.signo/.sh.signame could be set while a trapcommand is executed. In this case, it would reflect the signal that triggered the shell trap handler. I further propose to set $? to the special value 128 in case that si_status has a value that would result in $? == 0 after truncation to 8 bits. Please comment this proposal. |
|
(0002731) joerg (reporter) 2015-06-24 11:53 edited on: 2015-07-07 10:20 |
Today, I published an updated Bourne Shell source at: http://sourceforge.net/projects/schilytools/files/schily-2015-07-07.tar.bz2 [^] Or the latest available tarball at: http://sourceforge.net/projects/schilytools/files/ [^] See also: http://sourceforge.net/projects/schilytools/files/AN-2015-07-07 [^] of the latest announcement file in the related tarball. The included Bourne Shell source now implements full support for the siginfo status. This includes exec()d programs, builtins and setting .sh.signo and .sh.signame to the causing signal while a trap(1) command is executed. I would be interested to get feedback on which platforms implement a usable waidid() implementation. Solaris works FreeBSD masks si_status with 0xFF HP-UX-10.20 masks si_status with 0xFF Linux masks si_status with 0xFF Mac OS X clears si_code and si_pid and masks si_status with 0xFFFFFF and then sign extends So from my current knowledge, only Solaris is following POSIX rules. If you need a program that may return arbitrary exit code, try: bourne-shell -c "exit <value>" or use the test program I add in the n ext note: |
|
(0002732) joerg (reporter) 2015-06-24 11:53 |
#include <stdlib.h> #include <unistd.h> #include <sys/wait.h> #include <stdio.h> /* * Non-standard compliant platforms may need * #include <signal.h> or something similar * in addition to the include files above. */ int main() { siginfo_t si; pid_t pid; int ret; if ((pid = fork()) < 0) exit(1); if (pid == 0) { _exit(1234567890); } ret = waitid(P_PID, pid, &si, WEXITED); printf("ret: %d si_pid: %ld si_status: %d si_code: %d\n", ret, (long) si.si_pid, si.si_status, si.si_code); if (pid != si.si_pid) printf("si_pid in struct siginfo should be %ld but is %ld\n", (long) pid, (long) si.si_pid); if (si.si_status != 1234567890) printf("si_status in struct signinfo should be %d (0x%x) but is %d (0x%x)\n", 1234567890, 1234567890, si.si_status, si.si_status); if (CLD_EXITED != 1) printf("CLD_EXITED is %d on this platform\n", CLD_EXITED); return (0); } |
|
(0002745) joerg (reporter) 2015-07-06 11:56 |
Let me add a hint to understand the background: The call waitid() has been introduced in 1989 with SVr4. On SVr4, it supports to retrieve the full 32 bits from the exit() call since it was introduced. Given the fact that POSIX does not intend to modify but rather describes existing features, it can be seen as a POSIX bug when POSIX claims that the exit code is always masked by 0xFF. |
|
(0002746) geoffclare (manager) 2015-07-06 14:39 |
Looking at the history of waitid() standardisation, there is an interesting change between SUSv1 and SUSv2. If you read SUSv1 carefully, the description of exit() and _exit() was misleading but did not actually conflict with SVR4 waitid() behaviour. It said: "If the parent process of the calling process is executing a wait(), wait3(), waitid() or waitpid(), and has neither set its SA_NOCLDWAIT flag nor set SIGCHLD to SIG_IGN, it is notified of the calling process’ termination and the low-order eight bits (that is, bits 0377) of status are made available to it." This required the low-order eight bits to be available, but said nothing about whether other bits might also be made available. In SUSv2 that part was the same, but the following sentence was added earlier in the description: "The values of status can be EXIT_SUCCESS or EXIT_FAILURE, as described in <stdlib.h>, or any implementation-dependent value, although note that only the range 0 through 255 will be available to a waiting parent process." (The sentence has since morphed into what is now the 2nd paragraph.) This was the point at which the conflict with SVR4 waitid() behaviour was introduced. |
|
(0002749) joerg (reporter) 2015-07-06 15:05 |
Do you remember why this change has been introduced? Are there any type of records about the discussion for SUSv2? |
|
(0002750) joerg (reporter) 2015-07-08 12:55 |
Today, July 8 2015, FreeBSD added support for the whole int from exit() in siginfo.si_status. |
|
(0002754) eblake (manager) 2015-07-15 17:45 |
In response to Note: 0002657, just as awk's system() needs an update, so does m4's sysval. Also, xargs is documented to have special behavior on commands that return 255, where exit(-1) has typically triggered this special behavior. Mapping exit(256) to 1 vs. mapping it to 255 (instead of the current truncation to 0) may have implications on this usage of xargs. |
|
(0002756) joerg (reporter) 2015-07-16 14:30 |
Let me first mention that my implementation in the Bourne Shell was mainly (in the first attempt) to be able get an impression on the implications. Now that the implementation exist and is complete since a week, it seems that See my Bourne Shell man page that is updated from time to time here: http://schillix.sourceforge.net/man/man1/bosh.1.html [^] m4's sysval seems to be as "simple" as the shell as we would just need to define new variables for si_status and si_code. system() and awk may be harder. |
|
(0002776) joerg (reporter) 2015-07-29 08:17 edited on: 2016-01-21 16:48 |
Some more notes: 1) meanwhile, I added "set -o fullexitcode" to the Bourne Shell to tell the shell not t mask the value in $?. I am in hope that Surceforge will allow uploads again soon, in order to be able to make the related source available. 2) it may be of interest that the ability to get access to all 32 bits from the exit() code existed in 1980 on UNOS (the first UNIX clone) already and was accessible via the cwait() function that has a remarkable similarity to the waitid() interface introduced in 1989. Here is the man page from UNOS: cwait() waits for child processes to exit or suspend :format int cwait(pidp,statp) int *pidp; /* where to fill in pid */ int *statp; /* where to fill in status */ :description cwait() waits for children to exit or suspend and indicates why the particular process exited (e.g. normal termination). :returns Returns the following values, depending on why cwait() came back: < 0 error (no children) 0 normal termination, exit code of child 1 ^C 2 killed 3 trap, particular trap in status 4 suspended, can be resumed or killed 5 exec failure, standard system error code in status 6 syserr, error detected in kernel, standard error code :notes A process that suspends still exists in the system and should eventually be killed to free up the resources it uses. |
|
(0003022) mirabilos (reporter) 2016-01-13 21:46 |
I have just been made aware of the move towards more than 8 bit of exit code, and I would like to object against doing that, for several reasons: – real existing code has been relying on the formulation of an exit code being an unsigned integer in the standard for long – wording in the standard suggested the exit code could only be in the range 0‥255, inclusive – waitid() may have existed for a long time, but it’s still not present in some contemporary operating systems (such as OpenBSD), and is reportedly broken in others (and apparently, this was caused by a mistake in some versions of the standard, which, while a bug that should arguably be fixed, people were *encouraged* to rely on – nobody appreciates a standard which changes what’s allowed and forbidden for every issue) – real existing code (both shell scripts and compiled C code) can rely on the masking (in fact, I vaguely recall writing shell scripts doing that) – changing the standard in such a backwards-incompatible way that *additionally* imposes requirements on the operating system causes no small amount of harm to portability (e.g. that of a shell implementation) – once a standard gets too removed from the real existing situation out in the field, the standard will lose all credibility Not part of my objection, but still needs to be said: one of the greatest strengths of mksh is that it behaves consistently even across platforms. This does include such things as restricting arithmetics to 32 bit everywhere, but it guarantees scripts a well-defined runtime environment. This means suggesting me to only not do the masking when the OS supports waitid() will not be well received, as it would introduce another point of diverging depending on the underlying operating system. |
|
(0003025) chet_ramey (reporter) 2016-01-15 15:12 |
I agree with Thorsten. This seems like more of a change for change's sake, rather than solving some problem or satisfying a need. |
|
(0003054) rhansen (manager) 2016-01-28 17:09 edited on: 2016-01-28 17:09 |
Interpretation response ------------------------ The standard clearly states that if a command terminates normally (not by a signal), the shell sets $? to the value retrieved for the command by the equivalent of the wait() function WEXITSTATUS macro, and conforming implementations must conform to this. Rationale: ------------- If an application calls exit(256), the WEXITSTATUS macro's effective modulo 256 operation will result in the shell behaving as if the application called exit(0) (the '?' special parameter will expand to 0 and the exit status will be treated as true for purposes of if, while, AND and OR lists, etc.). The concern is that this behavior will result in certain application errors being interpreted as success by the shell. While this behavior is unfortunate, all known existing implementations behave this way, and changing the standard to require the use of the full exit value would render these implementations non-conformant. In addition, some applications may rely on the modulo 256 behavior to report different flavors of success. Notes to the Editor (not part of this interpretation): ------------------------------------------------------- On page 2337-2338 lines 74309-74319 (XCU 2.8.2 Exit Status for Commands), change: If a command is not found, the exit status shall be 127. If the command name is found, but it is not an executable utility, the exit status shall be 126. Applications that invoke utilities without using the shell should use these exit status values to report similar errors.to: The exit status of a command shall be determined as follows: On page 3692 after line 126274 (XRAT C.2.8.2 Exit Status for Commands) insert the following new paragraphs: If a C application calls <tt>exit(256)</tt>, the command's exit status in the shell becomes zero due to the modulo 256 operation. Since zero is interpreted as "true" or "success" for if statements, AND and OR lists, <tt>set -e</tt>, and so on, applications should be careful to avoid exiting with a value that is a multiple of 256 unless the value is intended to be interpreted as true or success. |
|
(0003065) ajosey (manager) 2016-02-01 21:31 |
Interpretation Proposed: 1st February 2016 |
|
(0003092) ajosey (manager) 2016-03-08 19:22 |
Interpretation approved: 8 March 2016 |
|
Issue History |
|||
| Date Modified | Username | Field | Change |
| 2015-05-06 11:51 | joerg | New Issue | |
| 2015-05-06 11:51 | joerg | Status | New => Under Review |
| 2015-05-06 11:51 | joerg | Assigned To | => ajosey |
| 2015-05-06 11:51 | joerg | Name | => Jörg Schilling |
| 2015-05-06 11:51 | joerg | Section | => Shell command language |
| 2015-05-06 11:51 | joerg | Page Number | => 2324 |
| 2015-05-06 11:51 | joerg | Line Number | => 73758 |
| 2015-05-07 09:29 | stephane | Note Added: 0002657 | |
| 2015-05-07 09:47 | joerg | Note Added: 0002658 | |
| 2015-05-07 12:33 | joerg | Note Edited: 0002658 | |
| 2015-05-07 12:33 | joerg | Note Edited: 0002658 | |
| 2015-06-18 09:25 | joerg | Note Edited: 0002658 | |
| 2015-06-18 09:36 | joerg | Note Edited: 0002658 | |
| 2015-06-18 09:37 | joerg | Note Edited: 0002658 | |
| 2015-06-22 14:58 | joerg | Note Added: 0002728 | |
| 2015-06-22 15:01 | joerg | Note Edited: 0002728 | |
| 2015-06-22 15:16 | joerg | Note Edited: 0002728 | |
| 2015-06-24 11:53 | joerg | Note Added: 0002731 | |
| 2015-06-24 11:53 | joerg | Note Added: 0002732 | |
| 2015-06-24 12:29 | joerg | Note Edited: 0002731 | |
| 2015-07-02 08:45 | joerg | Note Edited: 0002728 | |
| 2015-07-02 10:30 | joerg | Note Edited: 0002731 | |
| 2015-07-06 11:56 | joerg | Note Added: 0002745 | |
| 2015-07-06 14:39 | geoffclare | Note Added: 0002746 | |
| 2015-07-06 15:05 | joerg | Note Added: 0002749 | |
| 2015-07-06 16:21 | rhansen | Relationship added | related to 0000690 |
| 2015-07-07 10:20 | joerg | Note Edited: 0002731 | |
| 2015-07-08 12:55 | joerg | Note Added: 0002750 | |
| 2015-07-15 17:45 | eblake | Note Added: 0002754 | |
| 2015-07-16 14:30 | joerg | Note Added: 0002756 | |
| 2015-07-29 08:17 | joerg | Note Added: 0002776 | |
| 2016-01-07 16:22 | eblake | Relationship added | child of 0000594 |
| 2016-01-13 21:46 | mirabilos | Note Added: 0003022 | |
| 2016-01-15 15:12 | chet_ramey | Note Added: 0003025 | |
| 2016-01-21 16:48 | joerg | Note Edited: 0002776 | |
| 2016-01-28 16:09 | Don Cragun | Relationship added | related to 0001026 |
| 2016-01-28 17:09 | rhansen | Note Added: 0003054 | |
| 2016-01-28 17:09 | rhansen | Note Edited: 0003054 | |
| 2016-01-28 17:12 | Don Cragun | Interp Status | => Pending |
| 2016-01-28 17:12 | Don Cragun | Final Accepted Text | => See Note: 0003054. |
| 2016-01-28 17:12 | Don Cragun | Status | Under Review => Interpretation Required |
| 2016-01-28 17:12 | Don Cragun | Resolution | Open => Accepted As Marked |
| 2016-01-28 17:12 | Don Cragun | Tag Attached: tc3-2008 | |
| 2016-02-01 21:31 | ajosey | Interp Status | Pending => Proposed |
| 2016-02-01 21:31 | ajosey | Note Added: 0003065 | |
| 2016-03-08 19:22 | ajosey | Interp Status | Proposed => Approved |
| 2016-03-08 19:22 | ajosey | Note Added: 0003092 | |
| 2019-10-21 09:32 | geoffclare | Status | Interpretation Required => Applied |
| Mantis 1.1.6[^] Copyright © 2000 - 2008 Mantis Group |  |
