Austin Group Defect Tracker

Aardvark Mark IV

Main | My View | View Issues | Change Log | Docs
  

Viewing Issue Simple Details [ Jump to Notes ] [ Issue History ] [ Print ]
ID Category Severity Type Date Submitted Last Update
0001602 [Issue 8 drafts] Shell and Utilities Objection Omission 2022-08-23 16:33 2024-06-11 09:12
Reporter kre View Status public  
Assigned To
Priority normal Resolution Accepted As Marked  
Status Closed   Product Version Draft 2.1
Name Robert Elz
Organization
User Reference
Section XCU 2.14 exit XCU 2.14 return
Page Number 2369, 2379
Line Number 76750, 77069
Final Accepted Text Note: 0006012
Summary 0001602: No definition of "executed in a trap action"
Description This defect report could also be filed against issue 7 TC2, and
probably earlier versions as well, but the wording has changed
so much in this area that going back to discuss text which has already
been altered seems pointless. Hence filed against draft 2.1.

The wording of exit, when dealing with the case that no specific
status to use is given, is:

    If n is not specified, the result shall be as if n were specified
    with the current value of the special parameter '?' (see Section 2.5.2),
    except that when exit is executed in a trap action, the value for the
    special parameter '?' that is considered ``current'' shall be the value
    it had immediately preceding the trap action.

The wording for return is substantially the same, though the effects, and
required interpretation, are different.

So, first to consider exit. It seems likely (this requires some guesswork)
that the idea here, is that when an EXIT trap (or others, will come to those
soon) is executed, if the trap command says just "exit" - which will then
cause the shell to terminate, the status it terminates with should be the
status that it would have exited with had there been no EXIT trap, and
the shell had simply exited. That's fine, understandable, and the way
shells have behaved for a long time.

Doing the same for other traps is more likely just a "the way we run the
trap command is like this, so this is what happens" which is also OK,
though for the other traps actually depending upon this for anything is
kind of pointless, as the script writer can never know exactly where the
script will be when the trap needs to be taken, so the exit status would
be more or less random at that point.

That all seems good, but it is only meaningful because of the phrase I
included above (which is not anywhere in the standard), "which will then
cause the shell to terminate". Consider an alternative use of exit:

cleanup() {
    # various stuff not relevant
    if (cd /somewhere || exit; # cleanup stuff in /somewhere )
    then # do some more cleanup stuff
    else report cleanup failed, manual cleanup required
    fi
}

(Ignore the syntax error caused by the ')' being treated as part of
what is a comment, that comment would really be code in a real script).

and then

    trap 'echo Finishing; cleanup; exit' EXIT

(in which the exit is pointless, but makes for a better discussion).

Now assume that the code in the script, sometime later, does "exit 0"
That sets the status ($?) to 0, and then runs the EDIT trap before the
shell goes away.

There's no real question but that the wording in question applies to
exit that is in the string passed as the arg to the trap command, but
what about the one, executed in a subshell, in the cleanup function.

In that one, we may assume that perhaps the function was written long
ago, when no traps were being used, and the usage was

    cleanup; exit 0

that being a pattern (perhaps with different values for the exit status
throughout). In that case, it is clear, the exception of being in a
trap action does not apply, there is none, and the exit in cleanup returns
the status of the the failed cd command (we know it failed, or the exit
would not being executed). But then someone decided that it would be
cleaner to use an EXIT trap, remove all the calls to cleanup (perhaps there
were some exit's in the code, rarely encountered, which had omitted the
cleanup, and that bug is being fixed). Now cleanup is being executed in
a trap action. That's clear. But is the exit within the subshell within
cleanup also "executed in a trap action" or only text that is actually in the
trap action string? If were were to decide that it is the latter, then
let's assume that instead of just returning, cleanup ended with "exit".
That one, not being in a sub-shell, would cause the shell to terminate.
But with what status, the status from the last command in cleanup() which
is likely what the author of that function intended, or the value of $?
that was current when the EXIT trap was taken?

We don't know, because nothing in the standard (or nothing I can find)
tells us.

For return, things get even messier. That's because in general, return
is only defined when executed in a function. Simply executing a random "return" in a trap string (like was done with exit) is only meaningful if
we caqn guarantee that the trap will only fire while a function is running.

That's certainly possible to do, but not all that common. In practice
most 'return's that get executed are to return from a function which has
just been called.

That is
    func_performed=false
    func() {
        if $func_performed; then return; fi
        # perform whatever the function does
        func_performed=true
    }

In that func() will return 0 every time, the return will, as $? at
that point will be 0, because we know that $func_performed must have
been true or the return would not be being executed.

Now imagine that func() is called by some tree of function calls,
perhaps several times in different circumstanced. The root of that
set of functions iis a function done() (not shown here).

If we were to now do "trap done INT"

what does that mean to that return in func() - it has no impact at all
on the final value of $? when the trap action completes (that's controlled
by the done() function) and nothing here has any impact at all upon any
function that may have been executing when the trap occurred. To do
that there would need to be a an explicit "return" in the trap string,
anything in any function that is called applies only to that function.

Is the return in func() now executed in a trap action, and so required to
return whatever status was in $? when the SIGINT trap occurred? That
might not have been 0, so (depending upon how func() is used inn unshown code,
might alter the execution flow in unexpected ways --- the commented code
from func() might even have "return N" for some N != 0, with the intent to
alter the flow - but the one shown was not planned to do that.

Again, we cannot answer that question, as the standard does not tell us
what "executed in a trap action" actually means.

Further there have been different interpretations of all of this, it isn't
(any longer anyway) simply a case of "well it is obvious, we just didn't write
it down" as (some, at least one) shell authors have taken that wording, and
actually made their shell do (what is really) absurd things, because of it
(when it had previously been sane).

There was a thread on the mailing list

   https://www.mail-archive.com/austin-group-l@opengroup.org/msg06011.html [^]

in which this was briefly discussed, but none of the participants in
that discussion were people able to actually change the wording.

Hence this defect report, to force some attention by those who matter.
Desired Action For exit, it should probably be "an exit command executed while a trap action
is being performed, which will cause the shell environment executing the
trap action to terminate".

That's closer to what might be workable, but even that isn't perfect.

Given that exit & return need two quite different definitions, the
magic phrase should probably simply be deleted from the standard,
and replaced inline by the appropriate definition.
Tags issue8
Attached Files

- Relationships

-  Notes
(0005941)
kre (reporter)
2022-08-23 18:11

 Ugh, Mantis... I had to cut & paste my entire report, as the first time
I tried to submit it, Mantis claimed a "security violation" or something,
(suggested I may have submitted already - which obviously, I did not),
which I suspect was caused simply by the (lengthy) period it took me to
fill in the form for this one (it was over an houir for sure). I have
encountered similar before.

Anyway, I managed to miss cutting the first paragraph of the desired
action, which was intended to be:

  For "return" the definition of "executed in the trap action" probably
  should be "appears literally in the argument action to the trap command"
  which established the trap action now being taken. No other return should
  be considered.

I also ended up having had enough of this one, and didn't proof read
the report (almost at all) - consequently there are errors all over
the place (but I think the meaning is reasonably clear, despite that).

Pity Mantis has no way for ordinary mortals to edit their defect reports,
like it does for notes that were added, or I'd go back and clean up some
of the worst typos...

I wonder if I ever mentioned what I think of Mantis?
(0005942)
kre (reporter)
2022-08-23 18:27

 A way to word things might be:

for exit, rather than "except when exit is executed in a trap action" say
"except when exit is executed in the same execution environment as that in
which a trap action was invoked, that trap action is still executing, and
the exit would cause that execution environment to terminate"

and for return, rather than "except when when return is executed in a trap
action" say instead "except when return is executed in the same execution
evnironment as that in which a trap action was invoked, while that trap action
is still executing, and the return would cause a function that was executing
when the trap was invoked to be terminated"

I believe those produce sane results, and allow the "executed in a trap
action" remain undefined, as those words will no longer appear (unless of
course they're also somewhere else I'm not aware of, in which case that part
of the standard, whatever it is, will need fixing as well).
(0005943)
hvd (reporter)
2022-08-25 01:10

 Your proposed behaviour actually does suggest the same behaviour for exit and return, and the same wording could be used for both. Something along the lines of simply

  If the [exit/return] command would cause the end of execution of a trap action, the last command is considered to be the command that executed immediately preceding the trap action.

would cover it. I agree that this behaviour would be sane and if POSIX is changed to require or allow this, I have no problem changing my shell to implement this. However, while it is unclear to me what the POSIX wording currently requires, I believe there is no legitimate reading of the current wording that matches your expectations; this would be a change, not a clarification.

Note that your proposed behaviour is to say that

  trap ':; (exit); echo $?' EXIT; false

will be required to print 0, as exit is not executed in the same execution environment as that in which the trap action was executed. My shell does print 0, yours prints 1 (tested with the system shell of yesterday's NetBSD amd64 installation CD image), as does dash (which my shell is based on). Please actually check before dismissing others' work as absurd or insane.
(0005945)
kre (reporter)
2022-08-25 15:20

 Three responses to issues in Note: 0005943 (in a different order than
they appear there).

First, wrt the proposed wording - yes, that looks like it would work, I
didn't ever consider it that way. Provided that we actually want the
wording for exit & return to be the same (which risks the later
interpretation "we know what those words mean in case A, case B uses
the same words, so they must mean the same thing there" which is OK if
that really is the intent, but we need to be certain that the two are
really meant to be identical in all respects, otherwise it can be better to
use slightly different wording, and avoid that problem (that's why I varied
the wording I used just a little, in the exit and return cases).

Second, it wasn't anyone's work I was dismissing as absurd or insane, it
is the result produced. That's is as true of the NetBSD sh doing it as
any other shell - our shell (not mine, I'm just the current (major) maintainer
but I wrote only a tiny fraction of the code in there) has done absurd things
many times in the past, and almost certainly still does. They get fixed,
eventually, when discovered, and if possible.

But yes, I should have done a lot more testing of various shells, but
other than using the EXIT trap (which cannot be used to test everything)
actually running tests for this is difficult, as in a script, there's no
way I know of, to force the "value for the special parameter '?' that is
considered ``current'' shall be the value it had immediately preceding
the trap action." to be any known value - and if we don't know what it is,
we cannot test that it is being used when required (whenever we believe that
to be). This is just due to the nature of the conditions which invoke
trap actions, other than EXIT, when they occur is unpredictable.

Interactively it is possible, as one can set an exit status, and then
just do nothing, until after the trap action has occurred (sending the
occasional \n to cause new prompts to be written, and allow the trap
to happen). That works, but is ***slow*** and tedious to do. Hence I
was prevaricating ... this defect report was discussed (in other places,
and in private e-mail, in the early days of this month - and I had promised
(just private e-mail I think) to submit this back then ... but waiting for
me to get the energy to do more testing (or almost any testing) kept delaying
it, and since there was no end in sight to that problem, I thought I should
just go ahead anyway - test results can always be added as a note (like this)
by me later (or by anyone else who can add notes, like hvd, as one example).



And third, for the most complex one to reply to - this is going to take some
time (ie: many lines following...)

I think the current wording can be read to to meet my expectations, perhaps
with just one minor change (I'd prefer to be incompat with any possible
reading in this one way, but if that's unacceptable, it would not be fatal).

Start by accepting that "executed in a trap action" might mean "the
exit (or return, as appropriate) must appear literally in the action
string of the trap command, for the condition which occurred, such that
the action is invoked. Whether it does mean that is unknown, but since
we are just looking for "any legitimate reading" this is certainly one
possibility.

Two problems arise from this, the first is the one I mention just above,
which is that if we had

    func()
    {
         # do something, then perhaps conditionally
         exit
    }

and we then do

    trap func $CONDITION

then by this reading, that exit in func would not be one that is executed
in the trap action, and so would exit with the status of the final command in
"do something" rather than the status from before the action was invoked,
which my proposed wording (and your version of that) would require.

So, yes, that would be a change. But we do not have to make that change,
we could define "executed in the trap action" to mean just what I said is
one possible reading, and leave it at that.

Applications which wanted to write code like the above is intended to be
would need instead to write


    func()
    {
         # do something, then perhaps conditionally
         exit $1
    }

    trap 'func $?' $CONDITION

where, since nothing has happened in the action when $? is expanded,
the value produced by expanding it must be the value it had immediately
before the trap action was invoked, just as it would be in

   (exit 13)
   eval 'func $?'

which is exactly what is supposed to happen with the (most recent) trap
command above, assuming that $CONDITION happened to arise just when (exit 13)
was finishing. $? will be 13, and func (as modified) will do exit 13.

I'd prefer not to have to go that route, but if we were to be required to
simply be explaining what the words have always meant, and we just didn't
know it, rather than changing anything, this would be one way.

The second issue is your example

    trap ':; (exit); echo $?' EXIT; false

In that, the "exit" is literally in the action string, so it might
seem that that one would be required (in this example) to do exit 1
(as you say the NetBSD sh currently does, and which I have just confirmed,
and which I can certainly change - but won't do until I at least get a
sense of the likely outcome of this defect report) rather than the exit 0
which you (correctly) interpreted as being the intent of my proposed
resolution (also your version).

Howver "might seem" is not "necessarily is". There that exit is
(quite clearly) executed in a subshell environment. That subshell
is executed in the trap action, but what is the state of the subshell
environment wrt "executing in a trap action". This is where we can get
creative (but still remain legitimate).

XCU 2.12 (on page 2350 in Issue 8 draft 2.1) defines what the shell
execution environment is. Whether or not a trap action is currently
executing is not in the list. Hence, that status is not part of the
shell execution environment.

XCU (also in 2.12, but this time on page 2351 of the same draft)
says of subshell environments

    A subshell environment shall be created as a duplicate of the shell
    environment, except that:

None of the exceptions is material here. Since the "executing in a
trap action:" is (as just discovered) not part of the shell execution
environment, there is no requirement that that status be duplicated in
the subshell environment. Hence doing exit 0 as your shell does (apparently)
is perfectly legitimate as the text is written now.

Further, you're not the only shell to act that way - bash bosh yash and zsh
all do as well, so does an ancient pdksh. But mksh and ksh93 do not
(nor as you point out do ash derived shells - other than yours). mksh and ksh93
are actively maintained (ksh93, in a sense) so it is reasonable to assume might
have been updated to match an apparent reading of this requirement in the
standard - but perhaps not the correct (intended) reading. Who knows?
I have no idea what ksh88 does with that, I'd kind of guess, based upon pdksh
and bosh's behaviour, that it might print 0 as well.

For return there are no issues like this, to cause the executing function
to return, the "return" in question must be literally in the action string
(and not in a subshell) so for this reading of what the current words
mean (or at least can be interpreted to mean) the proposed resolution for
return has no problems.
(0005946)
kre (reporter)
2022-08-25 15:49

 There is another defect that we might want to fix as part of this issue.

XCU 2.14/trap says

   The value of "$?" after the trap action completes shall be the value
   it had before the trap action was executed.

yet it is quite clear (from any reading) that XCU 2.14/exit and XCU2.14/return
mandate, than when given an explicit status to return, those two commands
do alter the value of "$?" after the trap action completes (when one of those
commands is the last executed in the trap action).

That's deliberate, it allows an trap action like the following

     trap 'cleanup || exit 1' EXIT

when invoked by

     exit 0

to cause the script to exit with status 1, rather than 0, in the event that
cleanup failed. That is intended, and desirable, I think.

The same applies to a trap action whose purpose is to make a function
return with a specific value, regardless of what the value of $? might
have been at the point in the function where the action was invoked.

The wording above (from XCU 2.14/trap) needs to be corrected.
(0006012)
geoffclare (manager)
2022-10-24 15:31
edited on: 2022-10-24 15:37

 On page 2369 line 76750 section 2.14 exit, change:
when exit is executed in a trap action
to:
if the exit command would cause the end of execution of a trap action


On page 2379 line 77069 section 2.14 return, change:
when return is executed in a trap action
to:
if the return command would cause the end of execution of a trap action

- Issue History
Date Modified Username Field Change
2022-08-23 16:33 kre New Issue
2022-08-23 16:33 kre Name => Robert Elz
2022-08-23 16:33 kre Section => XCU 2.14 exit XCU 2.14 return
2022-08-23 16:33 kre Page Number => 2369, 2379
2022-08-23 16:33 kre Line Number => 76750, 77069
2022-08-23 18:11 kre Note Added: 0005941
2022-08-23 18:27 kre Note Added: 0005942
2022-08-25 01:10 hvd Note Added: 0005943
2022-08-25 15:20 kre Note Added: 0005945
2022-08-25 15:49 kre Note Added: 0005946
2022-10-24 15:31 geoffclare Note Added: 0006012
2022-10-24 15:34 geoffclare Note Edited: 0006012
2022-10-24 15:34 geoffclare Note Edited: 0006012
2022-10-24 15:36 geoffclare Final Accepted Text => Note: 0006012
2022-10-24 15:36 geoffclare Status New => Resolved
2022-10-24 15:36 geoffclare Resolution Open => Accepted As Marked
2022-10-24 15:36 geoffclare Tag Attached: issue8
2022-10-24 15:37 geoffclare Note Edited: 0006012
2022-11-08 14:43 geoffclare Status Resolved => Applied
2024-06-11 09:12 agadmin Status Applied => Closed

Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker