Austin Group Defect Tracker

Aardvark Mark IV


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0001254 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2019-06-08 09:14 2022-09-27 15:29
Reporter stephane View Status public  
Assigned To
Priority normal Resolution Accepted As Marked  
Status Applied  
Name Stephane Chazelas
Organization
User Reference
Section 2.9.3.1 Asynchronous Lists
Page Number 2370 (in 2018 edition, not 2016)
Line Number the whole 2.9.3.1 section and also 76177 (execution environment), 115763 (wait), 128448 (rationale), 74883 ($!)
Interp Status Approved
Final Accepted Text Note: 0005888
Summary 0001254: "asynchronous list" description uses "command" instead of "AND-OR list"
Description That's a follow-up on my Note: 0004410 comment on 0000760.

2.9.3 correctly specifies that "an '&' separator or terminator shall cause asynchronous execution of the preceding AND-OR list."

In POSIX shells, rc-like shells and csh-like shells, zsh being the only exception that I know,

a && b || c &

is interpreted as:

{ a && b || c; } &

not

a && b || { c & }

But 2.9.3.1 then only talks about "commands" started asynchronously, not even pipelines let alone AND-OR lists.

The sentence: "When an element of an asynchronous list (the portion of the list ended by an <ampersand>, such as command1, above) is started by the shell, the process ID of the last command in the asynchronous list element shall become known in the current shell execution environment; see Shell Execution Environment. This process ID shall remain known until:..." doesn't make sense.


Without giving a proper definition of "asynchronous list", it suggests that in:

  false && cmd2 &

The pid of cmd2 (what's the pid of a command btw?) should somehow become known (as $! presumably) whenever cmd2 is started (if ever it's started which is unlikely in that example). What should $! be until then?

The wait utility description has similar wording, worse in that it suggests $! should be updated for every command run in the list.

I suspect at some point "asynchronous list" must have been (confusingly) defined in the grammar as a sequence of 1 more of "and-or-list &" which would explain. There's a clue about that in the rationale at line 128448:

<<
   The grammar treats a construct such as:

     foo & bar & bam &

   as one "asynchronous list", but since the status of each element is tracked by the shell, the term "element of an asynchronous list" was introduced to identify just one of the foo, bar, or bam portions of the overall list.
>>

So the "component of an asynchronous list" in 2.9.3.1 probably does not refer to the pipeline component of the asynchronously started AND-OR list, but to one AND-OR list component of a "and-or-list1 & and-or-list2 & and-or-list3 &".

AFAICT, in the current version of the spec, asynchronous_list is not defined in the grammar (it wasn't in SUSv2 either). "and_or" is.

Which process we get in $! is important as we want to know what the effect of killing that process will be and it's important that calling "wait" on that pid should give us the exit status of the AND-OR-list.
Desired Action - Clearly define what is meant by an asynchronous list. The definition suggested by the rationale (cmd1 && cmd2 & cmd3 || cmd4 & being *one* asynchronous list) but AFAICT absent from the current specification is confusing.

I'd rather it be one single "AND-OR list" run asynchronously with & as there's no reason to treat "list1 & list2 & list3 & list4" (with the exclusion of list4) as part of a single construct. If that definition is chosen, the text in 2.9.3.1 the wait utility and the rationale at least would have to be updated.

Calling it an "asynchronous *AND-OR* list" might even remove even more confusion .

- Update 2.9.3.1, and $!/wait/execution environment description to remove the reference to "command" and replace with "AND-OR list"

- Clarify that the pid stored in $! should be the id of a child process that runs a subshell to execute that and_or list, make it clear that that process will exit with the exit status of the and_or list, and that if the and_or list consists of a single pipeline, the last component of the pipe line will be executed by that pid (sleep 1 | sleep 2 & sleep 1; kill "$!" should stop the execution of "sleep 2", same for sleep 2 & sleep 1; kill "$!")
Tags issue8
Attached Files

- Relationships
related to 0000760Closed 1003.1(2013)/Issue7+TC1 asynchronous list assignment of stdin should depend on job control 
has duplicate 0001611Closed Issue 8 drafts exit status from fg is either badly specified or is simply wrong 
related to 0001675Applied Issue 8 drafts Copy and paste error in new job control section 

-  Notes
(0004416)
stephane (reporter)
2019-06-10 18:34

I think that it makes sense to address the other objections I raised in Note: 0004410 and Note: 0004411 of 0000760 in this same bug.

the new bug760 wording of:

> If job control is disabled (see set, −m), the standard input for
> an asynchronous list, before any explicit redirections are
> performed, shall be considered to be assigned to a file that has
> the same properties as /dev/null. This shall not happen if job
> control is enabled. In all cases, explicit redirection of
> standard input shall override this activity.

seems to assume "my" definition of "asynchronous list" as it doesn't make sense to speak of "the standard input for an asynchronous list" when that asynchronous list can be "cmd1 & cmd2 &". If "redirection of standard input shall override this activity", that would imply in

   cmd1 < foo & cmd2 &

cmd2's stdin would not be redirected to /dev/null.

IMO, after "asynchronous list" has been clearly defined as "AND-OR-list followed by &", and maybe renamed to "asynchronous AND-OR list", that whole paragraph should be changed to something like:

} If job control is disabled (see set, −m), an implementation may
} choose to assign the standard input of an asynchronous AND-OR
} list to an open file description that behaves as if /dev/null
} had been opened in read-only.
}
} This shall not happen if job control is enabled.
}
} Where and if that happens, that shall happen before any explicit
} redirection of any command in the asynchronous AND-OR list is
} processed.
}
} A portable application that needs to make sure that is done
} would need to make the /dev/null redirection explicitly:
}
} < /dev/null cmd1 | cmd2 && < /dev/null cmd3 &
}
} Or
}
} { cmd1 | cmd2 && cmd3 ; } < /dev/null &
}
} A portable application that needs to make sure it is not done
} would need to duplicate the original stdin outside of the
} asynchronous AND-OR list and restore it inside for instance as:
}
} { <&3 3<&- cmd1 | 3<&- cmd2 && <&3 3<&- cmd3 & } <&3
}
} Or:
}
} { { cmd1 | cmd2 && cmd3 ; } <&3 3<&- & }
}
} As whether it is done or not in:
}
} cmd1 | cmd2 && cmd3 &
}
} Is unspecified.
(0005845)
geoffclare (manager)
2022-06-06 09:48
edited on: 2022-06-06 09:51

Proposed changes...

On page 38 line 1270 section 3.33 Background Job, change:
See Background Process Group in [xref to 3.35].
to:
In the context of the System Interfaces volume of POSIX.1-202x, a background process group (see [xref to 3.35]).

In the context of the shell, a job that the shell is not waiting for before it executes further commands or, if interactive, prompts for further commands. A background job can be a job-control background job or a non-job-control background job. A job-control background job is a job that started execution (either in the background or the foreground) while job control was enabled and is currently in the background. A non-job-control background job is an asynchronous AND-OR list that started execution while job control was disabled and was assigned a job number. An implementation need not support non-job-control background jobs; that is, the shell may, but need not, assign job numbers to asynchronous AND-OR lists that start execution while job control is disabled.

<small>Note: Asynchronous AND-OR lists are defined in detail in [xref to XCU 2.9.3.1].</small>

<small>Note: See also [xref to 3.182 Foreground Job], [xref to 3.202 Job], [xref to 3.203 Job Control], and [xref to 3.379 Suspended Job].</small>

On page 38 line 1273 section 3.35, change:
Background Process Group (or Background Job)
to:
Background Process Group

On page 38 line 1275 section 3.35, add a new paragraph:
<small>Note: See also [xref to 3.33 Background Job].</small>

On page 62 line 1849 section 3.182 Foreground Job, change:
See Foreground Process Group in [xref to 3.184].
to:
In the context of the System Interfaces volume of POSIX.1-202x, a foreground process group (see [xref to 3.184]).

In the context of the shell, a job that the shell is waiting for before it executes further commands or, if interactive, prompts for further commands.

<small>Note: See also [xref to 3.33 Background Job], [xref to 3.202 Job], and [xref to 3.379 Suspended Job].</small>

On page 62 line 1852 section 3.184, change:
Foreground Process Group (or Foreground Job)
to:
Foreground Process Group

On page 62 line 1857 section 3.184, add a new paragraph:
<small>Note: See also [xref to 3.182 Foreground Job].</small>

On page 65 line 1924 section 3.202 Job, change:
A set of processes, comprising a shell pipeline, and any processes descended from it, that are all in the same process group.

<small>Note: See also [xref to XCU 2.9.2 Pipelines].</small>
to:
A background job, a foreground job, or a suspended job.

In the context of the shell, jobs are created when a list (see [xref to XCU 2.9.3 Lists]) is executed while job control is enabled, and may be created when an asynchronous AND-OR list is executed while job control is disabled.

<small>Note: Job control in the shell is defined in detail in [xref to XCU 2.11 Job Control].</small>

<small>Note: See also [xref to 3.33 Background Job], [xref to 3.182 Foreground Job], and [xref to 3.379 Suspended Job].</small>

After page 65 line 1931 section 3.203 Job Control, add:
The term is also used in connection with system interfaces that can be used by a command interpreter to implement job control (see for example [xref to setpgid()]).

<small>Note: Job control in the shell is defined in detail in [xref to XCU 2.11 Job Control].</small>

On page 66 line 1932,1935,1936 section 3.204 Job Control Job ID, change:
Job Control Job ID
to:
Job ID

On page 66 line 1933 section 3.204 Job Control Job ID, and
page 2881,2883 line 95038,95096 section kill, and
page 3717 line 127377 section C.1.7 Built-In Utilities, change:
job control job ID
to:
job ID

On page 94 line 2625 section 3.379 Suspended Job, change:
A job that has received a SIGSTOP, SIGTSTP, SIGTTIN, or SIGTTOU signal that caused the process group to stop. A suspended job is a background job, but a background job is not necessarily a suspended job.
to:
In the context of the System Interfaces volume of POSIX.1-202x, a job that has received a SIGSTOP, SIGTSTP, SIGTTIN, or SIGTTOU signal that caused the process group to stop.

In the context of the shell, a job, other than a non-job-control background job, that became suspended when a process returned a wait status to the shell indicating that the process was stopped by a SIGSTOP, SIGTSTP, SIGTTIN, or SIGTTOU signal.

A suspended job is a job-control background job, but a job-control background job is not necessarily a suspended job. A non-job-control background job is never a suspended job, even if it includes processes that have been stopped by a SIGSTOP, SIGTSTP, SIGTTIN, or SIGTTOU signal.

<small>Note: See also [xref to 3.33 Background Job], [xref to 3.202 Job], and [xref to 3.182 Foreground Job].</small>

On page 2350 line 74877 section 2.5.2 Special Parameters (?), after applying bug 1309 change:
Expands to the shortest representation of the decimal exit status of the pipeline (see [xref to 2.9.2]) that most recently completed execution and was not executed in a subshell environment. The value shall be set to 0 during initialization of the shell.
to:
Expands to the shortest representation of the decimal exit status of the pipeline (see [xref to 2.9.2]) executed from the current shell execution environment (not a subshell environment) that most recently either terminated or, optionally but only if the shell is interactive and job control is enabled, was stopped by a signal. If this pipeline terminated, the status value shall be its exit status; otherwise, the status value shall be the same as the exit status that would have resulted if the pipeline had been terminated by a signal with the same number as the signal that stopped it. The value of the special parameter '?' shall be set to 0 during initialization of the shell.

On page 2350 line 74883 section 2.5.2 Special Parameters (!), after applying bug 1052, change:
Expands to the shortest representation of the decimal process ID of the most recent background command (see [xref to 2.9.3]) executed from the current shell. (For example, background commands executed from subshells do not affect the value of "$!" in the current shell environment.) For a pipeline, the process ID is that of the last command in the pipeline.
to:
Expands to the shortest representation of the decimal process ID associated with the most recent asynchronous AND-OR list (see [xref to 2.9.3.1]) executed from the current shell execution environment, or to the decimal process ID of the last command specified in the currently executing pipeline in the job-control background job that most recently resumed execution through the use of bg, whichever is the most recent.

On page 2369 line 75644 section 2.9.2 Pipelines, change:
If the pipeline is not in the background (see [xref to 2.9.3.1]), the shell shall wait ...
to:
If the pipeline is not in the background (see [xref to 2.9.3.1] and [xref to 2.11 Job Control]), the shell shall wait ...

On page 2370 line 75680 section 2.9.3.1 Asynchronous Lists, change:
Asynchronous Lists

If a command is terminated by the control operator <ampersand> ('&'), the shell shall execute the command asynchronously in a subshell. This means that the shell shall not wait for the command to finish before executing the next command.

The format for running a command in the background is:
command1 & [command2 & ... ]
If job control is disabled (see set, -m), the standard input for an asynchronous list, before any explicit redirections are performed, shall be considered to be assigned to a file that has the same properties as /dev/null. This shall not happen if job control is enabled. In all cases, explicit redirection of standard input shall override this activity.

When an element of an asynchronous list (the portion of the list ended by an <ampersand>, such as command1, above) is started by the shell, the process ID of the last command in the asynchronous list element shall become known in the current shell execution environment; see [xref to 2.12]. This process ID shall remain known until:

  1. The command terminates and the application waits for the process ID.

  2. Another asynchronous list is invoked before "$!" (corresponding to the previous asynchronous list) is expanded in the current execution environment.


The implementation need not retain more than the {CHILD_MAX} most recent entries in its list of known process IDs in the current shell execution environment.

Exit Status

The exit status of an asynchronous list shall be zero.
to:
Asynchronous AND-OR Lists

If an AND-OR list is terminated by the control operator <ampersand> ('&'), the shell shall execute the AND-OR list asynchronously in a subshell environment. This subshell shall execute in the background; that is, the shell shall not wait for the subshell to terminate before executing the next command (if any); if there are no further commands to execute, the shell shall not wait for the subshell to terminate before exiting.

If job control is enabled (see set, -m), the AND-OR list shall become a job-control background job and a job number shall be assigned to it. If job control is disabled, the AND-OR list may become a non-job-control background job, in which case a job number shall assigned to it; if no job number is assigned it shall become a background command but not a background job.

A job-control background job can be controlled as described in [xref to XCU 2.11 Job Control].

The process ID associated with the asynchronous AND-OR list shall become known in the current shell execution environment; see [xref to 2.12]. This process ID shall remain known until any one of the following occurs (and, unless otherwise specified, may continue to remain known after it occurs).

  • The process terminates and the application waits for the process ID or the corresponding job ID (see [xref to wait]).

  • If the asynchronous AND-OR list did not become a background job: another asynchronous AND-OR list is invoked before "$!" (corresponding to the previous asynchronous AND-OR list) is expanded in the current shell execution environment.

  • If the asynchronous AND-OR list became a background job: the jobs utility reports the termination status of that job.

  • If the shell is interactive and the asynchronous AND-OR list became a background job: a message indicating completion of the corresponding job is written to standard error. If set -b is enabled, it is unspecified whether the process ID is removed from the list of known process IDs when the message is written or immediately prior to when the shell writes the next prompt for input.
The implementation need not retain more than the {CHILD_MAX} most recent entries in its list of known process IDs in the current shell execution environment.

If, and only if, job control is disabled, the standard input for the subshell in which an asynchronous AND-OR list is executed shall initially be assigned to an open file description that behaves as if /dev/null had been opened for reading only. This initial assignment shall be overridden by any explicit redirection of standard input within the AND-OR list.

If the shell is interactive and the asynchronous AND-OR list became a background job, the job number and the process ID associated with the job shall be written to standard error using the format:
"[%d] %d\n", <job-number>, <process-id>
If the shell is interactive and the asynchronous AND-OR list did not become a background job, the process ID associated with the asynchronous AND-OR list shall be written to standard error in an unspecified format.

Exit Status

The exit status of an asynchronous AND-OR list shall be zero.

The exit status of the subshell in which the AND-OR list is asynchronously executed can be obtained using the wait utility.

On page 2369 line 75701 section 2.9.3.2 Sequential Lists, change:
Sequential Lists

Commands that are separated by a <semicolon> (';') shall be executed sequentially. The format for executing commands sequentially shall be:
command1 [; command2] ...
Each command shall be expanded and executed in the order specified.

Exit Status

The exit status of a sequential list shall be the exit status of the last command in the list.
to:
Sequential AND-OR Lists

AND-OR lists that are separated by a <semicolon> (';') shall be executed sequentially. The format for executing AND-OR lists sequentially shall be:
aolist1 [; aolist2] ...
Each AND-OR list shall be expanded and executed in the order specified.

If job control is enabled, the AND-OR lists shall form all or part of a foreground job that can be controlled as described in [xref to XCU 2.11 Job Control].

Exit Status

The exit status of a sequential AND-OR list shall be the exit status of the last pipeline in the AND-OR list that is executed.

On page 2381 line 76151 section 2, insert a new subsection:

2.11 Job Control
Job control is defined (see [xref to 3.203 Job Control]) as a facility that allows users selectively to stop (suspend) the execution of processes and continue (resume) their execution at a later point. It is jointly supplied by the terminal I/O driver and a command interpreter. The shell is one such command interpreter and job control in the shell is enabled by [xref to set] -m (which is enabled by default in interactive shells). The remainder of this section describes the job control facility provided by the shell. Requirements relating to background jobs stated in this section only apply to job-control background jobs.

If the shell has a controlling terminal and it is the controlling process for the terminal session, it shall initially set the foreground process group ID associated with the terminal to its own process group ID. Otherwise, if it has a controlling terminal, it shall initially perform the following steps if interactive and may perform them if non-interactive:

  1. If its process group is the foreground process group associated with the terminal, the shell shall set its process group ID to its process ID (if they are not already equal) and set the foreground process group ID associated with the terminal to its process group ID.

  2. If its process group is not the foreground process group associated with the terminal (which would result from it being started by a job-control shell as a background job), the shell shall either stop itself by sending itself a SIGTTIN signal or, if interactive, attempt to read from standard input (which will generate a SIGTTIN signal if standard input is the controlling terminal). If it is stopped, then when it continues execution (after receiving a SIGCONT signal) it shall repeat these steps.


Subsequently, the shell shall change the foreground process group associated with its controlling terminal when a foreground job is running as noted in the description below.

When job control is enabled, the shell shall create one or more jobs when it executes a list (see [xref to 2.9.3 Lists]) that has one of the following forms:

  • A single asynchronous AND-OR list

  • One or more sequentially executed AND-OR lists followed by at most one asynchronous AND-OR list


For the purposes of job control, a list that includes more than one asynchronous AND-OR list shall be treated as if it were split into multiple separate lists, each ending with an asynchronous AND-OR list.

When a job consisting of a single asynchronous AND-OR list is created, it shall form a background job and the associated process ID shall be that of a child process that is made a process group leader, with all other processes (if any) that the shell creates to execute the AND-OR list initially having this process ID as their process group ID.

For a list consisting of one or more sequentially executed AND-OR lists followed by at most one asynchronous AND-OR list, the whole list shall form a single foreground job up until the sequentially executed AND-OR lists have all completed execution, at which point the asynchronous AND-OR list (if any) shall form a background job as described above.

For each pipeline in a foreground job, if the pipeline is executed while the list is still a foreground job, the set of processes comprising the pipeline, and any processes descended from it, shall all be in the same process group, unless the shell executes some of the commands in the pipeline in the current shell execution environment and others in a subshell environment; in this case the process group ID of the current shell need not change (or cannot change if it is the session leader), and consequently the process group ID that the other processes all share may differ from the process group ID of the current shell (which means that a SIGSTOP, SIGTSTP, SIGTTIN, or SIGTTOU signal sent to one of those process groups does not cause the whole pipeline to stop).

A background job that was created on execution of an asynchronous AND-OR list can be brought into the foreground by means of the fg utility (if supported); in this case the entire job shall become a single foreground job. If a process that the shell subsequently waits for is part of this foreground job and is stopped by a signal, the entire job shall become a suspended job and the behavior shall be as if the process had been stopped while the job was running in the background.

When a foreground job is created, or a background job is brought into the foreground by the fg utility, if the shell has a controlling terminal it shall set the foreground process group ID associated with the terminal as follows:

  • If the job was originally created as a background job, the foreground process group ID shall be set to the process ID of the process that the shell made a process group leader when it executed the asynchronous AND-OR list.

  • If the job was originally created as a foreground job, the foreground process group ID shall be set as follows when each pipeline in the job is executed:

    • If the shell is not itself executing, in the current shell execution environment, all of the commands in the pipeline, the foreground process group ID shall be set to the process group ID that is shared by the other processes executing the pipeline (see above).

    • If all of the commands in the pipeline are being executed by the shell itself in the current shell execution environment, the foreground process group ID shall be set to the process group ID of the shell.


When a foreground job terminates, or becomes a suspended job (see below), if the shell has a controlling terminal it shall set the foreground process group ID associated with the terminal to the process group ID of the shell.

Each background job (whether suspended or not) shall have associated with it a job number and a process ID that is known in the current shell execution environment. When a background job is brought into the foreground by means of the fg utility, the associated job number shall be removed from the shell's background jobs list and the associated process ID shall be removed from the list of process IDs known in the current shell execution environment.

If a process that the shell is waiting for is part of a foreground job that was started as a foreground job and is stopped by a catchable signal (SIGTSTP, SIGTTIN, or SIGTTOU):

  • If the currently executing AND-OR list within the list comprising the foreground job consists of a single pipeline in which all of the commands are simple commands, the shell shall either create a suspended job consisting of at least that AND-OR list and the remaining (if any) AND-OR lists in the same list, or create a suspended job consisting of just that AND-OR list and discard the remaining (if any) AND-OR lists in the same list.

  • Otherwise, the shell shall create a suspended job consisting of a set of commands, from within the list comprising the foreground job, that is unspecified except that the set shall include at least the pipeline to which the stopped process belongs. Commands in the foreground job that have not already completed and are not included in the suspended job shall be discarded.


<small>Note:Although only a pipeline of simple commands is guaranteed to remain intact if started in the foreground and subsequently suspended, it is possible to ensure that a complex AND-OR list will remain intact when suspended by starting it in the background and immediately bringing it into the foreground. For example:
command1 && command2 | { command3 || command4; } & fg
</small>

If a process that the shell is waiting for is part of a foreground job that was started as a foreground job and is stopped by a SIGSTOP signal, the behavior shall be as described above for a catchable signal unless the shell was executing a built-in utility in the current shell execution environment when the SIGSTOP was delivered, resulting in the shell itself being stopped by the signal, in which case if the shell subsequently receives a SIGCONT signal and has one or more child processes that remain stopped, the shell shall create a suspended job as if only those child processes had been stopped.

When a suspended job is created as a result of a foreground job being stopped, it shall be assigned a job number, and an interactive shell shall write, and a non-interactive shell may write, a message to standard error, formatted as described by the jobs utility (without the -l option) for a suspended job. The message may indicate that the commands comprising the job include commands that have already completed; in this case the completed commands shall not be repeated if execution of the job is subsequently continued. If the shell is interactive, it shall save the terminal settings before changing them to the settings it needs to read further commands.

When a process associated with a background job is stopped by a SIGSTOP, SIGTSTP, SIGTTIN, or SIGTTOU signal, the shell shall convert the (non-suspended) background job into a suspended job and an interactive shell shall write a message to standard error, formatted as described by the jobs utility (without the -l option) for a suspended job, at the following time:

  • If set -b is enabled, the message shall be written either immediately after the job became suspended or immediately prior to writing the next prompt for input.

  • If set -b is disabled, the message shall be written immediately prior to writing the next prompt for input.


Execution of a suspended job can be continued as a foreground job by means of the fg utility (if supported), or as a (non-suspended) background job either by means of the bg utility (if supported) or by sending the stopped processes a SIGCONT signal. The fg and bg utilities shall send a SIGCONT signal to the process group of the process(es) whose stopped wait status caused the shell to suspend the job. If the shell has a controlling terminal, the fg utility shall send the SIGCONT signal after it has set the foreground process group ID associated with the terminal (see above). If the fg utility is used from an interactive shell to bring into the foreground a suspended job that was created from a foreground job, before it sends the SIGCONT signal the fg utility shall restore the terminal settings to the ones that the shell saved when the job was suspended.

When a background job completes or is terminated by a signal, an interactive shell shall write a message to standard error, formatted as described by the jobs utility (without the -l option) for a job that completed or was terminated by a signal, respectively, at the following time:

  • If set -b is enabled, the message shall be written immediately after the job becomes suspended.

  • If set -b is disabled, the message shall be written immediately prior to writing the next prompt for input.


In each case above where an interactive shell writes a message immediately prior to writing the next prompt for input, the same message may also be written by a non-interactive shell, at any of the following times:

  • After the next time a foreground job terminates or is suspended

  • Before the shell parses further input

  • Before the shell exits

and renumber the later subsections.

On page 2381 line 76152 section 2.11 Signals and Error Handling, change:
asynchronous list
to:
asynchronous AND-OR list

On page 2381 line 76177 section 2.12 Shell Execution Environment, change:
Process IDs of the last commands in asynchronous lists known to this shell environment
to:
Background jobs and their associated process IDs, and process IDs of child processes created to execute asynchronous AND-OR lists while job control is disabled; together these process IDs constitute the process IDs ``known to this shell environment''. If the implementation supports non-job-control background jobs, the list of known process IDs and the list of background jobs may form a single list even though this standard describes them as being updated separately.

On page 2382 line 76198 section 2.12 Shell Execution Environment, change:
asynchronous lists
to:
asynchronous AND-OR lists

On page 2409 line 77104 section 2.14 set (-b), change:
It shall cause the shell to notify the user asynchronously of background job completions. The following message is written to standard error:

[...]

When the shell notifies the user a job has been completed, it may remove the job's process ID from the list of those known in the current shell execution environment; see [xref to 2.9.3.1 Asynchronous Lists]. Asynchronous notification shall not be enabled by default.
to:
When job control and -b are both enabled, the shell shall write asynchronous notifications of background job completions (including termination by a signal), and may write asynchronous notifications of background job suspensions. See [xref to 2.11 Job Control] for details. When job control is disabled, the -b option shall have no effect. Asynchronous notification shall not be enabled by default.

On page 2410 line 77149 section 2.14 set (-m), change:
This option shall be supported if the implementation supports the User Portability Utilities option. All jobs shall be run in their own process groups. Immediately before the shell issues a prompt after completion of the background job, a message reporting the exit status of the background job shall be written to standard error. If a foreground job stops, the shell shall write a message to standard error to that effect, formatted as described by the jobs utility. In addition, if a job changes status other than exiting (for example, if it stops for input or output or is stopped by a SIGSTOP signal), the shell shall write a similar message immediately prior to writing the next prompt. This option is enabled by default for interactive shells.
to:
This option shall be supported if the implementation supports the User Portability Utilities option. When this option is enabled, the shell shall perform job control actions as described in [xref to 2.11 Job Control]. This option shall be enabled by default for interactive shells.

On page 2539 line 82126 section bg, change:
If job control is enabled (see the description of set -m), the bg utility shall resume suspended jobs from the current environment (see [xref to 2.12]) by running them as background jobs.
to:
If job control is enabled (see the description of set -m), the shell is interactive, and the current shell execution environment (see [xref to 2.12]) is not a subshell environment, the bg utility shall resume suspended jobs from the current shell execution environment by running them as background jobs, as described in [xref to 2.11 Job Control]; it may also do so if the shell is non-interactive or the current shell execution environment is a subshell environment.

On page 2539 line 82130 section bg, delete:
Using bg to place a job into the background shall cause its process ID to become ``known in the current shell execution environment'', as if it had been started as an asynchronous list; see [xref to 2.9.3.1].

After page 2541 line 82206 section bg, add a new paragraph:
The bg and fg utilities are not symmetric as regards the list of process IDs known in the current shell execution environment. Whereas fg removes a process ID from this list, bg has no need to add one to this list when it resumes execution of a suspended job in the background, because this has already been done by the shell when the suspended background job was created (see [xref to 2.11 Job Control]).

On page 2786 line 91465 section fg, change:
If job control is enabled (see the description of set -m), the fg utility shall move a background job from the current environment (see [xref to 2.12]) into the foreground.
to:
If job control is enabled (see the description of set -m), the shell is interactive, and the current shell execution environment (see [xref to 2.12]) is not a subshell environment, the fg utility shall move a background job in the current execution environment into the foreground, as described in [xref to 2.11 Job Control]; it may also do so if the shell is non-interactive or the current shell execution environment is a subshell environment.

On page 2787 line 91507 section fg (EXIT STATUS), change:
The following exit values shall be returned:

0 Successful completion.

>0 An error occurred.
to:
If the fg utility succeeds, it does not return an exit status. Instead, the shell waits for the job that fg moved into the foreground.

If fg does not move a job into the foreground, the following exit values shall be returned:

>0 An error occurred.

On page 2870 line 94571 section jobs, change:
in the current session
to:
in the current shell execution environment

On page 2870 line 94575 section jobs, change:
The jobs utility shall display the status of jobs that were started in the current shell environment; see [xref to 2.12].

When jobs reports the termination status of a job, the shell shall remove its process ID from ...
to:
If the current shell execution environment (see [xref to 2.12]) is not a subshell environment, the jobs utility shall display the status of background jobs that were created in the current shell execution environment; it may also do so if the current shell execution environment is a subshell environment.

When jobs reports the termination status of a job, the shell shall remove the job from the background jobs list and the associated process ID from ...

On page 2870 line 94583 section jobs (-l), change:
This information shall include the job number, current job, process group ID, state, and the command that formed the job.
to:
See STDOUT for details.

On page 2870 line 94586 section jobs (-p), change:
Display only the process IDs for the process group leaders of the selected jobs.
to:
Display only the process IDs for the process group leaders of job-control background jobs and the process IDs associated with non-job-control background jobs (if supported).

On page 2870 line 94587 section jobs, change:
shall display the status of all stopped jobs, running background jobs and all jobs whose status has changed
to:
shall display the status of all background jobs, both running and suspended, and all jobs whose status has changed

On page 2871 line 94629 section jobs, change:
process group
to:
job

On page 2871 line 94652 section jobs, change:
If the -l option is specified, a field containing the process group ID shall be inserted before the <state> field. Also, more processes in a process group may be output on separate lines, using only the process ID and <command> fields.
to:
If the -l option is specified:

  • For job-control background jobs, a field containing the process group ID shall be inserted before the <state> field. Also, more processes in a process group may be output on separate lines, using only the process ID and <command> fields.

  • For non-job-control background jobs (if supported), a field containing the process ID associated with the job shall be inserted before the <state> field. Also, more processes created to execute the job may be output on separate lines, using only the process ID and <command> fields.

On page 2872 line 94668 section jobs, change:
the process group of a job
to:
the process group of a job-control background job

On page 2872 line 94687 section jobs, change:
The jobs utility is not dependent on the job control option, as are the seemingly related bg and fg utilities because jobs is useful for examining background jobs, regardless of the condition of job control. When the user has invoked a set +m command and job control has been turned off, jobs can still be used to examine the background jobs associated with that current session. Similarly, kill can then be used to kill background jobs with kill %<background job number>.
to:
The jobs utility is not dependent on job control being enabled, as are the seemingly related bg and fg utilities, because jobs is useful for examining background jobs, regardless of the current state of job control. When job control has been disabled using set +m, jobs can still be used to examine the job-control background jobs and (if supported) non-job-control background jobs that were created in the current shell execution environment. See also the RATIONALE for kill() and wait().

On page 2879 line 94945 section kill, change:
... process that was terminated by a signal, the signal_name corresponding to the signal that terminated the process shall be written.
to:
... process that was terminated or stopped by a signal, the signal_name corresponding to the signal that terminated or stopped the process shall be written.

On page 2880 line 94987 section kill, change:
A job control job ID (see [xref to XBD Section 3.204]) that identifies a background process group to be signaled. The job control job ID notation is applicable only for invocations of kill in the current shell execution environment; see [xref to 2.12].
to:
A job ID (see [xref to XBD Section 3.204]) that identifies a process group in the case of a job-control background job, or a process ID in the case of a non-job-control background job (if supported), to be signaled. The job ID notation is applicable only for invocations of kill in the current shell execution environment; see [xref to 2.12].

<small>Note: The job ID type of pid is only available on systems supporting the User Portability Utilities option or supporting non-job-control background jobs.</small>

After page 2883 line 95095 section kill, add a new paragraph:
The use of job ID notation is not dependent on job control being enabled. When job control has been disabled using set +m, kill can still be used to signal the process group associated with a job-control background job, or the process ID associated with a non-control background job (if supported), using kill %<background job number> See also the RATIONALE for jobs() and wait().

On page 3430 line 115763 section wait, change:
When an asynchronous list (see [xref to 2.9.3.1) is started by the shell, the process ID of the last command in each element of the asynchronous list shall become known in the current shell execution environment; see [xref to 2.12].
to:
The wait utility shall wait for one or more child processes whose process IDs are known in the current shell execution environment (see [xref to 2.12]) to terminate.

On page 3430 line 115773 section wait, change:
The known process IDs are applicable only for invocations of wait in the current shell execution environment.
to:
Once a process ID that is known in the current shell execution environment (see [xref to 2.12]) has been successfully waited for, it shall be removed from the list of process IDs that are known in the current shell execution environment. If the process ID is associated with a background job, the corresponding job shall also be removed from the list of background jobs.

On page 3430 line 115780 section wait, change:
The unsigned decimal integer process ID of a command, for which the utility is to wait for the termination.
to:
The unsigned decimal integer process ID of a child process whose termination the utility is to wait for.

On page 3430 line 115782 section wait, change:
A job control job ID (see [xref to XBD Section 3.204]) that identifies a background process group to be waited for. The job control job ID notation is applicable only for invocations of wait in the current shell execution environment; see [xref to 2.12]. The exit status of wait shall be determined by the last command in the pipeline.

<small>Note: The job control job ID type of pid is only available on systems supporting the User Portability Utilities option.</small>
to:
A job ID (see [xref to XBD Section 3.204]) that identifies a process group in the case of a job-control background job, or a process ID in the case of a non-job-control background job (if supported), to be waited for. The job ID notation is applicable only for invocations of wait in the current shell execution environment; see [xref to 2.12]. The exit status of wait shall be determined by the exit status of the last pipeline to be executed.

<small>Note: The job ID type of pid is only available on systems supporting the User Portability Utilities option or supporting non-job-control background jobs.</small>

On page 3431 line 115818 section wait, change:
If one or more operands were specified, all of them have terminated or were not known by the invoking shell, and the status of the last operand specified is known, then the exit status of wait shall be the exit status information of the command indicated by the last operand specified.
to:
If one or more operands were specified, all of them have terminated or were not known in the invoking shell execution environment, and the status of the last operand specified is known, then the exit status of wait shall be the status of the last operand specified.

On page 3431 line 115828 section wait, change:
The command identified by the last pid operand specified is unknown.
to:
The process ID specified by the last pid operand is not known in the invoking shell execution environment.

After page 3431 line 115838 section wait, add a new paragraph:
The use of job ID notation is not dependent on job control being enabled. When job control has been disabled using set +m, wait can still be used to wait for the process group associated with a job-control background job, or the process ID associated with a non-control background job (if supported), using wait %<background job number> See also the RATIONALE for jobs() and kill().

On page 3431 line 115839-115853 section wait, delete the text that begins with:
Historical implementations of interactive shells have discarded ...

and ends with:
... work without losing status on any of the jobs.

On page 3432,3433 line 115857,115858,115859,115863,115896 section wait, change:
asynchronous
to:
asynchronous AND-OR

On page 3432 line 115868 section wait, add a new paragraph:
Some historical shells returned from wait when a process stops instead of only when it terminates. This standard does not allow wait to return when a process stops for two reasons:

  1. The vast majority, if not all, shell scripts that use wait (without using an extension) expect it not to return until the process terminates.

  2. It is not possible to write a portable shell script that can correctly handle wait returning when a process stops, because an exit status indicating a process was stopped by a signal cannot be distinguished from one indicating that the process called exit() with the same value.
The standard developers considered allowing interactive shells to return from wait when a process stops, since the interactive user would see a message which would allow them to tell whether the process stopped or terminated. However, they decided that it would be inadvisable to introduce an inconsistency between interactive and non-interactive shells, particularly as the most likely use of wait in an interactive shell is to try out commands before putting them in a shell script. Implementations can provide an extension that could be used to request that wait returns when a process stops. It is recommended that any such extension uses a different method of returning information about the wait status of the process so that the information can be unambiguous. One suitable method would be an option that takes a variable name as an option-argument. The named variable would be set to a numeric value and the exit status of wait would indicate whether this value is an exit value or a signal number, and whether the signal terminated the process or stopped it. Such an extension would also provide a way for shell scripts to obtain the full exit value (as would be returned by waitid()).

On page 3433 line 115904 section wait, change FUTURE DIRECTIONS from:
None.
to:
A future version of this standard may add an option which takes a variable name as an option-argument, allowing wait to return information about the wait status of a process in an unambiguous way.

On page 3494 line 118117 section A.3 Definitions (Job Control), change:
which causes the foreground job to stop and the shell to begin prompting for new commands. The stopped job can be ...
to:
which causes the foreground process group to stop, and the shell to convert the corresponding foreground job to a suspended job and begin prompting for new commands. The suspended job can be ...

On page 3494 line 118121 section A.3 Definitions (Job Control), change:
If a background job attempts to access the login terminal (controlling terminal), it is stopped by the terminal driver and the shell is notified, which, in turn, notifies the user.
to:
If a background process group attempts to access the login terminal (controlling terminal), it is stopped by the terminal driver and the shell detects this and, in turn, suspends the corresponding background job and notifies the user.

On page 3494 line 118124 section A.3 Definitions (Job Control), change:
can continue the stopped job in the foreground
to:
can continue the suspended job in the foreground

On page 3494 line 118128 section A.3 Definitions (Job Control), change:
The interactive interface described previously can be accomplished using the POSIX.1 job control facilities in the following way.
to:
The job control features of the POSIX shell (described in [xref to XCU 2.11 Job Control) and of other shells can be implemented using the job control facilities of the System Interfaces volume of POSIX.1-202x in the following way.

On page 3494 line 118134 section A.3 Definitions (Job Control), change:
The shell places each job in a separate process group via the setpgid() function.
to:
The shell places the process(es) it creates for each job in a separate process group via the setpgid() function.

On page 3495 line 118144 section A.3 Definitions (Job Control), change:
all processes in the pipeline (job)
to:
all processes in the job

On page 3495 line 118146 section A.3 Definitions (Job Control), delete:
The shell itself is considered to be a job and is the sole process in its own process group.

On page 3495 line 118154 section A.3 Definitions (Job Control), change:
The shell, in turn, informs the terminal I/O driver via the tcsetpgrp() function. This indicates to the terminal I/O driver the process group ID of the foreground process group (job). When the current foreground job either stops or terminates, the shell places itself in the foreground via tcsetpgrp() before prompting for additional commands. Note that when a job is created the new process group begins as a background process group. It requires an explicit act of the shell via tcsetpgrp() to move a process group (job) into the foreground.
to:
The shell, in turn, informs the terminal I/O driver via the tcsetpgrp() function. This indicates to the terminal I/O driver the process group ID of the foreground process group. When the current foreground job either stops or terminates, the shell places its own process group in the foreground via tcsetpgrp() before prompting for additional commands. Note that when a job is created the new process group begins as a background process group. It requires an explicit act of the shell via tcsetpgrp() to move a process group into the foreground.

On page 3495 line 118167 section A.3 Definitions (Job Control), change:
To continue a stopped job, the shell sends the SIGCONT signal to the process group of the job. In addition, if the job is being continued in the foreground, the shell invokes tcsetpgrp() to place the job in the foreground before sending SIGCONT.
to:
To continue a suspended job, the shell sends a SIGCONT signal to the corresponding process group. In addition, if the job is being continued in the foreground, the shell invokes tcsetpgrp() to place the process group in the foreground before sending SIGCONT.

On page 3495 line 118175 section A.3 Definitions (Job Control), change:
Note that the terms ``job'' and ``process group'' can be used interchangeably. A login session that is not using the job control facilities can be thought of as a large collection of processes that are all in the same job (process group). Such a login session may have a partial distinction between foreground and background processes; that is, the shell may choose to wait for some processes before continuing to read new commands and may not wait for other processes. However, the terminal I/O driver will consider all these processes to be in the foreground since they are all members of the same process group.
to:
A login session that is not using the job control facilities can be thought of as a large collection of processes that are all in the same job. Such a login session may have a partial distinction between foreground and background processes; that is, the shell waits for some processes before continuing to read new commands and does not wait for other processes. However, the terminal I/O driver considers all these processes to be in the foreground since they are all members of the same process group.

On page 3495 line 118184 section A.3 Definitions (Job Control), change:
When a foreground (not background) job stops, the shell must sample and remember the current terminal settings so that it can restore them later when it continues the stopped job in the foreground
to:
When a foreground (not background) job is suspended, the shell needs to sample and remember the current terminal settings so that it can restore them later when it continues the suspended job in the foreground

On page 3495 line 118187 section A.3 Definitions (Job Control), change:
Because a shell itself can be spawned from a shell, it must take special action to ensure that subshells interact well with their parent shells.

A subshell can be spawned to perform an interactive function (prompting the terminal for commands) or a non-interactive function (reading commands from a file). When operating non-interactively, the job control shell will refrain from performing the job control-specific actions described above.
to:
Because a shell itself can be spawned from a shell, it must take special action to ensure that child shells interact well with their parent shells. A child shell can be spawned to perform an interactive function (prompting the terminal for commands) or a non-interactive function (reading commands from a file). When operating non-interactively, the job control shell will by default refrain from performing the job control-specific actions described above.

On page 3496 line 118196 section A.3 Definitions (Job Control), change:
An interactive subshell can be spawned from another job control-cognizant shell in either the foreground or background. (For example, from the C Shell, the user can execute the command, csh &.) Before the subshell activates job control ...
to:
An interactive child shell can be spawned from another job control-cognizant shell in either the foreground or background. (For example, the user can execute an interactive shell in the background by means of the command "sh &".) Before the child shell activates job control ...

On page 3742 line 128447 section C.2.9.3 Lists, change:
Asynchronous Lists

The grammar treats a construct such as:
foo & bar & bam &
as one ``asynchronous list'', but since the status of each element is tracked by the shell, the term ``element of an asynchronous list'' was introduced to identify just one of the foo, bar, or bam portions of the overall list.
to:
Asynchronous AND-OR Lists

On page 3742 line 128458 section C.2.9.3 Lists, change:
asynchronous lists
to:
asynchronous AND-OR lists

On page 3742 line 128466 section C.2.9.3 Lists, change:
Sequential Lists
to:
Sequential AND-OR Lists

On page 3747 line 128643 section C.2, add a new subsection:

C.2.11 Job Control
See also [xref to Job Control subsection of A.3 Definitions].

Shell implementations differ regarding how much of a foreground job is retained when it is converted to a suspended job. For example, given this foreground job:
sleep 10; echo foo; echo bar &
if this is suspended during execution of the sleep, ksh93 retains all of the commands in the suspended job and executes them when fg is used:
^Z[1] + Stopped                  sleep 10; echo foo; echo bar &
$ jobs
[1] + Stopped                  sleep 10; echo foo; echo bar &
$ fg
sleep 10; echo foo; echo bar 
foo
[1]     30686
bar
$
However, some other shells create a suspended job containing only the <tt>sleep 10</tt> command.

Some historical shells did not handle suspending a foreground AND-OR list well. They would treat the wait status of a process that indicated it had stopped as if it was a non-zero exit status and (if the next operator in the AND-OR list was ||) would execute the remainder of the AND-OR list at that point. This behavior is not allowed by the standard for two reasons:

  1. It does not meet the fundamental requirement of an AND-OR list that the decision on whether to execute each part (except the first) is made based on the exit status of the previous part when it completes.

  2. It can lead to data loss. For example, consider a user who often runs this command:
    generate_report > report.out || rm report.out
    with the intention that the incomplete results from a failed generate_report run are never retained in order that they can't be mistaken for a complete set of results. If one day the user decides to check on the progress of the command by stopping it and examining what has been written so far, they will find that the report.out file has already been removed.
and renumber the later subsections.

(0005888)
geoffclare (manager)
2022-07-14 15:50

Interpretation response
------------------------
The standard is unclear on this issue, and no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor.

Rationale:
-------------
Various aspects of asynchronous lists and job control are insufficiently specified in the standard.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
Make the changes in Note: 0005845
(0005894)
agadmin (administrator)
2022-07-18 15:18

Interpretation proposed: 18 July 2022
(0005958)
agadmin (administrator)
2022-09-01 16:25

Interpretation Approved: 1 September 2022

- Issue History
Date Modified Username Field Change
2019-06-08 09:14 stephane New Issue
2019-06-08 09:14 stephane Name => Stephane Chazelas
2019-06-08 09:14 stephane Section => 2.9.3.1 Asynchronous Lists
2019-06-08 09:14 stephane Page Number => 2370 (in 2018 edition, not 2016)
2019-06-08 09:14 stephane Line Number => the whole 2.9.3.1 section and also 76177 (execution environment), 115763 (wait), 128448 (rationale), 74883 ($!)
2019-06-10 18:34 stephane Note Added: 0004416
2019-07-15 15:38 Don Cragun Relationship added related to 0000760
2022-06-06 09:48 geoffclare Note Added: 0005845
2022-06-06 09:51 geoffclare Note Edited: 0005845
2022-07-14 15:50 geoffclare Note Added: 0005888
2022-07-14 15:51 geoffclare Interp Status => Pending
2022-07-14 15:51 geoffclare Final Accepted Text => Note: 0005888
2022-07-14 15:51 geoffclare Status New => Interpretation Required
2022-07-14 15:51 geoffclare Resolution Open => Accepted As Marked
2022-07-14 15:51 geoffclare Tag Attached: issue8
2022-07-18 15:18 agadmin Interp Status Pending => Proposed
2022-07-18 15:18 agadmin Note Added: 0005894
2022-09-01 16:25 agadmin Interp Status Proposed => Approved
2022-09-01 16:25 agadmin Note Added: 0005958
2022-09-27 15:29 geoffclare Status Interpretation Required => Applied
2022-10-31 16:39 geoffclare Relationship added has duplicate 0001611
2023-04-20 10:57 geoffclare Relationship added related to 0001675


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker