View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
00016001003.1(2016/18)/Issue7+TC2Rationalepublic2022-08-23 09:282024-06-11 09:07
Reportergeoffclare Assigned To 
PrioritynormalSeverityCommentTypeError
Status ClosedResolutionAccepted As Marked 
NameGeoff Clare
OrganizationThe Open Group
User Reference
SectionC.1.7 Built-In Utilities
Page Number3716
Line Number127366
Interp Status---
Final Accepted TextSee 0001600:0006015.
Summary0001600: Problems with the suggested single-line shell script: $0 "$@"
DescriptionOn the mailing list, Jakub Wilk reported a problem with the rationale in C.1.7 that talks about implementing the 16 (sic) regular built-ins as a single-line shell script:
$0 "$@"

As he pointed out, "This wouldn't actually work: $0 is the absolute path, so the script would recurse indefinitely."

Another problem is that these external utilities are supposed to be exec-able using "the exec family of functions", which means all six functions not just execlp() and execvp(), so the script needs a #! line (and can only meet the requirements on systems that support #!).

Finally, the number 16 is out of date, as there are now 20 utilities in the table in XCU 1.7.
Desired ActionChange:
However, it has been pointed out that all 16 utilities in the table can be provided with 16 links to a single-line shell script:
$0 "$@"
to:
However, it has been pointed out that, on systems that support the (non-standard but ubiquitous) "#!" mechanism to make scripts executable by the exec family of functions, all of the utilities in the table can be provided as links to a two-line shell script:
#! /path/to/sh
${0##*/} "$@"

Tagsissue8

Activities

kre

2022-08-23 14:32

reporter   bugnote:0005940

Last edited: 2022-08-23 14:36

 I will start with the completely irrelevant, and just point out that
this bug is filed against Issue7 TC2, in which there is no XCU 1.7 to
have a table to have grown more entries... Further in the drafts
for Issue 8, in which there is an XCU 1.7, which does have a table
showing a list of built in utilities, at least in D2.1, that table has
16 entries, not 20, but has absolutely nothing to do with the subject
of this issue.

That said, I agree, it is not a good idea to specify a specific number
of utilities here, XCU 1.6 (both issues) where the requirement is specified
doesn't say how many, it just says everything except the special built-in
utilities, so removing the explicit number from XRAT is a wise decision.

However, it while giving an example using #! /..../sh as a means to
implement this requirement provides one means to satisfy the requirements,
the standard doesn't mandate #!, and it might be possible, perhaps, for
there to be a system somewhere which doesn't support it. As best I can work
out, the only other ways that the applicable utilities (or most of them)
can be implemented is to make them links to the shell, (or some shell)
and have that shell use argv[0] to decide what to do. That is, if
argv[0] == "cd" (assuming C had a comparison operator that worked on strings
that way) then simply execute its built in cd command. Or, to write
a simple C program, which amounts to just

     main(int argc, char **argv)
     {
          char *command = NULL;
          size_t len = 1;

          while (argc-- >= 0) {
              command = realloc(command, (len += strlen(*argv) + 3));
              strcat(command, " '");
              strcat(command, *argv++);
              strcat(command "'");
          }
          execl("/path/to/sh", "-c", "--", command, NULL);
      }

Of course, with error checking, and the quoting dealing with the possibility
that one (or more) of the args might contain ' characters (that's all just
programming, not design). That or something functionally similar.

Assuming that is correct (I shall explain why just below) then XRAT probably
should also give examples that show implementing it those ways, or at least
mention them as an implementation technique.

In the first case (links to sh which then uses argv[0]) it would also need to
explain what to do if the shell was given shell type arguments along with
an argv[0] which refers to a built in command, which has precedence, and if it
is to be argv[0] what happens when it is neither "sh" nor one of the
built in commands (which I'd assume would just be unspecified).

The reason for these (or perhaps something else very much the same I'm unable
to think of at the minute) is that to implement (at least most of) the
normally built in utilities (as opposed to things like test, echo, printf and
pwd, which the shell often has built in just because they are used so much
in scripts, but which always also exist as filesystem commands) requires
that they be running in a shell environment. There are a few exceptions,
but they are rare.

There is also kill which is also always available as an external command,
though it also needs to be built into the shell to perform all its functions.

It isn't just to do what they are supposed to do which makes this
a requirement, the standard actually mandates it.

Eg: in XCU 2.14/alias (one of the affected built in commands), we see:

   An alias definition shall affect the current shell execution environment

"shall" so it is mandatory, and to affect the current shell execution
environment, there must be a current shell execution environment to affect.

And bg:

    If job control is enabled (see the description of set -m), the bg
    utility shall resume suspended jobs from the current environment

Of course, there will be no jobs to resume, but there must be a current
environment to not find any in, and that environment must have a "set -m"
which can enable job control. Because there will no jobs to resume,
the later wording, which is even more explicit, probably doesn't apply.

cd:
    The cd utility shall change the working directory of the current shell
    execution environment

command:
    The command utility shall cause the shell

fc:
    The fc utility shall list, or shall edit and re-execute, commands
    previously entered to an interactive sh.

Then fg, which is, unsurprisingly, very similar to bg

getopts:
    Each time it is invoked, the getopts utility shall place the value
    of the next option in the shell variable

hash:
    The hash utility shall affect the way the current shell environment

jobs:
    The jobs utility shall display the status of jobs that were started
    in the current shell environment;

kill is mentioned above.

read:
    The read utility shall read a single logical line from standard input
    into one or more shell variables.

type is the very rare case of a built in command, not always implemented
as a filesystem command, which actually could be, without there being a shell.
Its results would be meaningless however, as how a command name is interpreted
depends upon the environment of the shell which is interpreting it (what
is a built in command in one shell might be an alias in another, and a file
system command in a third, and defined as a function in a fourth - a shell
agnostic type command cannot possibly do more than test whether the name exists
in PATH, which while not completely useless, isn't of much benefit either.

ulimit is another rare case, perhaps the one normally built in command
(required to be built in, it is one of the intrinsic commands which the table
in XCU 1.7 lists) which it is possible to implement properly outside a
shell (the options which change limits affect only the ulimit process itself,
so they're useless, but the limits can be reported).

umask:
    The umask utility shall set the file mode creation mask of the current
    shell execution environment

unalias:
    The aliases shall be removed from the current shell execution environment;

wait:
    If the wait utility is invoked with no operands, it shall wait until all
    process IDs known to the invoking shell

That's 16. To get to 20 we'd need to add some of (not sure which
were the blessed extra 4) true false printf pwd echo (and perhaps other
commands which are sometime built in - basename dirname expr). Perhaps
Geoff will list the (currently) 20 utilities he believes the requirement
to apply to (here, not in the standard).

The magic 16 happen (not by coincidence I expect)_ to be exactly the ones
required to be intrinsic in Draft2.1 XCU 1.7 table 1-5. And aside from kill
(which is always avalable as a filesystem command), and perhaps ulimit,
and type - all required a shell environment in order to be implemented.

Aside from kill (which I agree should remain implemented as a filesystem
coimmand) it really makes no sense at all to implement the others that way,
and it is fanciful (even absurd) to imagine scripts running "nohup bg" or
"find ... -exec cd {} \;" or anything else like that, in any productive way
(as opposed to simply testing whether or not they can).

So, please, can we change the wording of XCU 1.6 so that it exempts the
utilities required to be intrinsic by the standard, other than kill,
from the requirement that they be implemented so as to be able to be
exec()'d ?

There's no point to it, and they almost all require a shell.

Other standard utilities built into a shell (whether declared by the
implementation to be intrinsic or not) can remain required to be available
via exec (and always are I believe). The relevance of this to this defect
report, is that if the standard were to be changed this way, there is no
need for any of this in XRAT at all. No-one needs to be told how to
implement printf, echo, true (even kill) any more than they need to be told
how to implement awk or cut or ls. No need to attempt to justify the
(really) unjustifiable, and no need to give implementation examples to show
how it might be done (to no-one's benefit) Just remove the lot.

geoffclare

2022-10-27 15:17

manager   bugnote:0006015

Change:
However, all of the standard utilities, including the regular built-ins but not the special built-ins described in [xref to 2.14], shall be implemented in a manner ...
to:
However, all of the standard utilities other than:

  • The special built-ins described in [xref to 2.14]
  • The intrinsic utilities named in [xref to Table 1-5], except for kill
shall be implemented, regardless of whether they are also implemented as regular built-ins, in a manner ...


On page 3615 line 124885-124906 section C.1.7, change:
All of the regular built-in utilities [...] These arguments were ultimately the most effective.
to:
Earlier versions of this standard required that all of the regular built-in utilities, including intrinsic utilities, could be exec-ed. This was always a contentious requirement, and with the introduction of intrinsic utilities the standard developers decided to exempt the utilities that this standard requires to be intrinsic, with the exception of kill. The kill utility is still genuinely useful when exec-ed, only lacking support for the % job ID notation, whereas examples given of uses for the other utilities that are now exempted were considered contrived (such as using cd to test accessibility of a directory, which can be done using test -x). If an application needs exec-able versions of some of the exempted intrinsic utilities, it can easily provide them itself, on systems that support the (non-standard but ubiquitous) "#!" mechanism to make scripts executable by the exec family of functions, as links to a two-line shell script:
#! /path/to/sh
${0##*/} "$@"

kre

2022-10-28 04:37

reporter   bugnote:0006016

Thanks for that resolution, it should come as no surprise
that I support it completely.

Issue History

Date Modified Username Field Change
2022-08-23 09:28 geoffclare New Issue
2022-08-23 09:28 geoffclare Name => Geoff Clare
2022-08-23 09:28 geoffclare Organization => The Open Group
2022-08-23 09:28 geoffclare Section => C.1.7 Built-In Utilities
2022-08-23 09:28 geoffclare Page Number => 3716
2022-08-23 09:28 geoffclare Line Number => 127366
2022-08-23 09:28 geoffclare Interp Status => ---
2022-08-23 14:32 kre Note Added: 0005940
2022-08-23 14:36 kre Note Edited: 0005940
2022-10-27 15:17 geoffclare Note Added: 0006015
2022-10-27 15:19 Don Cragun Final Accepted Text => See 0001600:0006015.
2022-10-27 15:19 Don Cragun Status New => Resolved
2022-10-27 15:19 Don Cragun Resolution Open => Accepted As Marked
2022-10-27 15:19 Don Cragun Tag Attached: issue8
2022-10-28 04:37 kre Note Added: 0006016
2022-11-08 14:41 geoffclare Status Resolved => Applied
2024-06-11 09:07 agadmin Status Applied => Closed