Austin Group Defect Tracker

Aardvark Mark IV


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0000465 [1003.1(2008)/Issue 7] Shell and Utilities Comment Clarification Requested 2011-06-18 11:01 2013-11-05 11:40
Reporter jrn View Status public  
Assigned To ajosey
Priority normal Resolution Open  
Status Under Review  
Name Jonathan Nieder
Organization
User Reference jrn.special
Section 2.14 Special Built-In Utilities
Page Number 2334
Line Number 73818-73820
Interp Status ---
Final Accepted Text
Summary 0000465: is the list of special built-ins exhaustive (is "local" special)?
Description After 0000383, names of special built-ins are not valid function names, so application authors could benefit from knowing which utilities could or could not be special built-ins.

Lines 73818-73820 say: "An implementation may choose to make any utility a built-in; however, the special built-in utilities described here differ from regular built-in utilities".

That might suggest that the intent is to make the properties of special built-ins apply only to the functions in this section (break, :, continue, ., eval, exec, exit, export, readonly, return, set, shift, times, trap, unset), but most shells treat some additional built-ins as special. In particular:

bash makes "source" a synonym for ".", and in particular it is special. (By contrast, ksh93 makes "source" a built-in alias for "command .".)

ksh93 makes alias, enum, hash, login, newgrp, typeset, unalias and various built-in aliases special. ("times" seems to be a non-special alias, by the way.)

pdksh and its descendants let assignments before "typeset" and its alias "local" apply to the current environment but does not treat those two commands as special for the sake of the command search.

The examples that prompted this were scripts like texi2dvi that use code like this:

 if [ "$SOME_CONDITION" ]; then local () { : ... fallback implementation ...; }; fi

which triggers a parse error ("Bad function name") in dash if local is a special builtin.
Desired Action After line 73827, add: "All built-in utilities not described here shall be regular built-ins."

--- or ---

On line 73123, change "name of a special built-in utility" to "name of a special built-in utility described in section 2.14"

On line 73820, remove "described here"

After line 73829, add: "An implementation may provide additional special built-in commands from the following table. Any utility that is neither described in this section nor listed in the following table shall not be a special built-in.

    alias, autoload, compound, enum, float, functions,
    hash, integer, local, login, nameref, newgrp, source,
    typeset, unalias
"
Tags No tags attached.
Attached Files

- Relationships
related to 0000767Under Reviewajosey Add built-in "local" 
related to 0000383Closedajosey function with the same name as a special builtin 

-  Notes
(0000863)
Don Cragun (manager)
2011-06-18 17:57

Check out the rationale in the second paragraph of the discussion in XRAT section
C.2.9.5 (P 3668, L124817-124829) when considering how to handle this issue.
The last sentence of that section: "Implementations should reserve the identifier
local (as well as typeset, as used in the KornShell) in case this local variable
mechanism is adopted in a future version of this standard." gives some insight
into what the authors of the standard intended.

Note also that XCU section 1.6 (P2296, L72314-72318) already makes it clear
that no standard utilities (other than those described in XCU section 2.14) are
allowed to be implemented as special built-ins.
(0000919)
nick (manager)
2011-08-04 16:15
edited on: 2011-08-04 16:16

This was discussed on the conference call, 2011-08-04. We agreed that the problem was a serious one, but no consensus was reached with respect to the proposed actions.

One possibility that was discussed was using some form of reserved namespace for application writers to use to avoid such problems, such as the use of uppercase letters or underscores in any function name.

(0000969)
nick (manager)
2011-09-22 15:37

For your information, this is input on Bug 465 from David Korn

Begin forwarded message:

> From: David Korn
> Date: 13 September 2011 19:34:43 GMT+01:00
>
>
> Subject: Re: Input request on shell bug
> --------
>
> Bug 0000465: is the list of special built-ins exhaustive (is "local" special)?
>
> Here are my comments relating to this bug. I think it will take some
> discussion to get a final resolution.
>
>
> This is a problem. ksh93 and other shells have special builtins that
> are not mentioned or accounted for in the standard and this has certain
> side effects.
>
> First of all, the standard defines a list of special built-ins in
> section 2.14.
> Special builtins have properties that differ from other utilities:
> 1. Variable assignment remain in effect after the utility completes.
> 2. They are not searched on the path.
> 3. They override any function of the same name.
> 4. A syntax or redirection error causes a script containing them to
> exit.
>
> As mentioned in the original 1992 Posix shell standard, the name typeset was
> a special built-in in one of the two historical shells, ksh.
>
> The standard also defined a list of regular built-in utilities that do
> not have the standard rules but which skip the path search.
>
> However, several POSIX shells have features which violate the current wording
> of the standard. Many have additional special built-in commands, such
> as local and source. In addition, most shells have let as a built-in
> for compatility with ksh although the builtin shell arithmetic is the
> standard.
>
> Some shells (bash and zsh) allow users to define functions whose name is
> the same as a special builtin to override the special builtin, for example
> shift() { echo shift;}; shift
> echos shift.
>
> ksh93 has added shorthands for many types which are defined as preset
> aliases (aliases define automatically when the shell is invoked).
> Users cannot define functions with these names without quoting the definition
> and use which violates the standard. The following is the list of ksh93
> preset aliases put in various categories:
> New variable types (alias for typeset with options).
> float='typeset -lE'
> nameref='typeset -n'
> integer='typeset -li'
> compound='typeset -C'
> autoload='typeset -fu'
> functions='typeset -f'
> These allow alias definitions as the first argument.
> command='command '
> nohup='nohup '
> These allow these special builtins to behave like regular utilities.
> redirect='command exec'
> source='command .'
> These implement utilities in the standard.
> fc=hist
> hash='alias -t --'
> type='whence -v'
> times='{ { time;} 2>&1;}'
> These implement utilies that were in historical ksh.
> history='hist -l'
> r='hist -s'
> stop='kill -s STOP'
> suspend='kill -s STOP $$'
>
> As you can see there are several problems. I think that a combination
> of changes to the standard and changes to the implementations could resolve
> these.
>
> Here are my recommendations:
> 1. State that an implementation may consider it an error to define a
> function whose name is a special built-in.
> 2. Add typeset, enum, let, local, builtin,integer, float, nameref, and
> compound to the list of possible special builtins.
> 3. Allow command and nohup to expand alias for the first argument.
> 4. Add source but do not specify whether it is a special built-in
> or not.
> 5. Add namespace to the list of possible reserved words in 2.4.
>
> Most of the others ksh could change to be built-ins or in the case of fc,
> a predefined function rather than a predefined alias.
> Another possible change is to unset an alias if a function is defined
> with that name.
>
> I am not sure how to handle redirect. It could be replaced with
> a predefined function
> redirect() command exec "$@"
> which would allow a function redefinition, but it wouldn't do a path search
> so that it would override a command named redirect on the users path.
>
>
>
> David Korn
> dgk@research.att.com
(0000973)
Don Cragun (manager)
2011-09-22 17:02
edited on: 2011-09-22 17:04

During the September 22, 2011 conference call we discussed the comments mad
by David Korn and Jonathon Nieder in e-mail sequence numbers 16488 and 16489.
Those e-mails and the discussion are summarized here:

*** From David Korn:
David's comments are included in Note: 0000969.

*** Response from Jonathan Nieder:

| Here are my recommendations:
| 1. State that an implementation may consider it an error to define a
| function whose name is a special built-in.
| 2. Add typeset, enum, let, local, builtin,integer, float, nameref, and
| compound to the list of possible special builtins.
| 3. Allow command and nohup to expand alias for the first argument.
| 4. Add source but do not specify whether it is a special built-in
| or not.
| 5. Add namespace to the list of possible reserved words in 2.4.

Thank you. All of the above suggestions sound sane to me (#1 was already
accepted as Bug 383).

...

| I am not sure how to handle redirect. It could be replaced with
| a predefined function
| redirect() command exec "$@"
| which would allow a function redefinition, but it wouldn't do a path search
| so that it would override a command named redirect on the users path.

That sounds fine. It could be added to 2.9.1.1c ("command search and
execution"), as an additional paragraph:

    If the command name is listed in the following table, a built-in utility
    with that name may be invoked.

*** From the September 22, 2011 conference call discussion:
We agree with the approach suggested by David and Jonathan.

The relationship to Bug 0000383 has been noted in the Relationships
section of this bug report.

Since some of the aliases, utilities, built-ins, and special built-ins listed in the table
David provided are not defined by the standard, we suggest that something be added
explicitly stating that none of these interfaces can be allowed to create special built-ins.

We ask that Jonathan (the original submitter of this bug report) provide a complete
set of changes needed to implement these restrictions taking into account the issues
listed above.

(0001959)
shware_systems (reporter)
2013-11-05 11:40

It appears the request in Note 0000973 wasn't noted by Jonathan, so this has languished. I'd be willing to take a shot at it but I think it needs more discussion. In looking it over it seems the resolution to this may need changes to the grammar, not just the descriptive text, based on the points raised in Note 0000969. I don't see the grammar accounting for alias override quoting and function invocations based on fname rather than utility NAME, both as part of the <simple-command> production, or how the space at the end of an alias value affects argument word parsing when the alias is encountered, or where it's legal that a TOKEN may be considered an alias name rather than a function or utility name.

I realize aliasing is supposed to be fairly transparent, but I think to accurately reflect the precedence of SBIs, function calls, and regular utilities, aliases can't be entirely invisible.

- Issue History
Date Modified Username Field Change
2011-06-18 11:01 jrn New Issue
2011-06-18 11:01 jrn Status New => Under Review
2011-06-18 11:01 jrn Assigned To => ajosey
2011-06-18 11:01 jrn Name => Jonathan Nieder
2011-06-18 11:01 jrn User Reference => jrn.special
2011-06-18 11:01 jrn Section => 2.14 Special Built-In Utilities
2011-06-18 11:01 jrn Page Number => 2334
2011-06-18 11:01 jrn Line Number => 73818-73820
2011-06-18 17:57 Don Cragun Note Added: 0000863
2011-08-04 16:15 nick Note Added: 0000919
2011-08-04 16:16 nick Note Edited: 0000919
2011-09-22 15:35 nick Relationship added related to 0000383
2011-09-22 15:37 nick Note Added: 0000969
2011-09-22 17:02 Don Cragun Note Added: 0000973
2011-09-22 17:03 Don Cragun Note Edited: 0000973
2011-09-22 17:04 Don Cragun Note Edited: 0000973
2013-10-11 08:31 geoffclare Relationship added related to 0000767
2013-11-05 11:40 shware_systems Note Added: 0001959


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker