Austin Group Defect Tracker

Aardvark Mark IV


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0001340 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Objection Clarification Requested 2020-05-04 16:28 2020-06-17 14:02
Reporter shware_systems View Status public  
Assigned To
Priority normal Resolution Accepted As Marked  
Status Applied  
Name Mark Ziegast
Organization SHware Systems Dev.
User Reference
Section XBD 8.3
Page Number 178
Line Number 5854-7
Interp Status Approved
Final Accepted Text Note: 0004871
Summary 0001340: PATH specification has an ambiguity.
Description PATH currently has:
If the pathname being sought contains a <slash>, the search through the path prefixes shall not be performed. If the pathname begins with a <slash>, the specified path is resolved (see Section 4.13, on page 111).

In discussing bug 1208 it was noted the first sentence is in error, as all pathname's being checked will have a <slash> between prefix and filename, and a prerequisite for PATH being referenced is that a filename string has no <slash> characters. Therefore this sentence is superfluous.

The second sentence does not include the case where a pathname does not begin with a <slash> and is a relative reference. This leaves ambiguous what root directory as an additional prefix that begins with a <slash> is presumed: the current directory '.', the root directory '/', or a user's home directory, ala '~', as example choices. Actual practice is these are also resolved in accordance with XBD 4.13, historically.
Desired Action Replace the two sentences with:

Each pathname to be sought shall be resolved in accordance with <xref>XBD 4.13 Pathname Resolution</>, on page 111.
Tags tc3-2008
Attached Files

- Relationships

-  Notes
(0004846)
kre (reporter)
2020-05-04 19:31

While I would not object to the definition of PATH being rewritten in
a slightly more logical form, nothing in the Description here makes
any sense.

I have no idea what discussions related to 0001208 happened to be, or
why an issue relating ti chdir in posix_spawn should care in the least
about PATH, but the "pathname being sought" is typically "ls" or "cat" or "sh"
none of which contains a '/' - so the first sentence is not "in error",
there is no prefix, just the command name (you do understand how PATH works,
right?)

When the pathname begins with a slash there is nothing to do, no search is
made (that's for when someone says /bin/sh) - when there is some other slash
in the pathname, resolution happens the same as for any other filename
reference - since there is no leading '/' (we know that, that case was
already handled) the path lookup starts at the current working directory.

There is no ambiguity here, and while, once again, the paragraph could be
more clearly written (and should xref XCU 2.9.1.1) it doesn't contain any
actual errors.
(0004847)
kre (reporter)
2020-05-04 19:33

Actually, are you perhaps under the impression that the paragraph
in question is about pathnames in general ?

It is about the environment variable PATH - a very specific thing.
(0004848)
shware_systems (reporter)
2020-05-04 20:05

The Desired Action has already been discussed and reached consensus, maybe with a couple wording tweaks. This report is simply the formal record the change is considered required, for the reasons given in the Description.

This came up because sh does do PATH searches and may use *spawn() to exec utilities. The new Final Text for Bug 1208 is related mostly to how PATH is applicable.
(0004849)
kre (reporter)
2020-05-05 03:00

Then the resolution is wrong, and wasn't properly considered.

You could *add* a sentence like that which is proposed, perhaps even
replacing the final sentence, but the earlier text (while it could be
improved) is not incorrect and should not be deleted.
(0004850)
shware_systems (reporter)
2020-05-05 03:16

In that context filename refers to the desired utility, pathname is prefix (possibly with appended slash) concatenated with filename. It was argued filename might replace pathname and keep the basic statement, but that just made it obvious it was redundant so is considered superfluous.
(0004851)
kre (reporter)
2020-05-05 04:39
edited on: 2020-05-05 05:13

All filenames are pathnames. But yes, I agree the switch in terminology
is one of the things I would fix if rewording this is considered worthwhile.

But it is clear that the "pathname" in the later sections is intended to
mean the same thing as the "filename" in the initial part.

The text is redundant (to some extent) if and only if an xref to XCU 2.9.1.1
is added to replace it... but I am not sure that that section (which explains
how sh uses PATH for lookups) necessarily applies precisely to all other uses,
as it includes references to built in utilities, functions, etc, so I think
it would be better to retain the XBD definition for those other purposes,
and simply include an xref to add clarity - perhaps something like

    See [xref XCU 2.9.1.1] for a description of how sh [xref XCU 2] handles
    PATH searches.

(wordsmithed as appropriate).

ps: Not relevant directly here, but to respond to a comment made in Note: 0004848
it is hard to see how a sh could ever use posix_spawn as Note: 0004830 implies
while retaining the view that some people have of how and when builtin
utilities should be executed. It would be possible if the shell simply
looks for a builtin before starting the PATH search, but if it needs to
combine the PATH search with builtin recognition, if it was relying upon
posix_spawn to do the PATH search, I fail to see how a builtin would ever
be able to be executed - the only way would be to do the PATH search twice,
and I don't think that works either (the only truly reliable way to test
whether a PATH element + filename combination works (from userland)
is to try to exec it, and there's no returning from that if it does work.

It also isn't clear how a posix_spawn PATH scanning implementation would ever
maintain the hash table properly (unless of course, in both cases, the PATH
resolution is done as a separate step, and posix_spawn is only ever called
(from sh) with a fully qualified pathname (ie: just to replace the fork/exec
sequence, and fd/dir manipulations that fit between those) but in that case,
there would be no motivation for fiddling the definition of PATH based upon
what posix_spawn sees, as it would only every hit the "contains a slash" and
"begins with a slash" sections (ie: PATH is irrelevant, and what's more,
Note: 0004830 is incorrect, see: Note: 0004830 is incorrect">0001341) - so if you were to
believe that this definition is solely for the purposes of posix_spawn then
I guess the change would make sense. But it isn't, it is also used by
popen() system() find env ...

[Aside: mantis seems to have issues formatting this note correctly, I have
tried several different ways to write the text, yet it always seems to mangle
that final "see Note: 0004830 is incorrect">0001341" reference. Oh, I see the trigger now, it is
because the referenced bug (1341) contains bugid and bugnote references in its
Summary line; is that not supported - it works in the issue itself?]

(0004854)
geoffclare (manager)
2020-05-05 08:40

The description in this bug is reasonably close to what was discussed in the telconference. However, the desired action is most definitely not what we thought Mark was going to put in this bug.

The statement "If the pathname being sought contains a <slash>, the search through the path prefixes shall not be performed" is both erroneous and redundant. It is erroneous because the thing that is sought is a filename, not a pathname, and therefore by definition cannot contain a slash. It is redundant because all of the places in the standard that require a PATH search to be done already state that no PATH search is done if there is a slash in the string that would be the subject of the search. I.e the description of PATH never needs to be read in the context of something with a slash being sought.

The statement "If the pathname begins with a <slash>, the specified path is resolved" is problematic because it only requires pathname resolution to be done for absolute pathnames, not relative ones. The meeting agreed that this needs to be changed in some way so that it applies to relative pathnames as well.

Having thought about the issue a little more, I believe just changing that sentence is not the right solution to the second problem. The time when pathname resolution is done is during the search, not after the pathname has been found. (I.e. it can only be "found" by performing pathname resolution on each constructed pathname.)

I will add another note with a new proposed resolution.
(0004855)
geoffclare (manager)
2020-05-05 08:46
edited on: 2020-05-05 08:47

New proposed resolution...

On line 5851 change:
The list shall be searched from beginning to end, applying the filename to each prefix, until an executable file with the specified name and appropriate execution permissions is found. If the pathname being sought contains a <slash>, the search through the path prefixes shall not be performed. If the pathname begins with a <slash>, the specified path is resolved (see Section 4.13, on page 111).
to:
The list shall be searched from beginning to end, applying the filename to each prefix and attempting to resolve the resulting pathname (see Section 4.13, on page 111), until an executable file with appropriate execution permissions is found.


(0004861)
shware_systems (reporter)
2020-05-05 16:19

I've no objection to the new wording; it was my understanding all the text preceding the two sentences quoted was considered adequate so I left it alone for original proposal.
(0004862)
kre (reporter)
2020-05-05 16:21

Re Note: 0004854

To take the first couple of issues in reverse order, as it makes
more sense (to me) this way...

    It is redundant because all of the places in the standard that require
    a PATH search to be done already state that no PATH search is done if
    there is a slash in the string that would be the subject of the search.

To me that has produced exactly the inverse to the correct response. If
everything that uses PATH needs to specify that it is only used when there
is no slash in the name, then lets simplify all of those, and codify the rules
for PATH processing in a single place, so everyone gets a *known* consistent
interpretation and we don't have people quibbling about why the wording for
the use in context X is subtly different from the wording for contest Y, and
deducing from that, that the behaviour is supposed to be different. And
if we're certain that the identical wording is used in every case, then that's
an even better reason for consolidating it and removing the duplication.

Further, this is the safe way to make the change - if we have missed one of
the uses (and so don't delete its redundant wording) then we're left with
some redundant text, but that's generally harmless. On the other hand if
we have missed a use which turns out not to specify that no path search is
done when the pathname being sought contains a slash, and we delete the
text in the definition of PATH which deals with that, then we've just caused
the standard to be broken. Since the standard is very large, and I'm sure
the word PATH appears in it very many times, I'd hate to guarantee (as in,
my life depends upon it) that we know every place where a path search is
specified, and if we are not 100% certain that we have found every single one,
then neither can we be certain that every single one has the "no slash"
qualification.

And last, it means that someone who simply wants to know what this environment
variable PATH is all about, and (reasonably) turns to XBD 8.3 to get the
answer, will actually discover the whole picture, and not end up missing
the crucial element that a search for a/b only looks for "a/b" and doesn't
try PATH[1]/a/b PATH[2]/a/b (etc) (using invented notation for the ':'
delimited sub-strings of PATH).

Next:
    It is erroneous because the thing that is sought is a filename,
    not a pathname, and therefore by definition cannot contain a slash.

This is part of the "poor wording" I remarked on in earlier notes. It isn't
really as bad (and isn't actually incorrect) it is just hard to read
correctly easily (and currently depends upon the reader correctly
understanding the difference between pathname and filename). PATH What we
do about it depends upon the resolution of the previous point. If we
agree to consolidate the definition of how PATH works (all of it) in XBD 8.3,
which is what I'd suggest should be done, then the thing that is sought is
a pathname, not a filename (as they're defined in XBD 3.170 and 3.271), and
the correct response to this problem is to fix that, and use "filename" only
after we have excepted the case where there's a slash in the pathname - and
I'd make the wording be clear about the change of terminology, and why
something perhaps like:

PATH This variable shall represent the sequence of path prefixes that
       certain functions and utilities apply in searching for an executable
       file. The prefixes shall be separated by a <colon> (':'). If the
       pathname being sought contains no slash ('/') characters, and hence
       is a filename, the list of prefixes shall be searched from beginning
       to end, applying the filename to each prefix, until an executable file
       with the specified name and appropriate execution permissions is found.
       When a non-zero-length prefix is applied to the filename, a <slash>
       shall be inserted between the prefix and the filename if the prefix
       did not end in <slash>. A zero-length prefix is a legacy feature that
       indicates the current working directory. It appears as two adjacent
       <colon> characters ("::"), as an initial <colon> preceding the rest
       of the list, or as a trailing <colon> following the rest of the list.
       A strictly conforming application shall use an actual pathname
       (such as .) to represent the current working directory in PATH.
       If the pathname being sought contains a <slash>, and so is not a
       filename, the search through the path prefixes shall not be performed.

and then continue as it is now, as amended by the feed for ...

   The statement "If the pathname begins with a <slash>, the specified
   path is resolved" is problematic because it only requires pathname
   resolution to be done for absolute pathnames, not relative ones.

More poor wording, and I have no problem with fixing that.

However, I don't think we should discard mention of what happens with a
PATH search when the pathname being sought contains a '/' and rely upon
the reader just understanding that since it says "filename" thatmust be
true, while being left in the dark (until they find a use of PATH search)
about what happens if there is a '/'. And then wonder why it is divided
up that way, and if perhaps different applications of PATH searching might
have different rules for what to do in that case.

That is, I don't believe the proposed resolution in Note: 0004855 is good enough.
(0004864)
geoffclare (manager)
2020-05-06 08:18

Re Note: 0004862

Seems I didn't look hard enough for the places that require a PATH search. I have now found a place that doesn't say PATH is not searched if there is a slash in the search string: find -exec (and -ok), where the requirement that a PATH search is done is only made via the PATH entry in ENVIRONMENT VARIABLES - it's not in the description of -exec. There is also a slight problem with fc -e and xargs because they both state that the utility to be invoked is specified by its name, implying that portable applications can't use fc -e or xargs to invoke a utility by means of a pathname that includes a slash. Once those are fixed to allow a pathname, they would also need to specify the no-slash rule for PATH searching, unless it is covered by the description of PATH.

So I am now in agreement that the description of PATH should handle this.

Your suggestion of "If the pathname being sought contains no slash ('/') characters, and hence is a filename, the list of prefixes shall be searched ..." is okay with me as a way to do that. (I didn't compare the rest of your new PATH description with the current one to see what else you changed - it would be helpful if you could propose the changes you want in smaller chunks so there is less unchanged text to compare.)

The problems with fc -e and xargs could either be addressed here or in a separate bug.
(0004866)
kre (reporter)
2020-05-06 14:39

Here are the editing instructions to get to text shown in Note: 0004862
as requested in Note: 0004864

Al of this applies to the description of the environment variable PATH
in XBD 8.3, on page 178 (2016 edition).

In lines 5843-4 delete the words "known only by a filename".

In line 5844, after the sentence that ends "...separated by a <colon> (':')."
insert the words (beginning a new sentence)
     If the pathname being sought contains no slash ('/') characters,
     and hence is a filename,
and then follow that by moving the sentence that starts on line 5851
     The list shall be searched...
(up to the end of that sentence, on line 5854)
     ...permissions is found.
While moving that sentence, change the capital 'T' in the leading "The"
to lower case, as it is no longer the beginning of a sentence, just a claise.

That's it, the rest of the text is unchanged (since in Note: 0004862 I did not
attempt to fix the "resolve the path" issue.

What I might do for that is to replace the sentence that runs from line
5855-7, viz:

    If the pathname begins with a <slash>, the specified path is
    resolved (see Section 4.13, on page 111).

and replace it with

     In each case, either the result of appending the filename to each prefix
     from PATH when the pathname (filename) contains no slash characters,
     or the pathname when it does contain one or more slash characters,
     is resolved (see Section 4.13, on page 111).

but I am less happy with that wording than the earlier part.
Someone please suggest something better.
(0004868)
geoffclare (manager)
2020-05-07 08:50
edited on: 2020-05-07 08:51

Here's a suggestion that combines kre's solution from Note: 0004866 for the filename/pathname problem with my suggestion from Note: 0004855 for the pathname resolution problem...

All of this applies to the description of the environment variable PATH in XBD 8.3, on page 178 (2016 edition).

In lines 5843-4 delete the words "known only by a filename".

In line 5844, after the sentence that ends "...separated by a <colon> (':')."
insert the new sentence
If the pathname being sought contains no slash ('/') characters, and hence is a filename, the list shall be searched from beginning to end, applying the filename to each prefix and attempting to resolve the resulting pathname (see Section 4.13, on page 111), until an executable file with appropriate execution permissions is found.

Delete the sentence that starts on line 5851
     The list shall be searched...
(up to the end of that sentence, on line 5854)
     ...permissions is found.

Replace the two sentences that run from line 5854-7, viz:
If the pathname being sought contains a <slash>, the search through the path prefixes shall not be performed. If the pathname begins with a <slash>, the specified path is resolved (see Section 4.13, on page 111).
with:
If the pathname being sought contains any <slash> characters, the search through the path prefixes shall not be performed and the pathname shall be resolved as described in Section 4.13, on page 111.

The end result of these edits is the following as the first paragraph of the PATH description:
This variable shall represent the sequence of path prefixes that certain functions and utilities apply in searching for an executable file. The prefixes shall be separated by a <colon> (':'). If the pathname being sought contains no slash ('/') characters, and hence is a filename, the list shall be searched from beginning to end, applying the filename to each prefix and attempting to resolve the resulting pathname (see Section 4.13, on page 111), until an executable file with appropriate execution permissions is found. When a non-zero-length prefix is applied to this filename, a <slash> shall be inserted between the prefix and the filename if the prefix did not end in <slash>. A zero-length prefix is a legacy feature that indicates the current working directory. It appears as two adjacent <colon> characters ("::"), as an initial <colon> preceding the rest of the list, or as a trailing <colon> following the rest of the list. A strictly conforming application shall use an actual pathname (such as .) to represent the current working directory in PATH. If the pathname being sought contains any <slash> characters, the search through the path prefixes shall not be performed and the pathname shall be resolved as described in Section 4.13, on page 111. If PATH is unset or is set to null, the path search is implementation-defined.


(0004869)
kre (reporter)
2020-05-07 10:40

That (Note: 0004868) is fine with me.
(0004871)
geoffclare (manager)
2020-05-11 15:19

Interpretation response
------------------------
The standard is unclear on this issue, and no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor.

Rationale:
-------------
The current text does not require constructed relative pathnames to be resolved, only absolute ones.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
Make the changes in Note: 0004868
(0004872)
ajosey (manager)
2020-05-11 15:27

Interpretation proposed: 11 May 2020
(0004888)
ajosey (manager)
2020-06-12 09:09

Interpretation approved: 12 June 2020

- Issue History
Date Modified Username Field Change
2020-05-04 16:28 shware_systems New Issue
2020-05-04 16:28 shware_systems Name => Mark Ziegast
2020-05-04 16:28 shware_systems Organization => SHware Systems Dev.
2020-05-04 16:28 shware_systems Section => XBD 8.3
2020-05-04 16:28 shware_systems Page Number => 178
2020-05-04 16:28 shware_systems Line Number => 5854-7
2020-05-04 19:31 kre Note Added: 0004846
2020-05-04 19:33 kre Note Added: 0004847
2020-05-04 20:05 shware_systems Note Added: 0004848
2020-05-05 03:00 kre Note Added: 0004849
2020-05-05 03:16 shware_systems Note Added: 0004850
2020-05-05 04:39 kre Note Added: 0004851
2020-05-05 05:03 kre Note Edited: 0004851
2020-05-05 05:04 kre Note Edited: 0004851
2020-05-05 05:11 kre Note Edited: 0004851
2020-05-05 05:13 kre Note Edited: 0004851
2020-05-05 08:40 geoffclare Note Added: 0004854
2020-05-05 08:46 geoffclare Note Added: 0004855
2020-05-05 08:47 geoffclare Note Edited: 0004855
2020-05-05 16:19 shware_systems Note Added: 0004861
2020-05-05 16:21 kre Note Added: 0004862
2020-05-06 08:18 geoffclare Note Added: 0004864
2020-05-06 14:39 kre Note Added: 0004866
2020-05-07 08:50 geoffclare Note Added: 0004868
2020-05-07 08:51 geoffclare Note Edited: 0004868
2020-05-07 10:40 kre Note Added: 0004869
2020-05-11 15:19 geoffclare Note Added: 0004871
2020-05-11 15:20 geoffclare Interp Status => Pending
2020-05-11 15:20 geoffclare Final Accepted Text => Note: 0004871
2020-05-11 15:20 geoffclare Status New => Interpretation Required
2020-05-11 15:20 geoffclare Resolution Open => Accepted As Marked
2020-05-11 15:20 geoffclare Tag Attached: tc3-2008
2020-05-11 15:27 ajosey Interp Status Pending => Proposed
2020-05-11 15:27 ajosey Note Added: 0004872
2020-06-12 09:09 ajosey Interp Status Proposed => Approved
2020-06-12 09:09 ajosey Note Added: 0004888
2020-06-17 14:02 geoffclare Status Interpretation Required => Applied


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker