Viewing Issue Advanced Details
1445 [Issue 8 drafts] Shell and Utilities Editorial Error 2021-01-21 18:11 2021-01-21 18:16
rhansen
 
normal  
New  
Open  
   
Richard Hansen
2.14
2311
74980-74981
Update "special built-in" to match changes from bug #854
Bug #854 updated many sections regarding PATH search for different types of utilities but missed a spot in XCU 2.14 (as mentioned in Note: 0003228).
On page 2311 lines 74980-74981 (XCU 2.14 2nd paragraph) change:
The term "built-in" implies that the shell can execute the utility directly and does not need to search for it.
to:
The term "built-in" implies that there is no need to execute a separate executable file because the utility is implemented in the shell itself.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1444 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Editorial Error 2021-01-19 10:42 2021-01-19 10:42
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
<wordexp.h>
466
16113
---
Broken cross-references to wordexp()
The SEE ALSO section on the <wordexp.h> page is supposed to have a cross-reference to the wordexp() function in XSH, but it instead points to XCU 2.6. The XSH pages for fnmatch() and glob() have the same problem.

This is happening because the troff source uses the same tag "wordexp" for both of these cross-reference targets.
Fix the broken wordexp() cross-references in SEE ALSO on the <wordexp.h>, fnmatch(), and glob() pages.

Note to the editor: there are many more correct xrefs to XCU 2.6 than there are broken xrefs to wordexp(), so the least disruptive way to fix this is to change the xref tag for wordexp() to "wordexp_xsh".
There are no notes attached to this issue.




Viewing Issue Advanced Details
1443 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Objection Omission 2021-01-19 10:40 2021-01-19 10:40
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
3.274 Pattern
77
2224
---
Definition of "pattern" omits some uses
The definition of "pattern" talks about pathname expansion, but not pathname matching or other uses of shell patterns, such as case statements or the four varieties of parameter expansion that provide for substring processing.

It should talk about shell pattern matching notation the same way it does about regular expression notation, including referring in the small-font note to the section that defines the notation (instead of the current reference to the section about pathname expansion).

The problem extends to the text "as a means of selecting various character strings or pathnames", which only covers a subset of the uses of patterns. It would be best just to omit this part, as the references in the small-font note can be followed to find out more about what each type of pattern is used for. (Although an addition to XCU 2.13 along the lines suggested below would help in this regard.)
On page 77 line 2224 section 3.274 Pattern, change:
A sequence of characters used either with regular expression notation or for pathname expansion, as a means of selecting various character strings or pathnames, respectively.
to:
A sequence of characters used either with regular expression notation or with shell pattern matching notation.

On page 77 line 2227 section 3.274 Pattern, change:
See also [xref to XCU 2.6.6 Pathname Expansion].
to:
Shell pattern matching notation is defined in detail in [xref to XCU 2.13 Pattern Matching Notation].

On page 2308 line 74866 section 2.13 Pattern Matching Notation, after:
The pattern matching notation described in this section is used to specify patterns for matching strings in the shell.
add a sentence:
This notation is also used by some other utilities (find, pax, and optionally make) and by some system interfaces (fnmatch(), glob(), and wordexp()).

There are no notes attached to this issue.




Viewing Issue Advanced Details
1442 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Comment Enhancement Request 2021-01-10 13:10 2021-01-10 22:42
andras_farkas
 
normal  
New  
Open  
   
Andras Farkas
getopts
2840
93538 and 93541
---
Improvement to getopts example, remove double negation
On getopts's page
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/getopts.html [^]
The example uses interesting double negation in the two tests:
if [ ! -z "$aflag" ]; then
if [ ! -z "$bflag" ]; then
! -z could be shortened to -n
Replace
if [ ! -z "$aflag" ]; then
with
if [ -n "$aflag" ]; then
and
if [ ! -z "$bflag" ]; then
with
if [ -n "$bflag" ]; then
There are no notes attached to this issue.




Viewing Issue Advanced Details
1441 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Enhancement Request 2021-01-07 16:56 2021-01-07 16:56
rhansen
 
normal  
New  
Open  
   
Richard Hansen
2.5.3 Shell Variables, PS4
2352
74965
---
PS4 should be printed for non-interactive shells as well
The specification of PS4 only applies to interactive shells. It should also apply to non-interactive shells.
On page 2352 line 74965 (XCU 2.5.3 PS4), change:
When an execution trace (set −x) is being performed in an interactive shell,
to:
When an execution trace (set −x) is being performed,

There are no notes attached to this issue.




Viewing Issue Advanced Details
1440 [1003.1(2016/18)/Issue7+TC2] System Interfaces Comment Omission 2021-01-04 15:49 2021-01-13 10:50
ciprian_craciun
 
normal  
New  
Open  
   
Ciprian Dorin Craciun
system(3)
N/A
N/A
---
Calling `system("-some-tool")` fails (although it is a valid `sh` command)
This report is in relation to the following `glibc` and Linux man-pages issues:
* https://sourceware.org/bugzilla/show_bug.cgi?id=27143 [^]
* https://bugzilla.kernel.org/show_bug.cgi?id=211029 [^]

However it seems that the root cause is the interaction mandated by the POSIX standard between `system(3)` and `sh(1)`.


In simple terms, calling `system("-some-tool")` has an unexpected behaviour when delegating that to `sh`, as in `sh -c -some-tool`. (For how to replicate this just see the first link.)


In more details:

* the `system(3)` specification requires that:
~~~~
# from: https://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html [^]

The system() function shall behave as if a child process were created using fork(), and the child process invoked the sh utility using execl() as follows:

execl(<shell path>, "sh", "-c", command, (char *)0);
~~~~

* however the `sh(1)` specification requires that:
~~~~
# from https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html [^]

sh -c [-abCefhimnuvx] [-o option]... [+abCefhimnuvx] [+o option]...
       command_string [command_name [argument...]]
~~~~

I.e. `sh(1)` allows extra flags between `-c` and the `command_string`; however the `system(3)` doesn't account for a command that might start with a hyphen.

Thus, depending on what the command might be, the outcome is either a missing command (if the given command resembles an option accepted by `sh`), or an unknown option (if the given command doesn't resemble an option accepted by `sh`).
The "optimal" solution would be to require `system(3)` to call `sh(1)` with an extra `--` between `-c` and the `command`, as in:
~~~~
execl(<shell path>, "sh", "-c", "--", command, (char *)0);
                                ^^^^
~~~~

Given that `sh(1)` already mandates a compliant shell to correctly handle `--`, this shouldn't have many unintended consequences.

Alternatively add a warning in the specification stating that a command starting with `-` has an undefined behaviour and should be worked-around by preperding a space to any such command.
Notes
(0005191)
Vincent Lefevre   
2021-01-06 13:20   
Since with -c, there must be at least a non-option argument, how about doing a correction/improvement in sh as follows? If -c is given and there is a single argument after -c, then this argument should be regarded as a command string.

(This could be done in addition to adding "--" on the system() side.)
(0005192)
ciprian_craciun   
2021-01-06 15:41   
> Since with -c, there must be at least a non-option argument, how about doing a correction/improvement in sh as follows? If -c is given and there is a single argument after -c, then this argument should be regarded as a command string.


I think adding such a requirement is not advisable for multiple reasons:

* first, none of the existing `sh` implementations (as reported in one of the tickets none of `bash`, `dash` or `zsh`) exhibit such a behaviour, therefore this change will all of a sudden break their POSIX compliance;

* second, although it solves the `system` corner-case, it opens other potential issues; imagine for example a script that calls `sh -c -x "${@}"`; now if the user doesn't provide an argument, then all of a sudden `sh` executes `-x` as a command, as opposed to until now when it didn't execute anything;


Personally I don't know what is the best course of action here, besides updating `system` to use the `--` separator which is backward compatible and shouldn't break virtually anything.

However, given how much "legacy" there is based on `sh`, I wouldn't change its behaviour, except perhaps giving a warning in the man page about the situation.
(0005193)
joerg   
2021-01-06 16:12   
Please note that this problem has been introduced by POSIX. It does not
exist in pre-POSIX shells.

In the classical Bourne Shell, -c wasn't even seen as a real option but
as some kind of prefix to the command argument.

Since the problem has been introduced by shells while implementing POSIX
requirements, I am fully open to set up a new POSIX requirement for shell
option parsing that would fix the parsing to make it compatible to the
historic Bourne Shell again.

This would however require to find a simple and still always working
algorithm. Just counting the arguments after -c does not work.

Note that with the classical Bourne Shell

    sh -x -c 'echo foo'

does not work, but requiring it to work by POSIX seems to be a useful
enhancement compared to the classical Bourne Shell and the fact that
POSIX requires

    sh -c -x 'echo foo'

to work also seems to be a useful extension.

As a result, it may be hard to fix this problem in the shell...
(0005194)
kre   
2021-01-06 16:13   
(edited on: 2021-01-06 16:22)
Re Note: 0005191 and Note: 0005192

Not only is the suggestion in Note: 0005191 not the right thing to do,
as showin in Note: 0005192 (it would also cause sh to not satisfy the
utility guidelines in XBD 12.1)

But beyond that, it would also be insufficient to solve the problem
in general, as there can be more than one arg after -c, and the first
of those might begin with a "-" - which means the shell would either
fail just the same way (just not perhaps when called from system()) or
would run the wrong thing as a command.

This needs to be fixed in system(), or in the application, not in sh.

That could be done in system() adding a -- in the arg list, perhaps only if
the command string starts with a '-', or by prefixing a space to the arg
(though that would mean copying it, so probably would not be the method of
choice), or however else the system() function wants to do it.

Or we could simply note the potential issue in the APPLICATION USAGE of
system() and tell application authors not to attempt system() (or popen())
with a string starting with a '-' - there are a zillion ways (including
prependiong a space, or quoting the command name, or ...) the application
can avoid that ever happening.

Personally, I would put the onus on the application - the way system()
works has neevr really changed, the way sh works has never changed, it
has always been necessary for the application to deal with the combination
that results from that if they're doing something unusual (like starting
a command with a '-').

(0005199)
Vincent Lefevre   
2021-01-13 10:50   
And what about the invocation of $EDITOR and $VISUAL?

A proposed manpages patch for Debian at https://bugs.debian.org/866060 [^] has:

 .BR EDITOR / VISUAL
 The user's preferred utility to edit text files.
+Any string acceptable as a command_string operand to the
+.I sh -c
+command shall be valid.

Executing the command with "sh -c" will fail if it starts with "-". But I wonder how this is handled in practice in various implementations and whether something is missing in the spec, which is not very explicit.




Viewing Issue Advanced Details
1439 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Editorial Clarification Requested 2020-12-24 03:47 2021-01-16 11:27
dannyniu
 
normal  
New  
Open  
   
DannyNiu/NJF
<dlfcn.h>, <sys/mman.h>
many
many
---
The POSIX standard did not distinguish object and function pointers as did the C standard.
The C standard distinguishes function pointers from object pointers. The POSIX standard treat them as the same in <dlfcn.h> and in <sys/mman.h> where the return type of `dlsym` and `mmap` is `void *`.

However, the specification of ILP32 and LP64 memory models can be seen as implying the standard is treating function and object pointers as the same.
Explicitly state that function and object pointers are the same, and share the same process address space.
Notes
(0005189)
geoffclare   
2021-01-04 10:49   
(edited on: 2021-01-04 10:50)
This is effectively requesting that we revert to the situation before bug 0000074 was applied in Issue7 TC1. Prior to that, XSH7 included this section:
2.12.3 Pointer Types

All function pointer types shall have the same representation as the type pointer to void. Conversion of a function pointer to void * shall not alter the representation. A void * value resulting from such a conversion can be converted back to the original function pointer type, using an explicit cast, without loss of information.

Note: The ISO C standard does not require this, but it is required for POSIX conformance.

This had been added in the original Issue 7 because of dlsym(), but the decision made for bug 74 was that we should instead have a much narrower requirement specific to the conversions needed for dlsym() rather than this very general requirement.

This new bug also mentions mmap(), which I don't recall coming up before in this context. If there is existing application code that casts the mmap() return value to a function pointer, then we should consider adding an mmap() requirement similar to the current dlsym() one.

(0005195)
dannyniu   
2021-01-10 11:03   
(edited on: 2021-01-10 11:05)
If the decision of bug-0074 was made based on security, then I find it not very meaningful, as modern computers are capable of enforcing code/data access control in hardware, and many operating systems are also capable of emulating such access control in software should hardware support is lacking.

The other point which says:

> it need not necessarily be supported to try to examine
> the instructions that make a function by ...

While I can't fully understand it (probably due to my knowledge of English being limited), I think such examination is more of a feature.

Lastly, the "tiny code generator" (TCG) accelerator in QEMU is known to use mmap. I'm not sure if it actually acquires executable memory pages and calls function residing in it, but being a "code generator", I think it almost certainly does that. Although this last point isn't very compelling as QEMU is all about things that are hardware-specific (and thus implementation-specific).

My reason for reverting the decision of bug-0074 would only be for desiring an intuitive, simple, and straightforward memory model and type system.

(0005196)
joerg   
2021-01-10 14:53   
(edited on: 2021-01-10 16:31)
I don't believe that there is a need to change the mmap() requirements,
since the related tricks that may be needed could be hidden behind dlopen().

The mentioned change was needed to permit dlsym() to work both for
functions and data objects. The reason however was not security or
address space division but some older IBM mainframes had a different
bit-ness for code and data. The change requires the provider of a
POSIX platform to define the type void * to be useful for both,
function pointers and data pointers, while the C standard only requires
void * to work for data pointers.

I don't believe there is a need to change the current POSIX text
unless you like to explicitly permit mmap() to be useful for
self-managed mapped code segments. Even with that background, I cannot
see a problem, since mmap() already returns void * and there is PROT_EXEC.

(0005197)
shware_systems   
2021-01-10 16:29   
It is because of those hardware access controls that the need arises to distinguish between code and object pointers. With some processors, for each executable, a code pointer and an object pointer may have different bit widths. This is notably evident with the Intel 80286 processors using various memory models. A code pointer can be 32 bits, object pointers may only be 16 bits wide. For another file code pointers may be 16 bits, and object pointers may be 32. To support dlsym() this requires the storage for a void * to be 32 bits to be able to hold either possibility.

The other salient aspect to these is only object pointers are writable. Any data objects pointed to by code pointers is const qualified, enforced by the hardware on store dereferences if code attempts it. Because they can not be modified, there is no reason they need to be readable either. A compiler can as easily produce a reference to an area reserved for objects as into a function body and put the data there, after all. It is up to the compiler to manage function entry points and branch labels as rvalues only.

From the standard's perspective the only interfaces that should be modifying code pointers are the exec family and dlopen(), as trusted code doing load time relocation calculations. Any other code has to be presumed to have malicious intent in even trying to read a function body, as the security consideration.

While there are classes of applications where this isn't the intent, such as in-situ debuggers, the specification of these is considered out of scope for the standard. This is because they are so dependent on hardware knowledge and unspecified implementation support for them.
(0005202)
dannyniu   
2021-01-16 11:27   
<signal.h> describes siginfo_t and claims it has the following member:

void *si_addr // Address of faulting instruction




Viewing Issue Advanced Details
1438 [1003.1(2016/18)/Issue7+TC2] System Interfaces Editorial Clarification Requested 2020-12-17 19:19 2020-12-17 19:19
steffen
 
normal  
New  
Open  
   
steffen
iconv
1123
38001
---
iconv: clarify which "codeset" is meant
Nitpick that seems to sometimes cause issues in practice, as shown in thread on ML.
On page 1123 lines 38001 ff. change

If a sequence of input bytes does not form a valid character in the specified codeset, conversion shall stop after the previous successfully converted character.

to

If a sequence of input bytes does not form a valid character in the specified input codeset, conversion shall stop after the previous successfully converted character.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1437 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Enhancement Request 2020-12-15 21:02 2020-12-17 21:16
steffen
 
normal  
New  
Open  
   
steffen
Vol. 3: Shell and Utilities, Issue 7, make
(2975) 2989
(98708, 98742) 99308
---
make: (document .NOTPARALLEL and .WAIT special targets) in RATIONAL
Parallel, even massively parallel processing has become the widely supported and
used default, yet the standard make(1) does not document it.

If the -j command line option as requested in issue #1436 is standardized,
the standard should also mention ways to synchronize rule processing, in order
to support more complicated build relationships seen in real-life projects.

The suggestion given here can be used with the most widely used make(1)
implementations (GNU make, BSD make) on any tested platform (Linux, *BSD, SunOS
/ Solaris).

The make family rooted on the Sun side of likfe has has at least .WAIT: reserved
for decades, and in practice

  .NOTPARALLEL:
  .WAIT: # Luckily BSD make supports specifying this as target, too

  tangerine: config .WAIT build .WAIT test .WAIT install
  citron: config .WAIT build .WAIT install
  all: config .WAIT build

produces the desired outcome.
(
On page 2975, insert before line 98708

  .NOTPARALLEL
    When specified all rules specified in the given makefile are processed
    sequentially, as if -j has not been given.

On page 2975, insert before line 98472

  .WAIT
    When .WAIT appears in a dependency line, prerequisites that appear after it
    are built no sooner until all preceeding prerequisites have been built
    successfully.

On page 2975, change on line 98472 ff.

  The special targets .IGNORE, .POSIX, .PRECIOUS, .SILENT, and .SUFFIXES shall
  be specified without commands.

to

  The special targets .IGNORE, .NOTPARALLEL, .POSIX, .PRECIOUS, .SILENT,
  .SUFFIXES and .WAIT shall be specified without commands.
)

On page 2989, insert before line 99308

  The special targets .NOTPARALLEL and .WAIT represent different, rather
  incompatible synchronisation concepts of parallel build environments as
  used by the diverse implementations. By using an indirection both approaches
  can be supported: for implementations which use .NOTPARALLEL in a makefile to
  ensure sequential rule processing in the given file the full power of parallel
  processing will be available in recursive make instances invoked on makefiles
  without the directive, implementations which synchronize on the .WAIT special
  target will synchronize on it, for example:

    .NOTPARALLEL: # Process all rules sequentially
    .WAIT: # Define to satisfy "the other implementation"

    install: config .WAIT all
      echo Installing..
      ..
    all: config .WAIT build
    build:
      make -f makefile.build
    config:
      ./configure
Notes
(0005184)
psmith   
2020-12-17 17:10   
Just to note, GNU make does not support the .WAIT special target.
(0005185)
steffen   
2020-12-17 19:23   
Ach! I wish it would, i think the .WAIT approach "is better" (as in explicit, and does not need an indirection).
But you said on the ML that you do not plan to support it either.

This is why the first part above is in parenthesis, and this issue actually requests some words in RATIONALE instead.

The presented solution has the desired effect also in GNU make ("build" target uses parallel processing).
(0005186)
psmith   
2020-12-17 21:16   
I don't believe I said I didn't want GNU make to support it; I agree that it seems like a useful feature. I just said that given the way make's parallel support works I suspect implementation is not so easy and I didn't plan to work on it as I have other things on my (relatively small) plate.

But, if someone else were to get motivated to try to get this working that would be fine with me :). I wonder if creating some kind of internal/implicit prerequisite relationship that wasn't visible to users would be a simpler way to implement this... or not. Well, something for someone to consider.




Viewing Issue Advanced Details
1436 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Enhancement Request 2020-12-15 21:00 2020-12-15 21:00
steffen
 
normal  
New  
Open  
   
steffen
Vol. 3: Shell and Utilities, Issue 7, make
2969
98473
---
make: add "-j max_jobs" option to support simultaneous rule processing
Parallel, even massively parallel processing has become the widely supported and
used default, yet the standard make(1) does not document it.

P.S.:
Even though on SunOS/Solaris the first approach taken (long ago) involved
distribution of make jobs to different computers via a special "dmake" program,
it seems even there a default make(1) could follow an adjusted standard by doing
the equivalence of "exec dmake ARGUMENTS".
On page 2969, insert before line 2969

  -j max_jobs
    Specifies the maximum number of rule-processing jobs to run simultaneously.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1435 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Error 2020-12-15 14:50 2020-12-16 02:08
zackw
 
normal  
New  
Open  
   
Zack Weinberg
GNU
exec
(unknown)
(unknown)
---
execlp and execvp should not execute a command interpreter when other members of the exec family would fail with ENOEXEC
When execve would fail for a particular process image file and set errno to [ENOEXEC], execlp and execvp are specified to retry execution of that file as-if it were a shell script (see the text quoted as to-be-deleted, under "Desired Action").

This behavior exists only for historical reasons -- it predates the `#!` mechanism implemented by all current-generation Unixes -- and *may* constitute a security hole. On the computer where I'm typing this, the first eight bytes of a machine-code executable are consistently 7f 45 46 02 01 01 00. The behavior of `sh` when fed a file beginning with these bytes is not specified, as far as I can tell; I have observed at least one implementation which, when given an input file containing those eight bytes, attempts to execute a program whose name is "\177ELF\002\001\001" (C string notation). Suppose an unprivileged process that can create a program with that name in a location on a higher-privileged process's PATH. The higher-privileged process is believed only to run programs with known names, but one of the programs with a known name is corrupt and will be rejected by `execve` with ENOEXEC, causing `execvp` to attempt to run it as a shell script, and in turn to execution of the injected program named "\177ELF\002\001\001".

Yes, this exploit chain involves a questionable implementation of `sh` *and* two different system misconfigurations, but I think we all know that vendors will try to argue that each of those three things is harmless and does not need fixing.
From <https://pubs.opengroup.org/onlinepubs/9699919799/functions/exec.html>, [^] delete all of the following text from the DESCRIPTION section:

---
There are two distinct ways in which the contents of the process image file may cause the execution to fail, distinguished by the setting of errno to either [ENOEXEC] or [EINVAL] (see the ERRORS section). In the cases where the other members of the exec family of functions would fail and set errno to [ENOEXEC], the execlp() and execvp() functions shall execute a command interpreter and the environment of the executed command shall be as if the process invoked the sh utility using execl() as follows:

execl(<shell path>, arg0, file, arg1, ..., (char *)0);

where <shell path> is an unspecified pathname for the sh utility, file is the process image file, and for execvp(), where arg0, arg1, and so on correspond to the values passed to execvp() in argv[0], argv[1], and so on.
---

Also, in the ERRORS section, move the entry for [ENOEXEC] to the first group of errors ("The exec functions shall fail if: ...")
Notes
(0005173)
geoffclare   
2020-12-15 15:22   
There is no point changing execlp() and execvp() as there are many other ways that a shell can be invoked to try to execute such a file (e.g. system(), popen(), or just interactive use of a shell). The right place to deal with the issue is in the shell, and this was done in bug 0001226.

However, looking again at 1226 I see that there are two occurrences of the text that it fixes, and it only fixes one of them. Since that bug has already been applied, we should use this new bug as an opportunity to fix the other one.

On page 2368 line 75615 section 2.9.1.1, change:
If the executable file is not a text file, the shell may bypass this command execution. In this case, it shall write an error message, and shall return an exit status of 126.
to:
The shell may apply a heuristic check to determine if the file to be executed could be a script and may bypass this command execution if it determines that the file cannot be a script. In this case, it shall write an error message, and shall return an exit status of 126.
<small>Note: A common heuristic for rejecting files that cannot be a script is locating a NUL byte prior to a <newline> byte within a fixed-length prefix of the file. Since sh is required to accept input files with unlimited line lengths, the heuristic check cannot be based on line length.</small>
(0005174)
zackw   
2020-12-15 15:31   
The other C-level APIs you mention, system and popen, are expected to invoke a shell. The exec family, on the other hand, is expected *not* to invoke a shell. I think it is worth changing execlp and execvp just to eliminate this corner case where a shell might be invoked, contrary to expectations.
(0005175)
geoffclare   
2020-12-15 15:45   
Re Note: 0005174

No, only execl, execle, execv, and execve are expected not to invoke a shell. The execlp and execvp functions have always invoked a shell if the file cannot be executed directly and are very much expected to do so. Changing this would break a huge number of applications which rely on them executing a shell in cases where the shell script does not start with #! or the system does not support #!, including every strictly conforming application that includes a shell script to be executed using execlp or execvp (since strictly conforming applications cannot use #!)
(0005176)
zackw   
2020-12-15 15:53   
If you would prefer a more practical rationale for the proposed change, consider a program (a test suite driver, for instance) that wants to execute binaries that may have been cross-compiled and therefore may not be executable on the system where it's running, and distinguish ENOEXEC-type failure from other forms of failure. It runs on a wide variety of systems and therefore cannot rely on any binary-rejection heuristic being implemented by the shell. The only alternative is to avoid using execlp and execvp, but this means reimplementing PATH search by hand, and depending on the implementation language, it may not even be possible. (For instance, the Perl interpreter *only* exposes execvp and system to Perl programs.)
(0005177)
zackw   
2020-12-15 15:54   
On a closely related note, why on earth hasn't #! been standardized? I have *never* encountered a system that claims POSIX conformance but doesn't implement it, not even 25 years ago.
(0005178)
zackw   
2020-12-15 15:56   
> Changing this would break a huge number of applications

Please give one example of such an application.
(0005179)
shware_systems   
2020-12-15 16:11   
Those interfaces support the idea a utility may be implemented as a binary or a script, with binaries having attempt priority. The #! concept is an adjunct to this, not a replacement. It is for hardware and configuration dependant reasons like in the description why #! is precluded by the standard, however. There was a request to add it, a few years ago, and the issues discussed for rejecting it include factors out of scope for the standard.

It is also on the shell to provide a heuristic that recognizes sequences like in the description as being non-text, as the specified behavior, and reject processing it, per bug 1226. This is the same as the binary loader is expected to reject binaries for an incompatible processor. If it doesn't do this adequately it is a bug in that shell or loader, that I see, not a standard defect.
(0005180)
dannyniu   
2020-12-16 02:08   
(edited on: 2020-12-16 02:18)
As a read-only member on the mailing list who currently supervises the security logic of the web app deployed on our Linux server, I totally agree this is a serious potential security exploit.

I suggest a moderate modification. That is:

Change:

> execlp() and execvp() functions shall execute a command interpreter ...

To

> execlp() and execvp() functions may execute the "sh" command interpreter ...

This would allow implementations with additional access control deny certain scripts with say "security flags" attached to them when they're fetched from network or scanned by anti-virus software.





Viewing Issue Advanced Details
1434 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Comment Enhancement Request 2020-12-15 01:08 2020-12-15 02:09
andras_farkas
 
normal  
New  
Open  
   
Andras Farkas
pr
3107
103845-103847
---
Moving the -p option in the pr utility synopsis
In
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/pr.html [^]
-f and -p are put together as [-fp] in the synopsis. As far as I can tell, if I understand it right, this is a historical artifact from them being added to POSIX from the old SUS standard at the same time. -f is XSI while -p is normal (required, unshaded) so I feel -p can be moved into [-adFmrt] for [-adFmprt] since those are all the other regular flags that don't take arguments.

I originally noted this as a minor nit here:
https://austingroupbugs.net/view.php?id=1432 [^]

An additional thing that could be done might be to move -f too, in the sense of taking it from the very end of the synopsis, and putting it closer to the front, like so:
[-adFmprt] [-f]
Move -p into [-adFmrt] to make that section [-adFmprt]
Notes
(0005171)
andras_farkas   
2020-12-15 01:10   
Or, even, since a single flag can be XSI shaded in a group of flags, maybe even...
[-adfFmprt] or [-adFfmprt] (with XSI shading around the f)
(0005172)
andras_farkas   
2020-12-15 01:10   
Though if that should be a separate issue, I can make that. Or we can rename this issue to "moving multiple options in the pr synopsis" or something like that.




Viewing Issue Advanced Details
1433 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Clarification Requested 2020-12-12 00:16 2020-12-12 00:16
andras_farkas
 
normal  
New  
Open  
   
Andras Farkas
pr
---
The use of carriage-return in pr
On
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/pr.html [^]
I see that -p waits for <carriage-return>.

Is this intentional?

I see that OpenSolaris (System V), Unix 10th edition, FreeBSD, and NetBSD pr wait for <newline> ('\n') and not <carriage-return>. I did not find an implementation which waits for <carriage-return>. Also, in grepping the entire POSIX standard for carriage-return and \r I find very few uses of it, while uses of <newline> for user input are universal.
Figure out whether pr should truly wait for <carriage-return> or not.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1431 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Error 2020-12-10 16:00 2020-12-10 17:40
dirk_f
 
normal  
New  
Open  
   
Dirk Fieldhouse
XCU/4 Utilities/command
2596
84261ff
---
Incorrect xref for shell function lookup in specification of the 'command' utility

> The command utility shall cause the shell to treat the arguments as a simple command,
> suppressing the shell function lookup that is described in Section 2.9.1.1 (on page 2367), item 1b.

Should be "item 1c".

From Geoff Clare:
"It seems that the fault lies with bug 935 which added a new item 1b (and was applied via TC2)."

Change the text quoted in the Description above, as below:

"
The command utility shall cause the shell to treat the arguments as a simple command, suppressing the shell function lookup that is described in Section 2.9.1.1 (on page 2367), item 1c.
"

The issue continues in the latest Issue 8 draft, although the Section reference itself has been updated to 2.9.1.4. Is there an automatic process for forwarding this, or does it need a separate report?
Notes
(0005163)
geoffclare   
2020-12-10 17:40   
Re "Is there an automatic process for forwarding this, or does it need a separate report?"

It's automatic. Assuming this bug is resolved with a tc3-2008 tag, the fix will be applied in the Issue7TC3 branch in gitlab and then will automatically get merged into the Issue8 branch the next time I do a "git pull origin Issue7TC3" in that branch.




Viewing Issue Advanced Details
1430 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Error 2020-12-09 12:02 2020-12-09 12:02
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
write()
2311, 2314
73485, 73634
---
Text about interleaving data should say thread not process
Where the description of write() talks about data written to a pipe (or FIFO) being interleaved it refers to "other processes" but should say "other threads".

Also, the text on line 73481 before the bullet list says "pipe or FIFO" but the bullet list items only say "pipe".
On page 2311 line 73485 section write(), change:
Write requests of {PIPE_BUF} bytes or less shall not be interleaved with data from other processes doing writes on the same pipe. Writes of greater than {PIPE_BUF} bytes may have data interleaved, on arbitrary boundaries, with writes by other processes, whether or not the O_NONBLOCK flag of the file status flags is set.
to:
Write requests of {PIPE_BUF} bytes or less shall not be interleaved with data from other threads performing write operations on the same pipe. Writes of greater than {PIPE_BUF} bytes may have data interleaved, on arbitrary boundaries, with write operations by other threads, whether or not the O_NONBLOCK flag of the file status flags is set.

On page 2311 lines 73483-73500 section write(), change all occurrences of "pipe" to "pipe or FIFO".

On page 2314 line 73634 section write(), change:
interleaved with data from any other process
to:
interleaved with data from any other thread

There are no notes attached to this issue.




Viewing Issue Advanced Details
1429 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Omission 2020-12-02 16:57 2020-12-04 22:49
Konrad_Schwarz
 
normal  
New  
Open  
   
Konrad Schwarz
Siemens AG
recvfrom timeout
recvfrom
1
1
---
recvfrom does not specify the errno on timeout when the SO_RCVTIMEO option has been set
POSIX specifies the SO_RCVTIMEO socket option, that causes a receiving call on a socket to timeout when an error occurs. However, the recvfrom() (and presumably the related calls recv(), read() etc.) do not specify what errno is set to in this case.
According to the Linux socket(7) man page, EAGAIN/EWOULDBLOCK is returned in this case. (ETIMEDOUT would also kind of make sense and is what I assumed originally).

Add to the description of EGAIN in ERRORS the possibility of a timeout due to SO_RCVTIMEO.
Notes
(0005148)
geoffclare   
2020-12-03 10:39   
The required behaviour is specified in XSH 2.10.16 Use of Options:

P530 L18640 (SO_RCVTIMEO):
If a receive operation has blocked for this much time without receiving additional data, it shall return with a partial count or errno shall be set to [EAGAIN] or [EWOULDBLOCK] if no data were received.

P531 L18665 (SO_SNDTIMEO):
If a send operation has blocked for this much time, it shall return with a partial count or errno set to [EAGAIN] or [EWOULDBLOCK] if no data were sent.


We could either repeat this detail on each affected function page or just add a reference to 2.10.16.
(0005149)
Konrad_Schwarz   
2020-12-03 10:48   
My apologies; I think I was a bit tired yesterday.

I think the suggested reference would be fine.
(0005150)
shware_systems   
2020-12-03 11:25   
I'm more partial to requiring EAGAIN, not either, reserving EWOULDBLOCK for discovered conditions unrelated to timing such as hardware power failure on other end, if these are 2 different values. If they are the same value this would be a case for requiring them to be different.
(0005151)
kre   
2020-12-03 13:59   
Re Note: 0005150

XBD 14 <errno.h> says:

    [EAGAIN] Resource unavailable, try again (may be the same value as
          [EWOULDBLOCK]).

    [EWOULDBLOCK] Operation would block (may be the same value as [EAGAIN]).

In an I/O operation they mean the same thing, the two are very often the
same errno number (which is why the standard says they may be).

Apart from that (which I suppose could be undone, at the expense of breaking
many systems, as changing these values totally destroys binary compat for
the system) "reserving EWOULDBLOCK for discovered conditions unrelated to
timing such as hardware power failure on other end" is definitely not
EWOULDBLOCK - that's EIO.

The only argument against EAGAIN/EWOULDBLOCK here (and it is a weak one) is
that those generally occur immediately, rather than after a timeout period.
ETIMEDOUT (if it had its description changed from "connection timed out" to
"operation timed out" would probably fit a little better for an operation which
tries for a while, then eventually gives up.

But what really matters is not what anyone believes makes sense, but what
the actual implementations return.
(0005159)
shware_systems   
2020-12-04 18:02   
I would expect EIO to be about the system calling the interface, not about conditions on the other end of a connection. This is because there are codes like EAGAIN that are explicitly about the other end of a connection. Maybe some implementations don't differentiate and just use EIO as a blanket report code, given the terseness of "I/O error" as its definition in <errno.h>; maybe some have EAGAIN and EWOULDBLOCK as the same value, and use that where some other implementation might use EIO, but other systems may have 3 values. It is a definite portability issue on these systems if some return EAGAIN, and others EWOULDBLOCK; it forces application code to test for both constants, bloating both code and exec time, and does not give an application sufficient info about whether to retry or abort the attempt.

There's not a single one for indicating a timeout occurred but should complete eventually (nothing broke, latency is just higher than was expected), and a different one that says something happened where it can't tell if the operation might complete (something did break, but it wasn't corrected on other end before the timeout expired), and even a third for something broke on our end and any retry can't succeed until that's fixed.




Viewing Issue Advanced Details
1428 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Objection Omission 2020-11-27 14:56 2020-11-27 14:57
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
3
71,79
2069,2270
---
Add definitions of "negative" and "positive"
A discussion on the mailing list that arose from Note: 0005108 identified the need to add definitions of the terms "negative" and "positive".
On page 71 line 2069 section 3, insert:

3.239 Negative
When describing a value (not a sign), less than zero. Note that in the phrase ``negative zero'' it describes a sign, and therefore negative zero (also represented as -0.0) is not a negative value.
and renumber the later subsections.

On page 79 line 2270 section 3, insert:

3.285 Positive
When describing a value (not a sign), greater than zero. Note that in the common phrase ``positive zero'' (which is not used in this standard, although the representation +0.0 is) it describes a sign, and therefore positive zero (+0.0) is not a positive value.
and renumber the later subsections.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1427 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2020-11-26 15:21 2020-12-01 18:40
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
2.8.1 Consequences of Shell Errors
2364
75421
---
2.8.1 does not take into account "command"
The table in XCU 2.8.1 does not take account of the fact that when the command utility is used, it prevents the shell from aborting when an error occurs in a special built-in utility, as per this statement in the description of command:
If the command_name is the same as the name of one of the special built-in utilities, the special properties in the enumerated list at the beginning of Section 2.14 (on page 2384) shall not occur.

In the "Special built-in utility error" row of the table in 2.8.1 add a 1 superscript after "shall exit" in the "Non-Interactive Shell" column. Renumber the other superscripts. After the table insert a new note 1:
The shell shall exit only if the special built-in utility is executed directly. If it is executed via the command utility, the shell shall not exit.

Notes
(0005124)
kre   
2020-11-30 19:36   
(edited on: 2020-11-30 19:37)
It might be worth answering the question "what does happen when a shell
does not exit after an error which would normally be 'shall exit'?" in the
various situations where this can occur.

My impression (without a lot of testing for now) is that when the reason
the shell does not exit is because it is interactive, the normal behaviour
is to abort whatever processing was happening (the entire command/list/program
that has been input) and return to the top level, issue PS1, and wait for
more from the command input stream.

At least that's the case for the top level shell, my impression is that even
for interactive shells, when encountering a "shall exit" in a sub-shell
environment, that sub-shell exits (I'm not sure if this is specified somewhere,
it might be.)

On the other hand, when not exiting because "command" has turned a special
builtin error into a "not exit" condition, my (so far limited) testing
indicates that shells simply continue with whatever follows the "command"
command in the normal sequence. That's not at all the same as what happens
with an interactive shell where the same special builtin (but without the
"command" command being used) fails for whatever reason it is failing.

An easy test setup for this is to use the special builtin "eval" as that
one is trivially easy to make fail.

The two scenarios are, in an interactive shell

        eval 'nonsense ><; foobar'; printf 'continued\n'

and in any shell

        command eval 'nonsense ><; foobar'; printf 'continued\n'

and see wehen there are differences.

(0005125)
geoffclare   
2020-12-01 16:28   
Re Note: 0005124

Looks like you've identified a related problem with this text at the end of 2.8.1:
In all of the cases shown in the table where an interactive shell is required not to exit, the shell shall not perform any further processing of the command in which the error occurred.

I think this should change to:
In all of the cases shown in the table where an interactive shell is required not to exit and a non-interactive shell is required to exit, an interactive shell shall not perform any further processing of the command in which the error occurred.


(The subshell-of-an-interactive-shell behaviour is already the subject of bug 0001384.)
(0005129)
kre   
2020-12-01 18:40   
(edited on: 2020-12-01 19:13)
Re bugnote: 5125

First, for subshells, yes, 0001384 will take care of that.

For the rest, the proposed change leaves unspecified (I believe) just
what "the command" in "the command in which the error occurred" actually
is. Clearly the actual simple command (usually) which fails should not
be executed any further, but if that command is a part of a pipeline, or
a list, or is embedded in a function, or ... (there are many possibilities)
then is only that one simple command not executed, and everything else
should continue (an interesting concept with something like:

     grep foo /some/file | command eval 'nonsense ><;' | wc -l

and I'm not sure how I'd do that), or is something more than just "the
command" to not be executed, and if so, how much, and how do we specify
that.

I did a little more testing after sending Note: 0005124, and found that
with the command line

   eval 'nonsense ><; foobar'; printf 'continued\n'

given to an interactive shell, yash, bash, ksh93, and zsh print "continued"
whereas the ash derived shells (dash, FreeBSD (my version is getting old now)
and NetBSD), and bosh and mksh do not.

bosh and mksh also do not print "continued" when the command line
is changed to be:

   command eval 'nonsense ><; foobar'; printf 'continued\n'

whereas the ash derived shells do. The others (naturally) continue to
print it.

When not interactive, executing

    $SHELL -c "eval 'nonsense ><; foobar'; printf 'continued\n'"

ksh93 and zsh still print "continued" (as does bash when not in posix mode).
The other shells tested (including bash in posix mode) do not.

With the "command" prefix, that is:

    $SHELL -c "command eval 'nonsense ><; foobar'; printf 'continued\n'"

we cannot test zsh, as its "command" command ignores builtins (special or
not) and only runs filesystem commands, and my system (NetBSD) has no
filesystem command for "eval" (as a special builtin, one isn't required,
but the same happens with "eval cd ..." with zsh as we don't have a cd
filesystem command either). It does however still print "continued".

Of the others all except yash bosh and mksh print "continued" (yash
actually reports 2 syntax errors, one for each of the < and > redirect
operators which are both missing operands). It does that with or without
the "command" command being used.

My summary is that bosh and mksh simply return to the PS1 prompt level on
any "must exit" error, regardless of the use of "command" and then if
non-interactive just exit. That seems broken. Perhaps.

bash in non-posix mode basically ignores the "special" in special builtin
I think, which explains its behaviour then. Explaining what ksh93 and
yash do is harder.

For interactive shells there's a clear difference between the ash derived
shells and the others (the ones not already discounted as broken) over
just how much "command" should be aborted on an otherwise "must exit"
error.

Note that in the above tests, none of the shells attempted to run "foobar"
(ie: that never elicited a command not found error message ... there is no
"foobar" command (or function, or anything similar) on my system).


For the pipeline case above, I actually just tested:

    grep NetBSD /etc/motd | eval 'nonsense ><;' | wc -l
and
    grep NetBSD /etc/motd | command eval 'nonsense ><;' | wc -l

Much to my surprise all shells (really all) went ahead and ran the
pipeline (both cases) and printed "0" as the line count (The word
NetBSD does in appear in /etc/motd). The grep is also run, as when
I used (cut and paste from above) "/some/file" I got an error from
grep (all shells) about that file not existing. This might be explained
by the commands in a pipeline being run in a sub-shell environment, but
only really if each command in the pipeline is run in its own environment.

For the case of functions, given a function definition

    badfunc() {
        $1 eval 'nonsense ><;'
        printf 'continued\n'
    }

interactive shells printed "continued" exactly in the same cases as
above (when the function was called as "badfunc" and "badfunc command".

When that was run in a non-interactive shell

    $SHELL -c "badfunc() {
        $1 eval 'nonsense ><;'
        printf 'continued\n'
    }
    badfunc command
    badfunc
    printf 'done\n'"

again the results were more or less as above (including the difference
for bash in or not in posix mode) - except that the ash derived shells
seem to ignore the "command" use (passed as an arg to badfunc) never print
"continued" at all, and only issue one syntax error. bash in posix mode
generates a syntax error for each call of badfunc, but never prints "continued",
it does not print "done". mksh is similar to bash in posix mode here, but
does print "done". bash (in non-posix mode) ksh93, and zsh, are the only
shells that print "continued" (twice) along with 2 error messages (they also
all print done). yash probably is closest to being "correct" (whatever that
is) - it prints 2 syntax error messages, and nothing else.

All this looks like a giant mess to me. For what it is worth, this is
an area I have no problems with making changes to the way the NetBSD
shell works, if we can ever arrive at a sensible definition of what should
be done in all of these cases.





Viewing Issue Advanced Details
1426 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Objection Clarification Requested 2020-11-26 10:45 2020-11-26 10:52
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
4.12 Memory Synchronization
111
3066-3075
---
Clarifications regarding functions that synchronize memory
XBD 4.12 needs some clarifications regarding the conditions under which the functions in the table synchronize memory.

In Issue 6 it appears that the intent was that functions in the table always synchronize memory (on successful calls). A separate statement about pthread_once() was made after the table because it was only the first call in each thread (for a given pthread_once_t object) that needed to synchronize memory.

However, later changes have muddied the waters, in particular the addition of a paragraph about recursive mutexes, which now means that there are mutex functions in the table which do not always synchronize memory (on successful calls).

The statement "Unless explicitly stated otherwise, if one of the above functions returns an error, it is unspecified whether the invocation causes memory to be synchronized" has also become a problem following the addition of robust mutexes, since calls that return EOWNERDEAD lock the mutex and therefore should be required to synchronize memory.

Finally, the pthread_mutex_setprioceiling() function is missing from the table.
On page 111 line 3065 section 4.12, change:
The following functions synchronize memory with respect to other threads
to:
The following functions shall synchronize memory with respect to other threads on all successful calls

On page 111 line 3066-3075 section 4.12, remove all pthread_mutex_*() functions, pthread_cond_wait(), and pthread_cond_timedwait() from the table.
 
On page 111 line 3076-3079 section 4.12, change:
The pthread_once() function shall synchronize memory for the first call in each thread for a given pthread_once_t object. If the init_routine called by pthread_once() is a cancellation point and is canceled, a call to pthread_once() for the same pthread_once_t object made from a cancellation cleanup handler shall also synchronize memory.
to:
The pthread_once() function shall synchronize memory for the first successful call in each thread for a given pthread_once_t object. If the init_routine called by pthread_once() is a cancellation point and is canceled, a successful call to pthread_once() for the same pthread_once_t object made from a cancellation cleanup handler shall also synchronize memory.

On page 111 line 3080-3083 section 4.12, change:
The pthread_mutex_lock() function need not synchronize memory if the mutex type if PTHREAD_MUTEX_RECURSIVE and the calling thread already owns the mutex. The pthread_mutex_unlock() function need not synchronize memory if the mutex type is PTHREAD_MUTEX_RECURSIVE and the mutex has a lock count greater than one.
to:
The pthread_mutex_lock(), [RPP|TPP]pthread_mutex_setprioceiling(),[/RPP|TPP] pthread_mutex_timedlock(), and pthread_mutex_trylock() functions shall synchronize memory on all calls that acquire the mutex, including those that return [EOWNERDEAD]. The pthread_mutex_unlock() function shall synchronize memory on all calls that release the mutex.
<small>Note: If the mutex type is PTHREAD_MUTEX_RECURSIVE, calls to the locking functions do not acquire the mutex if the calling thread already owns it, and calls to pthread_mutex_unlock() do not release the mutex if it has a lock count greater than one.</small>
The pthread_cond_wait() and pthread_cond_timedwait() functions shall synchronize memory on all calls that release and re-acquire the specified mutex, including calls that return [EOWNERDEAD], both when the mutex is released and when it is re-acquired.
<small>Note: If the mutex type is PTHREAD_MUTEX_RECURSIVE, calls to pthread_cond_wait() and pthread_cond_timedwait() do not release and re-acquire the mutex if it has a lock count greater than one.</small>

On page 111 line 3084-3085 section 4.12, change:
Unless explicitly stated otherwise, if one of the above functions returns an error, it is unspecified whether the invocation causes memory to be synchronized.
to:
Unless explicitly stated otherwise, if one of the functions named in this section returns an error, it is unspecified whether the invocation causes memory to be synchronized.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1425 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2020-11-23 23:10 2020-11-23 23:10
kre
 
normal  
New  
Open  
   
Robert Elz
XCU 2.14 / exit
2399
76764-5
---
exit in an EXIT trap is not clearly specified
the page for the special builtin command "exit" says

    A trap on EXIT shall be executed before the shell terminates,
    except when the exit utility is invoked in that trap itself,
    in which case the shell shall exit immediately.

The issue is what "that trap itself" means.

Consider

   $SHELL -c 'trap "trap \"echo bar\" EXIT; echo foo; exit" EXIT'

When the exit command in the outer exit trap is executed, the EXIT
trap has been changed to be something different, that is, the
shell is not executing the (current) EXIT trap when that exit command
is executed, so by a literal reading of the standard, the inner trap
should fire, and the shell should print

    foo
    bar

before exiting.

That is what the FreeBSD and NetBSD shells, and bosh, mksh, and zsh do.
bash yash ksh93 and dash print only "foo", that is they seem to ignore
the "that" in "that exit trap" and treat the text as if it said

    except when the exit utility is invoked in an exit trap

The page in XCU 2.14 for trap says nothing pertinent to this issue
(I am not suggesting that it should, just noting it).
Since the "reference" shells seem to do this the broken way, it is
unlikely that the standard will have words (or an example) added to
reinforce the current wording (I am guessing).

But since the other shells are doing exactly what the standard requires
of them, it would be incorrect, and unfair to specify this so as to
require the currently non-standard behaviour that the reference shells
implement.

Given that, the best that can be done, I believe, is to add a new
sentence, immediately after the one quoted above (which is unchanged
in 202x drafts) something like

    It is unspecified whether an EXIT trap defined while executing
    a previous EXIT trap will be executed before the shell terminates
    after completing execution (via an exit command, or reaching the
    end) of the earlier EXIT trao.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1424 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Clarification Requested 2020-11-22 10:19 2020-11-23 09:35
rillig
 
normal  
New  
Open  
   
Roland Illig
make
(page or range of pages)
(Line or range of lines)
---
Inconsistent spelling of single suffix vs. single-suffix
* it is a single-suffix inference rule
* the single suffix inference rules
* The single-suffix inference rules
* one of the single suffixes appended
* SINGLE SUFFIX RULES

The words single-suffix should be spelled consistently.
Replace "single suffix" with "single-suffix" everywhere since it is meant to be an adjective.
Notes
(0005120)
geoffclare   
2020-11-23 09:35   
I have moved this bug to the Issue7+TC2 project, as this problem exists in C181.pdf.

The 2nd and 5th bullet points in the description are the cases that need a hyphen added. The 4th ("one of the single suffixes appended") does not need a hyphen, as "single suffixes" is not being used as an adjective there.




Viewing Issue Advanced Details
1423 [Issue 8 drafts] Shell and Utilities Objection Clarification Requested 2020-11-18 21:28 2020-11-19 14:40
joerg
 
normal  
New  
Open  
   
Jörg Schilling
make
2888
97001-97006
The make commandline argument macro::=value has unspecified bahavior
It is common accepted behavior that

  make macro=value

overwrites definitions from makefiles. This can be implemented in
two ways

- evaluate the command line argument before reading makefiles
  and mark the related macro readonly

- evaluate the command line argument after reading makefiles

Both methods result in the same result (even though the assumption
that command line arguments overwrite makefiles should be specified
in the standard). Note that the current text allows a make implementation
to evaluate command line macro assignments before reading makefiles
witout making them readonly and if this would be done, the assignments
in the makefiles would overwrite the commandline arguments.

With the command

  make macro::=value

the two possible implementations mentioned above give different
results.

My gut feeling is that people would expect "make 'macro::=$(macro2)'"
expands $(macro2) as defined in the makefiles instead of expanding
it as empty macro value, but this needs to be specified in the standard.
After line 97005 after the text "in the order they appear"

insert the text:

as if the command line macro definitions were evaluated after reading the makefiles
Notes
(0005115)
geoffclare   
2020-11-19 09:38   
The claim that the behaviour is unspecified is incorrect, and the suggested change would contradict existing text that specifies the behaviour here (not because it would require different behaviour, but because the existing text handles this a different way). See draft 1.1 page 2896 lines 97343-97361 noting in particular line 97344 "before the makefile(s) are read" and lines 97360-97361 "Macros defined in the makefile(s) shall not override macro definitions from source 1 or source 2."

Can we process this bug as withdrawn?
(0005116)
joerg   
2020-11-19 11:29   
The problem with that text is that it makes macro::=value on the
command line a useless feature since at that time, there are no
macro definitions that could be expanded.

So it would be a wise idea to remove macro::= value from the beginning
of line 97002.

This definitely needs to be discussed.

(0005117)
geoffclare   
2020-11-19 12:19   
There can be macro definitions from earlier command line macro assignments. E.g.:
echo 'all: ; echo $(BAR)' | make -f - FOO=1 BAR::='$(FOO)' FOO=2
is required to output 1 whereas with BAR='$(FOO)' it would output 2.
(0005118)
joerg   
2020-11-19 13:39   
(edited on: 2020-11-20 11:58)
Just because you are correct with this note, people would
intuitively expect

   make BAR::='$(FOO)'

to expand previous definitions from FOO in the makefiles.

I know this is harder to implement, but shouldn't we avoid
to standardize unexpected results?

(0005119)
geoffclare   
2020-11-19 14:40   
There is no such thing as "previous definitions from FOO in the makefiles". The standard clearly states that macro definitions are taken from the command line before the makefile(s) are read, as I pointed out in my first note.

Of course the usual "as if" rule applies here.




Viewing Issue Advanced Details
1422 [Issue 8 drafts] Shell and Utilities Objection Error 2020-11-18 13:49 2020-11-18 13:49
joerg
 
normal  
New  
Open  
   
Jörg Schilling
2894-2895
97277-97282
The current definition for the += assingment is in conflict with existing behavior of make and unpredictable/error-prone
The += operator has been introduced in January 1986 by SunPro Make.
It is defined to behave the same way as the = operator (to use delayed
expansion) regardless of how the assigned macro has been used before.

The current text in the standard does not seem to be compatible to long
existing behavior and it would be higly error-prone in complex makefiles
in special if these makefiles make use of the include directive.

Complex makefiles that make use of the include directive usually have
previous macro assignments in files from global make-include directories.
The author of a specific makefile thus cannot easily know whether there
was a previous = or ::= assinment for a specific macro since this would
require to manually scan many include files. Even worse: Which makefiles
are included is not easy to predic in object oriented makefile systems
that include different makefiles depending on the content of make
macros. As a result, the current standard text would cause
unpredictable results of the whole.

It seems to be better and more flexible to introduce a new operator +:= or
:+= for immediate expansion of the string to append, or to document this
work-around using the helper macro VAR2:

VAR2 := $(VAR3)
VAR = some text
VAR += $(VAR2)

instead of

VAR = some text
VAR += $(VAR3)

to enforce quasi-immediate expansion of the appended text.

In order to avoid an incompatible change for gmake users, we could
permit gmake to continue to support its deviating bahavior for the
non-standad := operator and require the POSIX ::= operator to behave
in a predictable way as proposed in Desired Action below.

P.S.:

Immediate expansion cannot be seen as completely new behavior because
SunPro Make in January 1986 already introduced immediate assignment in
contrast to delayed assignment for this:

VAR :sh= command

which is an immediate command substitution that calls the shell once
at parse time and this construct has been in use since then (using
the echo command) to emulate the behavior that recently has been
standardized for ::=

This problem was discovered while implementing ::= for SunPro Make
Change the text in line 97277-97282 to:

  If the macro named by string1 exists, then a <space> character followed
  by the unevaluated string2 shall be appended to the value currently
  assigned to the value of the macro named by string.



To make the standard compatible to the existing behavior.

Discuss the proposal for +:= or :+= and for the standard and write
a related proposal, or document the workaround mentioned above.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1421 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Comment Clarification Requested 2020-11-10 12:13 2020-11-10 19:49
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
make
2983
99076
---
make APPLICATION USAGE about default rules
The make APPLICATION USAGE section includes this statement:
The best way to provide portable makefiles is to include all of the rules needed in the makefile itself.
It seems that its purpose is to advise writers of portable applications not to rely on the default rules. (It is followed by other text relating to the default rules.) However, following the addition of include lines to the standard, it can be misinterpreted as advising against the use of include lines.

The problem could be fixed by changing "in the makefile itself" to "in the makefile(s), and any include files, themselves", but since the advice seems dubious I would prefer just to remove it (and make some adjustments to the text that follows it).
On page 2983 line 99076 section make (APPLICATION USAGE), change:
The best way to provide portable makefiles is to include all of the rules needed in the makefile itself. The rules provided use only features provided by other parts of this volume of POSIX.1-2017. The default rules include rules for optional commands in this volume of POSIX.1-2017. Only rules pertaining to commands that are provided are needed in an implementation's default set.
to:
The standard set of default rules use only features provided by other parts of this volume of POSIX.1-2017. They include rules for optional utilities in this volume of POSIX.1-2017, but only rules pertaining to utilities that are provided are needed in an implementation's default set.

Notes
(0005112)
psmith   
2020-11-10 19:49   
I agree that removing the advice altogether is the best plan.




Viewing Issue Advanced Details
1420 [1003.1(2004)/Issue 6] Shell and Utilities Editorial Clarification Requested 2020-11-07 20:27 2020-11-08 23:43
psmith
ajosey  
normal  
Under Review  
Open  
   
Paul Smith
GNU Project
make
---
Clarification: where are internal macros available?
In the section "Internal Macros" we see:

> The make utility shall maintain five internal macros that can be used in target and inference rules.

It is not clear from this text what is meant by "can be used in ... rules". In GNU make for example, internal macros (called "automatic variables" in the GNU make manual) are available only within the commands of a rule but not available within the prerequisites of a rule.

Does the statement "can be used in" mean to suggest that these macros can be used in the prerequisites list of a given rule as well?

If so, we must clarify what it means when a non-$@ variable is used in the prerequisites list.
Change the text:

The make utility shall maintain five internal macros that can be used in target and inference rules.

to:

The make utility shall maintain five internal macros that can be used in the commands of target and inference rules.
Notes
(0005103)
joerg   
2020-11-07 22:01   
Your seem to miss that the macros $< and $* are undefined in case of explicit rules.

smake does not expand $< or $* for explicit rules.

SunPro Make and gmake both expand these macros, but use completely incompatible rules to expand these macros in case of explicit rules. These rules result in the same expansion for some constraints and give different results for other constraints. Your text thus is inacceptable for standardization.

BTW: The background for this question was a question from a bug-tracking system that has been caused by an incompatibility in the BSD make program and was related to the use of $@ in the dependency list of an explicit target rule.

Make macros seen in the dependency list of a target rule are expanded by the parser already and at that time, $@ is undefined.
(0005104)
psmith   
2020-11-08 13:42   
I am aware of the difference in $< and $* in explicit rules. My suggested change is not related to that, and doesn't make this behavior any more or less defined WRT POSIX than it is now, so that is not relevant as to whether the suggested change is acceptable. If you wanted to make the handing of these internal macros inside of explicit rules more clear than it currently is, please file a separate issue and we can discuss there.

> Make macros seen in the dependency list of a target rule are expanded by the parser already and at that time, $@ is undefined.

You (and I) may _believe_ that, but the text of the standard doesn't _say_ that. Hence my request for clarification.

If you think the standard DOES say that already please quote the text that says so.

My suggested change ONLY makes clear that internal macros are not defined except for inside commands. It is not intended to address any other issue than that.
(0005105)
joerg   
2020-11-08 16:17   
In order to avoid missinterpretations with the dynamic macros, I like to make an alternate proposal that is aligned with the behavior of the UNIX make utility.

Change the following sentence on page 2898 line 97447 from:

      The make utility shall maintain five internal macros that can be used in
      target and inference rules.

into:

      The make utility shall maintain five internal macros that shall be expanded
      in the command sections of rules. Not all of these internal macros are
      expanded with all types of rules, see the detailed description below and
      the rationale section for the the System V dynamic dependency feature.
(0005106)
psmith   
2020-11-08 16:52   
I don't see the point in making this statement, since each explanation of the individual internal macros explains which types of rules it is required for and which it is not required for. For example, the $< macro description already says "The meaning of the $< macro shall be otherwise unspecified." (other than implicit rules and .DEFAULT), etc.

I don't think it's needed to refer to the rationale section from within the standard like this; the existing text is very clear on this point without it.

My concern here is specifically whether internal macros need to be assigned before the prerequisites are expanded, or not. The standard is not clear (enough) on this point, in my opinion.

Perhaps a change like this would be sufficient:

  The make utility shall maintain five internal macros that will be assigned
  while the commands of target and inference rules are evaluated, as described
  below.

Just to clearly point people to the individual macro discussions to follow.
(0005107)
joerg   
2020-11-08 21:30   
The proposed text from Note: 0005106 does not really contain a new
statement, since that information is already available if you
combine existing statements from different locations in the
standard text for the make utility.

On the other side, your proposal still does not seem to directly answer the
related question from your customer in your bug data base.

The dynamic macros are specified as returning values that are only
available while a target rule or an inference rule is evaluated.

On the other side, the standard already mentions that while reading
target rules by the parser, contained macro references in every
target rule need to be expanded immediately. At that time, a value
for $@ however is not available, so the rule

tgt: $@.c

needs to be expanded into

tgt: .c

while specifying

tgt: $$@.c

is required to expand to:

tgt: $@.c

at parse time and that may later be expanded as dynamic dependency
at the time the rule is evaluated. Well, in case the make program
implements support for the dynamic dependency feature that is not
required by POSIX.
(0005109)
psmith   
2020-11-08 23:43   
> At that time, a value for $@ however is not available,

You are _assuming_ that, because that's how implementations of make (that attempt to be POSIX compliant) work today: the standard does not _say_ that they are not available. That's the problem.

If you can find text in the standard that says that $@ is not set before prerequisites are expanded, please quote that text.

There's nothing in the standard that I can find that says that make doesn't have to work like this:

* Read in the rule definition line
* Expand the target
* Assign the internal macro $@ to the name of the target
* Expand the prerequisites

And in fact, the current wording of the standard is ambiguous enough that it could be argued that it IS required: the standard says that internal variables "can be used in target and inference rules". The prerequisites are definitely part of target and inference rules.

All I want in this issue is to clear this one thing up. I'm not trying to add new requirements, discuss $$@, etc. Merely clarifying that internal macros are only required to be available in the commands, not "in target and inference rules" generally, is sufficient and nothing else needs to be added.




Viewing Issue Advanced Details
1419 [1003.1(2004)/Issue 6] Shell and Utilities Editorial Error 2020-11-07 20:20 2020-11-07 20:20
psmith
ajosey  
normal  
Under Review  
Open  
   
Paul Smith
GNU Project
make
---
Missing newline / indentation in .SCCS_GET default rule definition
Apologies for not including the page and line number: I'm not sure where to get a copy of the standard with this information included.


The description of the .SCCS_GET special target says:

he application shall ensure that this special target is specified without prerequisites. If this special target is included in a makefile, the commands specified with this target shall replace the default commands associated with this special target (see Default Rules). The commands specified with this target are used to get all SCCS files that are not found in the current directory.

However, the example provided in the default rules section shows:

.SCCS_GET: sccs $(SCCSFLAGS) get $(SCCSGETFLAGS) $@

This shows the commands as prerequisites of the special target, which is not correct.
Change:

.SCCS_GET: sccs $(SCCSFLAGS) get $(SCCSGETFLAGS) $@


to:

.SCCS_GET:
        sccs $(SCCSFLAGS) get $(SCCSGETFLAGS) $@
There are no notes attached to this issue.




Viewing Issue Advanced Details
1418 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Comment Enhancement Request 2020-11-06 10:32 2020-11-24 10:09
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
ulimit
3326-3327
111991-112065
---
Add options to ulimit to match get/setrlimit()
Currently the ulimit utility only provides the means to query and set the RLIMIT_FSIZE soft limit. It should be able to query and set the hard limit, and also the soft and hard limits for the other resources that are specified for getrlimit() and setrlimit(). The -a option would also be useful.

The proposed additions are widely supported.
Replace the NAME, SYNOPSIS, DESCRIPTION, OPTIONS, and OPERANDS sections on the ulimit page with:

NAME
ulimit -- report or set resource limits

SYNOPSIS
[XSI]ulimit [-H|-S] -a

ulimit [-H|-S] [-cdfnstv] [newlimit][/XSI]

DESCRIPTION
The ulimit utility shall report or set the resource limits in effect in the process in which it is executed.

Soft limits can be changed by a process to any value that is less than or equal to the hard limit. A process can (irreversibly) lower its hard limit to any value that is greater than or equal to the soft limit. Only a process with appropriate privileges can raise a hard limit.

The value unlimited for a resource shall be considered to be larger than any other limit value. When a resource has this limit value, the implementation shall not enforce limits on that resource. In locales other than the POSIX locale, ulimit may support additional non-numeric values with the same meaning as unlimited.

The behavior when resource limits are exceeded shall be as described in the System Interfaces volume of POSIX.1-202x for the setrlimit() function.

OPTIONS
The ulimit utility shall conform to [xref to XBD 12.2].

The following options shall be supported:

-H
Report hard limit(s) or set only a hard limit.
-S
Report soft limit(s) or set only a soft limit.
-a
Report the limit value for all of the resources named below and for any implementation-specific additional resources.
-c
Report, or set if the newlimit operand is present, the core image size limit(s) in units of 512 bytes. [RLIMIT_CORE]
-d
Report, or set if the newlimit operand is present, the data segment size limit(s) in units of 1024 bytes. [RLIMIT_DATA]
-f
Report, or set if the newlimit operand is present, the file size limit(s) in units of 512 bytes. [RLIMIT_FSIZE]
-n
Report, or set if the newlimit operand is present, the limit(s) on the number of open file descriptors, given as a number one greater than the maximum value that the system assigns to a newly-created descriptor. [RLIMIT_NOFILE]
-s
Report, or set if the newlimit operand is present, the stack size limit(s) in units of 1024 bytes. [RLIMIT_STACK]
-t
Report, or set if the newlimit operand is present, the per-process CPU time limit(s) in units of seconds. [RLIMIT_CPU]
-v
Report, or set if the newlimit operand is present, the address space size limit(s) in units of 1024 bytes. [RLIMIT_AS]
Where an option description is followed by [RLIMIT_name] it indicates which resource for the getrlimit() and setrlimit() functions, defined in the System Interfaces volume of POSIX.1-202x, the option corresponds to.

If neither the -H nor -S option is specified:

  • If the newlimit operand is present, it shall be used as the new value for both the hard and soft limits.

  • If the newlimit operand is not present, -S shall be the default.


If no options other than -H or -S are specified, the behavior shall be as if the -f option was (also) specified.

OPERANDS
The following operand shall be supported:

newlimit
Either an integer value to use as the new limit(s) for the specified resource, in the units specified in OPTIONS, or a non-numeric string indicating no limit, as described in the DESCRIPTION section. Numerals in the range 0 to the maximum limit value supported by the implementation for any resource shall be syntactically recognized as numeric values.

Replace the STDOUT section on the ulimit page with:
The standard output shall be used when no newlimit operand is present.

If the -a option is specified, the output written for each resource shall consist of one line that includes:

  • A short phrase identifying the resource (for example "file size").

  • An indication of the units used for the resource.

  • The ulimit option used to specify the resource.

  • The limit value.


The format used within each line is unspecified, except that the format used for the limit value shall be as described below for case where the -a option is not specified.

If the -a option is not specified:

  • If the resource being reported has a numeric limit, the limit value shall be written in the following format:
    "%1d\n", <limit value>
    where <limit value> is the value of the limit in the units specified in OPTIONS.

  • If the resource being reported does not have a numeric limit, in the POSIX locale the following format shall be used:
    "unlimited\n"

Replace the APPLICATION USAGE, EXAMPLES, and RATIONALE sections on the ulimit page with:

APPLICATION USAGE
Since ulimit affects the current shell execution environment, it is always provided as a shell regular built-in. If it is called with an operand in a separate utility execution environment, such as one of the following:
nohup ulimit -f 10000
env ulimit -S -c 10000
it does not affect the limit(s) in the caller's environment.

See also the APPLICATION USAGE for [xref to getrlimit()].

EXAMPLES
Set the hard and soft file size limits to 51200 bytes:
ulimit -f 100

Save and restore a soft resource limit (where X is an option letter specifying a resource):
saved=$(ulimit -X)

...

ulimit -X -S "$saved"

Execute a utility with a CPU limit of 5 minutes (using an asynchronous subshell to ensure the limit is set in a child process):
(ulimit -t 300; exec utility_name </dev/null) &
wait $!

RATIONALE
The ulimit utility has no equivalent of the special values RLIM_SAVED_MAX and RLIM_SAVED_CUR returned by getrlimit(), as ulimit is required to be able to output, and accept as input, all numeric limit values supported by the system.

Add XCU ulimit to the SEE ALSO section on the getrlimit() page.
Notes
(0005110)
geoffclare   
2020-11-09 10:38   
(edited on: 2020-11-24 10:04)
Updated proposal...

Replace the NAME, SYNOPSIS, DESCRIPTION, OPTIONS, and OPERANDS sections on the ulimit page with:

NAME
ulimit -- report or set resource limits

SYNOPSIS
[XSI]ulimit [-H|-S] -a

ulimit [-H|-S] [-cdfnstv] [newlimit][/XSI]

DESCRIPTION
The ulimit utility shall report or set the resource limits in effect in the process in which it is executed.

Soft limits can be changed by a process to any value that is less than or equal to the hard limit. A process can (irreversibly) lower its hard limit to any value that is greater than or equal to the soft limit. Only a process with appropriate privileges can raise a hard limit.

The value unlimited for a resource shall be considered to be larger than any other limit value. When a resource has this limit value, the implementation shall not enforce limits on that resource. In locales other than the POSIX locale, ulimit may support additional non-numeric values with the same meaning as unlimited.

The behavior when resource limits are exceeded shall be as described in the System Interfaces volume of POSIX.1-202x for the setrlimit() function.

OPTIONS
The ulimit utility shall conform to [xref to XBD 12.2], except that:
  • The order in which options other than -H, -S, and -a are specified may be significant.

  • Conforming applications shall specify each option separately; that is, grouping option letters (for example, -fH) need not be recognized by all implementations.

The following options shall be supported:

-H
Report hard limit(s) or set only a hard limit.
-S
Report soft limit(s) or set only a soft limit.
-a
Report the limit value for all of the resources named below and for any implementation-specific additional resources.
-c
Report, or set if the newlimit operand is present, the core image size limit(s) in units of 512 bytes. [RLIMIT_CORE]
-d
Report, or set if the newlimit operand is present, the data segment size limit(s) in units of 1024 bytes. [RLIMIT_DATA]
-f
Report, or set if the newlimit operand is present, the file size limit(s) in units of 512 bytes. [RLIMIT_FSIZE]
-n
Report, or set if the newlimit operand is present, the limit(s) on the number of open file descriptors, given as a number one greater than the maximum value that the system assigns to a newly-created descriptor. [RLIMIT_NOFILE]
-s
Report, or set if the newlimit operand is present, the stack size limit(s) in units of 1024 bytes. [RLIMIT_STACK]
-t
Report, or set if the newlimit operand is present, the per-process CPU time limit(s) in units of seconds. [RLIMIT_CPU]
-v
Report, or set if the newlimit operand is present, the address space size limit(s) in units of 1024 bytes. [RLIMIT_AS]
Where an option description is followed by [RLIMIT_name] it indicates which resource for the getrlimit() and setrlimit() functions, defined in the System Interfaces volume of POSIX.1-202x, the option corresponds to.

If neither the -H nor -S option is specified:

  • If the newlimit operand is present, it shall be used as the new value for both the hard and soft limits.

  • If the newlimit operand is not present, -S shall be the default.

If no options other than -H, -S, or -a are specified, the behavior shall be as if the -f option was (also) specified.

If the newlimit operand is present and more than one option other than -H or -S is specified, the behavior is unspecified.

If any option other than -H or -S is repeated, the behavior is unspecified.

OPERANDS
The following operand shall be supported:

newlimit
Either an integer value to use as the new limit(s) for the specified resource, in the units specified in OPTIONS, or a non-numeric string indicating no limit, as described in the DESCRIPTION section. Numerals in the range 0 to the maximum limit value supported by the implementation for any resource shall be syntactically recognized as numeric values.

Replace the STDOUT section on the ulimit page with:
The standard output shall be used when no newlimit operand is present.

If the -a option is specified, or the -a option is not specified and more than one option other than -H or -S is specified, the output written for each resource shall consist of one line that includes:

  • A short phrase identifying the resource (for example "file size").

  • An indication of the units used for the resource, if the corresponding option description in OPTIONS specifies the units to be used.

  • The ulimit option used to specify the resource.

  • The limit value.

The format used within each line is unspecified, except that the format used for the limit value shall be as described below for the case where a single limit value is written. If -a is not specified and output is written for more than one resource, it is unspecified whether the lines are written in the order that the corresponding options were specified or in the order that the lines would be written for the -a option.

If a single limit value is to be written; that is, the -a option is not specified and at most one option other than -H or -S is specified:

  • If the resource being reported has a numeric limit, the limit value shall be written in the following format:
    "%1d\n", <limit value>
    where <limit value> is the value of the limit in the units specified in OPTIONS.

  • If the resource being reported does not have a numeric limit, in the POSIX locale the following format shall be used:
    "unlimited\n"

Replace the APPLICATION USAGE, EXAMPLES, and RATIONALE sections on the ulimit page with:

APPLICATION USAGE
Since ulimit affects the current shell execution environment, it is always provided as a shell regular built-in. If it is called with an operand in a separate utility execution environment, such as one of the following:
nohup ulimit -f 10000
env ulimit -S -c 10000
it does not affect the limit(s) in the caller's environment.

See also the APPLICATION USAGE for [xref to getrlimit()].

EXAMPLES
Set the hard and soft file size limits to 51200 bytes:
ulimit -f 100

Report the hard limits for file size and core image size (in the verbose format used for -a):
ulimit -H -f -c

Save and restore a soft resource limit (where X is an option letter specifying a resource):
saved=$(ulimit -X)

...

ulimit -X -S "$saved"

Execute a utility with a CPU limit of 5 minutes (using an asynchronous subshell to ensure the limit is set in a child process):
(ulimit -t 300; exec utility_name </dev/null) &
wait $!

RATIONALE
The ulimit utility has no equivalent of the special values RLIM_SAVED_MAX and RLIM_SAVED_CUR returned by getrlimit(), as ulimit is required to be able to output, and accept as input, all numeric limit values supported by the system.

Add XCU ulimit to the SEE ALSO section on the getrlimit() page.

(0005121)
geoffclare   
2020-11-24 10:09   
(edited on: 2020-11-24 10:13)
Following discussions on the mailing list, I have updated Note: 0005110 as follows:

* Applications are required to specify each option separately (as for c99).

* Unspecified behaviour if a resource-selection option is repeated.

* Output lines only need to include the units where the corresponding option description specifies the units to be used.

One issue that remains is whether to keep the current requirement for multiple resource-selection options with no newlimit operand or to make the behaviour unspecified. This needs to be decided in a teleconference.





Viewing Issue Advanced Details
1417 [Issue 8 drafts] Shell and Utilities Editorial Clarification Requested 2020-11-03 17:14 2020-11-04 09:49
joerg
ajosey  
normal  
Under Review  
Open  
   
Jörg Schilling
Fraunhofer FOKUS
make
2910
97964
Need to make sure that makefiles with += and ?= are portable to all make implementations
In 0000330 we added support for += and ?= but forgot to mention that there need to be a <space> character before *= and ?=.

The background is that some make implementations as an extension support the characters '+' and '?' to appear in macro names and as a result, the parser needs a leading space to properly detect += and ?=.

Note that += is supported (wihth leading space) in SunPro Make since January 1986.
In draft 1.1 on page 2910 after line 97964 add:

Some make implementations (e.g. SunPro Make) support the characters '+' and '?' to appear in macro names and as a result, the parser needs a leading space to properly detect += and ?=. A conforming makefile needs to have a <space> character before += or ?=.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1416 [1003.1(2016/18)/Issue7+TC2] System Interfaces Editorial Error 2020-11-03 10:45 2020-11-03 10:46
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
getrlimit()
1086
36850-36881
---
Misuses of "may" on the getrlimit() page
The getrlimit() page has some uses of "may" that should be "can" or "shall", as they do not relate to optional behaviour of implementations. (I.e. the word has been used with its informal English-language sense rather than its formal definition in the Terminology section.)

There is some overlap with the bug 0001141 change to the RLIMIT_CORE description, but that is an issue8 tagged bug, and this one should be tagged tc3-2008, so this needs to be retrospectively applied "before" bug 1141.
On page 1086 line 36850 section getrlimit(), change:
Soft limits may be changed by a process to any value that is less than or equal to the hard limit. A process may (irreversibly) lower its hard limit to any value that is greater than or equal to the soft limit.
to:
Soft limits can be changed by a process to any value that is less than or equal to the hard limit. A process can (irreversibly) lower its hard limit to any value that is greater than or equal to the soft limit.

On page 1086 line 36861 section getrlimit(), change:
This is the maximum size of a core file, in bytes, that may be created by a process.
to:
This is the maximum size of a core file, in bytes, that can be created by a process.

(Note to the editor: when merging this with the bug 0001141 change, the end result should be just the change from "may" to "can" in the Issue 8 draft text.)

On page 1086 line 36871 section getrlimit(), change:
This is the maximum size of a file, in bytes, that may be created by a process.
to:
This is the maximum size of a file, in bytes, that can be created by a process.

On page 1086 line 36877 section getrlimit(), change:
This is a number one greater than the maximum value that the system may assign to a newly-created descriptor. If this limit is exceeded, functions that allocate a file descriptor shall fail with errno set to [EMFILE]. This limit constrains the number of file descriptors that a process may allocate.
to:
This is a number one greater than the maximum value that the system shall assign to a newly-created descriptor. If this limit is exceeded, functions that allocate a file descriptor shall fail with errno set to [EMFILE]. This limit constrains the number of file descriptors that a process can allocate.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1414 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Clarification Requested 2020-10-25 17:51 2020-10-26 10:35
kre
 
normal  
New  
Open  
   
Robert Elz
XSH 3.rewind
1821 (for info: in 202x d1.1 it is page 1740)
90114 (in 202x d1.1 it is line 57200)
---
rewind() clears the error indicator, but when?
rewind() is defined to be (void) fseek(stream, 0L, SEEK_SET)

(line 90114): except that rewind() shall also clear the error indicator.

That's fine, as it stands, but nothing here says whether the error
indicator is cleared before, or after, the fseek() call (or whatever
equivalent code the implementation uses in its stead).

This matters if the fseek() call sets the error indicator, which it can do.

Now we all know that the error indicator (after a rewind()) is always clear,
so it must be cleared after the fseek() (or equivalent) happens. But what
"we all know" is not in the standard, and should be.
In line 90114 (quoted in the description) insert the word "subsequently"
between the words "also" and "clear" to produce:

   except that rewind() shall also subsequently clear the error indicator.
Notes
(0005064)
geoffclare   
2020-10-26 10:35   
The text in question is derived directly from C99, so we should liaise with the C committee on this issue.




Viewing Issue Advanced Details
1413 [Issue 8 drafts] Shell and Utilities Objection Error 2020-10-24 22:01 2020-10-24 22:01
philip-guenther
 
normal  
New  
Open  
   
Philip Guenther
OpenBSD
printf's APPLICATION USAGE
3038
102845 - 102848
incorrect description of how a hexadecimal character constant can be terminated in ISO C
The text says:

> In the ISO C standard, the "##" concatenation
> operator can be used to terminate a constant and follow it with a hexadecimal character to be
> written. In the shell, concatenation occurs before the printf utility has a chance to parse the end
> of the hexadecimal constant.

The ## operator is neither necessary nor sufficient. All that's needed is use of string literal concatenation.

Indeed, this was an example in (at least) drafts of the C99 standard, in the specification of string literals lexical elements:

# EXAMPLE This pair of adjacent character string literals
#
# "\x12" "3"
#
# produces a single character string literal containing the two characters whose values are '\x12' and '3',
# because escape sequences are converted into single members of the execution character set just prior to
# adjacent string literal concatenation.
Replace the sentence:
  In the ISO C standard, the "##" concatenation operator can be used to terminate a constant and follow it with a hexadecimal character to be written.

With something like:
  In the ISO C standard, the constant and its following hexadecimal character can be placed in separate string literals as escape sequence processing occurs before string literal concatenation.

(I'm not really happy with that phrasing; better suggestions welcome!)
There are no notes attached to this issue.




Viewing Issue Advanced Details
1412 [Online Pubs] Shell and Utilities Editorial Error 2020-10-21 05:50 2020-10-21 07:59
andras_farkas
 
normal  
New  
Open  
   
Andras Farkas
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/pax.html [^]
pax
Potential missing newline in pax examples section
In the following text on
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/pax.html [^]

The following commands:
mkdir newdirpax -rw olddir newdir
copy the olddir directory hierarchy to newdir.

There seems to be a missing newline between newdir and pax.
Fix line to the following:
mkdir newdir
pax -rw olddir newdir
Notes
(0005056)
Don Cragun   
2020-10-21 07:59   
In the PDF for the 2017 edition, the text shown above appears correctly on P3094, L103297-103298




Viewing Issue Advanced Details
1411 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Enhancement Request 2020-10-19 22:15 2020-10-20 08:43
rillig
 
normal  
New  
Open  
   
Roland Illig
here-documents
(page or range of pages)
(Line or range of lines)
---
Missing paragraph separator in definition of here-documents
The current text reads:

If the redirection operator is "<<-", all leading <tab> characters shall be stripped from input lines and the line containing the trailing delimiter. If more than one "<<" or "<<-" operator is specified on a line, the here-document associated with the first operator shall be supplied first by the application and shall be read first by the shell.

This looks as if the second sentence would depend on the first sentence, which is not the case. Since these two sentences are completely independent, each sentence should be in a separate paragraph.
Add a paragraph break between the sentences.
Notes
(0005055)
Don Cragun   
2020-10-20 00:46   
In the PDF version of Issue 7 + TC2, this appears on P2362, L75374-75377.




Viewing Issue Advanced Details
1410 [1003.1(2016/18)/Issue7+TC2] System Interfaces Editorial Enhancement Request 2020-10-16 14:56 2020-10-16 14:56
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
asctime(), ctime()
600, 727
20871, 24759
---
Remove asctime_r() and ctime_r()
The obsolescent interfaces removed by bug 0001330 did not include asctime_r() and ctime_r() because they are in the current C2x draft standard. However, in the October 2020 WG14 meeting, the C committee agreed to remove them from C2x, which means we can now go ahead and remove them.

As with the other removals, we don't need to spell out the changes in detail with page and line numbers; the high level instructions in the desired action are sufficient.
Remove the asctime_r() and ctime_r() functions and all references to them (except in change history).
There are no notes attached to this issue.




Viewing Issue Advanced Details
1409 [1003.1(2013)/Issue7+TC1] System Interfaces Editorial Enhancement Request 2020-10-08 16:48 2020-10-09 08:38
alanc
 
normal  
New  
Open  
   
Alan Coopersmith
fstatat()
951
32669
---
Example code for stat() could use better function name
The example code for stat() currently has:

/* Print out type, permissions, and number of links. */
printf("%10.10s", sperm (statbuf.st_mode));
printf("%4d", statbuf.st_nlink);

It appears that "sperm" is supposed to be a user-provided function to
make a string representing the permission bits of the mode value, but
this isn't clear, and trying to do an internet search for "sperm" to
find if this function exists somewhere is extremely unhelpful.
Use a better function name for the user provided function - perhaps strmode()
that actually exists on some systems, or a clearer made-up-name such as
permission_string() or mode_to_string(). Perhaps update the comment to
note that this is a user-supplied function, not part of the standard.
Notes
(0005039)
geoffclare   
2020-10-09 08:38   
(edited on: 2020-10-09 08:41)
The comment indicates that the type is included, so the new name should include "mode" not "perm". The name strmode() should not be used for a user-supplied function as str[a-z] is a reserved prefix for <string.h>. So I suggest either str_mode() or mode_string().

Also, the %4d is wrong because is assumes nlink_t is int (or promotes to int). That line should change to:
printf(" %4ju", (uintmax_t)statbuf.st_nlink);
The later lines that print st_gid and st_uid have the same problem.





Viewing Issue Advanced Details
1408 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Enhancement Request 2020-09-30 20:48 2020-10-05 21:50
steffen
 
normal  
New  
Open  
   
steffen
Vol. 3: Shell and Utilities, mailx
2952
97836
---
mailx: add mta-bcc-ok internal variable
My mailx clone (will) document(s) it like this:

 mta-bcc-ok
           (Boolean) In violation of RFC 5322 some MTAs do not remove ‘Bcc:’
           header lines from transported messages after having noted the
           respective receivers for addressing purposes. (The MTAs Exim and
           Courier for example require the command line option -t to enforce
           removal.) Setting this variable asserts that the MTA performs
           stripping.
Adapt an internal variable mta-bcc-ok, default unset.
Notes
(0005015)
steffen   
2020-10-01 15:48   
To reiterate what i said on the ML, i think this would be a sensible addition because POSIX mailx allows creation of Bcc: headers via the ~b and ~h command escapes as well as via the askbcc internal variable.
(0005018)
geoffclare   
2020-10-02 13:59   
I'm struggling to understand why any user would want to set this variable. What would they gain from it? Whatever benefit it is, it would need to be weighed against the risk of a Bcc not being stripped if the MTA is later changed (by a system administrator) to one that doesn't strip it. (Or do you expect them to check that the MTA hasn't changed every time they send an email with an embarrassing-if-leaked Bcc?)

As regards adding it to the standard, there is the problem of how would a user decide whether it is safe to set it. We would have to require that the system documentation provides enough information for users to be able to decide. And if the system has multiple MTAs available, the documentation would naturally provide the relevant info for each one, necessitating that the user somehow work out which MTA is installed (or if multiple MTAs are installed, which one mailx will use).
(0005019)
steffen   
2020-10-02 21:51   
But .. first: as of today all mailx codebases known to me (that i looked into the last couple of years) except mine pass the readily prepared message through as-is. This potentially includes Bcc headers if they were generated.

  Remark: it is not only embarrassing-if-leaked, but many people of the free software world work for a very large free mail provider and due to the smartness of their system they easily "detected" that i have created multiple accounts a decade ago, and if suddenly the name of the one account happens to appear as a injection in visible headers of messages sent via another account, then this is not _only_ embarassing, but .. very funny! Heartbleed here.

Of course, "my" MUA has a complete "infrastructure" to allow users to specify the actually used MTA, and in this context the variable as such makes sense.
It would be nice if POSIX would support it, but like i said on the ML

  Important would be an equivalence to or some words about the
  *mta-bcc-ok* setting of my one, since POSIX mailx supports Bcc:
  via ~b, ~h and *askbcc*:

For completeness sake, we support a complete "infrastructure":

     mta Select an alternate Mail-Transfer-Agent by either specifying the
           full pathname of an executable (a ‘file://’ prefix may be given),
           or [Option]ally a SMTP aka SUBMISSION protocol URL:
...lots of text...

     mta-arguments
           Arguments to pass through to a file-based mta[473] (Mail-Transfer-
           Agent), parsed according to Shell-style argument quoting[23] into
           an array of arguments which will be joined onto MTA options from
           other sources, for example ‘? set mta-arguments='-t -X "/tmp/my
           log"'’.

Even if POSIX would offer configuration of the actually used (file-based) MTA, this one here requires a (more or less) complete (and correct) shell-style parser, which is not an easy task.

     mta-no-default-arguments
           (Boolean) Avoids passing standard command line options to a file-
           based mta[473] (please see there).

     mta-no-receiver-arguments
           (Boolean) By default all receiver addresses will be passed as com‐
           mand line options to a file-based mta[473]. Setting this variable
           disables this behaviour to aid those MTAs which employ special
           treatment of such arguments. Doing so can make it necessary to
           pass a -t via mta-arguments[475], to testify the MTA that it should
           use the passed message as a template.

     mta-argv0
           Many systems use a so-called mailwrapper(8)[728] environment to
           ensure compatibility with sendmail(1)[729]. This works by inspect‐
           ing the name that was used to invoke the mail delivery system. If
           this variable is set then the mailwrapper (the program that is
           actually executed when calling the file-based mta[473]) will treat
           its contents as that name.

     mta-bcc-ok
           (Boolean) In violation of RFC 5322 some MTAs do not remove ‘Bcc:’
           header lines from transported messages after having noted the
           respective receivers for addressing purposes. (The MTAs Exim and
           Courier for example require the command line option -t to enforce
           removal.) Setting this variable asserts that the MTA performs
           stripping.

(Maybe i can add a word of description for my MUA here.)
All mailx codebases i know have to duplicate the entire message in order to filter out the Bcc line when passing messages through to the MTA.

It can be made a bit simpler for less feature-rich implementations which do not support signing etc., however, they could just call puthead() without GBCC set,

And it has to be said that many mailx codebases support the internal variable "sendmail" instead of our mta like so

  if ((cp = value("sendmail")) != NOSTR)
    cp = _PATH_SENDMAIL;

(Research Unix v10 uses MAIL as a fallback, which is "/bin/mail".)
(0005020)
steffen   
2020-10-02 21:53   
I said

  It can be made a bit simpler for less feature-rich implementations which do not support signing etc., however, they could just call puthead() without GBCC set,

I have not actually tried it, but .. i think this should do it.
(0005021)
steffen   
2020-10-02 22:14   
(However, for *record* the full message has to be produced...)
(0005024)
geoffclare   
2020-10-05 08:22   
Perhaps the most appropriate solution would be for POSIX simply to state that Bcc is not included in the message that the "mail delivery system" (as it calls the MTA) delivers, and leave how this is achieved up to the implementation.
(0005028)
steffen   
2020-10-05 17:30   
Ok, how about that.

On page 2948, append after line 97657 )

  When the message is typed and the end of the message is encountered, the message shall be passed to the mail delivery software.

)

  The list of blind carbon copy (Bcc) recipients shall not be included as part of the generated message.
(0005029)
steffen   
2020-10-05 17:32   
P.S.: maybe instead "but shall be passed by other means [not covered by this specification]".

P.P.S.: sorry for the false parenthesis.
(0005031)
steffen   
2020-10-05 21:50   
But maybe you are right and this is just beyond the standard, and this issue should be closed as invalid.

It surely is a fault of MTAs aka mail delivery software, unfortunately it is real behaviour of (at least) two widely deployed POSIX/Unix programs around (of the four that i would count as big ones), and this not only affects mailx but also other mail user agents (including the one you, Geoff Clare, use).

Distributions patch(ed) some software to work around this, but applications also get regulary patched/changed to allow dealing with the situation lately.
The behaviour as such is in the world for many years.

Hm.
Maybe instead of the above a word a sentence like

  The mail delivery software shall interpret passed messages according to the rules of RFC 5322.

would be an acceptable way to deal with the situation?
RFC 5322 is very clear (as all standards before were if i recall correctly), and has a discussion in the "Security considerations" section, too:

   Many implementations use the "Bcc:" (blind carbon copy) field,
   described in section 3.6.3, to facilitate sending messages to
   recipients without revealing the addresses of one or more of the
   addressees to the other recipients. Mishandling this use of "Bcc:"
   may disclose confidential information that could eventually lead to
   security problems through knowledge of even the existence of a
   particular mail address.

Good night.




Viewing Issue Advanced Details
1407 [Online Pubs] Shell and Utilities Objection Error 2020-09-29 13:17 2020-10-02 10:52
dennisw
 
normal  
Applied  
Accepted  
   
Dennis Wölfing
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/expr.html [^]
expr
HTML edition has expr table in reversed order
In the HTML edition the table in the EXTENDED DESCRIPTION of the expr utility has its cells in the opposite order than the PDF.
Note that the order in the table determines the precedence, so the order matters.
Reverse the order of the table cells so that it matches the PDF.
Notes
(0005017)
agadmin   
2020-10-02 10:52   
The table in extended description has been corrected.




Viewing Issue Advanced Details
1406 [1003.1(2016/18)/Issue7+TC2] System Interfaces Editorial Clarification Requested 2020-09-28 21:26 2020-09-29 23:32
djdelorie
 
normal  
New  
Open  
   
DJ Delorie
Red Hat Inc
open_memstream
https://pubs.opengroup.org/onlinepubs/9699919799/functions/open_memstream.html [^]
n/a
---
clarification of SEEK_END when current pointer doesn't match buffer size
Consider a stream created by open_memstream(), where 16 bytes are written, fseek(0,SEEK_POS) to rewind, then write 4 bytes, and fflush(). At this point, the value pointed to by the sizep argument to open_memstream() should be 4 (please confirm).
At this point in the state of the stream, what are the semantics of SEEK_END? What will be the "file size" if you fclose() at this point?
The example explicitly SEEK_SETs to the buffer size before fclose(), eliding the issue.
Please clarify if SEEK_END is relative to the current position or the current buffer length, and if it's changed by a call to fflush() at that time.
Please clarify if a SEEK_SET to set the current pointer less than the current buffer size, itself (without read/write), changes the SEEK_END semantics, or the value stored in *sizep after fflush().
Notes
(0005011)
carlos   
2020-09-29 17:53   
This isn't an idle request, we are actively trying to harmonize the behaviour of implementations, particularly glibc, musl and FreeBSD to name three. The current wording upstream seems underspecified. At least one Red Hat customer has argued that it should match the behaviour of a normal file-backed stream to avoid confusion. It is clear to me that the intent of open_memstream was *not* to follow the behaviour of file-backed streams because of the special language and programming models that would use the API. Thus this request for guidance and clarification.
(0005012)
Don Cragun   
2020-09-29 20:54   
(edited on: 2020-09-29 20:55)
This bug has been moved from "Base Definitions and Headers" to "System Interfaces".

Noting that the rationale for open_memstream() refers to fmemopen(), I assume that the description of what happens when you use one of the fseek() or fseeko() functions on a memory stream are as described in fmemopen(). Note that in the upcoming revision ot the standard, part of the description of this behavior is changed by the resolution of 0000456.

But, the first paragraph of the description of open_memstream() only says that the stream is opened for writing. It doesn't specify whether it is supposed to use byte mode or text mode (and I believe the mode affects whether of not a NUL byte is written and writing that NUL byte determines whether or not the stream size changes).

(0005013)
djdelorie   
2020-09-29 23:32   
The reference to fmemopen() is noted, but as open_memstream() returns the *current* pointer in *sizep in the example, it clouds the issue of what the "end" of the data is - the API does not give the caller a way to determine the current buffer length if the current pointer is not coincident with it, and the NUL terminator is not specified, so having the "end of file" be somewhere undiscoverable seems problematic.
The example given by the standard explicltly uses ftell/fseek to position the current pointer at the "expected" end of data, implying that that would not otherwise be EOF had this not happened.
It is these differences in spec between open_memstream() and fmemopen() that confuse us, hence our request for clarification.




Viewing Issue Advanced Details
1405 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Clarification Requested 2020-09-25 11:12 2020-09-28 15:03
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
mailx
2944-2965
97476-98335
---
mailx header terminology needs to change to match RFC5322
The terminology used in relation to message headers is described in the first paragraph under "Commands in mailx" as follows:
In the following list, header refers to lines from the message header, as shown in the OUTPUT FILES section. Header-line refers to lines within the header that begin with one or more non-white-space characters, immediately followed by a <colon> and white space and continuing until the next line beginning with a non-white-space character or an empty line. Header-field refers to the portion of a header line prior to the first <colon> in that line.

This is confusingly different from the terminology used in RFC5322 (where "header field" means what the above calls "header-line", and "field name" means what the above calls "header-field") and also does not match many of the uses of the terms later in the mailx description. For example the description of the "discard" and "ignore" commands says:
Suppress the specified header fields when writing messages.

By a strict reading, this requires that the part prior to the first <colon> is suppressed, leaving the colon and the remainder of the header-line.

There are also some uses of "headers" where either "header-lines" or "header summaries" is intended, and uses of "line" where "header-line" is intended.

Finally, there is inconsistency as to whether the ":" is included when specifying header fields by name.
Everywhere on the mailx page, change all occurrences of "header-line" to "header field".

On page 2944 line 97476 section mailx (-F option), change:
... found first on the To: line in the mail header.
to:
... found first in the To field in the message header.

On page 2945 line 97536 section mailx (LC_CTYPE), change:
header-field comparisons
to:
header-field name comparisons

On page 2946 line 97573 section mailx (LC_CTYPE), change:
screenful of headers
to:
screenful of header summaries

On page 2949 line 97706 section mailx (deleted state), change:
search the subject lines of messages
to:
search the Subject header fields of messages

On page 2950 line 97733 section mailx (/string message specification), change:
All messages with string in the subject line (case ignored).
to:
All messages with string in the Subject header field (case ignored).

On page 2951 line 97779 section mailx (ask variable), change:
Prompt for a subject line on outgoing mail
to:
Prompt for a value for the Subject header field on outgoing mail

On page 2953 line 97862 section mailx (showto variable), change:
write the information from the To: line instead of the From: line in the header summary
to:
write the information from the To field instead of the From field in the header summary

On page 2953 line 97876 section mailx (Commands in mailx), change:
In the following list, header refers to lines from the message header, as shown in the OUTPUT FILES section. Header-line refers to lines within the header that begin with one or more non-white-space characters, immediately followed by a <colon> and white space and continuing until the next line beginning with a non-white-space character or an empty line. Header-field refers to the portion of a header line prior to the first <colon> in that line.
to:
In the following list, header refers to lines from the message header, as shown in the OUTPUT FILES section. Header field refers to a line or set of lines within the header that begins with one or more non-white-space characters immediately followed by a <colon> and white space, and continuing up to and including a <newline> that immediately precedes either the next line beginning with a non-white-space character or an empty line. Field name refers to the portion of a header field prior to the first <colon>.

On page 2955 line 97943,97944 section mailx (discard command), change:
header-field
to:
field-name

On page 2955 line 97945 section mailx (discard command), change:
Suppress the specified header fields when writing messages. Specified header-fields shall be added to the list of suppressed header fields. Examples of header fields to ignore are status and cc. The fields shall be included when the message is saved. The Print and Type commands shall override this command. The comparison of header fields shall be in a case-insensitive manner. If no arguments are specified, write a list of the currently suppressed header fields to standard output; the listing need not reflect the same order of header fields that were entered.
to:
Suppress header fields with the specified field names when writing messages. Specified field-name arguments shall be added to the list of suppressed field names. Examples of field names to ignore are status and cc. The header fields shall be included when the message is saved. The Print and Type commands shall override this command. The comparison of field names shall be performed in a case-insensitive manner. If no arguments are specified, write a list of the currently suppressed field names to standard output; the listing need not reflect the same order of field names that were entered.

On page 2956 line 97999 section mailx (followup command), change:
The subject line shall be taken from the first message
to:
The Subject header field shall be taken from the first message

On page 2957 line 98006 section mailx (headers command), change:
Display Header Summary
to:
Display Header Summaries

On page 2957 line 98008-98011 section mailx (headers command), change three occurrences of "headers" to "header summaries".

On page 2958 line 98062 section mailx (Print command), change:
Display Message with Headers
to:
Display Message with Header

On page 2958 line 98065 section mailx (Print command), change:
Write the specified messages, including all header lines, to standard output. Override suppression of lines by ...
to:
Write the specified messages, including all header fields, to standard output. This command shall override suppression of header fields by ...

On page 2959 line 98085 section mailx (Reply command), change:
The subject line shall be formed by concatenating Re:<space> (unless it already begins with that string) and the subject from the first message.
to:
The value for the Subject header field shall be formed by concatenating Re:<space> (unless it already begins with that string) and the value of the Subject header field from the first message.

On page 2959 line 98093 section mailx (reply command), change:
The subject line shall be formed by concatenating Re:<space> (unless it already begins with that string) and the subject from the message.
to:
The value for the Subject header field shall be formed by concatenating Re:<space> (unless it already begins with that string) and the value of the Subject header field from the message.

On page 2959 line 98099 section mailx (retain command), change:
header-field
to:
field-name

On page 2959 line 98100 section mailx (retain command), change:
Retain the specified header fields when writing messages. This command shall override all discard and ignore commands. The comparison of header fields shall be in a case-insensitive manner. If no arguments are specified, write a list of the currently retained header fields to standard output; the listing need not reflect the same order of header fields that were entered.
to:
Retain header fields with the specified field names when writing messages. This command shall override all discard and ignore commands. The comparison of field names shall be performed in a case-insensitive manner. If no arguments are specified, write a list of the currently retained field names to standard output; the listing need not reflect the same order of field names that were entered.

On page 2961 line 98170 section mailx (z command), change:
The number of headers written
to:
The number of header summaries written

On page 2962 line 98210 section mailx (~f command escape), change:
The specified messages shall be inserted into the current message without alteration. This command escape also shall insert message headers into the message with field selection affected by ...
to:
The specified messages, including their headers, shall be inserted into the current message without alteration. The header fields included in each header shall be affected by ...

On page 2963 line 98214 section mailx (~F command escape), change:
except that all headers shall be included in the message
to:
except that all header fields shall be included in the message headers

On page 2963 line 98217 section mailx (~h command escape), change:
If standard input is a terminal, prompt for a Subject line and the To, Cc, and Bcc lists. Other implementation-defined headers may also be presented for editing. If the field is written with an initial value, it can be edited as if it had just been typed.
to:
If standard input is a terminal, prompt for values for the Subject, To, Cc, and Bcc header fields. Other implementation-defined header fields may also be presented for editing. If the header field is written with an initial value, it can be edited as if it had just been typed.

On page 2963 line 98222 section mailx (~m command escape), change:
Insert the specified messages into the message, prefixing non-empty lines with the string in the indentprefix variable. This command escape also shall insert message headers into the message, with field selection affected by ...
to:
Insert the specified messages, including their headers, into the current message, prefixing non-empty lines with the string in the indentprefix variable. The header fields included in each header shall be affected by ...

On page 2963 line 98226 section mailx (~M command escape), change:
except that all headers shall be included in the message
to:
except that all header fields shall be included in the message headers

On page 2963 line 98241 section mailx (~s command escape), change:
Set the subject line to string.
to:
Set the value for the Subject header field to string.

On page 2964 line 98293 section mailx (APPLICATION USAGE), change:
The retain command discards all header-fields except those explicitly retained. The discard command keeps all header-fields except those explicitly discarded. If headers exist on the retained header list, discard and ignore commands are ignored.
to:
The retain command discards all header fields except those explicitly retained. The discard command keeps all header fields except those explicitly discarded. If field names exist on the retained field names list, discard and ignore commands are ignored.

On page 2965 line 98321 section mailx (RATIONALE), change:
The restriction on a subject line being ...
to:
The restriction on the value for a Subject header field being ...

On page 2965 line 98335 section mailx (RATIONALE), change:
Some headers found in incoming mail
to:
Some header fields found in incoming mail
Notes
(0005006)
kre   
2020-09-28 15:03   
This isn't specifically about the issue (I have no problems with the
proposed changes) but since the paragraphs that relate to the discard/ignore
and retain commands are being rewritten anyway, it might be worth also fixing
them to take one consistent view of what they do.

The most obvious example of the issue is shown above (the discard cmd):

    Suppress header fields with the specified field names when writing
    messages. Specified field-name arguments shall be added to the list of
    suppressed field names. Examples of field names to ignore are status
    and cc. The header fields shall be included when the message is saved.
    The Print and Type commands shall override this command.

If the discard command adds field names to a list, it makes no sense for the
Print and Type commands to override the command (what would that mean
exactly?) What those commands would do is ignore the list, they don't
affect anything related to adding field names to the list (which still exists
unaltered for other commands, even after a Print command has "overridden"
the discard command).

Alternatively, it would be possible to skip the list implementation detail
and simply specify that the commands cause the field names listed to be
ignored when they would otherwise be written by other commands, in which case
the P/T commands can override it (though if going that route, care needs to
be taken to allow the command to be issued multiple times, and that all
of them still apply).

This split personality of how these things work appears all over the place,
including in the relationship with the retain command.

It might also be worth being more precise about what "written" means, I
believe these only apply to messages being displayed on stdout, right?

Just list the commands to which these lists (or commands) apply, and then
the sentence about "The header fields shall be included when the message
is saved." can be deleted, which would be a good thing, as it kind of
suggests that only the fields named in discard commands shall be included
when a message is saved, and that's certainly not what is intended ... but
if that isn't what it means, why the different wording style for saving
messages and the "override" for the P/T commands?

Note: all of this is just about the way the text is written, I don't think
there's any real doubt here as to what is supposed to actually happen.




Viewing Issue Advanced Details
1404 [1003.1(2016/18)/Issue7+TC2] System Interfaces Editorial Clarification Requested 2020-09-24 00:44 2020-09-24 00:44
dalias
 
normal  
New  
Open  
   
Rich Felker
musl libc
pthread_mutex_destroy
(page or range of pages)
(Line or range of lines)
---
Is there a requirement to destroy mutex before object lifetime ends?
The specification of pthread_mutex_destroy establishes conditions under which behavior of call to pthread_mutex_destroy is undefined, but does not cover whether there are conditions under which failure to call it is undefined. If the lifetime of a pthread_mutex_t object with automatic or allocated storage duration (or mmap/MAP_ANON) ends (end of execution of block, call to free, munmap, etc.) without a call to pthread_mutex_destroy, is or should the behavior be undefined?

I tend to believe it should be undefined, because the intent of having init/destroy functions seems to be to allow implementation flexibility that would include things like keeping a linked list of all existant objects for error-checking (necessary for diagnostic of double init, etc.), using allocation in pthread_mutex_init (in which failure to call pthread_mutex_destroy is a memory leak, which although not inherently fatal, is bad), and so on.
Possibly add text to description:

"If the lifetime of an initialized pthread_mutex_t object ends without a call to pthread_mutex_destroy, the behavior is undefined."

It's likely that this proposal needs further discussion, particularly for the case of process-shared objects living in MAP_SHARED (and maybe also MAP_ANON) memory where the end of the object lifetime is not well-defined, and where it may be desirable to allow omission of pthread_mutex_destroy because of software that relies on the shared memory ceasing to exist when the last user unmaps it.

If an action is taken on this, similar actions are probably needed for other pthread object types.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1403 [Online Pubs] System Interfaces Editorial Error 2020-09-20 01:46 2020-09-23 05:32
andras_farkas
 
normal  
Applied  
Accepted  
   
Andras Farkas
https://pubs.opengroup.org/onlinepubs/9699919799/functions/psignal.html [^]
psiginfo
psiginfo page has broken links to siginfo.h header
On
https://pubs.opengroup.org/onlinepubs/9699919799/functions/psignal.html [^]
there are two links to a siginfo.h header in the Rationale section. They don't go to the right location. Most likely, they simply shouldn't be hyperlinks at all
Remove both hyperlinks, or make them point to the right page.
Notes
(0004998)
andras_farkas   
2020-09-20 01:49   
Oops, I linked psignal.html
https://pubs.opengroup.org/onlinepubs/9699919799/functions/psiginfo.html [^]
But it is on psiginfo.html too.
(0005001)
agadmin   
2020-09-23 05:24   
These corrections have been made and the online spec updated including download bundles.




Viewing Issue Advanced Details
1402 [Aardvark Bugs] Aardvark Mk IV Editorial Enhancement Request 2020-09-20 01:42 2020-10-12 08:09
andras_farkas
 
normal  
New  
Open  
   
Andras Farkas
No 2018 project for bugs, on Aardvark here
Hi! I've been meaning to mark a new bug. However, when selecting a project here on this website, I can select 2008, 2013, and 2016, but not 2017! (2018?)
This means I can't quite select the most accurate descriptor for what a bug applies to.

My apologies if the project selected (Aardvark Bugs) is incorrect. By the way, the category selections (for aardvark bugs) doesn't contain an Aardvark Mark IV category, only III, and earlier.
Add project for 2018 version of standard into the bug tracker.
Notes
(0004999)
geoffclare   
2020-09-21 08:28   
The 2016 and 2018 editions are both "Issue7+TC2"; they have identical page and line numbers. (The only difference after the frontmatter is that they refer to themselves as POSIX.1-2008 and POSIX.1-2017, respectively).

I have renamed the project so it has "2016/18" instead of "2016" in the name.

The addition of an Aardvark Mark IV category still needs to be done.
(0005042)
geoffclare   
2020-10-09 11:21   
Changing category to (newly created) Aardvark Mk IV.
(0005043)
msbrown   
2020-10-09 21:00   
I can make a new Project if needed, please advise.
(0005044)
shware_systems   
2020-10-09 21:43   
For the odd instance where a report may be filed about the frontmatter, procedurally I think it is better to have separate tags for the editions to handle the different roman numeral page numbers also. Where a tag is relevant to a specific OpenGroup pdf file name for page and line numbers, such as C181 for 2018 edition, that could be incorporated in the name too.
(0005046)
geoffclare   
2020-10-12 08:09   
(edited on: 2020-10-12 08:13)
Re: Note: 0005043 Please don't add a new project. As I said in my first bug note, the 2016 and 2018 editions are both "Issue7+TC2", so having separate projects for them makes no sense. The change I made to the project name is sufficient to solve the problem.

Re: Note: 0005044 Frontmatter bugs are very rare (8 out of over 1400, but none so far against Issue7+TC2), and in any case the frontmatter doesn't have line numbers so submitters have to be more descriptive of where the problem text is. We often have bugs submitted with no page and line numbers and we handle those by setting the fields later. If anyone submits an Issue7+TC2 frontmatter bug, we can easily set the page number to, e.g. "xlv (2016), xl (2018)" if we feel the need.





Viewing Issue Advanced Details
1401 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Enhancement Request 2020-09-19 18:35 2020-10-01 10:44
steffen
 
normal  
New  
Open  
   
Steffen Nurpmeso
Vol. 3: Shell and Utilities, Issue 7, mailx
2959
98093
---
reply command uses obsolete wording
As has come up on the list, the included

  Mail a reply message to all recipients included in the header of the message.

is based upon thirty and more years old code, which also always contradicted the email RFCs, from RFC 822 on.
BSD Mail has been corrected to use RFC 822 compatible handling with

  Author: Craig Leres <leres>
  AuthorDate: 1983-06-05 17:20:23 -0800
  Commit: Craig Leres <leres>
  CommitDate: 1983-06-05 17:20:23 -0800

    Modified to use the "From:" line (if one exists) when replying rather
    than the "From" line (i.e. the first line in the message).

Also the handling of Reply-To: (same standard) has been introduced even earlier (meaning that it replaced all recipients, not only the entries of From:)

  Author: Kurt Shoens <kas>
  AuthorDate: 1981-02-06 09:35:41 -0800
  Commit: Kurt Shoens <kas>
  CommitDate: 1981-02-06 09:35:41 -0800

    Changed reply to send only to reply-to field if such given
On line 98093, change

  Mail a reply message to all recipients included in the header of the message.

to

  Mail a reply message to the sender and all recipients included in the header of the message.
Notes
(0005004)
geoffclare   
2020-09-28 09:28   
(edited on: 2020-10-02 16:07)
The desired action only corrects the additional fourth problem raised on the mailing list after the initial three. Here is a proposal that corrects all four...

On page 2952 line 97834 section mailx (metoo variable), change:
Suppress the deletion of the login name of the user from the recipient list when replying to a message or sending to a group.
to:
Suppress the deletion of the user's login name and any alternative addresses from the recipient list when replying to a message or sending to a group.

On page 2954 line 97918 section mailx (alternates command), change:
(See also the metoo variable.) Declare a list of alternative names for the user's login. When responding to a message, these names shall be removed from the list of recipients for the response. The comparison of names shall be in a case-insensitive manner. With no arguments, alternates shall write the current list of alternative names.
to:
Declare a list of alternative addresses for the address consisting of the user's login name. When responding to a message or sending to a group, if the metoo variable is unset these alternative addresses shall be removed from the list of recipients. The comparison of addresses shall be performed in a case-insensitive manner. With no arguments, alternates shall write the current list of alternative addresses.

On page 2956 line 97996 section mailx (followup command), after applying bug 1405 change:
In the lowercase form, respond to a message, recording the response in a file whose name is derived from the author of the message. See also the save and copy commands and outfolder.

In the capitalized form, respond to the first message in the msglist, sending the message to the author of each message in the msglist. The Subject header field shall be taken from the first message and the response shall be recorded in a file whose name is derived from the author of the first message. See also the Save and Copy commands and outfolder.

Both forms shall override the record variable, if set.
to:
The followup and Followup commands shall be equivalent to reply and Reply, respectively, except that:
  • They shall ignore the record variable.

  • The followup command shall record the response in a file whose name is derived from the author of the message.

  • The Followup command shall record the response in a file whose name is derived from the author of the first message in the msglist.

See also the save and copy commands and outfolder.

On page 2959 line 98082 section mailx, after applying bug 0001405 change:
Reply to a Message List

Synopsis:
R[eply] [msglist]
R[espond] [msglist]
Mail a reply message to the sender of each message in the msglist. The value for the Subject header field shall be formed by concatenating Re:<space> (unless it already begins with that string) and the value of the Subject header field from the first message. If record is set to a filename, the response shall be saved at the end of that file.

See also the flipr variable.

Reply to a Message

Synopsis:
r[eply] [message]
r[espond] [message]
Mail a reply message to all recipients included in the header of the message. The value for the Subject header field shall be formed by concatenating Re:<space> (unless it already begins with that string) and the value of the Subject header field from the message. If record is set to a filename, the response shall be saved at the end of that file.

See also the flipr variable.
to:
Reply to a Message or a Message List

Synopsis:
r[eply] [message]
r[espond] [message]
R[eply] [msglist]
R[espond] [msglist]
Mail a reply message to one or more addresses taken from certain header fields in the specified message or message list. If the flipr variable is unset, these commands shall behave as described below. If the flipr variable is set, commands in the lowercase form shall behave as described below for commands in the capitalized form, and vice versa; the synopsis forms shown above shall also be swapped accordingly.

The recipients of the reply message shall be determined by first constructing an initial list of recipients and then modifying it to form the list that is in effect when mailx enters input mode.

In the capitalized form, the initial list of recipients shall be taken from the header of each message in the msglist as follows:
  • If the header contains a Reply-To field, the address or addresses in that field shall be added to the list.

  • Otherwise, the address or addresses in the From field of the header shall be added to the list.

In the lowercase form, the initial list of recipients shall be taken from the header of the message as follows:
  • If the header does not contain a Reply-To field, all of the addresses in the From, To, and Cc fields shall be included in the list.

  • Otherwise, it is implementation-defined whether all of the addresses in the Reply-To, To, and Cc fields are included in the list or only the address or addresses in the Reply-To field.

The initial list of recipients shall be marked for placement in the header fields of the reply message as follows. Recipient addresses taken from a From or Reply-To header field shall be marked for placement in the To field of the reply message. Recipient addresses taken from a Cc header field shall be marked for placement in the Cc field of the reply message. Recipient addresses taken from a To header field shall be marked for placement in either the To or the Cc field of the reply message. Implementations shall provide a way to place them in the To field. Implementations may, but need not, provide an implementation-defined way to place them in the Cc field.

The modifications applied to the initial list of recipients shall be as follows:
  • If the metoo variable is unset, addresses consisting of the login name of the user and any alternative addresses declared using the alternates command shall be removed from the list.

  • The set of recipients marked for placement in the To header field of the reply message shall have duplicates within that set removed.

  • The set of recipients marked for placement in the Cc header field of the reply message shall have duplicates within that set removed and may have recipients that are also marked for placement in the To field removed.

The values for the To and Cc header fields of the reply message shall be constructed from the addresses in the modified list of recipients that are marked for placement in each of those fields.

The value for the Subject header field of the reply message shall be formed from the value of the Subject header field of the message or the first message in msglist by prefixing it with Re:<space>, unless it already begins with that string.

The values of the To, Cc, and Subject header fields set as described above can be modified by the user after mailx enters input mode through the use of the ~t, ~c, ~s, and ~h command escapes.

If record is set to a pathname, the response shall be saved at the end of that file.


(0005005)
kre   
2020-09-28 14:16   
Re Note: 0005004

I would suggest swapping the order of the "The initial list of recipients"
and "The modifications applied to the initial list" paragraphs, the logical
order is to first collect the addresses, then remove any that should be
removed, and then place them in their appropriate outgoing fields.

To:
    The set of recipients marked for placement in each header field of the reply
    message shall have duplicates removed.

I'd suggest adding
    Where duplicate addresses originated from different header fields in
    the original message(s) retain the copy that originated from any
    Reply-To header field, and if none did, from the From header field,
    and if none, from the To header field, and finally from the Cc
    header field.

That is for the purposes of assigning addresses to outgoing fields, mailx
shouldn't be replying to a message that is
      From: addr1
      To: addr2
      cc: addr1
and constructing a reply with addr1 in the cc field, because that's
where it was seen when the duplicate from the From field was deleted.

Aside from that, this looks fine, though (unrelated to the issue in the
bug report) it might be better to be clearer in the text...

    Mail a reply message to one or more addresses ...

that these commands don't simply mail a reply message, they construct a
template reply message, to which the user is expected to add the actual
reply, and after that's done, they mail the reply message (if it wasn't
aborted). This is hinted at with phrases like "after mailx enters input mode"
but nothing ever says why that would happen, or even that it should.

That should probably be somewhere generic (not part of the R/r command descr)
as it also applies to the "m" and "followup" commands, something perhaps like

     When commanded to mail a message, mailx shall construct a template
     message, and then prompt the user to complete it, by supplying the
     body of the message, and optionally modifying the template header.
     See [xref whatever section] for the commands available in input mode.

It may also be worth considering whether something should also be done with
the "followup" (fo/F) commands akin to what is being done with reply/respond.
(I didn't even know followup existed -- it has been about 40 years since I
last seriously used mailx (then just called Mail)).
(0005007)
steffen   
2020-09-28 20:34   
I do not understand

  Otherwise, it is implementation-defined whether all of the addresses in the Reply-To, To, and Cc fields are included in the list or only the address or addresses in the Reply-To field.

given that already RFC 822 states in 4.4.4 that Reply-To: is meant to replace the content of From:. The examples in A.2 ORIGINATOR ITEMS support this.

It is true this misuse of Reply-To: was advocated and such examples have been seen, i imagined because of missing standardization of the Mail-Followup-To: header which is meant to define an exclusive list of reply addresses, but given that already the email standard RFC 822 from 1982 was clear i would not standardize undefined behaviour in POSIX in 2020.
Software should follow the RFC.

Regarding metoo/alternates i have no time to look the next couple of days.
But it is clear that metoo processing historically _only_ applied to the login name of the user, not to alternates or even full email address matching.
(0005008)
geoffclare   
2020-09-29 10:57   
Re Note: 0005007 As I pointed out in the mailing list discussion, RFC 5322 explicitly states that it does not require any particular behaviour here. It says (in 3.6.3 after it describes how the destination fields should be set when creating a reply):
Note: Some mail applications have automatic reply commands that include the destination addresses of the original message in the destination addresses of the reply. How those reply commands behave is implementation dependent and is beyond the scope of this document. In particular, whether or not to include the original destination addresses when the original message had a "Reply-To:" field is not addressed here.
(0005009)
kre   
2020-09-29 13:52   
Re Note: 0005007 - I agree with Geoff (Note: 0005008) - but aside from that
we should not be legislating here, so regardless of what the RFCs (or any
other standards from any other bodies) say, what this standard should
reflect is what the implementations actually do, unless they are clearly buggy.

For this, it is clear that there are differences in the ways that the
various mailx implementations treat Reply-To ... I wish there weren't
and that everyone used it properly, but they don't, so the standard
cannot pretend that they do.
(0005010)
steffen   
2020-09-29 16:00   
Regarding metoo: neither in BSD Mail nor in Tenth Edition Research Unix is there any relationship in between metoo and alternates. metoo only control elimination of "myself" ($LOGNAME in POSIX) from alias expansions.

Having said that, it is much more useful if metoo is extended and covers the vaporisation of the user including any alternates and expanded aliases!
I think i will implement this when i have time.

Also, undocumented and non-standardized is the convention present in BSD-Mail and Research Unix mailx that a reverse solidus prefix of entries of alias lists prevents further alias expansion. I think i'll add an issue at times.
(0005014)
geoffclare   
2020-10-01 10:44   
I have updated Note: 0005004 to clarify the de-duplication requirement and allow cross-header de-duplication. It seemed simpler to specify this in terms of which outgoing header they are marked for placement in, instead of (as kre suggested) where they came from.




Viewing Issue Advanced Details
1400 [1003.1(2016/18)/Issue7+TC2] System Interfaces Editorial Error 2020-09-17 06:10 2020-09-17 06:10
mkerrisk
 
normal  
New  
Open  
   
Michael Kerrisk
man7.org
getgroups()
1042
35515
---
Typo in variable name (nogroups ==> ngroups)
In the EXAMPLES section of getgroups(), there is the following:

[[
gid_t *group;
int nogroups;
long ngroups_max;
ngroups_max = sysconf(_SC_NGROUPS_MAX) + 1;
group = (gid_t *)malloc(ngroups_max *sizeof(gid_t));
ngroups = getgroups(ngroups_max, group);
]]

A variable called "nogroups" is declared, but later the variable used in line 35519 is called "ngroups". To fix this (in a way that employs variable names that are also consistent with line 35516), "nogroups" should be changed to "ngroups".

Note that the same issue is present in Issue 8, draft 1, and reported as https://austingroupbugs.net/view.php?id=1399 [^]
 In line 35515, change "nogroups" to "ngroups".
There are no notes attached to this issue.




Viewing Issue Advanced Details
1398 [Issue 8 drafts] System Interfaces Editorial Omission 2020-09-03 15:30 2020-10-08 10:41
nick
 
normal  
Applied  
Accepted  
   
Nick Stoughton
USENIX
nms-missing-redirect
mkostemp
1283
43007
New API mkostemp() needs redirect page
The mkdtemp() function also documents mkostemp() and mkstemp(). The latter of these has a redirect page (page 1294) but the new mkostemp() function, which is alphabetically between mkdtemp() and mkstemp() has none.
Add a new page before page 1294:

NAME
  mkostemp - create a unique file

[CX]
SYNOPSIS
  #include <stdlib.h>
  
  int mkostemp(char *template, int flag);
[/CX]

DESCRIPTION
  Refer to [xref]mkdtemp()[/xref].
There are no notes attached to this issue.




Viewing Issue Advanced Details
1397 [1003.1(2008)/Issue 7] System Interfaces Objection Error 2020-08-30 20:53 2020-09-01 08:31
sebor
ajosey  
normal  
Under Review  
Open  
   
Martin Sebor
strndup
unknown
unknown
---
strndup incorrectly implies argument must be a string
Because it doesn't make use of the term array, and because it refers to "the length of s," the specification for strndup() suggests the s argument is required to be a nul-terminated string rather than an arbitrary array of characters. However, the canonical implementation of the function is to call strnlen() with the size argument as the bound to determine the number of characters to copy (see also APPLICATION USAGE). Thus the function accepts as the first argument an array that need not be a nul-terminated string.
Correct strndup() to make it explicit that s doesn't have to point to a nul-terminated string, e.g., like so:

The strndup() function shall be equivalent to the strdup() function, duplicating the <ins>smaller of the number of bytes in the array to which s points, not including any terminating NUL character, or the value of the maxlen argument, </ins><del>provided s</del> in a new block of memory allocated as if by using malloc(), ...
Notes
(0004963)
geoffclare   
2020-09-01 08:31   
This should be closed as a duplicate of bug 0001019.




Viewing Issue Advanced Details
1396 [1003.1(2016/18)/Issue7+TC2] System Interfaces Editorial Enhancement Request 2020-08-28 12:24 2020-09-02 08:53
rillig
 
normal  
New  
Open  
   
Roland Illig
3
(page or range of pages)
(Line or range of lines)
---
Table cells are incorrect
The page for basename contains a table with examples.

The table has a border and separating lines between the cells.
Some of these lines go straight through what should be a single cell.
The headings are split into 4 rows each, but that should be 1 row.
The last example is split over 2 rows.
Does the last example really contain a line break in the pathname?
Fix the formatting of the table.

The marked cell "" should contain the words "empty string" instead, without the quotes, to align with the syntax of the other cells at the right half of the table.
2020-08-28 14_22_07-basename.png (23 KB) 2020-08-28 12:24
Notes
(0004950)
agadmin   
2020-08-28 19:00   
I made layout changes to mirror the pdf version. I think the bug wants further changes. Note the download bundles have not yet been updated.
(0004951)
rillig   
2020-08-28 20:38   
Thank you, that looks much better now.

At the bottom of the table, the example looks as if basename() had to trim spaces. This looks unintended. Furthermore, the space before "test" is missing on the right side, but it is there on the left side.
(0004954)
agadmin   
2020-08-29 07:12   
I'll need some interpretation of what the last line in the table is as its unclear from the document source and pdf output

Is it

"/home/dwc// test"

or

"/home/dwc//
test"

or

"home/dwc//test"
(0004961)
geoffclare   
2020-09-01 08:27   
Column 1 in the last line should contain "home/dwc//test" (same as column 4 except for the quotes).
(0004962)
geoffclare   
2020-09-01 08:27   
I agree with the request in the Desired Action to change «""» to «empty string» but this needs to change in the PDF and therefore can't be handled in the "Online Pubs" project.

I suggest that once the table formatting has been dealt with, we move this bug to the Issue7+TC2 project to address this other change.
(0004964)
agadmin   
2020-09-01 18:39   
I have now updated the table formatting. I have still to update the download bundles
(0004965)
agadmin   
2020-09-02 05:38   
The download bundles have been updated
(0004966)
geoffclare   
2020-09-02 08:53   
I have moved this bug from Online Pubs to Issue7+TC2.

Proposed change ...

On page 625 line 21588 section basename(), change column 4 from:
""
to:
empty string




Viewing Issue Advanced Details
1395 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2020-08-26 14:36 2020-08-26 15:32
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
awk
2486
79925-79930
---
awk radix character in alternative conversion to atof()
In POSIX.2-1992 awk was required to convert string values to numeric values by the equivalent of:
setlocale(LC_NUMERIC, "");
numeric_value = atof(string_value);
Later when atof() was required to support hex floating point, "INF", and "NAN", this was changed to make the support of those new things optional in awk by saying the conversion is done either as above or by an alternative conversion that is a description of how atof() used to work in C90.

At least, that's what was intended. Unfortunately, the description of this alternative conversion refers to <period> instead of the radix character.
On page 2486 line 79925 section awk, change:
... optionally containing a <period>
to:
... optionally containing a radix character

On page 2486 line 79928 section awk, change:
The sequence starting with the first digit or the <period> (whichever occurs first) is interpreted as a floating constant of the C language, and if neither an exponent part nor a <period> appears, a <period> is assumed to follow the last digit in the string.
to:
The sequence starting with the first digit or the radix character (whichever occurs first) is interpreted as a floating constant of the C language, except that the radix character shall be used in place of a <period>, and if neither an exponent part nor a radix character appears, a radix character is assumed to follow the last digit in the string.

Notes
(0004946)
shware_systems   
2020-08-26 15:32   
(edited on: 2020-08-26 15:33)
A floating constant of the C language, as a subset of strings in general, does presume <period> is always the radix character, according to the C grammar. POSIX extending the definition, to allow other chars, it seems should be CX shaded, in interface and utility descriptions, or these be reworded to defer directly to strtod(), which has appropriate text and shading already.





Viewing Issue Advanced Details
1394 [Issue 8 drafts] System Interfaces Objection Omission 2020-08-26 09:18 2020-10-08 10:39
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
getenv()
1011
34750
Another way the getenv() return string is modifiable
Bug 0000188 added some text that acknowledges the application can modify the string returned by getenv() if it is part of a string previously added to the environment using putenv().

However, applications can also arrange for getenv() to return a known-modifiable string by setting environ to point to a suitable array of strings.
Change:
The application shall ensure that it does not modify the string pointed to by the getenv() function, [XSI]unless it is part of a string previously added to the environment using putenv()[/XSI].
to:
The application shall ensure that it does not modify the string pointed to by the getenv() function, [CX]unless it is part of a modifiable object previously placed in the environment by assigning a new value to environ[/CX] [XSI]or by using putenv()[/XSI].

There are no notes attached to this issue.




Viewing Issue Advanced Details
1393 [Issue 8 drafts] Shell and Utilities Objection Error 2020-08-24 19:15 2020-10-08 10:37
McDutchie
 
normal  
Applied  
Accepted As Marked  
   
Martijn Dekker
https://github.com/ksh93 [^]
XCU 3
2510
82523-82525
Note: 0005003
'command' should not be treated as a declaration utility
The current POSIX draft adds three lines to the specification of the 'command' utility, lines 82523-82525:

| The command utility shall be treated as a declaration utility if the
| first argument passed to the utility is recognized as a declaration
| utility. In this case, subsequent words of the form name=word shall
| be expanded in an assignment context. See Section 2.9.1.1 (on page
| 2292).

(As an aside, if I were to be pedantic, I could argue that the phrasing "if the first argument passed to the utility is recognized as a declaration utility" effectively makes this behaviour unspecified; a 'command' utility implementation could be written not to recognise any utility as such and this phrasing would allow that. I am assuming that this is not the intended meaning.)

This addition effectively makes the 'command' utility into a kind of "Schrödinger's declaration utility" that is simultaneously a declaration utility and not a declaration utility. I think that is a bad idea, for the following three reasons:


***1.*** In the currently published release version of POSIX, 'command' is a regular utility (grammatically no different from, say, 'ls' or 'printf'), and there is nothing that allows regular utilities to treat their arguments specially in a shell grammar sense. So it follows that declaration utility behaviour *must* be disabled along with all other special behaviour by the 'command' utility as specified in the release version.

Consequently, the behaviour specified in the draft is incompatible with the behaviour specified in the current release version, as it is not possible to be simultaneously compliant with both.

(A similar argument could be made for 'export' and 'readonly' themselves, but at least they are already special built-in utilities, so the inconsistently is not so serious.)


***2.*** It does not specify sufficiently widespread current practice. For the following test:

   $shell -c 'v="a y=b"; command export x=$v; echo "[$x] [$y]"'

bash, bosh (Schily), dash < 0.5.11, NetBSD sh, pdksh (including OpenBSD (k)sh), yash, and 'zsh --emulate sh' output:

   [a] [b]

meaning they do *not* treat 'command export' as a declaration utility.

Dash >= 0.5.11, its derivative gwsh, FreeBSD sh, ksh93, and mksh output

   [a y=b] []

meaning they do treat 'command export' as a declaration utility. But together they are not enough to establish a de facto standard behaviour.


***3.*** Since 'command' is specified as a regular utility, both of the following *must* work:

    c='command'; "$c" export foo=bar
    set -- command export; "$@" foo=bar

So, in the draft version, this effectively means that the shell must determine at runtime whether to treat 'command' as a declaration utility, and not at parse time. And that requirement is both innovative and unreasonable.

For the following test:

    $shell -c 'v="a y=b"; set -- command export; "$@" x=$v; echo "[$x] [$y]"'

only dash 0.5.11 and its derivative gwsh currently manage to output '[a y=b] []' meaning they recognise 'command export' as a declaration utility at runtime, and all other shells (except ksh93 which breaks completely[*]) output '[a] [b]' meaning they don't.

[*] ksh93 (which I currently maintain the only actively and openly developed fork of) has a bug that breaks this test case altogether -- the 'export' utility is not run at all; instead, the equivalent of 'command -v export' is run. It would be good if I could implement a clean fix, which would disassociate the 'command' utility from the parser altogether and treat it as a regular built-in utility as currently specified in the release version.
Delete lines 82523-82525.

Alternatively, change these lines to the following:

"If the first argument passed to the command utility is the name of a declaration utility (see Section 2.9.1.1 on page 2292), it is unspecified whether subsequent words of the form name=word are expanded in an assignment context."
Notes
(0004943)
McDutchie   
2020-08-24 19:16   
Argh. Could someone please edit the summary to say "'command' should NOT be treated as a declaration utility"? Thanks and apologies.
(0004944)
Don Cragun   
2020-08-24 23:09   
Re Note: 0004943: Done,
(0004945)
geoffclare   
2020-08-25 08:25   
I have moved this to the Issue 8 drafts project and set the product version to Draft 1.
(0004971)
McDutchie   
2020-09-04 03:53   
I had written: "So, in the draft version, this effectively means that the shell must determine at runtime whether to treat 'command' as a declaration utility, and not at parse time. And that requirement is both innovative and unreasonable."

I'd just like to expound this point a little. On shells where shell assignments allow special grammar like "a=(foo bar)" (which are bash, ksh93, mksh and zsh), the 'command' utility must be integrated into the parser if it is to preserve a declaration command's assignment behaviour including grammatical properties. ksh93 does this; bash, mksh, and zsh do not. So

    command typeset a=(1 2 3)

works on ksh93, but throws a syntax error on the others (zsh attempts and fails to interpret it as some kind of zsh-style pattern).

Of course this can only work on ksh93 because its command builtin is integrated into the parser (in ways that also cause multiple bugs including one serious bug, see footnote above). The assignments are recognised at parse time, not at runtime. If assignments can use special grammar such as parentheses, then it is not feasible or even possible to do this at runtime. Dash manages it because it doesn't support assignments with special syntax.

So, I think ksh93 is a case study on the reasons why it is a bad idea for 'command' to be anything else than a plain old regular builtin utility.
(0004972)
geoffclare   
2020-09-04 09:34   
Re: Note: 0004971 There is no requirement for this to be determined at runtime. The APPLICATION USAGE on page 2513 line 82660 says that the shell need only perform lexical analysis of the next argument when deciding if command should be treated as a declaration utility.

(There is an editorial error here: "command" should be in italics. We should fix that if the text here is otherwise unchanged by this bug.)
(0004973)
joerg   
2020-09-04 11:43   
Re: Note: 0004971

bosh only looks at the flags in it's builtin command data structures if a builtin is a special builtin and in this case does not make variable assignments a push operation that is popped after the builtin command terminated.

My impression is that this best fits to the previous shell behavior.

BTW: is there a list with all ksh93 related problems?
(0005002)
joerg   
2020-09-24 14:55   
It seems that before 0000351 no shell had a special treatment for quoting for the arguments of "export".

So a requirement for this behavior looks like invention.
(0005003)
geoffclare   
2020-09-24 15:55   
On page 2332 line 75618 section 2.14 export (APPLICATION USAGE), add a new paragraph:
In shells that support extended assignment syntax, for example to allow an array to be populated with a single assignment, such extensions can typically only be used in assignments specified as arguments to export if the command word is literally export, and not if it is some other word that expands to export. For example:
# Shells that support array assignment as an extension generally support this:
export x=(1 2 3); echo ${x[0]}  # outputs 1
# But generally do not support this:
e=export; $e x=(1 2 3); echo ${x[0]}  # syntax error

On page 2332 line 75637 section 2.14 export (RATIONALE), add a new paragraph:
Some implementations extend the shell's assignment syntax, for example to allow an array to be populated with a single assignment, and in order for such an extension to be usable in assignments specified as arguments to export these shells have export as a separate token in their grammar. This standard only permits an extension of this nature when the input to the shell would contain a syntax error according to the standard grammar. Note that although export can be a separate token in the shell's grammar, it cannot be a reserved word since export is a candidate for alias substitution whereas reserved words are not (see [xref to 2.3.1]).

On page 2335 line 75721 section 2.14 readonly (APPLICATION USAGE), change:
None.
to:
In shells that support extended assignment syntax, for example to allow an array to be populated with a single assignment, such extensions can typically only be used in assignments specified as arguments to readonly if the command word is literally readonly, and not if it is some other word that expands to readonly. For example:
# Shells that support array assignment as an extension generally support this:
readonly x=(1 2 3); echo ${x[0]}  # outputs 1
# But generally do not support this:
r=readonly; $r x=(1 2 3); echo ${x[0]}  # syntax error

On page 2335 line 75739 section 2.14 readonly (RATIONALE), add a new paragraph:
Some implementations extend the shell's assignment syntax, for example to allow an array to be populated with a single assignment, and in order for such an extension to be usable in assignments specified as arguments to readonly these shells have readonly as a separate token in their grammar. This standard only permits an extension of this nature when the input to the shell would contain a syntax error according to the standard grammar. Note that although readonly can be a separate token in the shell's grammar, it cannot be a reserved word since readonly is a candidate for alias substitution whereas reserved words are not (see [xref to 2.3.1]).

On page 2514 line 82700 section command (RATIONALE), change:
... or that prevents function lookup on b or c.
to:
... or that prevents function lookup on b or c. However, some implementations extend the shell's assignment syntax, for example to allow an array to be populated with a single assignment, and in order for such an extension to be usable in assignments specified as arguments to export and readonly these shells have those utility names as separate tokens in their grammar. When command is used to execute these utilities it also needs to be a separate token in the grammar so that the same extended assignment syntax can still be recognized in this case. This standard only permits an extension of this nature when the input to the shell would contain a syntax error according to the standard grammar, and therefore it cannot affect how '|' and ';' are parsed in the example above. Note that although command can be a separate token in the shell's grammar, it cannot be a reserved word since command is a candidate for alias substitution whereas reserved words are not (see [xref to 2.3.1]).




Viewing Issue Advanced Details
1392 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Clarification Requested 2020-08-14 12:55 2020-08-21 17:12
mohd_akram
 
normal  
New  
Open  
   
Mohamed Akram
find
2797
91951-91956
---
find(1): clarify whether -perm ops - and + should follow chmod
Currently, BSD/macOS find(1) behave like chmod when it comes to the - and + operators of -perm:

$ umask
0022
$ touch file
$ chmod +w file
$ find file -perm -+w
file

GNU find behaves differently (https://savannah.gnu.org/bugs/?58654): [^]

$ umask
0002
$ touch file
$ chmod +w
$ find file -perm -+w

GNU find treats = and + as equivalent, while BSD find doesn't (which is IMO more useful).
Clarify the behavior of find(1) -perm with respect to the - and + operators.
Notes
(0004925)
joerg   
2020-08-15 11:15   
My interpretation of the standard matches the one from BSD even though that differs from Solaris. But Solaris uses the man page text from POSIX....

BTW: libfind behaves the same as BSD with this detail.

Given that the standard mentions the mask, I believe it is exact enough.
(0004926)
geoffclare   
2020-08-19 09:22   
> Given that the standard mentions the mask, I believe it is exact enough.

I disagree. It only mentions the mask in the statement about '=':
'=' shall set the appropriate mode bits, without regard to the contents of the file mode creation mask of the process.
It does not mention the mask in the statements about '+' and '-', therefore it is implicitly unspecified whether the mask is used for those.

I think we should issue a "standard is silent" interpretation, thus allowing both behaviours for Issue 7, but we should make a change in Issue 8 to require the BSD behaviour.
(0004929)
joerg   
2020-08-19 10:32   
Given that the find man page points to chmod(1) and that it explicitly mentions '=' to differ somehow from '+', I see this as a hint that -perm must behave similar to chmod(1) which includes the file creation mask.

BTW: When I wrote my parser in July 2004 for libfind, I did not check existing find implementations but only the POSIX standard. The background for writing libfindhas been the missing usability of gfind and find from Solaris at that time. So the behavior of libfind is just a consequence of following the standard and what is expected for orthogonal behavior.

Regardless of the way you interpret this, we seem to basically agree that the libfind/BSD behavior is the right way to go for the future.

P.S. The Solaris implementation behaves the same as gfind for this detail, but I am happy to change this.
(0004930)
geoffclare   
2020-08-19 11:32   
(edited on: 2020-08-19 11:34)
Re: Note: 0004929

1. The find man page only points to chmod for the "format" of the mode operand, i.e. the syntax not the semantics.

2. '=' and '+' do indeed differ, but only in that '=' is required to ignore the mask whereas '+' can either use it or ignore it.

3. I'm not sure there is even a hint here; readers not sufficiently familiar with standardese might think so, but those experienced in interpreting the standard should see that there is nothing at all said about the mask in relation to '+'. Anyway, a "hint" is not a requirement.

(0004941)
mohd_akram   
2020-08-21 16:47   
(edited on: 2020-08-21 16:53)
I think the title I put should be tweaked - the issue is actually + and = (- seems to behave the same due to the 000 starting point). I had posted the = example in the filed GNU bug but forgot about it - hopefully this link will work: https://savannah.gnu.org/bugs/?58654 [^]

On macOS:

$ chmod =w file
$ find file -perm =w
file

On Fedora:

$ chmod =w file
$ find file -perm =w

It seems that macOS does not ignore the mask even with =, mimicking chmod's behavior. It feels like it would be simpler to defer to chmod in both syntax and semantics because the "symbolic_mode" operand is fairly complicated. This could entail removing the "without regard to the contents of the file mode creation mask of the process" clause (letting "appropriate" do all the heavy-lifting instead) and adding at the end that it is "implementation-defined whether + and = ignore the file mode creation mask when `who` is not specified".

(0004942)
shware_systems   
2020-08-21 17:12   
Yes, link works this time :-)




Viewing Issue Advanced Details
1391 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2020-08-13 19:58 2020-08-14 09:21
kre
 
normal  
New  
Open  
   
Robert Elz
2.9.1.1 1.c (2.9.1.4 1.c in Issue 8 draft 1)
2367 (2294 in Issue 8 draft 1)
75564-7 (74159-62 in Issue 8 draft 1)
---
Unclear language in specification of utilities implemented as functions
I do not believe there is likely to be any disagreement over what
should be done here, though there may be some over whether the
current language is unclear. My opinion on that is that if anyone
can reasonably read the text in a way different from what is intended,
and if it is easily possible to correct the text to improve clarity,
then that should always be done.

Anyway, 2.9.1.1 1.c (unchanged in Issue 8, other than the section number)
states
If the command name matches the name of a function known to
this shell, the function shall be invoked as described in Section 2.9.5.
If the implementation has provided a standard utility in the form of a
function, it shall not be recognized at this point. It shall be invoked
in conjunction with the path search in step 1e.


While this is somewhat fanciful, as to the best of my knowledge, no
shell implements any standard utilities as functions, the issue is
if the shell did implement some standard utility as a function (something
like dirname might be a possibility) and the user (application) has
also defined a function of the same name (overriding the implementation
provided definition), what should happen.

My expectation is that the "If the implementation has provided a standard
utility in the form of a function, it shall not be recognized at this point."
text means that the standard utility, implemented as a function, should
not be recognised, but that a user defined function of the same name still
should be.

However, it is not unreasonable to read it as prohibiting (at this point)
invoking any function with the same name as that of a standard utility
implemented as a function ("it shall not be recognized at this point",
is the antecedent of "it" the "command name" or the "standard utility
implemented as a function". While the latter is almost certainly what
was intended, either is actually a justifiable reading of the text.

This becomes problematic, as if a user defined function is not
invoked in 2.9.1.1 1.c then it never is, as 2.9.1.1 1.e.i only allows
for two cases, a where "If the system has implemented the utility as a
built-in or as a shell function, ..." or b "Otherwise, the shell executes
the utility in a separate utility environment" (the text here has altered
in Issue 8 draft 1, but the effect is the same). Neither of those
allow invoking a user defined function. Further, it would be strange
if any user defined function (regardless of its name) depended upon a
successful PATH search before it could be invoked.

Clarifying the language of 2.9.1.1 1.c will avoid this problem.

Note that if the resolution of 0001389 is to accept the proposed
text, or something substantially similar, then this issue becomes
moot, as all references to standard utilities implemented as
functions will have been removed, leaving 2.9.1.1 1.c simply
specifying the invocation of a function.

It should also be noted, that while the standard attempts to
specify what should be done with standard utilities implemented
as functions, it is not actually documenting any known standard
behaviour there, as there are no known (to me anyway) instances
of standard utilities implemented as functions. Rather than
documenting the standard behaviour, what happened is an attempt to
specify how such things should behave, if any ever were to exist.
That's not the standard, that's a flight of fancy.
Option 1.

In section 2.9.1.1 1.c (2.9.1.4 1.c in Issue 8 draft 1) delete
all after the first sentence. That is retain

If the command name matches the name of a function known to
this shell, the function shall be invoked as described in Section 2.9.5.

and delete the rest, viz:

If the implementation has provided a standard utility in the form of a function,
it shall not be recognized at this point. It shall be invoked in conjunction
with the path search in step 1e.

It would also probably make sense to delete the reference to standard
utilities implemented as functions from 2.9.1.1 1.e but that is not
essential for the purposes of this issue.

This option recognises that absent implementation, it is premature to
standardise the behaviour of standard utilities implemented as functions,
as no-one has any idea what that behaviour would be.

Option 2:

In 2.9.1.1 1.c change the words:
it shall not be recognized at this point.
to:
that standard utility shall not be recognized at this
point, whereas an application defined function of the same name is.


Also be change the word
It

in the following sentence (line 75567) into
Standard utilities implemented as functions
.

I recommend option 1.
Notes
(0004922)
geoffclare   
2020-08-14 09:21   
(edited on: 2020-08-14 09:24)
I think there is a misunderstanding here of what "If the implementation has provided a standard utility in the form of a function" means. The description talks about "if the shell did implement some standard utility as a function" which has implications that are not quite the same.

I believe the thinking behind "the implementation has provided a standard utility in the form of a function" is that an implementation could define a function in an implementation-provided shell startup file. Some implementations do this with aliases (such as the misguided alias rm='rm -i'); the standard allows a similar thing with functions. I suppose a function could also be predefined internally by the shell itself. However, the crucial thing is that it is just a function definition like any other; it is not special (other than being already defined by the time the shell reads commands from the user/script, and the requirement about when it is recognised). So if the application defines a function of the same name, it simply replaces the one provided by the implementation. The application could also delete the function using "unset -f".

I think a suitable change to clarify this would be to change:
If the implementation has provided a standard utility in the form of a function, it shall not be recognized at this point.
to:
If the implementation has provided a standard utility in the form of a function, and that function definition still exists (i.e. has not been removed using unset -f or replaced via another function definition with the same name), it shall not be recognized at this point.






Viewing Issue Advanced Details
1390 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Omission 2020-08-13 19:54 2020-08-14 09:01
kre
 
normal  
New  
Open  
   
Robert Elz
XCU 2.9.1.1 (In Issue 8 draft 1, section 2.9.1.4)
2367-8 (in Issue 8 draft 1: 2293-4)
75545-75627 (In Issue 8 Draft 1, lines 74140-74183)
---
No method to invoke a non-standard built-in utility
The procedures in XCU 2.9.1.1 (which is 2.9.1.4 in Issue 8 draft 1)
and specifically sub-section 1 of that section, specify how to
choose an actual command to invoke given a command name that does
not contain a slash.

All shells implement some built-in commands. The standard
specifies the procedures to invoke the special built-in commands,
a set of specific built-in commands (in Issue 8 called Intrinsic
built-in commands) and built-in commands that are standard utilities.
The specification of the last form is incorrect, as noted in
0001389 but that is immaterial here.

However some shells (most shells?) provide extra built-in utilities
that are not standard utilities, not intrinsic utilities, and not
special built-in utilities. The standard as currently written
provides no mechanism to allow such non-standard built-in utilities
to be invoked.

If 0001389 is resolved as accepted, either as written, or in some
substantially similar fashion, this bug can be closed, as it will
have already been resolved.

If 0001389 is rejected, then a change will be needed to allow for
non-standard built-in utilities to be invoked.

This is because (sub-sections of 2.9.1.1 1)
        a does not apply, the built-in is not a special built-in
        b can be assumed not to apply, at least most of the time,
          the new built-in will not have one of the designated names.
        c does not apply, the command to be invoked is not defined as
          a function
        d does not apply, the command name of the relevant utility is
          not one of those listed.
        e is all that remains, and specifies a PATH search to locate the
          utility. That search fails, as the built-in in question is
          not implemented as a file system command. Note that XCU 1.6
          requires ant standard utility implemented as a built-in to
          also be available for use via one of the exec() family of
          functions, but nothing is required in a similar fashion for
          non-standard utilities. A shell is entitled to implement a
          new non-standard utility as a built-in without making it also
          available in the filesystem. Since this search fails
          2.9.1.1 1.e.ii applies, and specifies:
If the search is unsuccessful, the command shall fail with
an exit status of 127 and the shell shall write an error message.


That is, any attempt to invoke a non-standard built-in utility is
required to fail (as written, this is not optional, it is mandatory).

Shells that implement non-standard built-in utilities (most of them)
do not act like that, and instead invoke the utility, just like any
other built-in utility.

An example of such a built-in is the "jobid" built-in from the NetBSD (and
FreeBSD) shells, which maps between shell job identifiers (%% etc) and
lead process id's (process group id) of the job (and so makes no sense at
all to implement outside the shell, as job identifiers exist only as an
internal shell notation).

The standard needs to be amended to correct this omission.
In section 2.9.1.1 1.d after the words
If the command name matches the name of the type or ulimit
utility, or of a utility listed in the following table,

add the new clause
or matches the name of any non-standard
built-in utility [xref XCU 1.6],


As amended (to become 2.9.1.4 1.d in Issue 8 draft 1), the relevant change
is to add after the words:
If the command name matches the name of an intrinsic utility
(see Section 1.7, on page 2268),

add the new clause
or matches the name of any non-standard
built-in utility [xref XCU 1.6],


Note that the words to be added are the same, but the text that precedes
them has altered between Issue 7 and Issue 8. In either case the text
that follows
that utility shall be invoked.

is unchanged, as is the list of command names that follows in Issue 7.
Notes
(0004921)
geoffclare   
2020-08-14 09:01   
"The standard as currently written provides no mechanism to allow such non-standard built-in utilities to be invoked."

This was solved in Issue 8 draft 1 by allowing implementations to add to the list of intrinsic utilities. See XCU 1.7 (P2268 L73117) in the draft: "Whether any additional utility is considered an intrinsic utility is implementation-defined."
All the implementation needs to do is document the additions.

This practice is not recommended (through the use of "should not" on L73119) but is allowed.




Viewing Issue Advanced Details
1389 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2020-08-13 19:52 2020-08-20 15:31
kre
 
normal  
New  
Open  
   
Robert Elz
2.9.1.1 1.c (2.9.1.4 1.c in Issue8 Draft 1), 2.9.1.1 1.d, 2.9.1.1 1.e.i.a
2367 (2335 in Issue8 draft 1)
75564-73 75577-78 (74159-64 74168-71 in Issue8 draft 1)
---
Shell command search procedure is incorrect
POSIX currently specifies that normal built-in standard utilities be
invoked only when the PATH search locates that utility.
 
There are all kinds of problems with this, but the most important
is that it does not document the standard behaviour (rather it
documents what it seems some people would like the standard behaviour
to be).

I set PATH to a list of directories that does not include /bin
(which, on my system, is where the standard utilities ls and test
are to be found - for completeness here, printf is in /usr/bin which
remained in $PATH, though the results below do not show that).
(I am not showing the value of my PATH with /bin removed, as what's
there isn't actually relevant for anything here).

Then I ran the following script, which tests every (posixish) shell
I have access to that I could remember I have...

for SHELL in /bin/sh /bin/ksh bash 'bash -o posix' fbsh dash bosh yash \
        pdksh mksh ksh93 zsh 'zsh --emulate sh'
do
        $SHELL -c '
                printf "\n\nTesting %s : \n" '"'$SHELL'"'
                ls /bin/test
                /bin/ls /bin/test
                IFS=:
                for p in $PATH
                do
                        [ -x $p/test ] && echo found test in $p
                done
                test x = x || echo weird
        '
done >/tmp/results 2>&1

It needs full pathnames for the standard NetBSD shells /bin/sh and /bin/ksh
as /bin isn't in PATH any more - /bin/ksh and pdksh are more or less the same
thing, though probably have slightly different sets of bugs fixed.
fbsh is the FreeBSD shell, the rest I think are well known. I don't
have a copy of ksh88.


In that the initial printf is just for documentation, the two ls
lines that follow are to show first that "ls" is not found (as /bin
is not in $PATH, and second that /bin/test exists - that is, that
is where the test utility would normally be found). From those
two lines I would expect (in every shell) to see some kind of error
about "ls" not being found, and then "/bin/test" as the output
from the 2nd /bin/ls command (and indeed, that is what happens,
as shown below).

IFS is set to break apart $PATH, and the for loop simply
verifies that there is no other "test" command anywhere else
that might be invoked. If there is another test, the printf
would tell us where it was located, if there isn't, then there
will be no output. Obviously all shells should do the same thing
here, and they do, none finds a "test" anywhere else.

Note that there is actually a possible issue here with the '['
command (which also lives in /bin) so a shell might complain that
'[' is not found. None do.

The final line is the important one, if a shell implemented the
specification in 2.9.1.1 1.c and 2.9.1.1 1.e.i.a "correctly", there
should be a "test not found" error (in some format or other,
just like the "ls not found" error) here, as the built-in version
of the standard utility test is only supposed to be located if test
is found in $PATH (which we have just seen it will not be).
The "|| echo weird" is just in case some non-standard test were
to be invoked, and there "echo" (which also lives in /bin) could
also result in "command not found" - this we don't know, as there
is no attempt made to execute that "echo" command, the test
command (in every shell I tested - I had hoped that in one or
two might not have test built in) is a built-in command, and
is executed (IMO correctly) here (and operates correctly,
returning a status of 0, so the || part is never attempted).

The results showing all of this appear bust below. To me this
is clear evidence that the current specification is not specifying
the standard behaviour, and should be fixed.


Testing /bin/sh :
ls: not found
/bin/test


Testing /bin/ksh :
/bin/ksh: ls: not found
/bin/test


Testing bash :
bash: ls: command not found
/bin/test


Testing bash -o posix :
bash: ls: command not found
/bin/test


Testing fbsh :
fbsh: ls: not found
/bin/test


Testing dash :
dash: 1: ls: not found
/bin/test


Testing bosh :
bosh: ls: not found
/bin/test


Testing yash :
yash: no such command `ls'
/bin/test


Testing pdksh :
pdksh: ls: not found
/bin/test


Testing mksh :
mksh: ls: not found
/bin/test


Testing ksh93 :
ksh93: ls: not found [No such file or directory]
/bin/test


Testing zsh :
zsh:1: command not found: ls
/bin/test


Testing zsh --emulate sh :
zsh:1: command not found: ls
/bin/test



First, if considering Issue8 draft 1, all references below to
section 2.9.1.1 should be interpreted as references to section 2.9.1.4

In section 2.9.1.1 1.c (page 2367) delete the sentences:
If the implementation has provided a standard utility
in the form of a function, it shall not be recognized at this point.
It shall be invoked in conjunction with the path search in step 1e.


Replace the entire text of section 2.9.1.1 1.d (which differs dramatically
between Issue 7 and Issue 8 draft 1, so the old text is not quoted
here), (page 2367) with:
If the command name matches the name of a utility built
into the shell, that utility shall be invoked.


In section 2.9.1.1 1.e replace the entire first part of the text of
sub-section i (including the entire contents of sub-sub-sections 'a' and 'b')
down to, but not including the paragraph that begins
Once a utility has been searched for and found

The text to be removed is not included here, as it differs between
Issue 7 and Issue 8 draft 1. The replacement text should be:
If the search is successful the shell shall execute the
utility as described in Section 2.9.1.6.

[This is using the section numbering for the reference as it appears
in Issue 8 draft 1, where 2.9.1.6 is the section entitled:
Non-built-in Utility Execution.]

These changes will make the standard match what shells actually
implement, rather than some pipe dream about what they should implement.
Notes
(0004920)
geoffclare   
2020-08-14 08:47   
Although it goes into more detail, this is effectively a repeat of bug 0000854 and should be closed as a duplicate.
(0004924)
joerg   
2020-08-14 10:39   
(edited on: 2020-08-19 10:36)
Just in case this is not known, ksh93 implements PATH search for builtin commands and this was a requirement for ksh93 integration into OpenSolaris in 2008 after a long and heaty discussion. The reason for that requirement was to retain POSIX compliance even though basic standard utilities are built into the standard shell.

If a shell does not implement PATH search for additional builtins, we would need to forbid shells to add more builtins than the minimal set.

ksh93 added many additional builtins like cat, chown, cmp, ... and POSIX grants that a user that if he has private version of system utilities, the shell needs to use them if they are first in PATH.

If a shell adds more builtins than currently documented, this would no longer work as historically builtin are always found first.

As a consequence, we created a list of tainted utilities (see 0000854) that already may be builtins.

If you like to add more builtins from the list of standard utilities, you either need to implement PATH search for these builtins, or you are forbidden to implement these utilities as builtins.

In ksh93, builtin commands are managed with the builtin named "builtin".

Call builtin for a list....e.g.

builtin
:
.
[
/bin/cat
/bin/chown
/bin/cksum
/bin/cmp
/bin/comm
/bin/cut
/bin/dirname
/bin/fold
/bin/getconf
/bin/head
/bin/join
/bin/logname
/bin/mkdir
/bin/mkfifo
/bin/mktemp
/bin/paste
/bin/pathchk
/bin/rev
/bin/rmdir
/bin/sum
/bin/sync
/bin/tail
/bin/tee
/bin/tty
/bin/uniq
/bin/wc
/bin/xgrep
/sbin/sync
/usr/ast/bin/basename
/usr/ast/bin/cat
/usr/ast/bin/chgrp
/usr/ast/bin/chmod
/usr/ast/bin/chown
/usr/ast/bin/cksum
/usr/ast/bin/cmp
/usr/ast/bin/comm
/usr/ast/bin/cp
/usr/ast/bin/cut
/usr/ast/bin/date
/usr/ast/bin/dirname
/usr/ast/bin/egrep
/usr/ast/bin/expr
/usr/ast/bin/fds
/usr/ast/bin/fgrep
/usr/ast/bin/fmt
/usr/ast/bin/fold
/usr/ast/bin/grep
/usr/ast/bin/head
/usr/ast/bin/id
/usr/ast/bin/join
/usr/ast/bin/ln
/usr/ast/bin/logname
/usr/ast/bin/md5sum
/usr/ast/bin/mkdir
/usr/ast/bin/mkfifo
/usr/ast/bin/mktemp
/usr/ast/bin/mv
/usr/ast/bin/paste
/usr/ast/bin/pathchk
/usr/ast/bin/readlink
/usr/ast/bin/rev
/usr/ast/bin/rm
/usr/ast/bin/rmdir
/usr/ast/bin/stty
/usr/ast/bin/sum
/usr/ast/bin/sync
/usr/ast/bin/tail
/usr/ast/bin/tee
/usr/ast/bin/tty
/usr/ast/bin/uname
/usr/ast/bin/uniq
/usr/ast/bin/wc
/usr/ast/bin/xgrep
/usr/sbin/sync
/usr/xpg4/bin/basename
/usr/xpg4/bin/egrep
/usr/xpg4/bin/fgrep
/usr/xpg4/bin/grep
/usr/xpg4/bin/id
/usr/xpg4/bin/rm
/usr/xpg4/bin/tail
/usr/xpg6/bin/expr
alarm
alias
bg
break
builtin
cd
command
....

Note that the builtins bound to /usr/ast/bin are not fully POSIX compliant and there fore are not bound to a path that appears in the default PATH.

If you do not care about POSIX compliance or if you do not like to add more builtins to your shell, you may do what ever you like. If you however like to add more builtins, you need to follow long existing rules about PATH search.

(0004934)
McDutchie   
2020-08-20 15:31   
ksh93 does not follow the POSIX rules either, and never has done:

(1) ksh93 has several builtins (including builtin aliases, something POSIX doesn't allow either), not bound to any path search, that are "not allowed" by this rule, i.e.: which are neither a standard utility, nor in the list of names reserved as "unspecified" in XCU 2.9.1.4(b). They are: alarm, compound, enum, float, functions, integer, nameref, r, redirect.

(2) Several POSIX standard regular builtins are *never* subject to a path search in ksh93, although POSIX specifies that they must be. These are: [, echo, printf, sleep, test, type, ulimit.

What's more, path-bound builtins in ksh93 behave exactly like external commands. The only difference is their path doesn't actually exist in the file system. But that is a mere implementation detail. POSIX has no business specifying or regulating implementation details.

So, if this path search rule exists because of ksh93, then it is based on a fundamental misunderstanding of what ksh93 does.

In the real world, there are zero shells that actually manage to follow this rule. Not even the deliberately pedantic 'yash -o posix' fully manages, though it tries hard. Bash, Busybox ash, dash, FreeBSD sh, NetBSD sh, mksh, pdksh (OpenBSD sh), and zsh all ignore this rule completely. So does yash in its sane default mode.

This entire path search rule for builtins should quite simply be removed. I suppose it won't do much damage if POSIX decides to keep it; I'm confident the overwhelming majority of shells will continue to ignore it completely. The real damage is to POSIX's reputation as a standard to be taken seriously.




Viewing Issue Advanced Details
1388 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Omission 2020-08-11 14:17 2021-01-21 22:17
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
yacc
3456
116732
---
yacc description does not say who declares yyerror() and yylex()
The description of yacc talks about the functions yyerror() and yylex() in various places, but nowhere does it state who is responsible for declaring them. This means that, in practice, a portable application has to declare them in the .y file (if it does not define them) in case yacc does not provide declarations of them in the code file and the compiler that will be used treats calls to undeclared functions as an error.

It doesn't take much thought about the reasons why prototypes were added to the C language to come to the conclusion that the greatly preferable solution is for yacc to be required to supply prototypes for yyerror() and yylex() that match the yyerror() definition in its library and the yylex() definition produced by lex, even (some might say especially) if the .y file includes definitions of those functions to be used instead of the library version of yyerror() and a lex-generated yylex().

There is also no statement about a declaration of main(), but the situation for main() is quite different from the above two functions. Although in theory an application could call yacc's library version of main() from code in a .y file, it is questionable why any application (other than a test suite) would do so, in particular because that version of main() does not accept any arguments and it calls exit() -- it does not return -- and therefore is of little use recursively. An application that provides its own main() could call it recursively, but can reasonably be expected to ensure it does not call main() without previously defining or declaring it. In addition, since main() has multiple different allowed prototypes, if yacc were to output a declaration it would have to be a non-prototype one:

int main();

so that there is no risk of a clash with a definition of main() that has a different prototype, but there does not seem much point in it producing such a declaration. (Or it could check whether the .y file contains a definition of main() and output a prototype declaration if it does not contain one, but that seems impractical.)

The simplest solution is just not to allow yacc to provide a declaration of main().
On page 3456 line 116732 section yacc (Code File), change:
It also shall contain a copy of the #define statements in the header file.
to:
It also shall contain prototype declarations of the yyerror() and yylex() functions, and a copy of the #define statements in the header file, prior to any code copied from within <tt>%{</tt> and <tt>%}</tt> in the declarations section in grammar.

On page 3456 line 116734 section yacc (Code File), add a new paragraph:
The code file shall not contain a declaration of the main() function, unless one is present within <tt>%{</tt> and <tt>%}</tt> in the declarations section in grammar.

On page 3469 line 117335 section yacc (RATIONALE), add new paragraphs:
Earlier versions of this standard did not require the code file created by yacc to contain declarations of yyerror() and yylex(). This meant that portable applications that did not define them had to declare them in the grammar file, to ensure they would not be diagnosed by the compiler as being called without being declared, but this was not stated in those versions of the standard either. The standard developers decided it was preferable for yacc to include the declarations in the code file and this is now a requirement.

Earlier versions of this standard were also silent about a declaration of main(). However, the equivalent solution was not adopted since a declaration of main() would only be needed if it is called recursively by an application. Although in theory an application could call the yacc library version of main() from code in a grammar file, it is questionable why any application (other than a test suite) would do so, in particular because that version of main() does not accept any arguments and it calls exit() -- it does not return -- and therefore is of little use recursively. An application that includes its own definition of main() could call it recursively, but can reasonably be expected to ensure it does not call main() without previously defining or declaring it. An additional complication is that main() has multiple different allowed prototypes. The standard developers decided the simplest solution was not to allow yacc to provide a declaration of main() in the code file.

Notes
(0004919)
shware_systems   
2020-08-11 16:53   
A collateral issue is ensuring yyin is declared as a FILE *, if yyparse() is linked to a yylex() generated by lex, in a manner accessible to code calling yyparse(). This adds an implied requirement on the .c or .h file produced to do a #include <stdio.h> to get the defining declaration of FILE.

The example main() for lex assumes such a declaration of yyin is produced at the top or bottom of the %% section, but there is no requirement this be an extern the header produced by yacc could reference. The lex description just has it shall be used, nothing on how it is to be declared, actually. The main() example for yacc doesn't declare or assign stdin to it either, assuming yylex() does this by default somehow on first call if generated by lex, or accesses stdin directly in a yylex() implemented in the programs section of the .y file.

Maybe this has been addressed as part of some other bug, but I don't remember any such discussion offhand.
(0004928)
Konrad_Schwarz   
2020-08-19 10:08   
Aren't default implementations of these symbols defined in the -ly library?
(0004931)
shware_systems   
2020-08-19 16:42   
(edited on: 2020-08-19 16:44)
Yes, but when a compiler requires the absence of any main() to be found to format an a.out as a dynamic library, rather than a utility executable, the use of liby and libl is precluded so the compile doesn't see the main() in those libraries. As such, the output of yacc or lex needs to be compile-able in a manner that doesn't require any library to provide those prototypes implicitly.

(0005205)
rhansen   
2021-01-21 17:17   
On page 3456 line 116732 section yacc (Code File), change:
It also shall contain a copy of the #define statements in the header file.
to:
It also shall contain function prototypes for the yyerror(), yylex(), and yyparse() functions, and a copy of the #define statements in the header file, prior to any code copied from within <tt>%{</tt> and <tt>%}</tt> in the declarations section in grammar.


On page 3456 line 116734 section yacc (Code File), add a new paragraph:
The code file shall not contain a declaration of the main() function, unless one is present within <tt>%{</tt> and <tt>%}</tt> in the declarations section in grammar.


On page 3469 line 117335 section yacc (RATIONALE), add new paragraphs:
Earlier versions of this standard did not require the code file created by yacc to contain declarations of yyerror(), yylex(), and yyparse(). This meant that portable applications that did not define them had to declare them in the grammar file, to ensure they would not be diagnosed by the compiler as being called without being declared, but this was not stated in those versions of the standard either. The standard developers decided it was preferable for yacc to include the declarations in the code file and this is now a requirement.

Earlier versions of this standard were also silent about a declaration of main(). However, the equivalent solution was not adopted because a declaration of main() would only be needed if it is called recursively by an application. Although in theory an application could call the yacc library version of main() from code in a grammar file, it is questionable why any application (other than a test suite) would do so, in particular because that version of main() does not accept any arguments and it calls exit()—it does not return—and therefore is of little use recursively. An application that includes its own definition of main() could call it recursively, but can reasonably be expected to ensure it does not call main() without previously defining or declaring it. An additional complication is that main() has multiple different allowed prototypes. The standard developers decided the simplest solution was to disallow yacc from providing a declaration of main() in the code file.
(0005206)
nick   
2021-01-21 21:17   
From Akim Demaille (maintainer of GNU Bison):


I agree that yyparse should be part of the declarations in the header
file (or in the implementation file if there is no header).

However I dislike that yylex and yyerror be prototyped. We don't do
it in Bison because:

- the user might decide that yylex be static, and it's actually not
  unfrequent that it is. The user might want yylex to return an enum
  of the valid expected tokens. The parser itself simply does not
  care, and forcing a prototype onto the user will just be more
  constraints for her. She might even have played with #define
  yylex() to pass additional arguments, or re#defined yylex to foolex,
  etc.

  In my humble opinion the parser is merely a consumer of
  yylex, it is not the provider. It is up to the provider to provide
  the prototype. For instance *lex* should be in charge of declaring
  yylex, not Yacc.

- the user provides her yyerror and she is perfectly welcome to
  make it a variadic function that starts with a format string,
  instead of a plain string. The parser itself only calls this
  function with a single argument, but what if the user wanted
  something more generic for some other calls to yyerror that she
  wrote herself.

  Granted, there is something dangerous here if the parser was to
  pass a message with percent-signs in it. But Yacc does not
  issue such messages.

  Again, Yacc does not provide this function, it *requires* it.
  So I disagree that Yacc should provide the prototype.

Pushing prototypes onto the user restricts her freedom. Her
freedom to make things static, her freedom to use attributes
to defeat warnings about ignored arguments, her freedom to
simply comply with a contract the way she wants. Call it
duck-typing if you want, but my opinion is that the contract
between the user and yyparse is much weaker than what a
prototype would actually enforce.

(Not to mention things that are outside the scope of Yacc,
such as passing additional arguments to yylex to avoid, for
instance, the use of globals for semantic values. But yes,
this out of the scope of Yacc.)


Note that since people are currently providing the prototypes
themselves, there's a good chance that the ones provided by
new Yaccs might be incompatible. There will be backward
compatibility issues. And portability issues when from one
machine to another you get–or don't get–the prototype,
because of varying versions of Yacc.



The routines provided in the library are merely an instance of
the set of possibles. And to be clear, I think -ly is vastly
useless. It is simply not worth the trouble to have to find a
library for main and yyerror. Someone who is ready to
spend time learning Yacc to generate a parser for a grammar is
certainly competent enough to write main and yyerror.


I also see references to yyin. I'm not sure I understand
exactly what is suggested, but yyin is out of scope. Yacc
does not care about yyin at all. yyparse knows nothing
about chars, it only knows about tokens. People routinely
make parser of strings, not just FILE*. yyin is irrelevant
to Yacc and should not be referred to in Yacc's specifications.
(0005207)
dickey   
2021-01-21 22:17   
Without a prototype, a C compiler cannot provide useful diagnostics when the number or type of parameters differs from what yacc requires.

Deciding whether to require that a prototype be static or extern is outside the scope of this question, because it's possible to provide prototyping information conditionally, e.g., by a macro which defines the return type and parameters. This is an example of what has been defined by Berkeley yacc starting in 2010:

/* Parameters sent to lex. */
#ifdef YYLEX_PARAM
# define YYLEX_DECL() yylex(void *YYLEX_PARAM)
# define YYLEX yylex(YYLEX_PARAM)
#else
# define YYLEX_DECL() yylex(void)
# define YYLEX yylex()
#endif

/* Parameters sent to yyerror. */
#ifndef YYERROR_DECL
#define YYERROR_DECL() yyerror(const char *s)
#endif
#ifndef YYERROR_CALL
#define YYERROR_CALL(msg) yyerror(msg)
#endif

Those are generated, and for some non-yacc features the macros differ, but provide an application developer with a way to refer to yylex and yyerror consistently with the parser's assumptions.




Viewing Issue Advanced Details
1387 [1003.1(2008)/Issue 7] System Interfaces Editorial Clarification Requested 2020-08-10 16:46 2021-01-21 16:25
rhansen
ajosey  
normal  
Interpretation Required  
Accepted As Marked  
   
Richard Hansen
malloc
1295 (Issue 7 2018 edition)
43161 (Issue 7 2018 edition)
Proposed
Note: 0005203
Should EAGAIN be acceptable for malloc failure?
On failure, the implementations of malloc from Solaris, OpenSolaris, OpenIndiana, illumos, etc. set errno to either ENOMEM or EAGAIN (see https://illumos.org/man/3c/malloc [^] ). It seems to me that EAGAIN is used for at least a subset of ordinary out-of-memory conditions, so on the surface these implementations appear to be non-conforming.

I am unfamiliar with the implementation details, and the phrase "Insufficient storage space is available" could be interpreted in a few subtly different ways, so perhaps one could argue that EAGAIN is only used for error cases other than "Insufficient storage space is available" (which is permitted by the standard).

IIUC, Solaris has behaved this way for a very long time. If the implementations are considered to be non-conforming, then it might make more sense to change the standard to permit EAGAIN than to change the implementations.

Link to HTML version of Issue 7 2018 edition: https://pubs.opengroup.org/onlinepubs/9699919799/functions/malloc.html [^]
On line page 1295 line 43161, change:

  [ENOMEM] Insufficient storage space is available.

to:

  [ENOMEM], [EAGAIN] Insufficient storage space is available.

A similar change would be required for every other function that allocates memory, assuming the implementations of those functions on Solaris and friends use malloc and leave errno unmodified on malloc failure.
Notes
(0004914)
rhansen   
2020-08-10 17:09   
A problem arises if we were to replace all instances of [ENOMEM] with [ENOMEM], [EAGAIN]: Some functions already specify [EAGAIN] for other error conditions. These include:
  • accept
  • fclose
  • fflush
  • fgetc
  • fgetwc
  • fork
  • fputc
  • fputwc
  • mlock
  • mlockall
  • mmap
  • mprotect
  • open
  • openat
  • posix_trace_create
  • posix_trace_create_withlog
  • pread
  • pthread_barrier_init
  • pthread_cond_init
  • pthread_key_create
  • pthread_mutex_init
  • pthread_rwlock_init
  • pthread_rwlockattr_init
  • pthread_spin_init
  • read
  • recv
  • recvfrom
  • recvmsg
  • sendmsg
  • sendto
(0004915)
Don Cragun   
2020-08-10 17:39   
Rather than:

      [ENOMEM], [EAGAIN] Insufficient storage space is available.

I would prefer to see:

     [EAGAIN] Allocating the requested storage space would cause the thread to be blocked.
     [ENOMEM] Insufficient storage space is available.
(0004916)
alanc   
2020-08-10 17:45   
I believe the Solaris behavior originally came from passing through sbrk() failures without checking/changing the reported errno value, and thus distinguishes between hitting some limit, for which trying again is not worthwhile, vs. waiting for other processes to exit or otherwise free up memory.

(0004918)
shware_systems   
2020-08-11 01:08   
EAGAIN, if added as Don proposes, should be a may fail case, not both as shall fail. As ENOMEM is currently synonymous with a null return with most platforms, adding a symbolic non-NULL ptr, i.e. AGAIN_PTR, would be in keeping with the C standard only using the return value to indicate errors. The implementation-defined errno possible with 0 size allocs also needs wording that it doesn't conflict with either of these, as well.
(0004923)
joerg   
2020-08-14 09:50   
(edited on: 2020-08-14 09:51)
Re:Note: 0004916

Hi Alan, do you know where EAGAIN is created in the Solaris kernel while running sbrk()?

Due to the object oriented design of the address space administration in SunOS, it is hard to find the right location in the code.

What I however can say is that since SunOS-4.0 (from late 1987), kmem_alloc() with a flag of KM_NOSLEEP returns EAGAIN in case that the operation would result in a sleep.

The anon pages segment driver vm_anon.c however only returns ENOMEM, the sgvn driver seg_vn.c returns ENOMEM in plenty of cases (e.g. with shared mappings that are not expected to apply to sbrk()), but also ENOMEM in other cases.

(0004927)
Konrad_Schwarz   
2020-08-19 09:53   
Shouldn't this fall under the following provision in https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_03? [^]

"Implementations may support additional errors not included in this list, may generate errors included in this list under circumstances other than those described here, or may contain extensions or limitations that prevent some errors from occurring."
(0004932)
rhansen   
2020-08-19 19:57   
> Shouldn't this fall under the following provision in [...]

Yes, if you interpret illumos's EAGAIN case to be distinct from "insufficient storage space is available." To me, it doesn't feel any different, but I can see how others would feel otherwise.

If I saw Don's suggested wording (Note: 0004915) in the standard, then I would interpret "insufficient storage space is available" more narrowly than I do now.
(0004933)
Konrad_Schwarz   
2020-08-20 07:23   
But isn't this a distinction without a difference? In vast majority of
cases, resolution of the EAGAIN error depends on the actions of other
processes in the system over which the application has no control.

For fork(), I think a case can be made that in Unix, many processes are
short lived and therefore a retry after a limited period can be worthwhile
(given a fixed-size process table). Similar for open file descriptors.

For memory, in a system with long-lived, memory hogging processes
(e.g., large databases), retry after memory allocation failure
does not seem worthwhile.

If POSIX lists EAGAIN as an alternative for ENOMEM everywhere
ENOMEM is documented, scrupulous application programmers will
have to handle EAGAIN explicitly, presumably differently from the ENOMEM
case (i.e., by sleeping and then retrying).

As it is now, if malloc() returns EAGAIN, this would be
handled by fully-conforming code under the "additional errors"
case, e.g., with perror(). This gives the system administrator the
input that more memory or more swap space needs to be installed --
it's not something that the application can handle in a useful way.
(0004935)
rhansen   
2020-08-20 17:58   
(edited on: 2020-08-20 18:10)
> But isn't this a distinction without a difference?

I agree with your analysis. That's why it feels wrong to me that Solaris-based systems use EAGAIN—they should use ENOMEM instead.

I see these options:
  • Change Solaris-based systems to use ENOMEM instead of EAGAIN. I prefer this option, but it is unlikely to happen.
  • Declare Solaris-based systems to be non-conforming and live with that.
  • Try to convince ourselves that the Solaris EAGAIN error condition is completely different than "insufficient storage space is available", and therefore the behavior does not run afoul of "Implementations shall not generate a different error number from one required by this volume of POSIX.1-2017 for an error condition described in this volume of POSIX.1-2017." I don't like this option because I don't think the error condition is completely different, for the reasons you state in your note.
  • Change the standard to mention EAGAIN as a possible error with a subtly different meaning ("insufficient storage space right now, but it might succeed if you try again later" vs. "a hard limit would be exceeded so there will never be sufficient storage space"). I don't like this option because retries are not generally useful so why make the distinction? In my opinion this makes the standard slightly less useful. The only reason to choose this option is to allow Solaris to be declared conforming without changing its implementation.


(0004936)
alanc   
2020-08-20 18:14   
There is an open bug report against Solaris for this:
15109791 malloc(3C) fails with EAGAIN where ENOMEM is expected
and it could be fixed if needed, it's just never been a high priority
since few applications do anything differently for EAGAIN vs. ENOMEM
return values - most just print the error message from them.

That bug notes the historical distinction was:

     ENOMEM
           The physical limits of the system are exceeded by size
           bytes of memory which cannot be allocated.

     EAGAIN
           There is not enough memory available to allocate size
           bytes of memory; but the application could try again
           later.

but also notes Solaris has multiple malloc implementations, and not all
of them made this distinction. (We're currently up to 8 different malloc
library options in Solaris 11.4 - see the ALTERNATIVE IMPLEMENTATIONS section
at the end of:
https://docs.oracle.com/cd/E88353_01/html/E37843/malloc-3c.html#scrolltoc [^] .)
(0004937)
rhansen   
2020-08-20 18:24   
(edited on: 2020-08-20 19:44)
"So you're telling me there's a chance. YEAH!" :)
(reference: https://www.imdb.com/title/tt0109686/quotes/qt0995799 [^] )

Joking aside, it's good news that Oracle isn't totally opposed to changing the behavior.

(0004938)
rhansen   
2020-08-20 18:30   
Is that Solaris bug report publicly accessible?
(0004939)
alanc   
2020-08-20 19:42   
Sorry, Oracle only makes Solaris bug reports accessible to customers with
support contracts, not to the general public.

(And yes, there's a chance, given a good reason, but that would only affect
 future support updates to Solaris 11.4, not decades of past releases that
 have had this behavior in.)
(0004940)
rhansen   
2020-08-20 19:48   
(edited on: 2020-08-20 19:49)
> there's a chance, given a good reason

Do you know if POSIX conformance would be a good enough reason by itself?

> that would only affect future support updates to Solaris 11.4,
> not decades of past releases that have had this behavior

From a POSIX perspective that's OK—only changing new releases is good enough to avoid rewording the standard.

(0005203)
geoffclare   
2021-01-21 16:11   
(edited on: 2021-01-22 09:30)
Interpretation response
------------------------
The standard clearly states that implementations may use any error number that is applicable to a particular failure, and conforming implementations must conform to this.

Rationale:
-------------
Some implementations return EAGAIN if the resource (memory) is temporarily unavailable. This is acceptable under the general rules for error numbers in section 2.3 of XSH.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
On page 637 line 22025 (calloc() rationale), change:
None.
to:
See the RATIONALE for [xref to malloc()].

On page 1295 line 43167 (malloc() rationale), change:
None.
to:
Some implementations set errno to [EAGAIN] to signal memory allocation failures that might succeed if retried and [ENOMEM] for failures that are unlikely to ever succeed, for example due to configured limits. XSH section 2.3 permits this behavior, since when multiple error conditions described there are simultaneously true there is no precedence between them.

On page 1788 line 57907 (realloc() rationale), change:
None.
to:
See the RATIONALE for [xref to malloc()].


(0005204)
agadmin   
2021-01-21 16:25   
Interpretation proposed: 21 Jan 2021




Viewing Issue Advanced Details
1386 [Issue 8 drafts] System Interfaces Objection Error 2020-08-08 19:43 2020-10-08 10:32
eggert
 
normal  
Applied  
Accepted As Marked  
   
Paul Eggert
UCLA
strftime
1964-1965
63876-63887
Note: 0005000
strftime RETURN VALUE and ERRORS problems
The RETURN VALUE and ERRORS section of strftime have been revised recently, and when I changed the tzdb reference implementation to support these revisions I found some problems.

1. The [EOVERFLOW] description requires strftime to fail if the requested time does not fit into time_t. This disallows a strftime implementation that internally uses an integer wider than time_t, e.g., a strftime operating in a mixed 32- and 64-bit time_t environment that uses 64 bits internally even when called via a 32-bit API. [EOVERFLOW] should be a "may fail", not a "shall fail".

2. No errno value is specified if strftime fails due to exhausting the output buffer. This is a longstanding problem with strftime: portable code has no way to distinguish success from failure when strftime returns 0, and this is a real problem in practice. The simplest way I can see to fix this is to require that errno is unchanged when 0 is returned on success. I've just now changed the reference strftime implementation in tzdb to do this, which was an easy thing to do while adding support for the other errno changes in draft POSIX; see <https://mm.icann.org/pipermail/tz/2020-August/029187.html>. [^]

3. Unnecessary terminology switching between "null byte" and "NUL character".
In lines 63887-63878, change "If the total number of resulting bytes including the terminating null byte is not more than _maxsize_" to "If successful".

After line 63880, add "If successful and 0 is returned, errno is unchanged."

After line 63882, add (as a "shall fail"):

  [ERANGE] The total number of resulting bytes including the terminating NUL character is more than _maxsize_.

Move lines 63883-63884 to just after line 63887, so that [EOVERFLOW] is a "may fail" rather than a "shall fail".

After line 63906 "APPLICATION USAGE" add:

A return value of 0 may indicate either success or failure if a format consists entirely of conversion specifiers like "%z" that happen to be replaced by no characters. To distinguish between success and failure when strftime() returns 0, an application can set errno to 0 before calling strftime() and test whether errno is 0 afterwards.
Notes
(0004904)
eggert   
2020-08-08 19:53   
(edited on: 2020-08-08 22:43)
Also see 0001353, where I've commented on the technical differences between the desired action there and the one here.

(0004905)
shware_systems   
2020-08-08 21:39   
Re: #1
It does not matter the internal implementation, it can use a 128-bit long double if it wants, if the calculated value is not in the symbolic range, as integral seconds, TIME_MIN to TIME_MAX it is a shall fail condition. Perhaps other phrasing than "fit in" that better reflects this range is warranted but a section change is not, that I see.
(0004906)
kre   
2020-08-08 21:47   
(edited on: 2020-08-09 16:55)
Trivia, a %z conversion can never result in 0 bytes, I think you meant %Z
but a much better (more likely) example is %p.

The ERANGE error isn't needed, it is easy to handle the ambiguity, by
simply never providing a conversion that can legitimately result in 0
bytes as the result. That simply means supplying an irrelevant prefix
character (like a space) at the start of the format if there is any possibility
that a successful return might be 0 bytes, or even always), and then ignoring
it (using s+1 as the resulting string, and result-1 as the length) when a
result >= 1 is returned (with 0 meaning one of the error cases, always).

(0004907)
eggert   
2020-08-09 07:19   
"it can use a 128-bit long double if it wants, if the calculated value is not in the symbolic range, as integral seconds, TIME_MIN to TIME_MAX it is a shall fail condition."

Yes I know that is what the current draft spec says. However, there is no reason for the spec to say that, and that is why I objected. If the strftime implementation can output the correct string for %s even though the represented value is outside of time_t range, there's no good reason for POSIX to require the implementation to fail.
(0004909)
eggert   
2020-08-09 07:27   
"That simply means supplying an irrelevant prefix character (like a space) at the start of the format if there is any possibility that a successful return might be 0"

That is not such a simple thing in a general-purpose subroutine such as a one that accepts a strftime format, generates a date string with that format, and puts the date string in a user-specified buffer. Such a subroutine would have to allocate a buffer to store a near-copy of the format, and another buffer to store a near-copy of the returned string. This is awkward and inefficient and hardly anybody does this. It's more common to use code like this:

        cp[0] = '\1';
        result = strftime(cp, size, format, tmp);
        if (result != 0 || cp[0] == '\0')
            /* strftime worked */;

(this is code taken from tzdb date.c) which "generally" works (although it relies on undocumented strftime internals) and which we hope is good enough. It would be much better if the strftime API gave us a straightforward way to distinguish success from failure. And this is very easy for implementations to do.

"I think you meant %Z" - You're right, thanks.
(0004910)
shware_systems   
2020-08-09 12:06   
(edited on: 2020-08-09 21:57)
Re: Note: 0004907

The type for "seconds since the epoch" is time_t, so values outside its range are not valid specifications for that platform. For 32-bit time_t's this is the 2038 rollover issue. That you need 60 bits or so to represent seconds since the Big Bang, and counting, is a known bug of many implementations in not being able to represent all times of scientific interest in a time_t. You certainly need more than 32 bits if the range of tm_year is 32 bits, and these don't even cover the age of the planets, as a similar defect of most tm struct definitions. Yes, doing the conversion as if by mktime() is somewhat arbitrary, but it is consistent with other uses of "seconds since the epoch" elsewhere in the standard.

(0004911)
kre   
2020-08-09 16:45   
(edited on: 2020-08-09 16:47)
Re Note: 0004905 Note: 0004907 and Note: 0004910

Paul is correct, there is no reason to prohibit an implementation of strftime()
from accepting a struct tm that cannot be converted into a time_t. The spec
should allow an implementation to fail in that case, not require it to do so.
The purpose of strftime() is not tm validation (if it were so, then the shall
fail would make sense, but it isn't, so it doesn't).

(0004912)
kre   
2020-08-09 16:51   
Re Note: 0004909

You're right, the workaround can be difficult to handle in a poorly
designed subroutine. Redesigining the subroutine API can allow it to
be handled simply however. In practice there are few such interfaces,
and even fewer exist in places where a 0 length valid result is
plausible, so most of the time, handling strftime() as it now exists
is not a issue worth complaining about.
(0004913)
shware_systems   
2020-08-10 11:40   
(edited on: 2020-08-10 14:52)
Re: Note: 0004911
If a platform wants that functionality it can add a char *func(const tm *arg) interface for it. A proposal for adding this using a %Os specification might get support also, if implemented.

Apparently enough platforms that added %s as an extension were using mktime() as part of the code for the conversions so the standard reflects this as the existing practice. Someone that was part of the discussions would need to verify that. Since mktime() does validate the structure, strftime() also is required to validate it now, if %s is used.

(0004917)
kre   
2020-08-10 17:45   
(edited on: 2020-08-10 23:49)
Re Note: 0004913

    [...] extension were using mktime() as part of the code for the
    conversions so the standard reflects this as the existing practice.

That's fine, and allowing the error is fine too.

But:

    Since mktime() does validate the structure, strftime() also is
    required to validate it now, if %s is used.

In the specification of the 's' conversion, after "as if by a call to mktime()"
we should add

    except that errors that may be generated by mktime need not be
    detected, and no actual time_t value need be generated.

It is "as if" by mktime() - nothing requires that mktime() actually be
used (even for %s) - especially as:

    determined using the LC_TIME category of the current locale and by the
    values of zero or more members of the broken-down time structure pointed
    to by timeptr, as specified in brackets in the description. If any of the
    specified values are outside the normal range, the characters stored are
    unspecified.

That is, struct tm normalisation is not required of strftime() (even though
mktime() does that) as if any values are out of range, the results are
unspecified. So an implementation could calculate the number of seconds
by simply usig a series of multiplications and additions based upon the
fields of the tm struct (with appropriate checks for month lengths, leap years
etc) with the results in any integral type it chooses.

There should be no requirement for an EOVERFLOW to be generated, that error
should be moved into the "may fail" section. Forcing implementations to
be defective, just because old ones were is not desirable or productive.

(0005000)
rhansen   
2020-09-21 15:29   
(edited on: 2020-09-21 15:31)
On page 1961 line 63753 fix the shading bug.

On page 1962 line 63784 section strftime(), change:
Replaced by the number of seconds since the Epoch as a decimal number (see XBD Section 4.17, on page 93). [all members, as if by a call to mktime()]
to:
Replaced by the number of seconds since the Epoch as a decimal number, calculated as described for mktime(). [all members]

On page 1964 lines 63877-63878, change:
If the total number of resulting bytes including the terminating null byte is not more than maxsize
to:
If successful

On page 1964 line 63879, change:
Otherwise
to:
If successful, errno shall not be changed. Otherwise

On page 1964, after line 63882, add (as a "shall fail"):
[ERANGE] The total number of resulting bytes including the terminating NUL character is more than maxsize.

On page 1965, move lines 63883-63884 to just after line 63887, so that [EOVERFLOW] is a "may fail" rather than a "shall fail".

On page 1965 after line 63906 "APPLICATION USAGE" add:
A return value of 0 may indicate either success or failure if a format is the empty string or consists of conversion specifications such as <tt>%p</tt> or <tt>%Z</tt> that are replaced by no characters in the current locale or because of the current setting of tzname[], respectively. To distinguish between success and failure when strftime() returns 0, an application can set errno to 0 before calling strftime() and test whether errno is 0 afterwards.






Viewing Issue Advanced Details
1385 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Error 2020-07-29 10:44 2021-01-11 16:50
geoffclare
 
normal  
Resolved  
Accepted  
   
Geoff Clare
The Open Group
unlink()
2197
70220
---
unlink() text needs updating to account for mmap()
The mmap() description says:
The mmap() function shall add an extra reference to the file associated with the file descriptor fildes which is not removed by a subsequent close() on that file descriptor. This reference shall be removed when there are no more mappings to the file.
I believe the intention of this is that when unlink() removes the last link to the file, the space it occupies is not freed until no process has the file open and no process has it mapped. However, this is not reflected in the description of unlink(), which only refers to processes having the file open.
Change:
When the file's link count becomes 0 and no process has the file open, the space occupied by the file shall be freed and the file shall no longer be accessible. If one or more processes have the file open when the last link is removed, the link shall be removed before unlink() returns, but the removal of the file contents shall be postponed until all references to the file are closed.
to:
When the file's link count becomes 0 and no process has a reference to the file via an open file descriptor or a memory mapping (see [xref to mmap()]), the space occupied by the file shall be freed and the file shall no longer be accessible. If one or more processes have such a reference to the file when the last link is removed, the link shall be removed before unlink() returns, but the removal of the file contents shall be postponed until there are no such references to the file.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1384 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Omission 2020-07-29 09:02 2021-01-14 16:31
geoffclare
 
normal  
Resolved  
Accepted As Marked  
   
Geoff Clare
The Open Group
2.12 Shell Execution Environment
2382
76195-76202
---
Note: 0005198
Subshell of an interactive shell is effectively non-interactive
The standard is missing a statement to the effect that a subshell of an interactive shell behaves as a non-interactive shell (despite the fact that $- and "set +o" indicate it is interactive).

For example, in all shells I tried, the subshell in the following is terminated by SIGTERM. (Some wrote "Terminated" between the two lines, others didn't, but the $? value in all of them indicated termination with SIGTERM and none of them wrote "foo".)
$ (pid=$(sh -c 'echo $PPID'); kill -s TERM "$pid"; echo foo)
$ echo $?
If the subshell was behaving as per the standard's requirements for an interactive shell, the SIGTERM would be ignored, the echo would be executed, and $? would be 0.

The current statement about traps in 2.12 is also out of date with respect to the description of the trap utility.
On page 2382 line 76195 section 2.12 Shell Execution Environment,
after applying bug 1247 change:
A subshell environment shall be created as a duplicate of the shell environment, except that traps that are not being ignored shall be set to the default action.
to:
A subshell environment shall be created as a duplicate of the shell environment, except that:
  • Unless specified otherwise (see [xref to trap]), traps that are not being ignored shall be set to the default action.

  • If the shell is interactive, the subshell shall behave as a non-interactive shell in all respects other than the expansion of the special parameter '-' and the output of <tt>set -o</tt> and <tt>set +o</tt>, which shall continue to indicate that it is interactive.

Notes
(0005198)
rhansen   
2021-01-11 16:45   
(edited on: 2021-01-14 16:30)
On page 2382 line 76195 section 2.12 Shell Execution Environment,
after applying bug 1247 change:
A subshell environment shall be created as a duplicate of the shell environment, except that traps that are not being ignored shall be set to the default action.
to:
A subshell environment shall be created as a duplicate of the shell environment, except that:
  • Unless specified otherwise (see [xref to trap]), traps that are not being ignored shall be set to the default action.
  • If the shell is interactive, the subshell shall behave as a non-interactive shell in all respects except:
    • The expansion of the special parameter '-' may continue to indicate that it is interactive.
    • The <tt>set -n</tt> option may be ignored.

On page 2411 line 77159 (XCU 2.14 set -n) change:
An interactive shell may ignore this option.
to:
Interactive shells and subshells of interactive shells, recursively, may ignore this option.


(0005200)
rhansen   
2021-01-14 16:31   
(edited on: 2021-01-14 16:37)
Note: 0005198 was edited to change "and the output of <tt>set -o</tt> and <tt>set +o</tt> shall" to "may" in response to a comment on the mailing list:

https://www.mail-archive.com/austin-group-l@opengroup.org/msg07384.html [^]





Viewing Issue Advanced Details
1383 [Issue 8 drafts] System Interfaces Editorial Enhancement Request 2020-07-25 11:24 2020-10-08 10:27
dannyniu
 
normal  
Applied  
Accepted As Marked  
   
DannyNiu/NJF
Fork
878
29985
Note: 0004997
Make "Application Usage" less confusing.
The "Application Usage" section has new content that's
added in response to Bug 62, but right now it has a
confusing (at least on the first sight) wording that's
unfriendly to new readers not familiar to the modification
made to fork.

Specifically, it says:

> processing performed in the child before fork() returns

On the first sight, it seems nonsensical as child doesn't
even exist before fork returns, however, when reminded of
pthread_atfork handlers, it makes sense.
Change

> processing performed in the child before fork() returns

To

> processing performed in the child by the pthread_atfork()
> handlers before fork() returns
Notes
(0004900)
geoffclare   
2020-07-27 09:07   
This change was not made as a result of bug 0000062, it is from the unrelated bug 0001114.

Where the new text refers to "processing performed in the child before fork() returns" it means everything that the implementation of fork() does in the child between the point where it creates the child process and the point where it returns in the child process. This includes any processing related to the bullet list on lines 29903-29957.

Rather than adding a reference to pthread_atfork() (which would incorrectly imply that the text only applies to processing performed by atfork handlers) we should make it clear that the text applies to both fork() and _Fork().
(0004901)
geoffclare   
2020-07-27 09:13   
Suggested change...

On page 878 line 29980 section fork(), change:
When a multi-threaded process calls fork(), ...
to:
When a multi-threaded process calls fork() or _Fork(), ...

On page 878 line 29985 section fork(), change:
the processing performed in the child before fork() returns
to:
the processing performed in the child before fork() or _Fork() returns in the child
(0004902)
shware_systems   
2020-07-27 14:40   
Upon rereading 1114, I'm more inclined second change in Note 4901 should start 'any processing', not 'the processing', to be inclusive of any implementations that set up the child's data space entirely before handing it off to the process scheduler, where this is plausible. Use of 'the' implies there will always be post handoff manipulations.
(0004997)
rhansen   
2020-09-17 16:03   
On page 878 line 29980 (fork() application usage), change:
When a multi-threaded process calls fork(), although all memory that is addressable in the calling thread remains addressable in the child, there is no guarantee that the contents of thread-specific memory, such as stacks or thread-local storage, that was specific to other threads in the parent will still be the same in the child. This is because such memory could be treated as available in the child given that the threads it related to do not exist in the child, and consequently the processing performed in the child before fork() returns could make use of that memory. Therefore applications should avoid using in the child any pointers to thread-specific memory that were passed to the calling thread from other threads in the parent.
to:
When a multi-threaded process calls fork() or _Fork(), there is no guarantee that thread-specific memory (such as stacks or thread-local storage) associated with threads in the parent other than the calling thread will still be available in the child. This is because threads in the parent other than the calling thread do not exist in the child. Consequently, the implementation of fork() and _Fork() could remove that memory from the address space in the child, or reuse it for other purposes before returning or (in the case of fork()) calling any of the fork handlers registered by pthread_atfork(). Therefore, applications should avoid using any pointers to thread-specific memory in the child that were passed to the calling thread from other threads in the parent.




Viewing Issue Advanced Details
1382 [1003.1(2016/18)/Issue7+TC2] System Interfaces Editorial Clarification Requested 2020-07-24 15:16 2021-01-04 16:09
tydeman
 
normal  
Resolved  
Accepted As Marked  
   
Fred J. Tydeman
Tydeman Consulting
2018 edition of Issue 7 of Base Specifications
XSH: system interfaces: asin()
603 (or 651 of 3952)
20997
---
Note: 0005187
Ambiguous statement in asin() and other math functions
In looking at asin(), I am confused by the wording in line 20997
"If x is not returned,....."
Which x is it talking about? The one on line 20995 or any x?

Is line 20997 supposed to be a continuation of line 20996?

If yes, but it cannot be because of MX vs MXX shading,
then line 20997 should be changed to
"If a subnormal result is not returned,...."

I believe the same problem exists for many other math functions: at least:
asinh, atan, atanh, expm1, log1p, sin, sinh, tan, tanh
Clean up the wording. nextafter() might be a way to show both MX and MXX shading as continuous.
Notes
(0004899)
geoffclare   
2020-07-24 15:39   
This text came from bug 0000068 where it is shown as one paragraph:

    [MX] If x is subnormal, a range error may occur[/MX] [MXX] and x
    should be returned.[/MXX]
    [MX] If x is not returned, asin(), asinf(), and asinl() shall
    return an implementation-defined value no greater in magnitude
    than DBL_MIN, FLT_MIN, and LDBL_MIN, respectively.[/MX]

So the paragraph break is an editorial mistake in applying bug 68. If the paragraph break is changed to a line break, the result would look like the current change from MX to MXX on the previous line, where the shading is continuous.

Changing to "If a subnormal result is not returned,...." won't work because the standard allows a different subnormal (implementation-defined) value to be returned.

The other cases are presumably also all the result of incorrectly applied bug 68 edits.
(0005187)
geoffclare   
2020-12-18 10:16   
In the Dec 17th teleconference it was felt that changing the wording so there is only one shading change instead of two would improve the readability, but we ran out of time to check for all of the affected places. I have now looked through the resolution of bug 0000068 and can confirm that the functions listed in the description here are the only ones where an unwanted paragraph break was inserted when the shading changed from one margin code to another.

Proposed changes...

At P603 L20995 (asin), P605 L21059 (asinh), P609 L21166 (atan),
P615 L21324 (atanh), P802 L27261 (expm1), P1276 L42602 (log1p),
P1993 L64163 (sin), P1995 L64237 (sinh), P2112 L67777 (tan) and
P2115 L67865 (tanh), change:
[MX] If x is subnormal, a range error may occur[/MX]
[MXX] and x should be returned.[/MXX]

[MX] If x is not returned, asin(), asinf(), and asinl() shall return an implementation-defined value no greater in magnitude than DBL_MIN, FLT_MIN, and LDBL_MIN, respectively.[/MX]
to (substituting the appropriate function names for asin(), asinf() and asinl()):
[MXX]If x is subnormal, x should be returned.[/MXX]
[MX]If x is subnormal, a range error may occur and, if x is not returned, asin(), asinf(), and asinl() shall return an implementation-defined value no greater in magnitude than DBL_MIN, FLT_MIN, and LDBL_MIN, respectively.[/MX]

Note to the editor: where the shading changes from MXX to MX, use a line break (.br) not a paragraph break (.P).




Viewing Issue Advanced Details
1381 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Omission 2020-07-23 09:34 2021-01-15 16:26
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
ftruncate()
980
33344
---
ftruncate() missing EINVAL from POSIX.1-1996
POSIX.1-1996 specified the following error for ftruncate():
[EINVAL] The fildes argument does not refer to a file on which this operation is possible.
This was omitted when POSIX.1-1996 and SUSv2 were merged to form POSIX.1-2001/SUSv3.

For consistency, truncate() should also have an equivalent EINVAL error.
On page 980 line 33344 section ftruncate() EINVAL, change:
The length argument was less than 0.
to:
The length argument is less than 0 or the fildes argument refers to a file on which this operation is not possible (for example, a pipe, FIFO or socket).

On page 2178 line 69765 section truncate() EINVAL, change:
The length argument was less than 0.
to:
The length argument is less than 0 or the path argument refers to a file, other than a directory, on which this operation is not possible (for example, a FIFO or socket).

(The editors may also want to rearrange both ERRORS sections into alphabetical order.)
There are no notes attached to this issue.




Viewing Issue Advanced Details
1380 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Objection Clarification Requested 2020-07-22 14:23 2021-01-15 16:23
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
3.192 Hard Link, 3.208 Link
64, 66
1891, 1951
---
Definitions of Link and Hard Link don't match some usage
The term "link" is defined in XBD 3.208 as a synonym for "directory entry", and "hard link" is defined in XBD 3.192 is "The relationship between two directory entries that represent the same file"; it also says that a hard link is "the result of an execution of the ln utility (without the -s option) or the link() function".

This does not match how these terms are often used in the standard. There are many places where it uses "link" to refer to both hard links and symbolic links, e.g. in this paragraph on the fstatat() page:
The lstat() function shall be equivalent to stat(), except when path refers to a symbolic link. In that case lstat() shall return information about the link, while stat() shall return information about the file the link references.
(In the second sentence both uses of "the link" do not fit the definition; they are being used to refer back to "symbolic link" in the previous sentence.)

Also, by a strict reading of the definition, the term "hard link" should be used only in contexts where a file has multiple links, but the standard uses it to refer to cases that include the possibility of a file with only one directory entry, e.g. on the <sys/stat.h> page where st_nlink is described as the "number of hard links".

There are, of course, also places where the terms are used correctly (mainly in text that dates back to before symbolic links were added).

There are two ways the problem could be fixed:

1. Keep the current definitions and change the text in places where it misuses the terms.

2. Change the definitions, and the text in places where changing the definitions changed the meaning.

The proposed changes do the latter. This is mainly because the current definitions are at odds with the way "link" and "hard link" are commonly used outside the context of the standard. I believe changing the definitions would make the text more natural and easier to understand.
On page 53 line 1648 section 3.130 Directory Entry, change:
Directory Entry (or Link)
to:
Directory Entry (or Hard Link)

On page 56 line 1697 section 3.144 Empty Directory, change:
A directory that contains, at most, directory entries for dot and dot-dot, and has exactly one link to it (other than its own dot entry, if one exists), in dot-dot. No other links to the directory may exist.
to:
A directory that contains, at most, directory entries for dot and dot-dot, and has exactly one hard link to it other than its own dot entry (if one exists), in dot-dot. No other hard links to the directory can exist.

On page 64 line 1891 section 3.192 Hard Link, change:
The relationship between two directory entries that represent the same file; see also Section 3.130 (on page 53). The result of an execution of the ln utility (without the -s option) or the link() function.
to:
See Directory Entry in Section 3.130 (on page 53). A file can have multiple hard links as a result of an execution of the ln utility (without the -s option) or the link() function.

On page 66 line 1951 section 3.208 Link, change:
See Directory Entry in Section 3.130 (on page 53).
to:
In the context of the file hierarchy, either a hard link or a symbolic link.

In the context of the c99 utility, the action performed by the link editor (or linker).

<small>Note:
The c99 utility is defined in detail in the Shell and Utilities volume of POSIX.1-2017.</small>

On page 235 line 7917 section <errno.h>, change:
[EMLINK] Too many links.
to:
[EMLINK] Too many hard links.

On page 236 line 7967 section <errno.h>, change:
[EXDEV] Cross-device link.
to:
[EXDEV] Improper hard link.

On page 413 line 14030 section <tar.h> LNKTYPE, change:
Link.
to:
Hard link.


Cross-volume changes to XSH ...

On page 484 line 16712 section 2.3 Error Numbers (EMLINK), change:
Too many links.
to:
Too many hard links.

On page 488 line 16865 section 2.3 Error Numbers (EXDEV), change:
Improper link. A link to a file on another file system was attempted.
to:
Improper hard link. Creation of a hard link to a file on another file system was attempted.

On page 965 line 32816 section fstatat(), change:
st_nlink shall be set to the number of links to the file
to:
st_nlink shall be set to the number of hard links to the file

On page 965 line 32823 section fstatat(), change:
the number of (hard) links to the symbolic link
to:
the number of hard links to the symbolic link

On page 1243 line 41492 section link(), change:
link one file to another file
to:
hard link one file to another file

On page 1243 line 41500,41503,41507,41525,41531 section link(), change:
a new link
to:
a new hard link

On page 1244 line 41545 section link(), change:
The number of links
to:
The number of hard links

On page 1244 line 41568 section link() EXDEV, change:
The link named by path2 and the file named by path1 are on different file systems and the implementation does not support links between file systems
to:
The file named by path1 and the directory in which the directory entry named by path2 is to be created are on different file systems and the implementation does not support hard links between file systems.

On page 1245 line 41590 section link() EXAMPLES, change:
Creating a Link to a File

The following example shows how to create a link to a file
to:
Creating a Hard Link to a File

The following example shows how to create an additional hard link to a file

On page 1245 line 41599 section link() EXAMPLES, change:
Creating a Link to a File Within a Program

In the following program example, the link() function links the
to:
Creating a Hard Link to a File Within a Program

In the following program example, the link() function hard links the

On page 1246 line 41617 section link() APPLICATION USAGE, change:
Some implementations do allow links between file systems.
to:
Some implementations do allow hard links between file systems.

On page 1246 line 41621 section link() RATIONALE, change:
Linking to a directory
to:
Creating additional hard links to a directory

On page 1246 line 41625 section link() RATIONALE, change:
allow linking of files on different file systems
to:
allow hard linking of files on different file systems

On page 1246 line 41627 section link() RATIONALE, change:
The exception for cross-file system links is intended to apply only to links that are programmatically indistinguishable from ``hard'' links.
to:
The exception for cross-file system hard links is intended to apply only to links that are programmatically indistinguishable from traditional hard links.

On page 1246 line 41634 section link() RATIONALE, change:
The AT_SYMLINK_FOLLOW flag allows for implementing both common behaviors of the link() function. The POSIX specification requires that if path1 is a symbolic link, a new link for the target of the symbolic link is created. Many systems by default or as an alternative provide a mechanism to avoid the implicit symbolic link lookup and create a new link for the symbolic link itself.

Earlier versions of this standard specified only the link() function, and required it to behave like linkat() with the AT_SYMLINK_FOLLOW flag. However, historical practice from SVR4 and Linux kernels had link() behaving like linkat() with no flags, and many systems that attempted to provide a conforming link() function did so in a way that was rarely used, and when it was used did not conform to the standard (e.g., by not being atomic, or by dereferencing the symbolic link incorrectly). Since applications could not rely on link() following links in practice, the linkat() function was added taking a flag to specify the desired behavior for the application.
to:
Earlier versions of this standard specified only the link() function, and required it to behave like linkat() with the AT_SYMLINK_FOLLOW flag. However, historical practice from SVR4 and Linux kernels had link() behaving like linkat() with no flags, and many systems that attempted to provide a conforming link() function did so in a way that was rarely used, and when it was used did not conform to the standard (e.g., by not being atomic, or by dereferencing the symbolic link incorrectly). Since applications could not rely on link() following symbolic links in practice, the linkat() function was added taking a flag to specify the desired behavior for the application.

On page 1816 line 58819 section rename(), change:
If the old argument points to the pathname of a file that is not a directory, the new argument shall not point to the pathname of a directory. If the link named by the new argument exists, it shall be removed and old renamed to new. In this case, a link named new shall remain visible to other threads throughout the renaming operation and refer either to the file referred to by new or old before the operation began. Write access permission is required for both the directory containing old and the directory containing new.

If the old argument points to the pathname of a directory, the new argument shall not point to the pathname of a file that is not a directory. If the directory named by the new argument exists, it shall be removed and old renamed to new. In this case, a link named new shall exist throughout the renaming operation and shall refer either to the directory referred to by new or old before the operation began. If new names an existing directory, it shall be required to be an empty directory.
to:
If the old argument names a file that is not a directory and the new argument names a directory, or old names a directory and new names a file that is not a directory, or new names a directory that is not empty, rename() shall fail. Otherwise, if the directory entry named by new exists, it shall be removed and old renamed to new. In this case, a directory entry named new shall remain visible to other threads throughout the renaming operation and refer either to the file referred to by new or old before the operation began.

(Note that the sentence about write access permission is intentionally
dropped as it duplicates line 58834.)

On page 1816 line 58837 section rename(), change:
If the link named by the new argument exists and the file's link count becomes 0
to:
If the new argument names an existing file and the file's link count becomes 0

On page 1817 line 58871 section rename() EEXIST, change:
The link named by new is a directory that is not an empty directory.
to:
The new argument names a directory that is not empty.

On page 1818 line 58883 section rename() ENOENT, change:
The link named by old does not name an existing file
to:
The old argument does not name an existing file

On page 1818 line 58905 section rename() EXDEV, change:
The links named by new and old are on different file systems and the implementation does not support links between file systems.
to:
The file named by old and the directory in which the directory entry named by new is to be created or replaced are on different file systems and the implementation does not support hard links between file systems.

On page 2197 line 70216 section unlink(), change:
The unlink() function shall remove a link to a file. If path names a symbolic link, unlink() shall remove the symbolic link named by path and shall not affect any file or directory named by the contents of the symbolic link. Otherwise, unlink() shall remove the link named by the pathname pointed to by path and shall decrement the link count of the file referenced by the link.
to:
The unlink() function shall remove the directory entry named by path and shall decrement the link count of the file referenced by the directory entry. If path names a symbolic link, unlink() shall remove the symbolic link and shall not affect any file named by the contents of the symbolic link.


Cross-volume changes to XCU ...

On page 2897 line 95692 section ln, change:
In the first synopsis form, the ln utility shall create a new directory entry (link) at the destination path specified by the target_file operand. If the -s option is specified, a symbolic link shall be created for the file specified by the source_file operand.
to:
In the first synopsis form, the ln utility shall create a new directory entry at the destination path specified by the target_file operand. If the -s option is specified, a symbolic link shall be created with the contents specified by the source_file operand (which need not name an existing file); otherwise, a hard link shall be created to the file named by the source_file operand.

On page 2897 line 95697 section ln, change:
In the second synopsis form, the ln utility shall create a new directory entry (link), or if the -s option is specified a symbolic link, for each file specified by a source_file operand, at a destination path in the existing directory named by target_dir.
to:
In the second synopsis form, the ln utility shall create a new directory entry for each source_file operand, at a destination path in the existing directory named by target_dir. If the -s option is specified, a symbolic link shall be created with the contents specified by each source_file operand (which need not name an existing file); otherwise, a hard link shall be created to each file named by a source_file operand.

On page 2899 line 95743,95745 section ln (-L & -P options), change:
create a (hard) link
to:
create a hard link

On page 3075 line 102555 section pax (-u option), change:
In copy mode, the file in the destination hierarchy shall be replaced by the file in the source hierarchy or by a link to the file in the source hierarchy if the file in the source hierarchy is newer.
to:
In copy mode, the file in the destination hierarchy shall be replaced if the file in the source hierarchy is newer.

On page 3087 line 103039 section pax (ustar Interchange Format), change:
1 (a link) or 2 (a symbolic link)
to:
1 (a hard link) or 2 (a symbolic link)

On page 3093 line 103232 section pax (CONSEQUENCES OF ERRORS), change:
cannot create a link to a file
to:
cannot create a hard link to a file

On page 3101 line 103596 section pax (RATIONALE), change:
Links are recorded in the fashion described here because a link can be to any file type. It is desirable in general to be able to restore part of an archive selectively and restore all of those files completely. If the data is not associated with each link, it is not possible to do this. However, the data associated with a file can be large, and when selective restoration is not needed, this can be a significant burden. The archive is structured so that files that have no associated data can always be restored by the name of any link name of any link, and the user may choose whether data is recorded with each instance of a file that contains data. The format permits mixing of both types of links in a single archive; this can be done for special needs, and pax is expected to interpret such archives on input properly, despite the fact that there is no pax option that would force this mixed case on output.
to:
Hard links are recorded in the fashion described here because a hard link can be to any file type. It is desirable in general to be able to restore part of an archive selectively and restore all of those files completely. If the data is not associated with each hard link, it is not possible to do this. However, the data associated with a file can be large, and when selective restoration is not needed, this can be a significant burden. The archive is structured so that files that have no associated data can always be restored by the name of any link name of any hard link, and the user can choose whether data is recorded with each instance of a file that contains data. The format permits mixing of hard links with data and hard links without data in a single archive; this can be done for special needs, and pax is expected to interpret such archives on input properly, despite the fact that there is no pax option that would force this mixed case on output.

On page 3200 line 107351 section rm (APPLICATION USAGE), change:
do not permit the removal of the last link
to:
do not permit the removal of the last hard link


Cross-volume changes to XRAT ...

On page 3491 line 117998 section A.3 Definitions, delete:
Directory Entry

Throughout POSIX.1-2017, the term ``link'' is used (about the link() function, for example) in describing the objects that point to files from directories.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1379 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2020-07-20 08:36 2021-01-15 15:55
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
pax
3078
102671-102680
---
pax environment variables and affirmative responses
The pax utility has the usual boilerplate in the ENVIRONMENT VARIABLES section that utilities which read yes/no responses have. However, it doesn't prompt for that type of response. (The responses for -i are pathnames, not a y/n response.)

Also, LC_COLLATE refers to the pattern operand and the -s option in the part about equivalence classes, but they are missing from the corresponding LC_CTYPE text about character classes.
On page 3078 line 102671 section pax, change:
... used in the pattern matching expressions for the pattern operand, the basic regular expression for the -s option, and the extended regular expression defined for the yesexpr locale keyword in the LC_MESSAGES category.
to:
... used in the pattern matching expressions for the pattern operand and the basic regular expression for the -s option.

On page 3078 line 102674 section pax, change:
Determine the locale for the interpretation of sequences of bytes of text data as characters (for example, single-byte as opposed to multi-byte characters in arguments and input files), the behavior of character classes used in the extended regular expression defined for the yesexpr locale keyword in the LC_MESSAGES category, and pattern matching.
to:
Determine the locale for the interpretation of sequences of bytes of text data as characters (for example, single-byte as opposed to multi-byte characters in arguments and input files), and the behavior of character classes used in the pattern matching expressions for the pattern operand and the basic regular expression for the -s option.

On page 3078 line 102680 section pax, change:
Determine the locale used to process affirmative responses, and the locale used ...
to:
Determine the locale used ...

There are no notes attached to this issue.




Viewing Issue Advanced Details
1378 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Omission 2020-07-17 15:39 2021-01-15 15:53
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
ex
2699
88112
---
ex LC_MESSAGES and affirmative responses
Utilities that read yes/no responses have some boilerplate in the LC_MESSAGES entry in ENVIRONMENT VARIABLES that is missing from the ex page.

It is needed because the s command has a c flag that asks for confirmation.

(Note that the LC_COLLATE and LC_CTYPE entries do not need to mention yesexpr explicitly the way some utilities do, since they already cover that as part of their general statement about regular expressions.)
Change:
Determine the locale that ...
to:
Determine the locale used to process affirmative responses, and the locale that ...

There are no notes attached to this issue.




Viewing Issue Advanced Details
1377 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Error 2020-07-17 09:32 2021-01-15 15:51
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
at
2474
79386
---
at -q addition from POSIX.2b incorrectly applied
The description of the at -q option says:
If -q is specified along with either of the -t time_arg or timespec arguments, the results are unspecified.

This text is the result of an editorial mistake when applying the changes from POSIX.2b, where the addition to be made was given as:
If -q b is specified ...

I.e. this is about what happens if you specify the batch queue and also specify a time/date.
Change:
If -q is specified
to:
If -q b is specified

There are no notes attached to this issue.




Viewing Issue Advanced Details
1376 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Error 2020-07-16 08:19 2021-01-13 16:06
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
asctime(), ctime(), gmtime(), localtime()
600, 727, 1113, 1265
20907, 24771, 37681, 42218
---
CX shading on C99 time conversion functions
The descriptions of asctime(), ctime(), gmtime(), and localtime() should not have CX shading on this paragraph:
The asctime(), ctime(), gmtime(), and localtime() functions shall return values in one of two static objects: a broken-down time structure and an array of type char. Execution of any of the functions may overwrite the information returned in either of these objects by any of the other functions.

The text is derived from the introductory paragraph for time conversion functions in C99 (7.23.3 para 1 in N1256). There is also a wording difference with C99 that is worth fixing.
On page 600 line 20907 section asctime(),
page 727 line 24771 section ctime(),
page 1113 line 37681 section gmtime(),
page 1265 line 42218 section localtime(), remove CX shading from:
The asctime(), ctime(), gmtime(), and localtime() functions shall return values in one of two static objects: a broken-down time structure and an array of type char. Execution of any of the functions may overwrite the information returned in either of these objects by any of the other functions.
and change it to:
The asctime(), ctime(), gmtime(), and localtime() functions shall return values in one of two static objects: a broken-down time structure and an array of type char. Execution of any of the functions that return a pointer to one of these object types may overwrite the information in any object of the same type pointed to by the value returned from any previous call to any of them.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1375 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Error 2020-07-15 14:08 2021-01-13 16:03
geoffclare
 
normal  
Applied  
Accepted As Marked  
   
Geoff Clare
The Open Group
scanf(), fwscanf()
950, 1004, 1006
32272, 34177, 34248, 34263
---
See Note: 0005183
*scanf() 'm' allocation char v. wchar_t problems
Various parts of the text relating to the 'm' allocation character on the fscanf() and fwscanf() pages do not correctly account for the difference between conversions that use the 'l' modifier and those that don't.

In some cases the text is improved by bug 0001173 but is still affected.
On page 950 line 32272 section fscanf(), after applying bug 1173 change:
... terminating null character.
to:
... terminating null character or wide character.

On page 1004 line 34177 section fwscanf(), after applying bug 1173 change:
... terminating null wide character.
to:
... terminating null character or wide character.

On page 1006 line 34248,34263 section fwscanf(), change two occurrences of:
wchar_t
to:
char
(This results in 's' and '[' each having one char and one wchar_t, the same as 'c'.)
Notes
(0005183)
nick   
2020-12-17 16:36   
On page 950 line 32272 section fscanf(), after applying bug 1173 change:...
... terminating null character.

to:
    
... terminating null character (or wide character).

    
On page 1004 line 34177 section fwscanf(), after applying bug 1173 change:
    
... terminating null wide character.

to:
    
... terminating null character (or wide character).

    
On page 1006 line 34248,34263 section fwscanf(), change two occurrences of:
wchar_t

to:
char




Viewing Issue Advanced Details
1374 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Error 2020-07-15 09:30 2021-01-13 16:01
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
ungetwc()
2195
70178
---
Value of file-position indicator after ungetwc()
The ungetwc() page says "The file-position indicator is decremented (by one or more) by each successful call to ungetwc(); if its value was 0 before a call, its value is unspecified after the call."

(Bug 0000701 changes "indicator is decremented" to "indicator for the stream shall be decremented".)

This implies that if the position is at 1 byte and ungetwc() pushes back a wide character that converts to 2 bytes, then the position is required to be set to -1.

The current text derives from XPG4 and does not match C99 (and is not CX shaded). The C99 text is the same as the original (1995) MSE spec. It says "the value of its file position indicator after a successful call to the ungetwc function is unspecified until all pushed-back wide characters are read or discarded".

It appears that when the MSE was incorporated into SUSv2 this discrepancy went unnoticed and the defective XPG4 text has been retained until now.
After applying bug 701, change:
The file-position indicator for the stream shall be decremented (by one or more) by each successful call to ungetwc(); if its value was 0 before a call, its value is unspecified after the call. The value of the file-position indicator after all pushed-back characters have been read shall be the same as it was before the characters were pushed back.
to:
The value of the file-position indicator for the stream after a successful call to ungetwc() is unspecified until all pushed-back wide characters are read or discarded; its value after all pushed-back wide characters have been read shall be the same as it was before the wide characters were pushed back.

Notes
(0004897)
shware_systems   
2020-07-15 14:22   
I think both XPG4 and the C standard are in error. The behavior is nominally unspecified only when the effective file position is less than the number of chars to be pushed back. Otherwise the position will be greater than or equal to zero. The ungetc() description reflects this generalization for its particular case where the char count is 1; 0 as position on entry is the only problematic value for most physical media.

Note both descriptions are only valid for the limiting case where the bit width for a byte of physical media equals the CHAR_BITS value. Additional complications neither standard addresses exist when this isn't the case. As exemplified by the <termios.h> header this isn't the general case for serial ports accessed via FILE * records, where the bit width may be from 5 to 8. For the C standard, where CHAR_BITS may be up to 15, this applies also when physical media uses 8 bit bytes, the expected behavior is left undefined, not even unspecified.




Viewing Issue Advanced Details
1373 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Error 2020-07-14 09:01 2021-01-13 15:58
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
test
3288
110620
---
The '[' utility does not conform to syntax guidelines 1 and 2
As noted in Note: 0004769 in bug 252, the description of the '[' form of the test utility needs to be updated to reflect the fact that it does not conform to Guidelines 1 and 2.
After:
The test utility shall not recognize the "--" argument in the manner specified by Guideline 10 in XBD Section 12.2 (on page 216).
add a new sentence:
In addition, when the utility name used is [ the utility does not conform to Guidelines 1 and 2.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1372 [1003.1(2016/18)/Issue7+TC2] Rationale Comment Error 2020-07-13 09:25 2021-01-13 15:57
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
A.3 (Mount Point)
3498
118283
---
Rationale says there is no mount point definition, but there is
XRAT A.3 has an entry for "Mount Point" with an asterisk, indicating that there is no definition for "Mount Point". This entry should have been removed when the definition of "Mount Point" was added to XBD.

There is also a cross-reference to this entry under "Root of a File System".
On page 3498 line 118283 section A.3, delete:
Mount Point*

The directory on which a ``mounted file system'' is mounted. This term, like mount() and umount(), was not included because it was implementation-defined.

On page 3501 line 118411 section A.3, change:
Implementation-defined; see Mount Point* (on page 3498).
to:
Commonly used to refer to a mount point; this standard uses the latter.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1371 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Objection Clarification Requested 2020-07-10 14:29 2020-12-18 09:16
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
3.370 Stream
93
2583
---
Definition of stream is XSH-specific
The definition of the term "stream" says it is a "file access object [...] as described by the ISO C standard" and says these objects can be created using fdopen(), etc.

This is XSH-specific, but the term needs to apply to XCU as well. In particular, the current definitions of "standard error", "standard input" and "standard output" are the ones from POSIX.2-1992 and they use the word "stream". In that standard, the definition of "stream" was:
An ordered sequence of characters, as described by the C Standard
which, although it also refers to the C Standard, does not associate "stream" with a file access object.

The definition also says streams are "associated with a file descriptor", but this is not true for streams opened with fmemopen() or open_memstream().
Change:
Appearing in lowercase, a stream is a file access object that allows access to an ordered sequence of characters, as described by the ISO C standard. Such objects can be created by the fdopen(), fmemopen(), fopen(), open_memstream(), or popen() functions, and are associated with a file descriptor.
to:
Appearing in lowercase, a stream is an ordered sequence of bytes, as described by the ISO C standard.

In the shell command language, each stream is associated with a file descriptor. These can be opened using redirection operators.

<small>Note:
Redirection is defined in detail in [xref to XCU 2.7].</small>
In the C language, each stream is accessed via a file access object and is either a stream associated with a file descriptor or a memory stream. A file access object associated with a file descriptor can be created by the fdopen(), fopen(), or popen() functions. A file access object for a memory stream can be created by the fmemopen() or open_memstream() functions.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1370 [Issue 8 drafts] Rationale Editorial Error 2020-07-10 09:21 2020-09-30 14:27
geoffclare
 
normal  
Applied  
Accepted As Marked  
   
Geoff Clare
The Open Group
B.1.1, C.1.1
3433, 3555
117421, 122714
See Note: 0004992.
Where change history starts
XRAT B.1.1 says:
The CHANGE HISTORY section for each entry details the technical changes that have been made to that entry from Issue 5. Changes between earlier versions of the base document and Issue 5 are not included.

and C.1.1 says:
The CHANGE HISTORY section for each utility describes technical changes made to that utility from Issue 5. Changes between earlier versions of the base document and Issue 5 are not included.

These statements are not actually true. The change history starts with an "Issue 5" subheading, but changes listed under that subheading give the changes made in Issue 5, not the changes from Issue 5. So the changes between Issue 4 Version 2 and Issue 5 are included, contrary to what the second sentence of each of those paragraphs says.

However, rather than fixing this text, perhaps what we should do is make it true. I.e. remove change history before the "Issue 6" subheading.

Looking back at how much change history was included in old versions:
  • XPG4 only had 1 heading: "Issue 4"
  • SUSv1 had 2: "Issue 4" and "Issue 4, Version 2"
  • SUSv2 had 3: "Issue 4", "Issue 4, Version 2", and "Issue 5"
  • SUSv3 had 2: "Issue 5" and "Issue 6"
  • SUSv4 had 3: "Issue 5", "Issue 6" and "Issue 7"
If we don't remove "Issue 5" in SUSv5, it will have 4 headings which is more than any previous version. Also, since Issue 6 was the POSIX.1/SUS merge, it seems like a good place to start the change history.
On each header, function, built-in utility, and utility page that has an "Issue 5" subheading under CHANGE HISTORY, delete from that heading down to (but not including) the next subheading.
Notes
(0004992)
rhansen   
2020-09-14 15:57   
On page 3433 line 117421 (XRAT B.1.1) change:
The CHANGE HISTORY section for each entry details the technical changes that have been made to that entry from Issue 5. Changes between earlier versions of the base document and Issue 5 are not included.
to:
The CHANGE HISTORY section for each entry details the technical changes that have been made in Issue 5 and later. Changes made before Issue 5 are not included.

On page 3555 line 122714 (XRAT C.1.1) change:
The CHANGE HISTORY section for each utility describes technical changes made to that utility from Issue 5. Changes between earlier versions of the base document and Issue 5 are not included.
to:
The CHANGE HISTORY section for each utility describes technical changes made to that utility in Issue 5 and later. Changes made before Issue 5 are not included.




Viewing Issue Advanced Details
1369 [Issue 8 drafts] Rationale Comment Error 2020-07-09 11:11 2020-09-30 14:25
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
A.8.3
3407
116496
COLUMNS and LINES rationale
Bug 0001185 made changes to the descriptions of COLUMNS and LINES in XBD, but did not change XRAT to match.
Change:
The default values for the number of column positions, COLUMNS, and screen height, LINES, are unspecified because historical implementations use different methods to determine values corresponding to the size of the screen in which the utility is run. This size is typically known to the implementation through the value of TERM, or by more elaborate methods such as extensions to the stty utility or knowledge of how the user is dynamically resizing windows on a bit-mapped display terminal.
to:
The default values for the number of column positions when COLUMNS is unset or null, and screen height when LINES is unset or null, are unspecified if the terminal window size cannot be obtained (from tcgetwinsize()) because historical implementations use different methods to determine the values.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1368 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Enhancement Request 2020-07-08 23:10 2021-01-14 16:46
steffen
 
normal  
Resolved  
Accepted As Marked  
   
Steffen Nurpmeso
Vol. 3: Shell and Utilities, at, batch
2479, 2523
79636 ff., 81458 ff.
---
Note: 0005168
Unworldly use of redirection and mailx(1) in at(1) and batch(1) examples.
The sequence

 at now + 1 hour <<!
 diff file1 file2 2>&1 >outfile | mailx mygroup
 !

(likewise for batch(1)) sends out a possibly empty message to the work group "mygroup", which is a very low quality example that seems unworldly.
If the -E option of mailx(1) becomes standardized (as requested in issue #1367) the above could become wonderful examples for the power of the *X shell environment, and simply be changed to

 at now + 1 hour <<!
 diff file1 file2 2>&1 >outfile | mailx -E mygroup
 !

(likewise for batch(1)). Otherwise change

 at now + 1 hour <<!
 diff file1 file2 2>&1 >outfile | mailx -E mygroup
 !

to

 2. This sequence, which demonstrates redirecting standard error to a pipe, and carefully avoiding of false conditional status codes, is useful in a command procedure (the sequence of output redirection specifications is significant),

 at now + 1 hour <<!
 exec >errfile
 diff file1 file2 2>&1 >outfile
 if [ -s errfile ]; then
   < errfile mailx mygroup
 fi
 !
Notes
(0005168)
rhansen   
2020-12-14 16:43   
(edited on: 2021-01-14 16:43)
After applying the changes from 0001347 make the following changes:

On page 2479 lines 79638-79640 (XCU at EXAMPLES) change:
at now + 1 hour <<!
diff file1 file2 2>&1 >outfile | mailx mygroup
!
to:
at now + 1 hour <<EOF
diff file1 file2 2>&1 >outfile | mailx -s "outfile update" mygroup
EOF
Note that this always sends mail when there has been an attempt to update outfile and the body of the message will be empty unless an error occurred.

3. The following shows how to capture both standard error and standard output:
at now + 1 hour <<EOF
{
    run-batch-procesing |
        mailx -s "batch processing output" mygroup
} 2>&1 | mailx -E -s "errors during batch processing" mygroup
EOF
and renumber the later examples.

On page 2523 lines 81460-81462 (XCU batch EXAMPLES) change:
batch <<!
diff file1 file2 2>&1 >outfile | mailx mygroup
!
to:
batch <<EOF
diff file1 file2 2>&1 >outfile | mailx -s "outfile update" mygroup
EOF
Note that this always sends mail when there has been an attempt to update outfile and the body of the message will be empty unless an error occurred.

3. The following shows how to capture both standard error and standard output:
batch <<EOF
{
    run-batch-procesing |
        mailx -s "batch processing output" mygroup
} 2>&1 | mailx -E -s "errors during batch processing" mygroup
EOF


(0005201)
rhansen   
2021-01-14 16:46   
Note: 0005168 was updated to address feedback sent to the mailing list:

https://www.mail-archive.com/austin-group-l@opengroup.org/msg07304.html [^]




Viewing Issue Advanced Details
1367 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Enhancement Request 2020-07-08 22:55 2020-12-16 16:41
steffen
 
normal  
Applied  
Accepted As Marked  
   
Steffen Nurpmeso
Vol. 3: Shell and Utilities, mailx
2943, 2944
97431, 97471 ff.
---
Note: 0005162
mailx: add -E option to discard (not send) empty messages
The mailx variants of Apple, NetBSD, OpenBSD as well as other Open Source incarnations, support a command line option -E that rejects sending empty messages, successfully.

This is very helpful in scripted use cases since possible error notifications will only be send out if necessary.

The backing value ("INTERNAL VARIABLE") is different (skipemptybody, dontsendempty, skipempty, to name a few), but the behaviour of the -E option itself is identical.

The only known mailx incarnation which does not support -E is Solaris (OpenIndiana inspected), but the implementation is simplicistic since mailx warns on empty messages, the code change would need to turn

  if (fsize(mtf) == 0 && hp->h_subject == NOSTR) {
    printf(gettext("No message !?!\n"));
    goto out;
  }

into

  if (fsize(mtf) == 0) {
    if (value("dontsendempty") != NOSTR)
      goto jout;
    if (hp->h_subject == NOSTR) {
      printf(gettext("No message !?!\n"));
      goto out;
    }
  }
On page 2943, line 97431, change

  mailx [−s subject] address...

into

  mailx [-E] [−s subject] address...

On page 2944, insert after line 97471

  -E Discard messages with an empty message body, successfully.
Notes
(0004895)
geoffclare   
2020-07-09 08:01   
Line 98259 in EXIT STATUS also needs to change. I suggest changing:
Successful completion; note that this status implies that all messages were sent, ...
to:
Successful completion; note that this status implies that all messages were sent, or successfully discarded (see -E), ...


(0004896)
steffen   
2020-07-09 13:25   
..and the Solaris / OpenIndiana should allow all-empty messages by default, nothing is wrong with them, they are the shortest possible mail-based notification ("ping"), which in practice is nice since in practice the MTA / LDA (Mail-Transfer-Agent, Local-Delivery-Agent) adds at least a so-called From_, but especially the former also a From: line, for example

  #?0|kent:steffen$ </dev/null mailx -:/ root
  mailx: No message, no subject; hope that's ok
  #?0|kent:steffen$ tail -n 13 /var/spool/mail/steffen

  From steffen@localhost Thu Jul 9 15:22:56 2020
  Received: from steffen (uid 1000)
          (envelope-from steffen@localhost)
          id a791
          by kent (DragonFly Mail Agent v0.13);
          Thu, 09 Jul 2020 15:22:56 +0200
  Date: Thu, 09 Jul 2020 15:22:56 +0200
  To: root
  User-Agent: mailx v14.9.19
  Message-Id: <5f071a30.a791.1071c30e@kent>
  From: <steffen@localhost>

  #?0|kent:steffen$

The BSD based code does

        if (fsize(mtf) == 0) {
                if (value("skipempty") != NULL)
                        goto out;
                if (hp->h_subject == NULL || *hp->h_subject == '\0')
                        puts("No message, no subject; hope that's ok");
                else
                        puts("Null message body; hope that's ok");
        }

Which, finally, and as an off-topic note, makes me think the root of the related issue 0001368 was caused by experiences with SysV based mail.
(0005162)
rhansen   
2020-12-10 17:23   
On page 2943 line 97431 (mailx SYNOPSIS) change:
<tt>mailx [−s subject] address...</tt>
to:
<tt>mailx [-E] [−s subject] address...</tt>

On page 2944, insert after line 97471:
-E Discard messages with an empty message body.

On page 2964 line 98259 (mailx EXIT STATUS) change:
Successful completion; note that this status implies that all messages were sent, ...
to:
Successful completion; note that in Send Mode this status implies that all messages were either sent or successfully discarded (see -E), ...
(0005182)
geoffclare   
2020-12-16 16:41   
When applying this bug I also made an editorial change to page 2944 line 97467 from:
(Only the -s subject option shall be required on all systems...
to:
(Only the -E and -s subject options are required on all systems...




Viewing Issue Advanced Details
1366 [Issue 8 drafts] Rationale Comment Error 2020-07-08 15:20 2020-09-30 14:24
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
A.3 Definitions (XSI)
3379
115320
Out of date rationale for XSI definition
The rationale for the XSI definition says:
The term ``XSI'' has been used for 10 years in connection with the XPG series and the first and second versions of the base volumes of the Single UNIX Specification. The XSI margin code was introduced to denote the extended or more restrictive semantics beyond POSIX that are applicable to UNIX systems.
The use of "has" was okay when the text was written (for Issue 6), but is now a problem.
Change:
The term ``XSI'' has been used for 10 years ...
to:
When POSIX.1 and the Single UNIX Specification were merged, the term ``XSI'' had been used for over 10 years ...
There are no notes attached to this issue.




Viewing Issue Advanced Details
1365 [Issue 8 drafts] Shell and Utilities Objection Error 2020-07-07 09:36 2020-09-30 14:22
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
rm
3066
103824
rm -v description implies only operands are reported
The changes from bug 0001154 to add rm -v were supposed to require that rm reports each file that it removes. However, the description of -v is:
After file has been removed, write a message to standard output indicating that it has been removed.
which implies that rm -rv only reports the removal of file operands and does not report the removal of files within a directory specified as an operand.

The changes at line 103806 correctly state that each removed file is reported.

Line 103858 in STDOUT also has two minor problems:
  • Use of "the file" is odd, as if rm only ever removes one file.
  • The phrase "being removed" implies that something could be written before the removal attempt is made, rather than (as per the other changes) only after successful removal.

On page 3066 line 103824 section rm, change:
After file has been removed
to:
After each file has been removed

On page 3067 line 103858 section rm, change:
information about the file being removed
to:
information about each removed file
There are no notes attached to this issue.




Viewing Issue Advanced Details
1364 [Issue 8 drafts] Shell and Utilities Comment Clarification Requested 2020-07-06 10:45 2020-09-30 14:20
geoffclare
 
normal  
Applied  
Accepted As Marked  
   
Geoff Clare
The Open Group
2.7.2 Redirecting Output
2287
73909
Note: 0004991
use of noclobber with files in /tmp
Some time after bug 0001016 was approved as an interpretation, Note: 0003655 was added in which McDutchie observed that the statement "Performing these operations atomically ensures that the creation of lock files and unique (often temporary) files is reliable" is not true if the file is in a public directory such as /tmp, since another user could create a FIFO or device file with the same name.
On page 2287 line 73900 section 2.7.2, and
page 3587 line 124071 section C.2.7.2, add to the "Notes to Reviewers":
If option 2 is adopted in a future draft, note that the change from <URL> will need to be reapplied to the option 2 text.

where <URL> is the URL for this bug.

On page 2287 line 73909 section 2.7.2, change:
Performing these operations atomically ensures that the creation of lock files and unique (often temporary) files is reliable.
to:
Performing these operations atomically ensures that the creation of lock files and unique (often temporary) files is reliable, provided the files are created in a private directory (i.e. not in /tmp or similar).

On page 3588 line 124127 section C.2.7.2, change:
The standard developers consider this to be of less importance than ensuring that the creation of lock files is reliable.
to:
The standard developers consider this to be of less importance than ensuring that the creation of lock files is reliable (in a private directory).

Creation of lock files and unique (often temporary) files with noclobber set is only reliable provided the files are created in a private directory. If a directory such as /tmp is used where other users can create files, then another user could create a FIFO (or a device file, given sufficient privilege) with the same name, causing multiple redirections with noclobber set to open the existing non-regular file instead of one succeeding and the others failing.

Notes
(0004991)
rhansen   
2020-09-14 15:35   
(edited on: 2020-09-14 15:36)
On page 2287 line 73900 section 2.7.2, and
page 3587 line 124071 section C.2.7.2, add to the "Notes to Reviewers":
If option 2 is adopted in a future draft, note that the change from https://www.austingroupbugs.net/view.php?id=1364 [^] will need to be reapplied to the option 2 text.

On page 2287 line 73909 section 2.7.2, change:
Performing these operations atomically ensures that the creation of lock files and unique (often temporary) files is reliable.
to:
Performing these operations atomically ensures that the creation of lock files and unique (often temporary) files is reliable, with important caveats detailed in [xref to XRAT C.2.7.2].

On page 3588 line 124127 section C.2.7.2, change:
The standard developers consider this to be of less importance than ensuring that the creation of lock files is reliable.
to:
The standard developers consider reliable lock creation to be more important than the creation of symbolic link targets.

Creation of lock files and unique (often temporary) files with noclobber set is only reliable provided neither non-regular files nor symbolic links to non-regular files exist or are created in the same directory with the same names, and no other processes delete the files while still in use. If a directory such as /tmp is used for lock files, then another process could accidentally or maliciously create a FIFO (or a special file, given sufficient privilege) with the same name, causing multiple processes to simultaneously open the same lock file instead of one succeeding and the others failing.






Viewing Issue Advanced Details
1363 [Issue 8 drafts] System Interfaces Comment Enhancement Request 2020-07-03 13:44 2020-09-17 09:18
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
wait()
2154
69578
out of date wait() RATIONALE regarding core dump indication
The text:
Implementations that support implementation-defined actions, such as the creation of a file containing a core image, on termination of some processes traditionally provide a bit in the status returned by wait() to indicate that such actions have occurred.

is out of date because the way to query this using WCOREDUMP() has now been added to the standard.
Change:
Implementations that support implementation-defined actions, such as the creation of a file containing a core image, on termination of some processes traditionally provide a bit in the status returned by wait() to indicate that such actions have occurred.
to:
On implementations that support the creation of a file containing a core image on some process terminations, the WCOREDUMP(stat_val) macro indicates whether creation of a core image was attempted. If it returns a non-zero value this does not necessarily mean that the core image was created, only that it was attempted. For example, if the RLIMIT_CORE limit for the process is 0, this prevents creation of the file; WCOREDUMP(stat_val) returning non-zero in this case indicates that the file would have been created if the limit had not been 0.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1362 [Issue 8 drafts] System Interfaces Editorial Error 2020-07-03 09:35 2020-09-17 09:16
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
posix_spawn()
1428
47825-47851
small editorial fixes to posix_spawn()
In the ERRORS section there are several occurrences of:
(or, if the error occurs after the calling process successfully returns, the child process shall exit with exit status 127)

This is intended to state a requirement (hence the use of "shall") and therefore should not be in parentheses. Only one occurrence is new in Issue 8 draft 1, but the others need to be fixed as well.

Another small editorial fix that doesn't seem worth submitting a separate bug for is a use of the phrase "relative file names".
On page 1428 line 47825-47851 section posix_spawn(), change 6 occurrences of:
... (or, if the error occurs after the calling process successfully returns, the child process shall exit with exit status 127).
to:
...; or, if the error occurs after the calling process successfully returns, the child process shall exit with exit status 127.

On page 1426 line 47733 section posix_spawn(), change:
relative file names
to:
relative pathnames
There are no notes attached to this issue.




Viewing Issue Advanced Details
1361 [Issue 8 drafts] System Interfaces Objection Error 2020-07-02 13:52 2020-09-17 09:14
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
fork()
877
29947
fork() changes incomplete
The changes from bug 0000062 to add _Fork() and make fork() non-async-signal-safe missed some things on the fork() page:

The text "When the application calls fork() from a signal handler and any of the fork handlers registered by pthread_atfork() calls a function that is not async-signal-safe, the behavior is undefined" in one of the later bullet items is redundant now that fork() itself is not async-signal-safe.

There is a paragraph in RATIONALE explaining why the above text is there.
On page 877 line 29947 section fork(), delete:
When the application calls fork() from a signal handler and any of the fork handlers registered by pthread_atfork() calls a function that is not async-signal-safe, the behavior is undefined.

On page 879 line 30059 section fork(), delete:
While the fork() function is async-signal-safe, there is no way for an implementation to determine whether the fork handlers established by pthread_atfork() are async-signal-safe. The fork handlers may attempt to execute portions of the implementation that are not async-signal- safe, such as those that are protected by mutexes, leading to a deadlock condition. It is therefore undefined for the fork handlers to execute functions that are not async-signal-safe when fork() is called from a signal handler.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1360 [Issue 8 drafts] System Interfaces Objection Clarification Requested 2020-07-01 15:59 2020-09-17 09:12
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
fdopendir()
816, 818
27933, 28030
opendir() file descriptor requirements
There are three issues with the description of opendir() related to the fd associated with a directory stream.

1. The phrase "If the type DIR is implemented using a file descriptor" is problematic, given that dirfd() no longer has an ENOTSUP error. I.e. For Issue 8, DIR always has the ability to store a fd; the optionality now is not if but when: the fd can be opened either by opendir() or by dirfd(). The suggested changes refer to the fd as "associated with" the stream, as that is the phrase used on the dirfd() page.

2. It describes opening with O_DIRECTORY and then in a separate paragraph says that FD_CLOEXEC is set. This implies that setting FD_CLOEXEC can be done separately, whereas to avoid race conditions the open should be done as if with O_CLOEXEC. (As is already required for dirfd().)

3. The wording "applications shall only be able to open up to a total of {OPEN_MAX} files and directories" has the same problem that all the EMFILE descriptions used to have. (They were reworded to use "file descriptors available to the process".)

Issue 1 also affects closedir().

In addition to these issues in the normative text, there are problems in RATIONALE too.
On page 816 line 27933 section fdopendir(), change:
If the type DIR is implemented using a file descriptor, applications shall only be able to open up to a total of {OPEN_MAX} files and directories.

If the type DIR is implemented using a file descriptor, the descriptor shall be obtained as if the O_DIRECTORY flag was passed to open().

If the type DIR is implemented using a file descriptor and a directory stream is opened by a successful call to opendir(), the FD_CLOEXEC flag shall be set on the file descriptor; see [xref to <fcntl.h>].
to:
If opendir() opens a file descriptor for dirname to associate with the returned stream:
  • The descriptor shall be allocated as if the O_DIRECTORY and O_CLOEXEC flags were passed to open().

  • The descriptor shall be subject to the limit of {OPEN_MAX} file descriptors available to the process.

On page 818 line 28030 section fdopendir(), change:
Based on historical implementations, the rules about file descriptors apply to directory streams as well. However, this volume of POSIX.1-202x does not mandate that the directory stream be implemented using file descriptors. The description of closedir() clarifies that if a file descriptor is used for the directory stream, it is mandatory that closedir() deallocate the file descriptor. When a file descriptor is used to implement the directory stream, opendir() behaves as if the FD_CLOEXEC flag had been set for the file descriptor.
to:
Based on historical implementations, the rules about file descriptors apply to directory streams as well. However, this volume of POSIX.1-202x does not mandate that opendir() opens a file descriptor to associate with the stream; this may instead be done by the first call to dirfd(), thus avoiding the need to allocate a file descriptor if dirfd() is never called. Once a file descriptor has been associated with the stream, it is mandatory that closedir() deallocate the file descriptor. If opendir() opens a file descriptor to associate with the stream, it behaves as if the O_CLOEXEC flag for open() had been used, so that the FD_CLOEXEC flag is set for the file descriptor.

On page 819 line 28079 section fdopendir(), change:
if the type DIR is implemented using a file descriptor
to:
if it associates a file descriptor with the returned stream.

On page 667 line 23091 section closedir(), change:
If a file descriptor is used to implement type DIR, that file descriptor shall be closed.
to:
If there is a file descriptor associated with the stream (whether opened by opendir() or dirfd(), or passed to fdopendir() when creating the stream), that file descriptor shall be closed by closedir().

There are no notes attached to this issue.




Viewing Issue Advanced Details
1359 [Issue 8 drafts] System Interfaces Comment Error 2020-06-30 15:14 2020-09-16 15:16
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
dirfd()
712
24548
dirfd() rationale out of date
Bug 0000391 made changes to the normative text for dirfd() but did not update the RATIONALE to match.
Change:
The description uses the term ``a file descriptor'' rather than ``the file descriptor''. The implication intended is that an implementation that does not use an fd for opendir() could still open() the directory to implement the dirfd() function. Such a descriptor must be closed later during a call to closedir().

If it is necessary to allocate an fd to be returned by dirfd(), it should be done at the time of a call to opendir().
to:
On an implementation where reading from a directory stream does not use a file descriptor, opendir() need not allocate one to be returned by dirfd(). The implementation can instead delay the allocation of a suitable file descriptor until the first time dirfd() is called for the stream. A file descriptor allocated by dirfd() must be closed by closedir().

There are no notes attached to this issue.




Viewing Issue Advanced Details
1358 [Issue 8 drafts] System Interfaces Objection Error 2020-06-30 14:42 2020-09-16 15:14
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
basename()
593
20765
basename/dirname difference for thread termination
The description of basename() includes the sentence "The returned pointer might also be invalidated if the calling thread is terminated", but dirname() does not. Bug 0001064 removed it from dirname(), but didn't do so for basename(). I believe this was accidental, resulting from the relevant paragraph being in different places on the two pages (RETURN VALUE v. DESCRIPTION).
Delete:
The returned pointer might also be invalidated if the calling thread is terminated.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1357 [Issue 8 drafts] Base Definitions and Headers Objection Omission 2020-06-30 14:16 2020-09-16 15:13
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
<unistd.h>
429
14938
SEEK_* distinct values requirement
Bug 0000415 added SEEK_HOLE and SEEK_DATA with a requirement that they have distinct values, but they should be required to be distinct from each other and from SEEK_CUR, SEEK_END, and SEEK_SET.
Change:
with distinct values
to:
with values that are distinct from each other and from SEEK_CUR, SEEK_END, and SEEK_SET

Notes
(0004892)
shware_systems   
2020-06-30 14:59   
The 'with distinct values' I see as superfluous there, actually. No harm, just not really required for clarity. It belongs imo more with the paragraph where <unistd.h> shall declare them, as simply 'mutually exclusive values', possibly '... non-zero values'.




Viewing Issue Advanced Details
1356 [Issue 8 drafts] Base Definitions and Headers Editorial Enhancement Request 2020-06-29 12:47 2020-09-16 15:11
dannyniu
 
normal  
Applied  
Accepted As Marked  
   
DannyNiu/NJF
3.57 Character
37
1241
Note: 0004985
Our definition of character disagrees with that of Unicode.
In Draft 1, a character is being defined as:

> A sequence of one or more bytes representing
> a single graphic symbol or control code.

This definition falls apart when applied in the context of e.g. Arabic text.

The Unicode standard (version 13, page 15) says:

> The Unicode Standard draws a distinction
> between characters and glyphs. Characters are
> the abstract representations of the
> smallest components of written language
> that have semantic value.

Considering Unicode has a radically different goal than POSIX,
I propose the following new definition for consideration
(as the current wording may be intended, even not on purpose)

> A sequence of bytes that is considered an
> individual unit in text processing.

Explanation:

1) A sequence of bytes: remains the same as our original definition.

2) individual unit: graphic symbols in Arabic text are composed of parts that're sometimes stacked on top of each other. Older iOS that didn't take this into account caused the iPhone Arabic Glitch.

3) text processing: this can refer to terminal (and emulators) processing control sequences, and `wc -m` counting characters. Defining characters in terms of text processing lifts the burdon of relying on the concept of "code point" externally defined in the Unicode standard.

Consider applying the the proposed new definition.
Notes
(0004985)
geoffclare   
2020-09-10 15:30   
On page 37 line 1241 (XBD 3.57 Character), change:
A sequence of one or more bytes representing a single graphic symbol or control code.
to:
A sequence of one or more bytes representing a member of a character set.

On page 3357 line 114412 section A.3 (Character), change:
The term ``character’’ is used to mean a sequence of one or more bytes representing a single graphic symbol.
to:
The term ``character’’ is used to mean a sequence of one or more bytes representing a member of a character set.




Viewing Issue Advanced Details
1355 [Issue 8 drafts] Base Definitions and Headers Editorial Enhancement Request 2020-06-29 11:46 2020-09-16 15:08
dannyniu
 
normal  
Applied  
Accepted As Marked  
   
DannyNiu/NJF
2.1.5.2 XSI Option Groups
20
703
Note: 0004970
Better wording to replace "decoding algorithm"
In the Encryption Option Group, there's the text:

> Due to export restrictions on the decoding algorithm in some countries

The word "decoding" is inappropriate in that:

1. decoding is (strictly speaking) an unkeyed process (e.g. base64)

2. even if there's a key, encryption isn't necessarily the only functionality that may be restricted by the customs.

Some wording that I believe may be better include: "cipher algorithm" which specifically refers to encryption functionalities, or "cryptographic algorithm" which is completely generic.
Change "decoding" to "cipher" or "cryptographic".
Notes
(0004970)
rhansen   
2020-09-03 15:59   
On page 20 line 703 replace:
Due to export restrictions on the decoding algorithm in some countries,
with:
Due to export restrictions on the cryptographic algorithm in some countries,




Viewing Issue Advanced Details
1354 [Issue 8 drafts] System Interfaces Editorial Error 2020-06-28 15:12 2020-09-16 10:07
dennisw
 
normal  
Applied  
Accepted  
   
Dennis Wölfing
strftime
1966
63929-63930
Bug 1313 applied incorrectly
Bug 0001313 was applied incorrectly. It seems like the Desired Action was applied instead of the accepted text for that bug. Also an opening parenthesis is missing.
On page 1966 lines 63929-63930 change
±YYYYY-MM-DD)
to
YYYYY-MM-DD, i.e. with a 5 or more digit year)
Notes
(0004890)
geoffclare   
2020-06-29 09:46   
The format string was correctly updated (and the desired action here does not change it), but the missing opening parenthesis should be added.
(0004891)
geoffclare   
2020-06-29 09:48   
Ignore previous note. I see now that it was not talking about the format string, but the omission of the text "i.e. with a 5 or more digit year".




Viewing Issue Advanced Details
1352 [Issue 8 drafts] Base Definitions and Headers Comment Error 2020-06-28 15:06 2020-09-16 10:04
dennisw
 
normal  
Applied  
Accepted  
   
Dennis Wölfing
wchar.h
445
15632-15643
Outdated text in <wchar.h>
The <wchar.h> APPLICATION USAGE and RATIONALE sections contain text about <wctype.h> functions being declared in <wchar.h> for backwards compatibility. However these obsolescent declarations have been removed in draft 1, so this text is no longer needed.
On page 445 change lines 15632-15643 to:
APPLICATION USAGE
None.
RATIONALE
None.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1351 [Issue 8 drafts] Base Definitions and Headers Objection Omission 2020-06-28 15:04 2020-09-16 10:02
dennisw
 
normal  
Applied  
Accepted  
   
Dennis Wölfing
fcntl.h
225
7834
F_DUPFD_CLOFORK missing in <fcntl.h>
Bug 0001318 added F_DUPFD_CLOFORK to the fcntl() function. However the bug was missing a change to add the definition of F_DUPFD_CLOFORK in <fcntl.h>.
On page 225 after line 7834 add:
F_DUPFD_CLOFORK
Duplicate file descriptor with the close-on-fork flag FD_CLOFORK set.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1350 [Issue 8 drafts] Base Definitions and Headers Objection Error 2020-06-26 15:57 2020-09-16 10:15
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
<stdlib.h>
351
12251
O_* constants needed in <stdlib.h> for mkostemp()
This line:
The <stdlib.h> header shall define O_CLOEXEC, O_NOCTTY, and O_RDWR as described in <fcntl.h>.
was added by bug 0000593 with XSI shading because posix_openpt() is XSI.

However, O_CLOEXEC is also needed by mkostemp() which is mandatory. Several other constants are also needed by mkostemp().
On page 351 line 12251 section <stdlib.h>, change:
[XSI]The <stdlib.h> header shall define O_CLOEXEC, O_NOCTTY, and O_RDWR as described in <fcntl.h>.[/XSI]
to:
The <stdlib.h> header shall define the following symbolic constants as described in <fcntl.h>:

O_APPEND
O_CLOEXEC
O_CLOFORK
[SIO]O_DSYNC[/SIO]
[XSI]O_NOCTTY
O_RDWR[/XSI]
[SIO]O_RSYNC[/SIO]
O_SYNC

On page 1283 line 43031 section mkdtemp(), change:
contain additional flags (from <fcntl.h>) to be used
to:
contain additional flags to be used
Notes
(0004889)
dennisw   
2020-06-28 15:35   
0000598 reported the same issue already but was not resolved yet. As reported there dup3() from <unistd.h> also has this issue (it needs O_CLOEXEC and O_CLOFORK).




Viewing Issue Advanced Details
1349 [Issue 8 drafts] Base Definitions and Headers Editorial Enhancement Request 2020-06-26 15:35 2020-09-16 09:58
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
1.1 Scope
4
72
Where to obtain ISO/IEC standards (footnote)
The footnote here says:
ISO/IEC documents can be obtained from the ISO office: ...
and gives a street address in Geneva.

It would make more sense to give a URL for their online store.


Change the line to:
ISO/IEC documents can be obtained from https://www.iso.org/store.html [^] .
(with the URL as a clickable link).
There are no notes attached to this issue.




Viewing Issue Advanced Details
1348 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Error 2020-06-25 08:54 2020-12-16 16:02
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
test
3288
110615
---
"the exec() family of functions" should not include "()"
The phrase "the exec() family of functions" does not fit the normal
convention of omitting the "()" when referring to a family of functions.
Change:
the exec() family of functions
to:
the exec family of functions
There are no notes attached to this issue.




Viewing Issue Advanced Details
1347 [1003.1(2016/18)/Issue7+TC2] System Interfaces Editorial Clarification Requested 2020-05-28 19:50 2020-12-16 16:00
dalias
 
normal  
Applied  
Accepted As Marked  
   
Rich Felker
musl libc
stdin
2017
64733
---
Note: 0005161
stderr access mode - "is expected to be" is not defined
The specification of the standard FILE streams includes the (CX shaded) text:

"The stderr stream is expected to be open for reading and writing."

As far as I can tell, the wording "is expected to be" is not defined anywhere in the standard, and is unclear as to which party may use the expectation and which is responsible for ensuring that it is satisfied.

In addition, it's unclear whether the intent was that the expectation apply to the underlying open file description behind fd 2, or the stderr stdio FILE stream. The latter does not seem useful since dual-mode streams are only usable on seekable files (there's no way to switch from reading to writing without a successful seek) but the use of the word "stream" in the above-quoted suggests it should be read that way. Moreover, even if the stream is open for reading and writing, that does not automatically imply that the file descriptor is.
Clarify which party (implementation, invoking application, or something else) the obligation is on and which party may have the expectation.

Clarify what the consequences are if the expectation is not satisfied.

Clarify whether the statement applies to the FILE stream, the underlying open file description's mode, or both.
Notes
(0004880)
geoffclare   
2020-05-29 10:53   
(edited on: 2020-05-29 12:16)
After a bit of digging (with help from Andrew J) it appears that this wording arose as a result of ERN 40 against XSH6 draft 1, which can be seen here:

http://www.opengroup.org/austin/docs/austin_34r1.txt [^]

and says:
 Problem:
 How is stderr opened: input only (not likely), output only (of course)
 or both (there's the rub)?

 Action:
 My preference: "stderr is opened for input and output, although it is
 expected that under normal circumstances it will not be used for input".
 I'm OK with "stderr is opened for output.  Implementations may open it
 for input as well, but a conforming application should not expect that."

It was submitted by someone called Donn with an Interix email address, which I assume is Donn Terry. If Donn reads this perhaps he can remember what led him to raise this.

My initial guess was that it is is related to the more utility reading commands from "standard error", but since more doesn't have to be implemented in C (and anyway could use read() rather than stdio) there would have been no need to raise the matter in connection with C's stderr but only for file descriptor 2 (as inherited by the login shell).

(0004881)
dalias   
2020-05-29 20:26   
FYI this issue was opened as a result of discussion on a question posted on Stack Overflow, https://stackoverflow.com/questions/62052909 [^] where it was found that at least glibc, FreeBSD, and OpenBSD all have the stderr FILE stream in write-only mode. So it seems that an interpretation applying the text in question as a requirement on the implementation's definition of stderr would be contrary to fairly widespread existing practice.
(0005161)
rhansen   
2020-12-10 16:53   
(edited on: 2020-12-10 17:02)
On page 65 before line 1912 (XBD "interactive shell") insert a new definition and renumber the remaining:
Interactive Device

A terminal device.

Note: This definition is intended to align with the ISO C standard's use of "interactive device".

On page 496 lines 17224-17228 (XSH 2.5 Standard I/O Streams) and page 2017 lines 64723-64727 (XSH stderr, stdin, stdout DESCRIPTION) change:
At program start-up, three streams shall be predefined and need not be opened explicitly: standard input (for reading conventional input), standard output (for writing conventional output), and standard error (for writing diagnostic output). When opened, the standard error stream is not fully buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device.
to:
At program start-up, three streams shall be predefined and already open: stdin (standard input, for conventional input) for reading, stdout (standard output, for conventional output) for writing, and stderr (standard error, for diagnostic output) for writing. When opened, stderr shall not be fully buffered; stdin and stdout shall be fully buffered if and only if [CX]the file descriptor associated with the stream is determined not to be associated with an interactive device.[/CX]

On page 2017 line 64733 (XSH stdin DESCRIPTION), change:
The stderr stream is expected to be open for reading and writing.
to:
These file descriptors are often all associated with a single open file description which has access mode O_RDWR (e.g., in the case of a terminal device for a login shell). However, the stderr, stdin, and stdout streams need not be opened for both reading and writing at program start-up in this case.

On page 2017 line 64747 (stdin SEE ALSO) add isatty().





Viewing Issue Advanced Details
1346 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Comment Enhancement Request 2020-05-26 13:44 2020-12-16 16:33
markh
 
normal  
Applied  
Accepted As Marked  
   
Mark Harris
1.7.1
8
236
---
See Note: 0005157.
Require support for CLOCK_MONOTONIC
CLOCK_MONOTONIC was introduced in Issue 6, but only as an optional feature. However many applications require the use of a clock that cannot have negative time jumps.

A monotonic clock is not an obscure feature, and is widely available via non-POSIX interfaces. For example, C++ applications (since C++11) can rely on std::chrono::steady_clock, Java applications (since Java 7) can rely on System.nanoTime(), Python applications (since Python 3.5) can rely on time.monotonic(), macOS and iOS applications can rely on mach_absolute_time(), and Windows applications (since Windows 2000) can rely on QueryPerformanceCounter().

Many applications targeting POSIX just assume that CLOCK_MONOTONIC is available, but others that are trying to be more portable use checks that are difficult to get right and test, and may still fail on a system conforming to the latest revision of POSIX if CLOCK_MONOTONIC works with some interfaces (like clock_gettime()) but not others (like clock_nanosleep()). The addition of new interfaces such as pthread_cond_clockwait() (#1216) will make correct checking even more complex for applications that wish to use the new interfaces, but also support the possibility allowed by the standard that they may not be usable with CLOCK_MONOTONIC. A properly functioning monotonic clock is important and applications relying on Issue 8 should not be burdened with additional checks or fallbacks; they should be able to rely on a monotonic clock being available, as C++ applications, Java applications, Python applications, and macOS, iOS, and Windows applications have been able to do for years.
For Issue 8, require support for the Monotonic Clock option, and require that CLOCK_MONOTONIC be supported by all standard interfaces that accept a clock id except clock_settime().
Notes
(0005157)
Don Cragun   
2020-12-03 17:21   
(edited on: 2020-12-03 17:33)
All page and line numbers are against Issue 7 2018 edition (C181).

Delete page 8 lines 235-240 (XBD 1.7.1 Codes, MON) and remove the MON shading throughout:
  • page 426 lines 14470-14474 (XBD <time.h>)
  • page 511 lines 17861-17865 (XSH 2.8.5 Clocks and Timers)
  • page 512 line 17872 (XSH 2.8.5 Clocks and Timers)
  • page 590 lines 20564-20565 (XSH aio_suspend() description)
  • page 680 line 23320 (XSH clock_getres() errors)


On page 18 line 586 (XBD 2.1.3 POSIX Conformance), move _POSIX_MONOTONIC_CLOCK to a new list of symbolic constants for Issue 8 under page 17 line 552.

Delete page 23 line 818 and page 24 line 826.

On page 275 line 9235 (<limits.h>) change:
If the Monotonic Clock option is supported, the ...
to:
The ...

and remove the [MON] shading.

On page 436 lines 14851 - 14854, change
_POSIX_MONOTONIC_CLOCK
The implementation supports the Monotonic Clock option. If this symbol is defined in <unistd.h>, it shall be defined to be −1, 0, or 200809L. The value of this symbol reported by sysconf( ) shall either be −1 or 200809L.
to
_POSIX_MONOTONIC_CLOCK
The implementation supports a monotonic clock. This symbol shall always be set to the value 20yymmL.

and remove the [MON] shading.

On page 679 line 23272 (XSH clock_getres() description), change:
If the Monotonic Clock option is supported, all ...
to:
All ...


and remove the [MON] shading.

On page 1986 lines 63922-63923 (XSH sigtimedwait() ) change:
If the Monotonic Clock option is supported, the ...
to:
The ...


and remove the [MON] shading.

On page 2151 lines 68909-68910 change:
All implementations shall support a clock_id of CLOCK_REALTIME. If the Monotonic Clock option is supported, implementations shall support a clock_id of CLOCK_MONOTONIC.
to:
All implementations shall support CLOCK_REALTIME and CLOCK_MONOTONIC as values for clock_id.


and remove the [MON] shading.

On page 3485 line 117753 section A.2.1.3 change:
The following options from the previous version of this standard are now mandatory
to:
The following options from previous versions of this standard are now mandatory


and add _POSIX_MONOTONIC_CLOCK to the list following that text.

On page 3779 line 129738 section D.3.4, change:
The system supports the Monotonic Clock option.


This option ...
to:
Support for a monotonic clock is mandatory in POSIX.1-202x.


This facility ...






Viewing Issue Advanced Details
1345 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2020-05-24 22:48 2020-12-16 15:27
j_willie
 
normal  
Applied  
Accepted As Marked  
   
J William Piggott
date
163, 2634-2638
5224-5228, 85673-85823
---
Note: 0004893
date(1) default format
PREFACE
For simplicity I will state XPG3 positions as fact, although I do not actually
have access to the XPG3 standard. The XPG3 references are not imperative to
addressing this issue, they merely give plausible explanation for the current
situation. Anyone with access to the XPG3 standard is welcome to confirm or
deny. Finally, I'm new here and will likely do something wrong; I plead for
patience and guidance, please.

Terms:
 default format: when calling date(1) without arguments.

There are two closely related issues for the date(1) default format:

a) the default format is ambiguous:
   Stated: shall be the locale specific 'date and time'.
   Not Stated: what locale element represents it.

   Line 85674:
    By default, the current date and time shall be written.
   Line 85805:
    ... the [default format] output in the POSIX locale ...
   Lines 85864-85892
    EXAMPLES [also illustrate locale specific default formats]

b) the STDOUT POSIX locale format string is unreachable
   Line 85807 date "+%a %b %e %H:%M:%S %Z %Y"

PLAUSIBLE EXPLANATION
_______________________________________
XPG3

XPG3 specified the date(1) default output format as %C [uppercase 'C'].
     It was bound to the date_fmt locale element:

date(1)
%C locale’s date and time representation as produced by date(1)

STDOUT
 When no formatting operand is specified, the output in the POSIX locale shall
 be equivalent to specifying: date "+%a %b %e %H:%M:%S %Z %Y"

POSIX Locale
# date and time representation as produced by date(1) (%C)
# "%a %b %e %H:%M:%S %Z %Y"
date_fmt " ...

NOTE: the format in date(1) STDOUT was reachable via LC_TIME date_fmt; %C
_______________
XPG4

In XPG4 %C was redefined to Century number and date_fmt was dropped from the
POSIX locale. However, the change seemed to overlook establishing a new date(1)
default format specifier and did not fix the dangling reference to the deleted
date_fmt in the date(1) STDOUT section.
_______________________________________
IMPACT
The required default date(1) POSIX locale format being unavailable has lead to
implementation creativity. Glibc and coreutils have partially reimplemented the
XPG3 date_fmt as the default date(1) format without binding it to a format
specifier. There have been several posts to the glibc mailing list asking what
are the expected values for d_t_fmt and date_fmt. Nobody knows. I think the
reason date_fmt was reintroduced is to resolve the unreachable format string
the standard requires in the date(1) STDOUT section.
_______________________________________
SOLUTION

a) The de facto date(1) default format must be %c (LC_TIME d_t_fmt).
   It is the only locale keyword that satisfies the required 'date and time'.
   The several date(1) EXAMPLES seem to support this position.

b) date(1) STDOUT should not be restating the POSIX locale. It should
   simply state %c, and let the POSIX locale speak for itself.

   That leaves the conflict between the current format string listed in STDOUT
   and the value for the POSIX locale d_t_fmt. I propose, that because common
   date(1) implementations include %Z in the default output, (as the current
   standard requires); it should be added to the POSIX locale's LC_TIME d_t_fmt
   value.
XBD Page 163
Section 7.3.5.3 LC_TIME Category in the POSIX Locale
Line 5224 replace with:
# "%a %b %e %H:%M:%S %Z %Y"

Line 5228 replace with:
<space><percent-sign><Z><space><percent-sign><Y>"

XCU Page 2634
Section date DESCRIPTION
Lines 85673-85676 replace with:
The date utility shall write the date and time to standard output <XSI>or
attempt to set the system date and time.</XSI> By default, the current date and
time shall be written in the locale dependent %c format. If an operand
beginning with '+' is specified, the output format of date shall be controlled
by the conversion specifications and other text in the operand.

XCU Page 2637
Section date STDOUT
Lines 85805-85807 replace with:
The default standard output shall be in the locale dependent %c format(see
[xref to XSH 3 strftime()] and [xref to XBD 7.3.5.3 LC_TIME Category in the
POSIX Locale]).

XCU Page 2638
Section date APPLICATION USAGE
Lines 85821-85823 replaced with:
The default standard output may be difficult to parse, as it is locale
specific. It may contain <newline> characters and be vastly different from what
is defined for the POSIX locale.

Notes
(0004886)
shware_systems   
2020-06-08 14:49   
(edited on: 2020-06-08 16:06)
On systems that do not support the POSIX2_LOCALEDEF option, it is platform specific what the default formats are for any locales in addition to POSIX the platform makes available. As unspecified platform elections not modifiable by a user there would be no documentation requirement either. I suspect date_fmt was rejected because it constrained how a date utility for such a fixed set of locales might be written. Some implementations might just compile a table of format strings directly into the utility, for example, and not make any references to data setlocale() makes available to a process.

To keep this flexibiity when POSIX2_LOCALEDEF is supported means a platform has to document a) whether date only supports locales provided by the platform, and if so which format is used when a locale created by localedef is referenced; and if not b) the implementation-defined means of mapping a locale name that represents the output of localedef to the platform specific means of providing additional format strings. A platform may elect to extend localedef to accomplish this, others may leverage catget() or use a simple text file format.

Because of these variations I think the Desired Action is too limiting. Adding an LD sensitive documentation requirement is more backwards compatible, I feel.

(0004893)
geoffclare   
2020-07-02 10:57   
(edited on: 2020-07-02 12:02)
The POSIX locale default date utility format:
%a %b %e %H:%M:%S %Z %Y
contains two pieces of information beyond the minimum of date and time required for other locales: the day name (%a) and the time zone (%Z). Most implementations include these in the default date output for their implementation-provided locales.

Since many users would expect to see day name and time zone information in the default date output (particularly if they are used to the traditional behaviour that was standardised for the POSIX locale), as a minimum we should add something to APPLICATION USAGE about this. We should also consider recommending that implementations include them (either via "should" in normative text or a statement in RATIONALE). Here is a suggested set of changes that does the latter...

On page 2638 line 85823 section date, add a paragraph to APPLICATION USAGE:
Since the default date utility format for locales other than the POSIX or C locale is not required to include anything beyond the date and time, whereas for the POSIX or C locale it also includes the day name and time zone, it may be necessary to specify a format (or override the locale-selection environment variables) to ensure this information is included when desired.

On page 2640 line 85914 section date, add these paragraphs to RATIONALE:
Although this standard only requires the default date utility format, for locales other than the POSIX or C locale, to include the date and time, it is common for implementations to include day name and time zone information as well. (For the POSIX locale this is required, with the day name in %a format at the beginning and the time zone in %Z format before the year.) Implementations are encouraged to include the day name (in %a or %A format) and the time zone (in %Z or %z format) in the default date utility format for all of the locales they provide.

Some implementations have a date_fmt locale keyword (see [xref to XBD 7.3.5]) as an extension, to specify the default date utility format for each locale. On such implementations, if the localedef utility is used to create a locale that does not have this information, the date utility must by default still produce output for that locale that includes both the time and the date.


(0004894)
geoffclare   
2020-07-02 11:02   
For the record, most of the historical information given in the Description of this bug is factually incorrect. See the mailing list for details.




Viewing Issue Advanced Details
1344 [1003.1(2008)/Issue 7] System Interfaces Editorial Enhancement Request 2020-05-20 10:34 2020-05-21 13:09
mkerrisk
ajosey  
normal  
Under Review  
Open  
   
Michael Kerrisk
man7.org
XSH
n/a
n/a
---
Addition of setresuid()/setresgid()/getresuid()/getresgid()
setresuid()/setresgid()/getresuid()/getresgid() are implemented on a number systems including at least Linux, FreeBSD, OpenBSD, and HP-UX. (Notably, they are not present on Solaris, so far as I know.)

Adding these interfaces to POSIX would be valuable for a number of reasons:

* The semantics of the existing APIs for modifying credentials are
problematic. The semantics of setuid()/setgid() depend on whether the
process is privileged, so that the API either changes just the
effective ID, or all of real/effective and saved set IDs. The only
POSIX-specified way to change saved set IDs is to use
setreuid()/setregid(). But those APIs depend on a bizarre rule to
determine whether or not the saved set ID is modified. These sorts of
funny behaviors are invitations for programmers to make mistakes, and
in this case such mistakes have obvious security implications.

* By contrast with the former point, the semantics of the changes made
by setresuid() and setresgid() are simple and transparent: one
argument per credential, with "-1" being used to signify "no change".
No semantics that vary according to whether the process is privileged
and no funny rules.

* getresuid()/getresgid() provide the only means of explicitly
retrieving the save set-UID/GID.
1. Add specifications of setresuid()/setresgid()/getresuid()/getresgid()
2. Add SEE ALSO entries in relevant other pages (getuid(), geteuid(), setuid(), setreuid(), getgid(), getegid(), setgid(), setregid(), <unistd.h>)
3. Add prototypes to <unistd.h>
4. Add to "XSI_USER_GROUPS" in "E.1 Subprofiling Option Groups"(?)

I will attempt 1; presumably 2, 3, 4 can be written up as boilerplate editing directions (which I can attempt, but may need some assistance).
Notes
(0004879)
mkerrisk   
2020-05-21 13:09   
On page 448 (<unistd.h> Declarations), after line 15419, insert

int getresgid(gid_t *rgid, gid_t *egid, gid_t *sgid);
int getresuid(uid_t *ruid, uid_t *euid, uid_t *suid);


On page 448 (<unistd.h> Declarations), after line 15443, insert

int setresgid(gid_t rgid, gid_t egid, gid_t sgid);
int setresuid(uid_t ruid, uid_t euid, uid_t suid);


On page 451 (<unistd.h> SEE ALSO) at lines 15579-15581, insert the following
entries into the list in sorted order:

getresgid(), getresuid(), setresgid(), setresuid()

On page 1028 (getegid() SEE ALSO) at line 35033, insert the following
entries into the list in sorted order:

getresgid(), getresuid(), setresgid(), setresuid()

On page 1032 (geteuid() SEE ALSO) at line 35171, insert the following
entries into the list in sorted order:

getresgid(), getresuid(), setresgid(), setresuid()

On page 1033 (getgid() SEE ALSO) at line 35210, insert the following
entries into the list in sorted order:

getresgid(), getresuid(), setresgid(), setresuid()

On page 1104 (getuid() SEE ALSO) at line 37410, insert the following
entries into the list in sorted order:

getresgid(), getresuid(), setresgid(), setresuid()

On page 1890 (setegid() SEE ALSO) at line 61214, insert the following
entries into the list in sorted order:

getresgid(), getresuid(), setresgid(), setresuid()

On page 1893 (seteuid() SEE ALSO) at line 61308, insert the following
entries into the list in sorted order:

getresgid(), getresuid(), setresgid(), setresuid()

On page 1894 (setgid() SEE ALSO) at line 61345, insert the following
entries into the list in sorted order:

getresgid(), getresuid(), setresgid(), setresuid()

On page 1917 (setregid() SEE ALSO) at line 61847, insert the following
entries into the list in sorted order:

getresgid(), getresuid(), setresgid(), setresuid()

On page 1919 (setreuid() SEE ALSO) at line 61908, insert the following
entries into the list in sorted order:

getresgid(), getresuid(), setresgid(), setresuid()

On page 1929 (setuid() SEE ALSO) at line 62155, insert the following
entries into the list in sorted order:

getresgid(), getresuid(), setresgid(), setresuid()


(Depending on whether these APIs should be part of XSI_USER_GROUPS)
On page 3794 (Subprofiling Option Groups) at lines 130234-130234,
insert the following entries into the list in sorted order:

getresgid(), getresuid(), setresgid(), setresuid()


At page 1086, insert the specifications for getresuid() and getresgid():

NAME

getresgid - get real group ID, effective group ID, and saved set-group-ID

SYNOPSIS
#include <unistd.h>

int getresgid(gid_t *rgid, gid_t *egid, gid_t *sgid);

DESCRIPTION

The getresgid() function shall return the calling process's real
group ID, effective group ID, and saved set-group-ID, storing them
in the locations pointed to by, respectively, the arguments rgid,
egid, and sgid.

RETURN VALUE

Upon successful completion, 0 shall be returned. Otherwise, -1
shall be returned and errno set to indicate the error.

ERRORS

No errors are defined.

SEE ALSO

getegid(), geteuid(), getgid(), getresuid(), getuid(), setegid(),
setgid(), seteuid(), setregid(), setresgid(), setresuid(),
setreuid(), setuid()

XBD <sys/types.h>, <unistd.h>


NAME

getresuid - get real user ID, effective user ID, and saved set-user-ID

SYNOPSIS
#include <unistd.h>

int getresuid(uid_t *ruid, uid_t *euid, uid_t *suid);

DESCRIPTION

The getresuid() function shall return the calling process's real
user ID, effective user ID, and saved set-user-ID, storing them in
the locations pointed to by, respectively, the arguments ruid,
euid, and suid.

RETURN VALUE

Upon successful completion, 0 shall be returned. Otherwise, -1
shall be returned and errno set to indicate the error.

ERRORS

No errors are defined.

SEE ALSO

getegid(), geteuid(), getgid(), getresgid(), getuid(), setegid(),
setgid(), seteuid(), setregid(), setresgid(), setresuid(),
setreuid(), setuid()

XBD <sys/types.h>, <unistd.h>



At page 1918, insert the specifications for setresuid() and setresgid():

NAME

setresgid - set real group ID, effective group ID, and saved set-group-ID

SYNOPSIS
#include <unistd.h>

int setresgid(gid_t rgid, gid_t egid, gid_t sgid);

DESCRIPTION

The setresgid() function shall change the calling process's real
group ID, effective group ID, and saved set-group-ID, respectively,
to the values specified by rgid, egid, and sgid.

If one of the arguments is -1, the corresponding group ID shall
not be changed.

Only a process with appropriate privileges can set the real group
ID, effective group ID, and saved set-group-ID to any valid value.


A non-privileged process may set its real group ID, effective
group ID, and saved set-group-ID, each to one of the values that
it currently holds in its real group ID, effective group ID, or
saved set-group-ID.

The real group ID, effective group ID, and saved set-group-ID may
be set to different values in the same call.

RETURN VALUE

Upon successful completion, 0 shall be returned. Otherwise, -1
shall be returned and errno set to indicate the error, and none of
the group IDs shall be changed.

ERRORS

The setresgid() function shall fail if:

[EINVAL]

The value of the rgid, egid, or sgid argument is invalid or out-of-range.

[EPERM]

The calling process does not have appropriate privileges,
and an attempt was made to change the real group ID,
effective group ID, or saved set-group-ID to a value that
is not currently present in one of those IDs.

SEE ALSO

getegid(), geteuid(), getgid(), getresgid(), getresuid(),
getuid(), setegid(), seteuid(), setgid(), setregid(), setresuid(),
setreuid(), setuid()

XBD <sys/types.h>, <unistd.h>

NAME

setresuid - set real user ID, effective user ID, and saved set-user-ID

SYNOPSIS
#include <unistd.h>

int setresuid(uid_t ruid, uid_t euid, uid_t suid);

DESCRIPTION

The setresuid() function shall change the calling process's real
user ID, effective user ID, and saved set-user-ID, respectively,
to the values specified by ruid, euid, and suid.

If one of the arguments is -1, the corresponding user ID shall not
be changed.

Only a process with appropriate privileges can set the real user
ID, effective user ID, and saved set-user-ID to any valid value.

A non-privileged process may set its real user ID, effective user
ID, and saved set-user-ID, each to one of the values that it
currently holds in its real user ID, effective user ID, or saved
set-user-ID.

The real user ID, effective user ID, and saved set-user-ID may be
set to different values in the same call.

RETURN VALUE

Upon successful completion, 0 shall be returned. Otherwise, -1
shall be returned and errno set to indicate the error, and none of
the user IDs shall be changed.

ERRORS

The setresuid() function shall fail if:

[EINVAL]

The value of the ruid, euid, or suid argument is invalid or out-of-range.

[EPERM]

The calling process does not have appropriate privileges,
and an attempt was made to change the real user ID,
effective user ID, or saved set-user-ID to a value that is
not currently present in one of those IDs.

SEE ALSO

getegid(), geteuid(), getgid(), getresgid(), getresuid(),
getuid(), setegid(), seteuid(), setgid(), setregid(), setresgid(),
setreuid(), setuid()

XBD <sys/types.h>, <unistd.h>




Viewing Issue Advanced Details
1342 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2020-05-05 08:50 2021-01-04 12:14
kre
 
normal  
Interpretation Required  
Accepted As Marked  
   
Robert Elz
XCU 2.2.3
2346
74721
Approved
Note: 0005123
Aliases in command substitutions are handled differently when the command subst is quoted vs not quoted
A 1997 interpretation resulted in the addition of the words:

   not including the alias substitutions in Section 2.3.1,

to the rules for processing a command substitution embedded within double
quotes (XCU 2.2.3).

No similar change was made to section 2.3 (bullet point 5) which handles
the general rules for token recognition when a command substitution is
detected, unquoted.

This means that different rules apply to recognising a command substitution
that happens in a double-quoted string than one that appears unquoted.

This is neither rational, nor what shells implement.

Apart from the unfortunate difference made to the two cases, the 1997
interpretation was almost certainly correct for its time - the relevant shells
of the mid 1990's did not expand aliases while searching for the terminating ')'
of a $() command substitution (backquoted command substitutions are much
simpler and have none of these issues). Shells of the time didn't parse
the command substitution at all while searching for the ')' - they relied
upon '(' and ')' counting.

In the intervening 20-something years that technique has been shown to be
fatally flawed, not only do (some - unlikely but possible) aliases potentially
break this, but case statements (not using the optional '(' preceding the
pattern) and here documents (which can contain almost anything - but can
only be recognised as being a here document by parsing the accompanying
command) break this simple scheme - shells must fully parse a $() command
substitution to have any hope of correctly finding the terminating ')' in
all cases.

Note that whether the shell retains the results of this parse, or simply
copies the command substitution string literally, saving it to be parsed
again later, is immaterial to this issue - the correct ')' must be located
to properly terminate the command substitution, whatever is done with the
text of the command substitution, or the results of parsing it, during that
process.
In line 74271, on page 2346 of XCU 2.2.3 (2016 edition) delete the words:

    not including the alias substitutions in Section 2.3.1,

This is needed regardless of any other changes that might be made, as even
if it is (was) the correct behaviour, it was applied in the wrong place.

That is all that is needed to handle this issue.

However, as an option, in XCU 2.3, bullet point 5, on page 2348, after
the sentence (lines 74774-6):

    While processing the characters, if instances of expansions or quoting are
    found nested within the substitution, the shell shall recursively process
    them in the manner specified for the construct that is found.

add a new sentence:

    It is unspecified whether aliases [xref XCU 2.3.1] shall be expanded
    while performing this process.

It needs to be unspecified, rather than "shall not process" as most shells
do process aliases at this point, as they are when the command substitution
is eventually parsed and executed, if different rules apply strange hard
to fathom errors can occur.

However, not all shells do, so making it unspecified might be the wise choice
for now - even if such shells tend to be broken in this regard. Since alias
processing when the command substitution was in a double quoted string was
(or is) prohibited by the standard, suddenly making it mandated when there are,
or might be, some shells which actually implemented this prohibition would
also seem harsh.

If this option is adopted, two further changes should be made.

In the application notes (wherever they are for this) there should be an
admonition

     Applications shall not include aliases which contain unbalanced
     syntax components in any $() command substitution.

What that means can be expanded if deemed appropriate. The idea is that
trivial aliases "alias l=ls" (etc) are harmless, and not a problem, it is
only ones like "alias switch=case", "alias subshell='('", or
"alias forever='while true do;'" that cause problems. The "unbalanced"
is because an alias like "alias set13='(exit 13)'" is not a problem,
despite having parentheses in its value.

And second, in the Future Directions (wherever that is for sh) it should
say

     A future version of this standard may require alias expansion
     while scanning for the terminating ')' in a $() command substitution.

Last, some kind of explanation of all that happened here should be added
in the rationale (somewhere in XRAT I assume).


An alternative option open to the group who decides these things would be
to simply delete aliases from the standard completely. Since I doubt this
one will happen, I won't bother providing the changes needed to implement it.
Notes
(0004856)
geoffclare   
2020-05-05 10:57   
We also need to do something about the statement for $() command substitution that "Any valid shell script can be used for command". This is not true because a shell script which defines an alias which changes the syntax, and then uses it, might not be parsed correctly without the alias command being executed.
For example a script containing:
alias subshell='('
subshell echo foo )
subshell echo bar )
works fine (note you need POSIXLY_CORRECT set for bash), but:
echo $(
alias subshell='('
subshell echo foo )
subshell echo bar )
)
does not.
(0004859)
kre   
2020-05-05 15:36   
Re Note: 0004856

I suspect that's already handled by the (new, I forget which bug it is)
wording that specifies when alias commands take effect.

This is just the same issue as prevents

if [ $SHELL = myshell ]; then
    alias xx=mycommand
    xx foo bar
fi

from working. Or from defining, and then using, an alias in a function.

Aliases are evil - let's delete them!!!
(0004860)
geoffclare   
2020-05-05 15:52   
Re Note: 0004859

Bug 0000953 does indeed change that text, but the new text still has the same problem. It now says:
It is unspecified whether command is parsed and executed as a program (as for a shell script) or is parsed as a single compound_list that is executed after the entire command has been parsed. With the $(command) form any valid program can be used for command, except a program consisting solely of redirections which produces unspecified results.
(0004863)
kre   
2020-05-05 17:43   
OK, we can add text that makes it clear (just in case it wasn't
already obvious) that the entire command in a command substitution is
parsed before any of it is executed. And given that, alias commands
in the command substitution cannot work - the command substitution is
a subshell environment, so the parent shell cannot be affected, and
any aliases defined there get defined too late to be used within the
command substitution.

Alias in a sub-shell block '(' code ')' have the same uselessness.

Aliases in general are largely useless, let's delete them!!!
(0005123)
geoffclare   
2020-11-30 17:04   
Interpretation response
------------------------
The standard states that alias substitutions are applied when finding the terminating ')' of a command substitution when the command substitution is not within double quotes, but they are not applied when the command substitution is within double quotes, and conforming implementations must conform to this. However, concerns have been raised about this which are being referred to the sponsor.

Rationale:
-------------
Historically some shells used simple parenthesis counting to find the terminating ')' and therefore did not account for aliases. However, such shells never conformed to POSIX, which has always required recursive parsing (see XCU 2.3 item 5). It is likely that observing the behavior of these non-conforming shells was what led to the erroneous addition of "not including the alias substitutions in Section 2.3.1" to XCU 2.2.3 after PASC interpretation 1003.2 #157.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------

On page 2346 line 74721 section 2.2.3, delete:
not including the alias substitutions in Section 2.3.1,

On page 2357 line 75182 section 2.6.3, change:
Command substitution allows the output of a command to be substituted in place of the command name itself. Command substitution shall occur when the command is enclosed as follows
to:
Command substitution allows the output of one or more commands to be substituted in place of the commands themselves. Command substitution shall occur when command(s) are enclosed as follows

On page 2357 line 75190 section 2.6.3, change:
with the standard output of the command
to:
with the standard output of the command(s)

On page 2358 lines 75202-75204 section 2.6.3, after applying bug 953, change:
It is unspecified whether command is parsed and executed as a program (as for a shell script) or is parsed as a single compound_list that is executed after the entire command has been parsed. With the $(command) form any valid program can be used for command, except a program consisting solely of redirections which produces unspecified results.
to:
It is unspecified whether the commands string is parsed and executed incrementally as a program (as for a shell script), or is parsed as a single compound_list that is executed after the string has been completely parsed. In addition, it is unspecified whether the terminating ')' of the $(commands) form can result from alias substitution. With the $(commands) form any syntactically correct program can be used for commands, except that:

  • If the commands string consists solely of redirections, the results are unspecified.

  • If the commands string is parsed as a single compound_list, before any commands are executed, alias and unalias commands in commands have no effect during parsing (see [xref to 2.3.1]). Strictly conforming applications shall ensure that the commands string does not depend on alias changes taking effect incrementally as would be the case if parsed and executed as a program.

  • The behavior is unspecified if the terminating ')' is not present in the token containing the command substitution; that is, if the ')' is expected to result from alias substitution.

Throughout section 2.6.3 change any remaining occurrences of "command" to "commands" or "the commands string" as appropriate.

On page 3730 line 127967 section C.2.6.3, change:
   
While the newer "$()" form can process any kind of valid embedded script, the backquoted form ...
to:
While the newer "$()" form can process any kind of valid embedded script (with a few caveats; see below), the backquoted form ...

On page 3730 line 127985 section C.2.6.3, change:
If command is of the form ...
to:
If the commands string is of the form ...

On page 3730 line 127990 section C.2.6.3, after:
... it is unspecified to have a script within "$()" that has only redirections.
add the following new paragraph:
In POSIX.2-1992 the $(commands) form of command substitution only had unspecified behavior for a commands string consisting solely of redirections. However, two additional unspecified cases have since been added with relation to aliases:

  1. Implementations are permitted to parse the entire commands string before executing any of it, and in this case alias and unalias commands in commands have no effect during parsing. For example, the following commands:
    alias foo='echo "hello globe"'
    echo $(alias foo='echo "Hello World"';foo)

    produce the output "hello globe" if the entire commands string is executed as an entire command and produce the output "Hello World" if the commands string is executed incrementally.

  2. Although existing aliases are required to be expanded when the shell parses the input that follows the "$(" in order to find the terminating ')' (see [xref to 2.3 Token Recognition]), it is unspecified whether the terminating ')' can result from alias substitution. For example, with this script:
    alias foo="echo foo )"
    echo $(foo ; echo bar

    some shells output lines containing <tt>foo</tt> and <tt>bar</tt> whereas other shells report a syntax error because they do not find a terminating ')' for the command substitution.
(0005137)
agadmin   
2020-12-02 17:04   
Interpretation proposed: 2 December 2020
(0005190)
agadmin   
2021-01-04 12:14   
Interpretation approved: 4 Jan 2021




Viewing Issue Advanced Details
1340 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Objection Clarification Requested 2020-05-04 16:28 2020-06-17 14:02
shware_systems
 
normal  
Applied  
Accepted As Marked  
   
Mark Ziegast
SHware Systems Dev.
XBD 8.3
178
5854-7
Approved
Note: 0004871
PATH specification has an ambiguity.
PATH currently has:
If the pathname being sought contains a <slash>, the search through the path prefixes shall not be performed. If the pathname begins with a <slash>, the specified path is resolved (see Section 4.13, on page 111).

In discussing bug 1208 it was noted the first sentence is in error, as all pathname's being checked will have a <slash> between prefix and filename, and a prerequisite for PATH being referenced is that a filename string has no <slash> characters. Therefore this sentence is superfluous.

The second sentence does not include the case where a pathname does not begin with a <slash> and is a relative reference. This leaves ambiguous what root directory as an additional prefix that begins with a <slash> is presumed: the current directory '.', the root directory '/', or a user's home directory, ala '~', as example choices. Actual practice is these are also resolved in accordance with XBD 4.13, historically.
Replace the two sentences with:

Each pathname to be sought shall be resolved in accordance with <xref>XBD 4.13 Pathname Resolution</>, on page 111.
Notes
(0004846)
kre   
2020-05-04 19:31   
While I would not object to the definition of PATH being rewritten in
a slightly more logical form, nothing in the Description here makes
any sense.

I have no idea what discussions related to 0001208 happened to be, or
why an issue relating ti chdir in posix_spawn should care in the least
about PATH, but the "pathname being sought" is typically "ls" or "cat" or "sh"
none of which contains a '/' - so the first sentence is not "in error",
there is no prefix, just the command name (you do understand how PATH works,
right?)

When the pathname begins with a slash there is nothing to do, no search is
made (that's for when someone says /bin/sh) - when there is some other slash
in the pathname, resolution happens the same as for any other filename
reference - since there is no leading '/' (we know that, that case was
already handled) the path lookup starts at the current working directory.

There is no ambiguity here, and while, once again, the paragraph could be
more clearly written (and should xref XCU 2.9.1.1) it doesn't contain any
actual errors.
(0004847)
kre   
2020-05-04 19:33   
Actually, are you perhaps under the impression that the paragraph
in question is about pathnames in general ?

It is about the environment variable PATH - a very specific thing.
(0004848)
shware_systems   
2020-05-04 20:05   
The Desired Action has already been discussed and reached consensus, maybe with a couple wording tweaks. This report is simply the formal record the change is considered required, for the reasons given in the Description.

This came up because sh does do PATH searches and may use *spawn() to exec utilities. The new Final Text for Bug 1208 is related mostly to how PATH is applicable.
(0004849)
kre   
2020-05-05 03:00   
Then the resolution is wrong, and wasn't properly considered.

You could *add* a sentence like that which is proposed, perhaps even
replacing the final sentence, but the earlier text (while it could be
improved) is not incorrect and should not be deleted.
(0004850)
shware_systems   
2020-05-05 03:16   
In that context filename refers to the desired utility, pathname is prefix (possibly with appended slash) concatenated with filename. It was argued filename might replace pathname and keep the basic statement, but that just made it obvious it was redundant so is considered superfluous.
(0004851)
kre   
2020-05-05 04:39   
(edited on: 2020-05-05 05:13)
All filenames are pathnames. But yes, I agree the switch in terminology
is one of the things I would fix if rewording this is considered worthwhile.

But it is clear that the "pathname" in the later sections is intended to
mean the same thing as the "filename" in the initial part.

The text is redundant (to some extent) if and only if an xref to XCU 2.9.1.1
is added to replace it... but I am not sure that that section (which explains
how sh uses PATH for lookups) necessarily applies precisely to all other uses,
as it includes references to built in utilities, functions, etc, so I think
it would be better to retain the XBD definition for those other purposes,
and simply include an xref to add clarity - perhaps something like

    See [xref XCU 2.9.1.1] for a description of how sh [xref XCU 2] handles
    PATH searches.

(wordsmithed as appropriate).

ps: Not relevant directly here, but to respond to a comment made in Note: 0004848
it is hard to see how a sh could ever use posix_spawn as Note: 0004830 implies
while retaining the view that some people have of how and when builtin
utilities should be executed. It would be possible if the shell simply
looks for a builtin before starting the PATH search, but if it needs to
combine the PATH search with builtin recognition, if it was relying upon
posix_spawn to do the PATH search, I fail to see how a builtin would ever
be able to be executed - the only way would be to do the PATH search twice,
and I don't think that works either (the only truly reliable way to test
whether a PATH element + filename combination works (from userland)
is to try to exec it, and there's no returning from that if it does work.

It also isn't clear how a posix_spawn PATH scanning implementation would ever
maintain the hash table properly (unless of course, in both cases, the PATH
resolution is done as a separate step, and posix_spawn is only ever called
(from sh) with a fully qualified pathname (ie: just to replace the fork/exec
sequence, and fd/dir manipulations that fit between those) but in that case,
there would be no motivation for fiddling the definition of PATH based upon
what posix_spawn sees, as it would only every hit the "contains a slash" and
"begins with a slash" sections (ie: PATH is irrelevant, and what's more,
Note: 0004830 is incorrect, see: Note: 0004830 is incorrect">0001341) - so if you were to
believe that this definition is solely for the purposes of posix_spawn then
I guess the change would make sense. But it isn't, it is also used by
popen() system() find env ...

[Aside: mantis seems to have issues formatting this note correctly, I have
tried several different ways to write the text, yet it always seems to mangle
that final "see Note: 0004830 is incorrect">0001341" reference. Oh, I see the trigger now, it is
because the referenced bug (1341) contains bugid and bugnote references in its
Summary line; is that not supported - it works in the issue itself?]

(0004854)
geoffclare   
2020-05-05 08:40   
The description in this bug is reasonably close to what was discussed in the telconference. However, the desired action is most definitely not what we thought Mark was going to put in this bug.

The statement "If the pathname being sought contains a <slash>, the search through the path prefixes shall not be performed" is both erroneous and redundant. It is erroneous because the thing that is sought is a filename, not a pathname, and therefore by definition cannot contain a slash. It is redundant because all of the places in the standard that require a PATH search to be done already state that no PATH search is done if there is a slash in the string that would be the subject of the search. I.e the description of PATH never needs to be read in the context of something with a slash being sought.

The statement "If the pathname begins with a <slash>, the specified path is resolved" is problematic because it only requires pathname resolution to be done for absolute pathnames, not relative ones. The meeting agreed that this needs to be changed in some way so that it applies to relative pathnames as well.

Having thought about the issue a little more, I believe just changing that sentence is not the right solution to the second problem. The time when pathname resolution is done is during the search, not after the pathname has been found. (I.e. it can only be "found" by performing pathname resolution on each constructed pathname.)

I will add another note with a new proposed resolution.
(0004855)
geoffclare   
2020-05-05 08:46   
(edited on: 2020-05-05 08:47)
New proposed resolution...

On line 5851 change:
The list shall be searched from beginning to end, applying the filename to each prefix, until an executable file with the specified name and appropriate execution permissions is found. If the pathname being sought contains a <slash>, the search through the path prefixes shall not be performed. If the pathname begins with a <slash>, the specified path is resolved (see Section 4.13, on page 111).
to:
The list shall be searched from beginning to end, applying the filename to each prefix and attempting to resolve the resulting pathname (see Section 4.13, on page 111), until an executable file with appropriate execution permissions is found.


(0004861)
shware_systems   
2020-05-05 16:19   
I've no objection to the new wording; it was my understanding all the text preceding the two sentences quoted was considered adequate so I left it alone for original proposal.
(0004862)
kre   
2020-05-05 16:21   
Re Note: 0004854

To take the first couple of issues in reverse order, as it makes
more sense (to me) this way...

    It is redundant because all of the places in the standard that require
    a PATH search to be done already state that no PATH search is done if
    there is a slash in the string that would be the subject of the search.

To me that has produced exactly the inverse to the correct response. If
everything that uses PATH needs to specify that it is only used when there
is no slash in the name, then lets simplify all of those, and codify the rules
for PATH processing in a single place, so everyone gets a *known* consistent
interpretation and we don't have people quibbling about why the wording for
the use in context X is subtly different from the wording for contest Y, and
deducing from that, that the behaviour is supposed to be different. And
if we're certain that the identical wording is used in every case, then that's
an even better reason for consolidating it and removing the duplication.

Further, this is the safe way to make the change - if we have missed one of
the uses (and so don't delete its redundant wording) then we're left with
some redundant text, but that's generally harmless. On the other hand if
we have missed a use which turns out not to specify that no path search is
done when the pathname being sought contains a slash, and we delete the
text in the definition of PATH which deals with that, then we've just caused
the standard to be broken. Since the standard is very large, and I'm sure
the word PATH appears in it very many times, I'd hate to guarantee (as in,
my life depends upon it) that we know every place where a path search is
specified, and if we are not 100% certain that we have found every single one,
then neither can we be certain that every single one has the "no slash"
qualification.

And last, it means that someone who simply wants to know what this environment
variable PATH is all about, and (reasonably) turns to XBD 8.3 to get the
answer, will actually discover the whole picture, and not end up missing
the crucial element that a search for a/b only looks for "a/b" and doesn't
try PATH[1]/a/b PATH[2]/a/b (etc) (using invented notation for the ':'
delimited sub-strings of PATH).

Next:
    It is erroneous because the thing that is sought is a filename,
    not a pathname, and therefore by definition cannot contain a slash.

This is part of the "poor wording" I remarked on in earlier notes. It isn't
really as bad (and isn't actually incorrect) it is just hard to read
correctly easily (and currently depends upon the reader correctly
understanding the difference between pathname and filename). PATH What we
do about it depends upon the resolution of the previous point. If we
agree to consolidate the definition of how PATH works (all of it) in XBD 8.3,
which is what I'd suggest should be done, then the thing that is sought is
a pathname, not a filename (as they're defined in XBD 3.170 and 3.271), and
the correct response to this problem is to fix that, and use "filename" only
after we have excepted the case where there's a slash in the pathname - and
I'd make the wording be clear about the change of terminology, and why
something perhaps like:

PATH This variable shall represent the sequence of path prefixes that
       certain functions and utilities apply in searching for an executable
       file. The prefixes shall be separated by a <colon> (':'). If the
       pathname being sought contains no slash ('/') characters, and hence
       is a filename, the list of prefixes shall be searched from beginning
       to end, applying the filename to each prefix, until an executable file
       with the specified name and appropriate execution permissions is found.
       When a non-zero-length prefix is applied to the filename, a <slash>
       shall be inserted between the prefix and the filename if the prefix
       did not end in <slash>. A zero-length prefix is a legacy feature that
       indicates the current working directory. It appears as two adjacent
       <colon> characters ("::"), as an initial <colon> preceding the rest
       of the list, or as a trailing <colon> following the rest of the list.
       A strictly conforming application shall use an actual pathname
       (such as .) to represent the current working directory in PATH.
       If the pathname being sought contains a <slash>, and so is not a
       filename, the search through the path prefixes shall not be performed.

and then continue as it is now, as amended by the feed for ...

   The statement "If the pathname begins with a <slash>, the specified
   path is resolved" is problematic because it only requires pathname
   resolution to be done for absolute pathnames, not relative ones.

More poor wording, and I have no problem with fixing that.

However, I don't think we should discard mention of what happens with a
PATH search when the pathname being sought contains a '/' and rely upon
the reader just understanding that since it says "filename" thatmust be
true, while being left in the dark (until they find a use of PATH search)
about what happens if there is a '/'. And then wonder why it is divided
up that way, and if perhaps different applications of PATH searching might
have different rules for what to do in that case.

That is, I don't believe the proposed resolution in Note: 0004855 is good enough.
(0004864)
geoffclare   
2020-05-06 08:18   
Re Note: 0004862

Seems I didn't look hard enough for the places that require a PATH search. I have now found a place that doesn't say PATH is not searched if there is a slash in the search string: find -exec (and -ok), where the requirement that a PATH search is done is only made via the PATH entry in ENVIRONMENT VARIABLES - it's not in the description of -exec. There is also a slight problem with fc -e and xargs because they both state that the utility to be invoked is specified by its name, implying that portable applications can't use fc -e or xargs to invoke a utility by means of a pathname that includes a slash. Once those are fixed to allow a pathname, they would also need to specify the no-slash rule for PATH searching, unless it is covered by the description of PATH.

So I am now in agreement that the description of PATH should handle this.

Your suggestion of "If the pathname being sought contains no slash ('/') characters, and hence is a filename, the list of prefixes shall be searched ..." is okay with me as a way to do that. (I didn't compare the rest of your new PATH description with the current one to see what else you changed - it would be helpful if you could propose the changes you want in smaller chunks so there is less unchanged text to compare.)

The problems with fc -e and xargs could either be addressed here or in a separate bug.
(0004866)
kre   
2020-05-06 14:39   
Here are the editing instructions to get to text shown in Note: 0004862
as requested in Note: 0004864

Al of this applies to the description of the environment variable PATH
in XBD 8.3, on page 178 (2016 edition).

In lines 5843-4 delete the words "known only by a filename".

In line 5844, after the sentence that ends "...separated by a <colon> (':')."
insert the words (beginning a new sentence)
     If the pathname being sought contains no slash ('/') characters,
     and hence is a filename,
and then follow that by moving the sentence that starts on line 5851
     The list shall be searched...
(up to the end of that sentence, on line 5854)
     ...permissions is found.
While moving that sentence, change the capital 'T' in the leading "The"
to lower case, as it is no longer the beginning of a sentence, just a claise.

That's it, the rest of the text is unchanged (since in Note: 0004862 I did not
attempt to fix the "resolve the path" issue.

What I might do for that is to replace the sentence that runs from line
5855-7, viz:

    If the pathname begins with a <slash>, the specified path is
    resolved (see Section 4.13, on page 111).

and replace it with

     In each case, either the result of appending the filename to each prefix
     from PATH when the pathname (filename) contains no slash characters,
     or the pathname when it does contain one or more slash characters,
     is resolved (see Section 4.13, on page 111).

but I am less happy with that wording than the earlier part.
Someone please suggest something better.
(0004868)
geoffclare   
2020-05-07 08:50   
(edited on: 2020-05-07 08:51)
Here's a suggestion that combines kre's solution from Note: 0004866 for the filename/pathname problem with my suggestion from Note: 0004855 for the pathname resolution problem...

All of this applies to the description of the environment variable PATH in XBD 8.3, on page 178 (2016 edition).

In lines 5843-4 delete the words "known only by a filename".

In line 5844, after the sentence that ends "...separated by a <colon> (':')."
insert the new sentence
If the pathname being sought contains no slash ('/') characters, and hence is a filename, the list shall be searched from beginning to end, applying the filename to each prefix and attempting to resolve the resulting pathname (see Section 4.13, on page 111), until an executable file with appropriate execution permissions is found.

Delete the sentence that starts on line 5851
     The list shall be searched...
(up to the end of that sentence, on line 5854)
     ...permissions is found.

Replace the two sentences that run from line 5854-7, viz:
If the pathname being sought contains a <slash>, the search through the path prefixes shall not be performed. If the pathname begins with a <slash>, the specified path is resolved (see Section 4.13, on page 111).
with:
If the pathname being sought contains any <slash> characters, the search through the path prefixes shall not be performed and the pathname shall be resolved as described in Section 4.13, on page 111.

The end result of these edits is the following as the first paragraph of the PATH description:
This variable shall represent the sequence of path prefixes that certain functions and utilities apply in searching for an executable file. The prefixes shall be separated by a <colon> (':'). If the pathname being sought contains no slash ('/') characters, and hence is a filename, the list shall be searched from beginning to end, applying the filename to each prefix and attempting to resolve the resulting pathname (see Section 4.13, on page 111), until an executable file with appropriate execution permissions is found. When a non-zero-length prefix is applied to this filename, a <slash> shall be inserted between the prefix and the filename if the prefix did not end in <slash>. A zero-length prefix is a legacy feature that indicates the current working directory. It appears as two adjacent <colon> characters ("::"), as an initial <colon> preceding the rest of the list, or as a trailing <colon> following the rest of the list. A strictly conforming application shall use an actual pathname (such as .) to represent the current working directory in PATH. If the pathname being sought contains any <slash> characters, the search through the path prefixes shall not be performed and the pathname shall be resolved as described in Section 4.13, on page 111. If PATH is unset or is set to null, the path search is implementation-defined.


(0004869)
kre   
2020-05-07 10:40   
That (Note: 0004868) is fine with me.
(0004871)
geoffclare   
2020-05-11 15:19   
Interpretation response
------------------------
The standard is unclear on this issue, and no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor.

Rationale:
-------------
The current text does not require constructed relative pathnames to be resolved, only absolute ones.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
Make the changes in Note: 0004868
(0004872)
ajosey   
2020-05-11 15:27   
Interpretation proposed: 11 May 2020
(0004888)
ajosey   
2020-06-12 09:09   
Interpretation approved: 12 June 2020




Viewing Issue Advanced Details
1339 [Online Pubs] Main Index Editorial Error 2020-05-01 18:31 2020-05-04 15:09
andras_farkas
 
normal  
Applied  
Accepted  
   
Andras Farkas
bzip2 download has not been updated
I normally use the .tar.bz2 download here:
https://pubs.opengroup.org/onlinepubs/9699919799/download/index.html [^]
I thought "huh, that's weird, some bugs aren't fixed in it that are fixed on the site. Was it not regenerated?"
I checked all three other downloads, and those three all match each other as being up to date. But the .tar.bz2 download alone has not been regenerated for some time: at least half a year.
Place updated .tar.bz2 file on the website.
Notes
(0004844)
ajosey   
2020-05-04 15:09   
It seems bzip2 does not like to overwrite existing files. I have fixed the file generator to remove the files before it creates them.




Viewing Issue Advanced Details
1338 [Online Pubs] System Interfaces Editorial Error 2020-05-01 18:13 2020-05-04 17:11
andras_farkas
 
normal  
Applied  
Accepted  
   
Andras Farkas
creat, open
creat and open have similar missing BR element between includes
I was researching the OH (optional header) tag, since I was curious, sometime after this
https://austingroupbugs.net/view.php?id=1335 [^]
and noticed that both
https://pubs.opengroup.org/onlinepubs/9699919799/functions/creat.html [^]
https://pubs.opengroup.org/onlinepubs/9699919799/functions/open.html [^]
have the same small error.
They lack a BR (or newline) between the two includes.

Also, out of curiosity, why don't optional headers get a hyperlink to their XBD page like other headers do? I don't see why not.
Add BR element between the two includes.
Notes
(0004845)
ajosey   
2020-05-04 17:11   
These corrections have been made and the download bundles also updated.

The hyperlinking can always be improved.




Viewing Issue Advanced Details
1337 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Omission 2020-04-29 17:41 2020-12-09 10:06
eblake
 
normal  
Applied  
Accepted  
   
Eric Blake
Red Hat
accept vs setsockopt
XSH 2.10.16
528
18538
---
Clarify socket option values after accept()
While the standard is clear that all sockets have default settings for socket options, and that some defaults are implementation-defined (for example, "The default value for the SO_RCVBUF option value is implementation-defined, and may vary by protocol." line 18628), it is silent on whether a non-default socket option set on a listening socket is required/permitted to be inherited into the new socket created by accept(). As there are existing implementations which inherit some but not all socket options (see https://stackoverflow.com/questions/5968132/are-socket-options-inherited-across-accept-from-the-listening-socket), [^] the best course of action is to just clarify that portable applications cannot rely on option inheritance.

Also, the text states "All of the options have defaults" (line 18538) then contradicts itself "SO_TYPE has no default value" (line 18672).
[note: this change assumes that 0000411 adding accept4() is applied]

Change page 528 line 18538 (XSH 2.10.16 Use of Options) from:
All of the options have defaults.
to:
All of the options usable with setsockopt( ) have defaults. For each option where a default value is listed as implementation-defined, the implementation also controls whether a socket created by accept( ) or accept4( ) starts with the option reset to the original default value, or inherited as the value previously customized on the original listening socket.


At page 565 line 19882 [XSH accept() DESCRIPTION), add a paragraph:
It shall be implementation-defined which socket options, if any, on the accepted socket will have a default value determined by a value previously customized by setsockopt( ) on socket, rather than the default value used for other new sockets.


At page 569 line 19912 [XSH accept() APPLICATION USAGE), add a new paragraph:
Many socket options are described as having implementation-defined default values, which may differ according to the protocol in use by the socket. Existing practice differs on whether socket options such as SO_SNDBUF that were customized on the original listening socket will impact the corresponding option on the newly returned socket. Implementations are permitted to allow inheritance of customized settings where it makes sense, although the most portable approach for applications is to limit setsockopt( ) customizations to only the accepted socket.


At page 1924 line 62032 [XSH setsockopt() APPLICATION USAGE), add a new paragraph:
It is implementation-defined which socket options, if any, are inherited from a listening socket to an accepted socket by accept( ) or accept4( ).

There are no notes attached to this issue.




Viewing Issue Advanced Details
1336 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Error 2020-04-23 13:09 2020-12-18 09:15
dennisw
 
normal  
Applied  
Accepted As Marked  
   
Dennis Wölfing
getrusage
1089
36960-36962
Approved
Note: 0005113
getrusage should recursively include information about children of children
The specification for getrusage says that for RUSAGE_CHILDREN information about waited-for
children of the current process is returned. However existing implementations also recursively
include children of children.

In XBD 3.93 Child Process is defined as
A new process created (by fork( ), posix_spawn( ), or posix_spawnp( )) by a given process. A child
process remains the child of the creating process as long as both processes continue to exist.
which makes clear that children of children are not considered children of the current process.
On page 1089 lines 36960-36962 section getrusage, change
If the
value of the who argument is RUSAGE_CHILDREN, information shall be returned about
resources used by the terminated and waited-for children of the current process.
to
If the
value of the who argument is RUSAGE_CHILDREN, information shall be returned about
resources used by the terminated and waited-for children of the current process and
recursively the terminated and waited-for children thereof.
Notes
(0005113)
rhansen   
2020-11-12 16:26   
Interpretation response
------------------------
The standard states that getrusage() with RUSAGE_CHILDREN only reports on direct children, and conforming implementations must conform to this. However, concerns have been raised about this which are being referred to the sponsor.

Rationale:
-------------
No known implementation has ever worked this way.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------

On page 1089 lines 36960-36962 section getrusage, change
If the value of the who argument is RUSAGE_CHILDREN, information shall be returned about resources used by the terminated and waited-for children of the current process.
to:
If the value of the who argument is RUSAGE_CHILDREN, information shall be returned about resources used by the children of the current process that have terminated and been waited-for and their children that have terminated and been waited-for, recursively.
(0005114)
agadmin   
2020-11-13 12:23   
Interpretation proposed: 13 Nov 2020
(0005166)
agadmin   
2020-12-14 11:35   
Interpretation approved 14 Dec 2020.




Viewing Issue Advanced Details
1335 [Online Pubs] System Interfaces Editorial Error 2020-04-20 01:29 2020-04-30 16:27
andras_farkas
 
normal  
Applied  
Accepted  
   
Andras Farkas
fstatat
Missing newline or BR element on fstatat() page
On the fstatat() page:
https://pubs.opengroup.org/onlinepubs/9699919799/functions/fstatat.html [^]
The first line after SYNOPSIS looks like this:
[OH] #include <fcntl.h> #include <sys/stat.h>
It should look like this:
[OH] #include <fcntl.h>
#include <sys/stat.h>
Remove the space character between the two includes, and replace it with a HTML BR element between the first and second includes.
Notes
(0004826)
Don Cragun   
2020-04-20 01:51   
This appears correctly in the PDF on P965, L32786-32787.

I'll move this to the Online Pubs project.
(0004827)
andras_farkas   
2020-04-20 04:02   
Thanks, Don!
Where do I find the PDF, by the way? It's not available here:
https://pubs.opengroup.org/onlinepubs/9699919799/download/index.html [^]
(0004828)
Don Cragun   
2020-04-20 05:36   
(edited on: 2020-04-20 05:37)
Re: Note: 0004827:
You can buy the PDF for the standard from IEEE or ISO. (I don't remember what they charge, but it isn't cheap.)

If you have a login for opengroup.org, you can sign in, click on "Standards", click on "UNIX Standards", and click on "Learn More" under the heading UNIX BASE SPECIFICATIONS ISSUE 7 2018 EDITION to find pointers to the PDF and to the online HTML that you have already found.

(0004837)
ajosey   
2020-04-30 16:11   
Change applied to the html edition
(0004842)
ajosey   
2020-04-30 16:27   
The html download bundles have also been updated




Viewing Issue Advanced Details
1334 [Online Pubs] Rationale Editorial Error 2020-04-07 19:28 2020-04-08 08:29
andras_farkas
 
normal  
Applied  
Accepted As Marked  
   
Andras Farkas
Portability Considerations
Possible bad troff to HTML conversion in Portability Considerations section of XRAT
In D.2.10 Command Language:
https://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_port.html#tag_24_02_10 [^]
In the list of commands, we have:

sleep, tee, test, CONVERSION ERROR (.Cm) time *,1 true, wait,

The conversion error text seems erroneous. time is also not a hyperlink to time.html, unlike the other commands in the list.
Get rid of
"CONVERSION ERROR (.Cm) "
text and make time a hyperlink to the intended time utility.
Notes
(0004815)
Don Cragun   
2020-04-07 23:35   
In the PDF (P3771, L129440-129443) the only hyperlink is to Section 2.14. It would be nice if the utilities in that list were hyperlinks as in the on-line version.

The conversion error does not appear in the PDF. Note that time utility has a footnote in both the HTML and the PDF, but there is an extra space between "time" and "*" in the HTML that isn't in the PDF. I would guess that there is a bug in the .Cm macro processing when producing the HTML.
(0004819)
geoffclare   
2020-04-08 08:04   
(edited on: 2020-04-08 08:05)
It looks like the HTML translation tool doesn't correctly handle a footnote marker in a macro argument. The source here has:
.Cm time *,\*F
I looked for other places where "\*F" appears on a line beginning with "." and found there is only one other such place, and it also has a translation problem. It's in XRAT Section E Subprofiling Considerations under POSIX_SYMBOLIC_LINKS, where the troff source:
.Fn lchown ,\*F
is rendered in the HTML version as:

,,()href="#tag_foot_2">2

(0004820)
agadmin   
2020-04-08 08:12   
This conversion error and two other conversion errors in XRAT have been corrected.
(0004821)
agadmin   
2020-04-08 08:28   
The lchown conversion has also been fixed.

Files updated in XRAT:
V4_subprofiles.html
V4_xbd_chap12.html
V4_xbd_chap03.html
V4_port.html

Download bundles have been updated.




Viewing Issue Advanced Details
1333 [Online Pubs] Shell and Utilities Editorial Error 2020-04-07 19:00 2020-04-08 08:30
andras_farkas
 
normal  
Applied  
Accepted As Marked  
   
Andras Farkas
talk
talk.html page seems to have a missing newline in example text
In
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/talk.html [^]
the example text below

Message from <unspecified string>
talk: connection requested by your_addresstalk: respond with: talk your_address

seems to be missing a newline in the second of the two lines.

(Do forgive me for the noise if I'm wrong! I've never actually gotten talk to work on any machine I've used. And thanks so much for helping me out with my previous bug reports.)
Change the example text to

Message from <unspecified string>
talk: connection requested by your_address
talk: respond with: talk your_address

by adding a single newline.
Notes
(0004812)
shware_systems   
2020-04-07 19:08   
I get similar. At the least there should be a SPC between your_address in italics and second talk, if message can all fit on one line.
(0004813)
kre   
2020-04-07 19:29   
This looks to be an issue with the HTML translation, the PDF version is correct.
(0004814)
Don Cragun   
2020-04-07 23:11   
As kre said in Note: 0004813, this text appears correctly in the PDF on P3281, L110387-110389. Therefore this bug has been moved from the 1003.1(2016)/Issue7+TC2 Project to the Online Pubs Project.
(0004816)
Konrad_Schwarz   
2020-04-08 07:28   
The mailx page has similar issues:
in the "Commands in mailx" section, the variant command forms are shown
all in the same line without intervening space,
they should probably each be on a separate line.
(0004817)
agadmin   
2020-04-08 07:44   
The talk html page has been updated
(0004818)
agadmin   
2020-04-08 07:58   
The mailx page has been updated
(0004822)
agadmin   
2020-04-08 08:29   
The talk and mailx pages were updated in the utilities directory
The Download bundles have been updated.




Viewing Issue Advanced Details
1332 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Error 2020-04-07 10:51 2020-06-04 08:43
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
waitid
2236
71333
---
WEXITED should not be limited to processes that have exited
The description of WEXITED on the waitid() page (and <sys/wait>) is:

    Wait for processes that have exited.

The use of "exited" here means that waitid() calls with WEXITED are
not required to return status information for processes that have
been terminated by a signal.
At the specified location, and also at
page 409 line 13906 section <sys/wait.h>, change:

    Wait for processes that have exited.

to:

    Wait for processes that have terminated.
Notes
(0004811)
shware_systems   
2020-04-07 16:01   
I agree that could be misconstrued, but so could the desired action. I'd prefer:
Wait for processes that have exited normally, called abort(), or have been terminated due to a signal.




Viewing Issue Advanced Details
1331 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Error 2020-04-01 09:14 2020-06-04 08:41
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
pax
3082, 3084
102809, 102881
---
pax atime and mtime keyword descriptions should refer to st_atim and st_mtim
When the standard was updated to support finegrained timestamps, a change to pax was missed where it refers to the st_atime and st_mtime members of the stat structure but should now refer to st_atim and st_mtim.

Since the text uses the word "member" and there are no stat structure members called st_atime and st_mtime, it is not possible to misinterpret the text as referring somehow to the st_atime and st_mtime macros, and so this should be treated as a minor editorial error.

(There is also a reference to an "st_mtime field" in the ar RATIONALE section, but I believe that one should not be changed as it is describing the historical 4.4BSD ar format, and the 4.4BSD stat structure had an st_mtime member, not an st_mtim member.)
On page 3082 line 102809 section pax, change:
the st_atime member
to:
the st_atim member

On page 3084 line 102881 section pax, change:
the st_mtime member
to:
the st_mtim member
There are no notes attached to this issue.



Viewing Issue Advanced Details
1330 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Editorial Enhancement Request 2020-03-31 15:53 2020-05-29 15:35
geoffclare
 
normal  
Applied  
Accepted