Austin Group Defect Tracker

Aardvark Mark IV

Main | My View | View Issues | Change Log | Docs
  

Viewing Issue Simple Details [ Jump to Notes ] [ Issue History ] [ Print ]
ID Category Severity Type Date Submitted Last Update
0000662 [1003.1(2008)/Issue 7] System Interfaces Editorial Enhancement Request 2013-02-21 01:20 2022-08-18 16:14
Reporter dalias View Status public  
Assigned To ajosey
Priority normal Resolution Rejected  
Status Closed  
Name Rich Felker
Organization musl libc
User Reference
Section freopen
Page Number unknown
Line Number unknown
Interp Status ---
Final Accepted Text
Summary 0000662: Clarify or add file descriptor preservation and atomicity requirements for freopen
Description freopen is often recommended (including in the application usage text) for replacing the standard streams stdin/stdout/stderr:

"The freopen() function is typically used to attach the pre-opened streams associated with stdin, stdout, and stderr to other files."

However, there are two serious issues with such usage:

1. It's not clear that the new file descriptor needs to have the same value as the original one. This makes freopen useless for use with the standard streams whenever their standard file descriptor values might be used to access them, which is the case whenever an exec-family function or posix_spawn is used after having called freopen.

2. Even if the same file descriptor would be obtained for the newly opened file, there is no requirement that freopen replace it atomically; if freopen operates internally according to the description of its behavior on the abstract machine, then there would be a window after the old file descriptor is closed but before the new one is opened, during which another thread could race for the file descriptor.

I realize that freopen's specification is aligned with ISO C; however, ISO C does not have file descriptors to worry about, and up through C99, did not have threads/concurrency to worry about either. In order to make the recommended usage of freopen with the standard streams safe and practical, I believe these issues should be fixed.
Desired Action Add text requiring that, upon successful return from freopen, fileno(stream) be equal to its value before the call to freopen. I believe this requirement, along with the requirement of thread-safety, implies that the race condition (problem 2 above) is not allowed. If necessary, clarifying text that freopen is required to operate as if by first attempting to open the new file and then performing dup2 could be added.
Tags No tags attached.
Attached Files

- Relationships
related to 0000480Closedajosey freopen, fclose effect on multiple file handles 
related to 0000411Closedajosey adding atomic FD_CLOEXEC support 
related to 0000555Closedajosey the incestuous relationship of fclose(stderr) and STDERR_FILENO 

-  Notes
(0001466)
lacos (reporter)
2013-02-21 04:36

 (I hope Mantis won't re-flow my text.)

> 1. It's not clear that the new file descriptor needs to have the same
> value as the original one. This makes freopen useless for use with the
> standard streams whenever their standard file descriptor values might
> be used to access them, which is the case whenever an exec-family
> function or posix_spawn is used after having called freopen.

In this case the application (not the C library) should follow the rules
in "Interaction of File Descriptors and Standard I/O Streams"
<http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_05_01> [^]
to get the stream in question to a clean, well defined state, open() the
new file, and dup2() it over to the stream's fileno().

From that point on the file descriptor originally returned by open(),
the original fileno() of the stream, and the "retargeted" stdio stream
itself are all "handles" on the same open file description, hence
subject to the rules in "Interaction of File Descriptors and Standard
I/O Streams".

> Even if the same file descriptor would be obtained for the newly
> opened file, there is no requirement that freopen replace it
> atomically; if freopen operates internally according to the
> description of its behavior on the abstract machine, then there would
> be a window after the old file descriptor is closed but before the new
> one is opened, during which another thread could race for the file
> descriptor.

This windows is explicitly allowed / mandated by the specification, see
line 30884 (or
<http://pubs.opengroup.org/onlinepubs/9699919799/functions/freopen.html>): [^]

    The original stream shall be closed regardless of whether the
    subsequent open succeeds.
(0001467)
geoffclare (manager)
2013-02-21 10:27

 The standard is clear that implementations are actually not allowed to
do what the desired action is asking for, and the requested change
would make all existing implementations non-conforming.

If an application calls:

close(1);
freopen(file, "w", stderr);

then the standard requires that after these calls, fileno(stderr) is 1.
This is because "the file descriptor associated with the reopened
stream shall be allocated and opened as if by a call to open()", and
this includes the requirement that open() returns the lowest available
file descriptor.

Perhaps we should add some application usage warning about this, and
also advising that multithreaded processes should use dup2() instead.
(0001469)
eblake (manager)
2013-02-21 18:08

 In 0000555, we argued that an application calling close(1) followed by freopen() is non-compliant.
(0001471)
dalias (reporter)
2013-02-21 18:43

 In reply to note #0001466, I agree that the issue of the file descriptor being reassigned for another use applies when freopen fails. However, as long as the application ensures that the stream is never used after freopen fails (for example, immediately calling exit or _exit after freopen returns NULL), there is little or no inherent danger in another thread obtaining the same file descriptor number for some other use. (As an example, think of a thread performing an asynchronous DNS lookup that starts before the main thread's initialization and redirection of standard streams is performed.) The interesting case is when freopen succeeds, and in that case, it's compatible with the requirements of the standard for an implementation to ensure that there is no race window (e.g. by using dup2). What I'm suggesting is that this preferred behavior should not only be compatible with the standard, but required by the standard, as it's essential for safe multi-threaded use.
(0001473)
lacos (reporter)
2013-02-22 03:40

 In reply to note #0001471: if I understand correctly, code like the following is the concern:

if (0 != freopen("pathname", "a", stdout)) {
  use_fd(STDOUT_FILENO);
}

as in, fd STDOUT_FILENO (==1) may no longer underlie stdout, even though the reopen succeeded. For that case I would suggest

if (0 != freopen("pathname", "a", stdout)) {
  int fd;

  fd = fileno(stdout);
  assert(-1 != fd);
  use_fd(fd);
}
(0001474)
dalias (reporter)
2013-02-22 04:49

 The solution proposed in note #0001473 does not work when part of using the fd involves executing an external program that uses the standard file descriptors, for example using the system() function. With direct use of fork and exec or posix_spawn, it will be possible in many cases to perform the necessary remappings manually, but I believe there is already an interpretation stating that it's invalid to close (or dup2 or open over top of) file descriptor numbers that the application itself did not obtain as part of the spawn file actions, and that attempts to do so may interfere with internal usage of file descriptors by the implementation.
(0001475)
geoffclare (manager)
2013-02-22 09:55

 (Response to Note: 0001469)
I don't see anything in 0000555 that says close(1) followed by
freopen() is non-compliant. The resolution (Note: 0001240) says that
it is not atomic and advises the use of dup2() instead, but that does
not contradict my assertion that if an application calls close(1)
followed by freopen(file, "w", stderr) then the standard requires
fileno(stderr) to be 1. Note that by "calls" I mean that those
are the actual calls it makes, not that those calls are present
in that sequence in the source code (i.e. the requirement is
only true provided there is no fd manipulation done by other
threads or by a signal handler in between).
(0001477)
dalias (reporter)
2013-02-22 13:18

 If the only issue is what happens when the fd is already closed, I would have no problem conditioning the "same fd" requirement on there presently being a valid file descriptor associated with the stream. However, being that fclose(stream) and freopen(stream) are already specified to close the original file descriptor, it would be a serious programming error if there is the possibility that the file descriptor originally belonging to the stream was subsequently reassigned for other use after the original file descriptor was manually closed. Thus, I'm skeptical of this whole matter.
(0005935)
geoffclare (manager)
2022-08-18 16:10
edited on: 2022-08-18 16:11

 Not sure why I didn't reply to Note: 0001477 back in 2013, but it is conditional on "if the only issue is what happens when the fd is already closed", which is false. If there is a valid fd underlying the stream but a lower fd is available, freopen() is required to close the current fd, and then open the lower one and associate it with the stream.

- Issue History
Date Modified Username Field Change
2013-02-21 01:20 dalias New Issue
2013-02-21 01:20 dalias Status New => Under Review
2013-02-21 01:20 dalias Assigned To => ajosey
2013-02-21 01:20 dalias Name => Rich Felker
2013-02-21 01:20 dalias Organization => musl libc
2013-02-21 01:20 dalias Section => freopen
2013-02-21 01:20 dalias Page Number => unknown
2013-02-21 01:20 dalias Line Number => unknown
2013-02-21 04:36 lacos Note Added: 0001466
2013-02-21 10:27 geoffclare Note Added: 0001467
2013-02-21 18:04 eblake Relationship added related to 0000480
2013-02-21 18:06 eblake Relationship added related to 0000411
2013-02-21 18:08 eblake Note Added: 0001469
2013-02-21 18:09 eblake Relationship added related to 0000555
2013-02-21 18:43 dalias Note Added: 0001471
2013-02-22 03:40 lacos Note Added: 0001473
2013-02-22 04:49 dalias Note Added: 0001474
2013-02-22 09:55 geoffclare Note Added: 0001475
2013-02-22 13:18 dalias Note Added: 0001477
2022-08-18 16:10 geoffclare Note Added: 0005935
2022-08-18 16:11 geoffclare Note Edited: 0005935
2022-08-18 16:14 geoffclare Interp Status => ---
2022-08-18 16:14 geoffclare Status Under Review => Closed
2022-08-18 16:14 geoffclare Resolution Open => Rejected

Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker