View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000400 | 1003.1(2008)/Issue 7 | System Interfaces | public | 2011-03-24 16:37 | 2014-04-10 16:00 |
| Reporter | eblake | Assigned To | ajosey | ||
| Priority | normal | Severity | Objection | Type | Error |
| Status | Closed | Resolution | Accepted As Marked | ||
| Name | Eric Blake | ||||
| Organization | Red Hat | ||||
| User Reference | ebb.realloc | ||||
| Section | realloc | ||||
| Page Number | 1754 | ||||
| Line Number | 56027 | ||||
| Interp Status | Approved | ||||
| Final Accepted Text | 0000400:0000772 | ||||
| Summary | 0000400: realloc wording conflicts with C99 | ||||
| Description | The C committee considered three possibilities: 1. Do nothing and accept the fact that some C99 implementations may not be able to conform to the requirements of the C99 standard in this regard for compatibility reasons. 2. Change C to more closely align with the POSIX requirements so as to make the existing implementatins conforming. This resolution is being sought by the POSIX group and is also preferred by the implementers of the affected implementations. 3. Change both the C and POSIX standards to require realloc() to always attempt to allocate space even for zero-size requests. This would simplify the specification and, at least in theory, also make it easier to write portable programs. This option would require the support of the implementers of the affected implementations. Without such support this option becomes equivalent to option 1 above. The outcome of this discussion is that if realloc returns NULL, it has failed, and the original memory has not been freed. The POSIX words came from C90, and these were changed for C99. POSIX failed to pick up the change See N872 item 19c. Unfortunately this breaks several existing implementations. However, those implementer present (Sun/Oracle, Apple, IBM, HP) all agreed that they were prepared to change their implementations. NOTE: Since the text at the top of realloc() states: The functionality described on this reference page is aligned with the ISO C standard. Any conflict between the requirements described here and the ISO C standard is unintentional. This volume of POSIX.1-2008 defers to the ISO C standard. I am not sure we have the wiggle room to do anything but DEFECT situation. | ||||
| Desired Action | POSIX Change requested by C99: Page 1754 line 56030 Change: The realloc() function shall change the size of the memory object pointed to by ptr to the size specified by size. The contents of the object shall remain unchanged up to the lesser of the new and old sizes. If the new size of the memory object would require movement of the object, the space for the previous instantiation of the object is freed. If the new size is larger, the contents of the newly allocated portion of the object are unspecified. If size is 0 and ptr is not a null pointer, the object pointed to is freed. to The realloc() function shall deallocate the old object pointed to by ptr and return a pointer to a new object that has the size specified by size. The contents of the new object shall be the same as that of the old object prior to deallocation, up to the lesser of the new and old sizes. Any bytes in the new object beyond the size of the old object have indeterminate values. Page 1754 line 56046 change Upon successful completion with a size not equal to 0, realloc() shall return a pointer to the (possibly moved) allocated space. to Upon successful completion, realloc() shall return a pointer to the (possibly moved) allocated space. At line 56047-8 change: If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() shall be returned. to If size is 0, either: * a null pointer shall be returned <CX>and errno set to EINVAL</CX>, or * unique pointer that can be successfully passed to free() shall be returned, and the memory object pointed to by ptr shall be freed. Add after line 56049: If realloc() returns NULL, the memory referenced by ptr is not changed. Add after 56052: The realloc() function may fail if: EINVAL size is 0 and the implementation does not support 0 sized allocations. | ||||
| Tags | c99, tc1-2008 | ||||
| parent of | 0000374 | Closed | ajosey | 1003.1(2008)/Issue 7 | malloc(0) and realloc(p,0) must not change errno on success |
| parent of | 0000524 | Closed | ajosey | 2008-TC1 | The 2nd of 2 'If size is 0, either:' option was split into 2 options. |
| related to | 0000526 | Closed | ajosey | 1003.1(2008)/Issue 7 | Adopt C99 wording for zero size calloc(), malloc() et al. |
| related to | 0000688 | Closed | 1003.1(2013)/Issue7+TC1 | realloc(NULL, 0) vs. malloc(0) |
|
|
glibc is impacted by the outcome of this bug: http://sourceware.org/bugzilla/show_bug.cgi?id=12547 |
|
|
The line 56049 addition should be worded: If realloc() returns a null pointer, the memory referenced by ptr shall not be changed. I think the EINVAL error should be "shall fail", not "may fail", because although the behaviour of returning an EINVAL error for size 0 is optional, the implementation choice is stated in the error description: "... and the implementation does not support 0 sized allocations." |
|
|
A fourth possibility not covered by the C99's recommendations: Change line 56034 [realloc DESCRIPTION]: If size is 0 and ptr is not a null pointer, the object pointed to is freed. to: <CX>If size is 0 and ptr is not a null pointer, behavior is unspecified; portable applications shall ensure that a size of 0 is only used when ptr is a null pointer.</CX> Change line 56047 [RETURN VALUE]: If size is 0, to: If size is 0 and ptr is a null pointer, Change line 56058 [RATIONALE]: None. to two paragraphs: The C99 standard requires that for a non-null pointer ptr, realloc(ptr,0) will either return a zero-size allocation (as if by malloc(0)) or leave ptr unchanged (because a NULL return indicates failure). However, there are existing implementations that take a third approach of freeing ptr while still returning NULL. While these implementations do not comply with C99, they are still permitted by this version of POSIX. An application that assumes C99 semantics by calling free(ptr) after a NULL return from realloc(ptr,0) failure will end up with undefined behavior of freeing an already-freed pointer. Therefore, this standard completely avoids the portability issues by instead requiring that applications never use realloc with a non-null pointer but zero size. This restriction does not represent a conflict with the ISO C standard. An implementation may choose whether it wishes to comply to both C99 and POSIX (use of size 0 will have some guarantees per C99, although a conforming POSIX application will not trigger those guarantees), or just POSIX. Note that while there are several places in the standard where <CX> provides tighter implementation constraints (not all C99 implementations comply with POSIX), I think this would be the first case of the converse (a case where <CX> provides looser implementation constraints, that is, where a POSIX-compliant implementation need not obey C99. On the other hand, where this standard is tighter than C99, it implies that an application that conforms to this standard will not necessarily behave the same way in all other C99 environments; whereas this requirement to avoid the unspecified behavior of the looser implementation guarantees that an application written to this standard will never trigger portability problems whether run on a C99 environment or a legacy environment. Given the extreme reluctance to change existing implementations, this may be the only solution for POSIX to continue to cater to implementations such as glibc, while still permitting implementations that obey C99. |
|
|
Note also that this standard has already had a track record of marking some C99 requirements as obsolescent and not to be relied on by portable programs, such as gets(), which gives more precedence to the notion of declaring realloc(ptr,0) as <OB>. |
|
|
Sent to the SC22WG14 email reflector: The Austin Group has been studying this problem (see http://austingroupbugs.net/view.php?id=400) and trying to resolve any conflict with C. This was also examined by WG14 during the London meeting, where the resolution was "POSIX should conform to the requirements of C". The Austin Group agrees with this resolution. However, during our discussions, we have found the following in the C standard (using refs to C99, not C1x): Point 1: Objects cannot be zero sized. 6.2.6.1(2) Except for bit-fields, objects are composed of contiguous sequences of one or more bytes, Point 2: realloc(NULL, 0) is required to behave like malloc(0), and malloc states "7.20.3.3(2) The malloc function allocates space for an object whose size is specified by size and whose value is indeterminate." Since objects cannot be zero sized, returning a non-null pointer is at best questionable. However, this argues that realloc(p, 0) should always fail. Point 3: Explicit permission to succeed AND return NULL is granted: "7.20.3(1) If the size of the space requested is zero, the behavior is implementation- defined: either a null pointer is returned, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object." [this refers to calloc, malloc and realloc]. It has been claimed that if realloc(p,0) returns a null pointer, that indicates that "the new object cannot be allocated" and so the old object remains alive. However, that interpretation is not clear, and the two separate sentences in 7.20.3.1 about null pointers are distinct cases, and the second does not necessarily imply the first. The POSIX words currently follow the original C89 text, and not the C99 update. They state: "If size is 0 and ptr is not a null pointer, the object pointed to is freed. ...RETURNS... If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() shall be returned." The C99 words state "7.20.3.4(2) The realloc function deallocates the old object pointed to by ptr and returns a pointer to a new object that has the size specified by size." This would appear to allow an implementation to deallocate the space and then return NULL (without setting errno). Additionally, "7.20.3(3) ... or if the space has been deallocated by a call to the free or realloc function..." strongly suggests that realloc(ptr, 0) is a valid way to free ptr. So, the Austin Group currently does not believe that POSIX conflicts with C, and that it is permissible for realloc(p, 0) (where p is non-NULL) to free the space allocated and return NULL. In POSIX, it is acceptable to set errno to zero before the call, and test it afterwards to see if the pointer has been freed or not. Our current survey has shown that the following implementations free the pointer and return NULL : Glibc (GNU/Linux) AIX HP-UX Solaris OSF/1 There is strong resistance from at least one of these to changing their implementation, in the belief that this would make current, memory leak free, conforming code become both non-conforming and leaking. -- Nick Stoughton Austin Group/WG14 Liaison |
|
|
Reply on SC22WG14 reflector: What WG 14 has been saying is that that interpretation is incorrect. Referencing the latest C1x draft: 1. 7.22.3.5p4, "Returns," states "The realloc function returns a pointer to the new object (which may have the same value as a pointer to the old object), or a null pointer if the new object could not be allocated." In other words, null means failure. 2. 7.22.3p1 states "If the space cannot be allocated, a null pointer is returned. If the size of the space requested is zero, the behavior is implementation-defined: either a null pointer is returned, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object." This, together with the previous quote, means that the function can either succeed and return a valid pointer, or fail and return a null pointer. 3. 7.22.3.5p1 states "If memory for the new object cannot be allocated, the old object is not deallocated and its value is unchanged." This, together with the previous two quotes, means that if a null pointer is returned, the operation has failed and the old object is not deallocated. You mentioned the fact that objects cannot be zero sized. That is why the wording is the way it is. If we had zero-sized objects, we could just say that an allocation of size zero returns a pointer to just past the end of a zero-sized object (and that is why the pointer cannot be dereferenced). Instead, we have to say the same thing without talking about zero-sized objects. |
|
|
Current proposed wording: Page 1754 line 56030 Change: The realloc() function shall change the size of the memory object pointed to by ptr to the size specified by size. The contents of the object shall remain unchanged up to the lesser of the new and old sizes. If the new size of the memory object would require movement of the object, the space for the previous instantiation of the object is freed. If the new size is larger, the contents of the newly allocated portion of the object are unspecified. If size is 0 and ptr is not a null pointer, the object pointed to is freed. to The realloc() function shall deallocate the old object pointed to by ptr and return a pointer to a new object that has the size specified by size. The contents of the new object shall be the same as that of the old object prior to deallocation, up to the lesser of the new and old sizes. Any bytes in the new object beyond the size of the old object have indeterminate values. If the size of the space requested is zero, the behavior shall be implementation-defined: either a null pointer is returned, or the behavior shall be as if the size were some nonzero value, except that the returned pointer shall not be used to access an object. Page 1754 line 56046 change Upon successful completion with a size not equal to 0, realloc() shall return a pointer to the (possibly moved) allocated space. to Upon successful completion, realloc() shall return a pointer to the (possibly moved) allocated space. At line 56047-8 change: If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() shall be returned. to If size is 0, either: * a null pointer shall be returned <CX>and errno set to an implementation defined value</CX>, or * unique pointer that can be successfully passed to free() shall be returned, and the memory object pointed to by ptr shall be freed. The application shall ensure that the pointer is not used to access an object. Add after line 56049: If realloc() returns NULL <CX>and errno has been set to a ENOMEM</CX>, the memory referenced by ptr shall not be changed. At line 56056 (Application Usage), change: None. to The description of realloc() has been modified from previous versions of this standard to align with C99. Previous versions explicitly permitted a call to realloc(p, 0) to free the space pointed to by p and return NULL. While this behavior could be interpreted as permitted by this version of the standard, the C language committee have indicated that this interpretation is incorrect. Applications should assume that if realloc returns a null pointer, the space pointed to be p has not been freed. Since this could lead to double-frees, implementations should also set errno if a null pointer actually indicates a failure, and applications should only free the space if errno was changed. Change at line 56060 (Future Directions) from: None. to This standard defers to the C standard. While that standard currently has language that might permit realloc(p, 0), where p is not a null pointer, to free p while still returning a null pointer, the committee responsible for that standard is considering clarifying the language to explicitly prohibit that alternative. |
|
|
Interpretation response ------------------------ The standard states its realloc specification , and conforming implementations must conform to this. However, concerns have been raised about this which are being referred to the sponsor. Rationale: ------------- The C standards WG has given us guidance on this issue. Notes to the Editor (not part of this interpretation): ------------------------------------------------------- Take the actions as described in 0000400:0000746 |
|
|
Interpretation proposed 16 June 2011 for final 30 day review |
|
|
The interpretation is now approved. |
|
|
http://open-std.org/JTC1/SC22/WG14/www/docs/dr_400.htm |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2011-03-24 16:37 | eblake | New Issue | |
| 2011-03-24 16:37 | eblake | Status | New => Under Review |
| 2011-03-24 16:37 | eblake | Assigned To | => ajosey |
| 2011-03-24 16:37 | eblake | Name | => Eric Blake |
| 2011-03-24 16:37 | eblake | Organization | => Red Hat |
| 2011-03-24 16:37 | eblake | User Reference | => ebb.realloc |
| 2011-03-24 16:37 | eblake | Section | => realloc |
| 2011-03-24 16:37 | eblake | Page Number | => 1754 |
| 2011-03-24 16:37 | eblake | Line Number | => 56027 |
| 2011-03-24 16:37 | eblake | Interp Status | => --- |
| 2011-03-24 16:38 | eblake | Relationship added | parent of 0000374 |
| 2011-03-24 16:38 | eblake | Tag Attached: c99 | |
| 2011-03-24 16:38 | eblake | Tag Attached: tc1-2008 | |
| 2011-03-24 17:08 | eblake | Note Added: 0000723 | |
| 2011-03-25 09:48 | geoffclare | Note Added: 0000725 | |
| 2011-03-25 20:22 | eblake | Note Added: 0000726 | |
| 2011-03-25 20:26 | eblake | Note Added: 0000727 | |
| 2011-04-01 22:29 | nick | Note Added: 0000734 | |
| 2011-04-01 22:30 | nick | Note Added: 0000735 | |
| 2011-04-21 15:25 | nick | Note Added: 0000746 | |
| 2011-04-28 15:17 | nick | Note Edited: 0000746 | |
| 2011-04-28 15:17 | nick | Note Edited: 0000746 | |
| 2011-04-28 15:25 | msbrown | Interp Status | --- => Pending |
| 2011-04-28 15:25 | msbrown | Note Added: 0000772 | |
| 2011-04-28 15:25 | msbrown | Status | Under Review => Interpretation Required |
| 2011-04-28 15:25 | msbrown | Resolution | Open => Accepted As Marked |
| 2011-04-28 15:26 | msbrown | Final Accepted Text | => 0000400:0000746 |
| 2011-04-28 15:28 | msbrown | Final Accepted Text | 0000400:0000746 => 0000400:0000772 |
| 2011-06-16 10:16 | ajosey | Interp Status | Pending => Proposed |
| 2011-06-16 10:16 | ajosey | Note Added: 0000845 | |
| 2011-07-29 06:14 | ajosey | Interp Status | Proposed => Approved |
| 2011-07-29 06:14 | ajosey | Note Added: 0000907 | |
| 2011-12-06 16:12 | nick | Relationship added | parent of 0000524 |
| 2011-12-12 10:27 | geoffclare | Relationship added | related to 0000526 |
| 2013-04-16 13:06 | ajosey | Status | Interpretation Required => Closed |
| 2013-05-09 15:56 | eblake | Relationship added | related to 0000688 |
| 2014-04-10 16:00 | eblake | Note Added: 0002223 |
