View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0001949 | 1003.1(2024)/Issue8 | System Interfaces | public | 2025-09-21 12:31 | 2025-09-21 16:04 |
| Reporter | alx | Assigned To | |||
| Priority | high | Severity | Objection | Type | Enhancement Request |
| Status | New | Resolution | Open | ||
| Name | Alejandro Colomar | ||||
| Organization | Linux man-pages project | ||||
| User Reference | |||||
| Section | realloc, posix_memalign, aligned_alloc, malloc, calloc | ||||
| Page Number | I don't know. | ||||
| Line Number | I don't know. | ||||
| Interp Status | |||||
| Final Accepted Text | |||||
| Summary | 0001949: Restore the traditional realloc(3) specification | ||||
| Description | Name alx-p0001r0 - Restore the traditional realloc(3) specification Category Remove UB. Author Signed-off-by: Alejandro Colomar <alx@kernel.org> Cc: <bug-gnulib@gnu.org> Cc: <musl@lists.openwall.com> Cc: <libc-alpha@sourceware.org> Cc: наб <nabijaczleweli@nabijaczleweli.xyz> Cc: Douglas McIlroy <douglas.mcilroy@dartmouth.edu> Cc: Paul Eggert <eggert@cs.ucla.edu> Cc: Robert Seacord <rcseacord@gmail.com> Cc: Elliott Hughes <enh@google.com> Cc: Bruno Haible <bruno@clisp.org> Cc: JeanHeyd Meneide <phdofthehouse@gmail.com> Cc: Rich Felker <dalias@libc.org> Cc: Adhemerval Zanella Netto <adhemerval.zanella@linaro.org> Cc: Joseph Myers <josmyers@redhat.com> Cc: Florian Weimer <fweimer@redhat.com> Cc: Andreas Schwab <schwab@suse.de> Cc: Thorsten Glaser <tg@mirbsd.de> Cc: Eric Blake <eblake@redhat.com> Cc: Vincent Lefevre <vincent@vinc17.net> Cc: Mark Harris <mark.hsj@gmail.com> Cc: Collin Funk <collin.funk1@gmail.com> Cc: Wilco Dijkstra <Wilco.Dijkstra@arm.com> Cc: DJ Delorie <dj@redhat.com> Cc: Cristian Rodríguez <cristian@rodriguez.im> Cc: Siddhesh Poyarekar <siddhesh@gotplt.org> Cc: Sam James <sam@gentoo.org> Cc: Mark Wielaard <mark@klomp.org> Cc: "Maciej W. Rozycki" <macro@redhat.com> Cc: Martin Uecker <ma.uecker@gmail.com> Cc: Christopher Bazley <chris.bazley.wg14@gmail.com> Cc: <eskil@obsession.se> Cc: Daniel Krügler <daniel.kruegler@googlemail.com> Cc: Kees Cook <keescook@chromium.org> Cc: Valdis Klētnieks <valdis.kletnieks@vt.edu> History <https://www.alejandro-colomar.es/src/alx/alx/std/posix/alx-p0001.git/> alx-0029r0 (2025-06-17): - Initial draft. alx-0029r1 (2025-06-20): - Full rewrite after the recent glibc discussion. alx-0029r2 (2025-06-21): - Remove CC. Add CC. - wfix. - Drop quote. - Add a few more principles - Clarify why ENOMEM is used in this proposal, and make it optional. - Mention exceptional leak in code checking (size != 0). - Clarify that part of the description of realloc can be editorially removed after this change. alx-0029r3 (2025-06-23): - Fix diff missing line. - Remove ENOMEM from the proposal. - Clarify that ENOMEM should be retained by platforms already using it. - Add mention that LLVM's address sanitizer will catch the leak mentioned in r2. - Add links to real bugs (including an RCE bug). alx-0029r4 (2025-06-24): - Use a better link for the Whatsapp RCE. - s/Description/Rationale/ - wfix - Mention that glibc <2.1.1 had the BSD behavior. - Add footnote that realloc(3) may fail while shrinking. alx-0029r5 (2025-06-26): - It was glibc 2.1.1 that broke it, not glibc 2.2. - wfix - Mention in the footnote that the pointer may change. - Document why not go the other way around. It was explained several times during discussion, but people keep suggesting it. alx-0029r6 (2025-06-27; n3621): - Clarify that the paragraph about what happens when the size is zero refers to when the total size is zero (for calloc(3) that is nmemb*size). - s/Unix V7/V7 Unix/ - tfix. - wfix. Brno meeting (2025-08-27): - 9/13/6 - Along the lines: 21/1/5 - People recognized in the dinner after the meeting, and in the reflector, and in corridor discussions, that they hadn't understood the paper, and that it was more well thought than they initially thought. They would change their vote to be in favour with this proposal. alx-0029r7 (2025-09-21): - Add link. alx-p0001r0 (2025-09-21): - Fork POSIX proposal from ISO C proposal. See also <https://www.alejandro-colomar.es/src/alx/alx/wg14/alx-0069.git/> <https://nabijaczleweli.xyz/content/blogn_t/017-malloc0.html> <https://sourceware.org/pipermail/libc-alpha/1999-April/000956.html> <https://inbox.sourceware.org/libc-alpha/nbyurzcgzgd5rdybbi4no2kw5grrc32k63svf7oq73nfcbus5r@77gry66kpqfr/> <https://inbox.sourceware.org/libc-alpha/20241019014002.3684656-1-siddhesh@sourceware.org/T/#u> <https://inbox.sourceware.org/libc-alpha/qukfe5yxycbl5v7ooskvqdnm3au3orohbx4babfltegi47iyly@or6dgf7akeqv/T/#u> <https://github.com/bminor/glibc/commit/7c2b945e1fd64e0a5a4dbd6ae6592a7314dcd4b5> <https://github.com/llvm/llvm-project/issues/113065> <https://www.austingroupbugs.net/view.php?id=400> <https://www.austingroupbugs.net/view.php?id=526> <https://www.austingroupbugs.net/view.php?id=688> <https://sourceware.org/bugzilla/show_bug.cgi?id=12547> <https://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_400.htm> <https://www.open-std.org/jtc1/sc22/wg14/www/docs/n868.htm> <https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2438.htm> <https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2464.pdf> <https://pubs.opengroup.org/onlinepubs/9699919799.2008edition/functions/realloc.html> <https://pubs.opengroup.org/onlinepubs/9699919799.2013edition/functions/realloc.html> <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=120744> <https://lore.kernel.org/lkml/20220213182443.4037039-1-keescook@chromium.org/> <https://awakened1712.github.io/hacking/hacking-whatsapp-gif-rce/> <https://gbhackers.com/whatsapp-double-free-vulnerability/> Rationale The specification of realloc(3) has been problematic since the very first standards, even before ISO C. The wording has changed significantly, trying to forcedly permit implementations to return a null pointer when the requested size is zero. This originated from the intent of banning zero-sized objects from the language in C89, but that never worked well in retrospective, as we can see from the fallout. None of the specifications have been good, and C23 finally gave up and made it undefined behavior. The problem is not only theoretical. Programmers don't know how to use realloc(3) correctly, and have written weird code in their attempts. This has resulted in a lot of non-sensical code in configure scripts[1], and even bugs in actual programs[2]. [1] <https://codesearch.debian.net/search?q=%5Cbrealloc%5B+%5Ct%5D*%5B%28%5D%5B%5E%2C%5D*%2C%5B+%5Ct%5D0%5B%29%5D&literal=0> [2] <https://lore.kernel.org/lkml/20220213182443.4037039-1-keescook@chromium.org/> In some cases, this non-sensical code has resulted in RCEs[3]. [3] <https://awakened1712.github.io/hacking/hacking-whatsapp-gif-rce/> However, this doesn't need to be like that. The traditional implementation of realloc(3), present in V7 Unix, inherited by the BSDs, and currently available in a range of systems, including musl libc, doesn't have any issues regarding zero-size allocations. glibc --which uses an independent implementation rather than a Unix derivative-- also had this behavior originally; it changed to the current behavior in 1999 (glibc 2.1.1), only for compatibility with C89, even though ironically C99 was released soon after and removed the text that glibc was trying to comply with, and introduced some new text that was very confusing, and one of its interpretations would make the new glibc behavior non-conforming. Code written for platforms returning a null pointer can be migrated to platforms returning non-null, without significant issues. There are two kinds of code that call realloc(p,0). One hard-codes the 0, and is used as a replacement of free(p). This code ignores the return value, since it's unimportant. This code currently produces a leak of 0 bytes plus associated metadata on platforms such as musl libc, where it returns a non-null pointer. However, assuming that there are programs written with the knowledge that they won't ever be run on such platforms, we should take care of that, and make sure they don't leak. A way of accomplishing this would be to recommend implementations to issue a diagnostic when realloc(3) is called with a hardcoded zero. This is only an informal recommendation made by this proposal, as this is a matter of QoI, and the standard shouldn't say anything about it. This would prevent this class of minor leaks. Moreover, in glibc, realloc(p,0) may return non-null, in the case where p is NULL, so code must already take that into account, and thus code that simply takes realloc(p,0) as a synonym of free(p) is already leaky, as free(NULL) is a no-op, but realloc(NULL,0) allocates 0 bytes. The other kind of code is in algorithms that realloc(3) an arbitrary size, which might eventually be zero. This gets more complex. Here's the code that should be written for AIX or glibc: errno = 0; new = realloc(old, size); if (new == NULL) { if (errno == ENOMEM) free(old); goto fail; } ... free(new); Failing to check for ENOMEM in these platforms before freeing the old pointer would result in a double-free. If the program decides to continue using the old pointer instead of freeing it, it would result in a use-after-free. In the platforms where realloc(p,0) returns non-null, such as the BSDs or musl libc, it is simpler to handle it: new = realloc(old, size); if (new == NULL) { // errno is ENOMEM free(old); goto fail; } ... free(new); Whenever the result is a null pointer, these platforms are reporting an ENOMEM error, and thus it is superfluous to check errno there. Most code is written in this way, even if run on platforms returning a null pointer. This is because most programmers are just unaware of this problem. Part of the reason is also that returning a non-null pointer with zero bytes is the natural extension of the behavior, which is what programmers intuitively expect from libc; that is, if realloc(p,3) allocates 3 bytes, r(p,2) allocates two bytes, and r(p,1) allocates one byte, it is natural by induction to expect that r(p,0) will allocate zero bytes. Most algorithms naturally extend to 0 just fine, and special casing 0 is artificial. If the realloc(3) specification were changed to require that realloc(p,0) returns non-null on success, and that realloc(p,0) only fails when out-of-memory (and assuming the implementations will continue setting errno to ENOMEM), then code written for AIX or glibc would continue working just fine, since the errno check would be redundant with the null check. Simply, the conditional (errno == ENOMEM) would always be true when (new == NULL). Then, there are non-POSIX platforms that don't set ENOMEM. In those platforms, code might do this: new = realloc(old, size); if (new == NULL) { if (size != 0) free(old); goto fail; } ... free(new); That code would continue working with this proposal, except for a very rare corner case, in which it would leak. In the normal case, (size != 0) would never be true under (new == NULL), because a reallocation of 0 bytes would almost always succeed, and thus not return a null pointer under this proposal. However, in some cases, the system might not find space even for the small metadata needed for a 0-byte allocation. In such case, the (size != 0) conditional would prevent deallocating 'old', and thus cause a memory leak. This case is exceptional enough that it shouldn't stop us from fixing realloc(3). Anyway, on an out-of-memory case, the program is likely to terminate rather soon, so the issue is even less likely to have an impact on any existing programs. Also, LLVM's address sanitizer will soon able to catch such a leak: <https://github.com/llvm/llvm-project/issues/113065> This proposal makes handling of realloc(3) as straightforward as one would expect, with only two states: success or error. There are no in-between states. The resulting wording in the standard is also much simpler, as it doesn't need to define so many special cases. For consistency, all the other allocation functions are updated to both return a null pointer on error, and use consistent wording. Why not go the other way around? Some people keep asking why not go the other way around: why not force the BSDs and musl to return a null pointer if size is 0. This would result in double-free and use-after-free bugs, which can result in RCE vulnerabilities (remote code execution), which is clearly unacceptable. Consider this code, which is the usual code for calling realloc(3) in such systems: new = realloc(old, size); if (new == NULL) { free(old); goto fail; } ... free(new); If realloc(p,0) would return a null pointer and free the old block, then the third line would be a double-free bug. Prior art gnulib gnulib provides the realloc-posix module, which aims to wrap the system realloc(3) and reallocarray(3) functions so that they behave in a POSIX-complying manner. It previously behaved like glibc. After I reported that it was non-conforming to POSIX, we discussed the best way forward, which we agreed was the same direction that this paper is proposing now for C2y. The implementation was changed in gnulib.git d884e6fc4a60 (2024-11-04; "realloc-posix: realloc (..., 0) now returns nonnull") There have been no regression reports since then, as we expected. V7 Unix, BSD The proposed behavior is the one endorsed by Doug McIlroy, the author of the original implementation of realloc(3) in V7 Unix, and also present in the BSDs. glibc <= 2.1 glibc was implemented originally to return non-null. It was only in 1999, and purely to comply with the standards --with no requests by users to do so--, that the glibc maintainers decided to switch to the current behavior. Design decisions This change needs two changes, which can be applied all at once, or in separate steps. The first step would make realloc(p,s) be consistent with free(p) and malloc(s), including when p is a null pointer, when s is zero, and also when both corner cases happen at the same time. This change would already turn the implementations where malloc(0) returns non-null into the end goal we have. This would require changes to (at least) the following implementations: glibc, Bionic, Windows. The second step would be to require that malloc(0) returns a non-null pointer. This would require changes to (at least) the following implementations: AIX. This proposal has merged all steps into a single proposal. Caveats n?n:1 Code written in the near future should be careful, in case it can run on older systems that are not fixed to comply with this stricter specification. Thus, code written in the near future should call realloc(3) similar to this: realloc(p, n?n:1); When all existing implementations are fixed to comply with this stricter specification, that workaround can be removed. ENOMEM Existing implementations that set errno to ENOMEM must continue doing so when the input pointer is not freed. If they didn't, code that is currently portable to all POSIX systems errno = 0; new = realloc(old, size); if (new == NULL) { if (errno == ENOMEM) free(old); goto fail; } ... free(new); would leak on error. Since it is currently impossible to write code today that is portable to arbitrary C17 systems, this is not an issue in ISO C. - New code written for C2y will only need to check for NULL to detect errors. - Code written for specific C17 and older platforms that don't set errno will continue to work for those specific platforms. - Code written for POSIX.1-2024 and older platforms will continue working on POSIX C2y platforms, assuming that POSIX will continue mandating ENOMEM. - Code written for POSIX.1-2024 and older will not be able to be run on non-POSIX C2y platforms, but that could be expected. The only important thing is that platforms that did set ENOMEM should continue setting it, to avoid introducing leaks. | ||||
| Desired Action | Proposed wording Based on Issue 8. posix_memalign() @@ DESCRIPTION, p3 If the size of the space requested is 0, -the behavior is implementation-defined: -either a null pointer shall be returned in memptr, -or the behavior shall be as if the size were some non-zero value, except that the behavior is undefined if the value returned in memptr is used to access an object. @@ RETURN VALUE, p2 If size is 0, -either: - * - posix_memalign() shall not attempt to allocate any space, - in which case - either an implementation-defined error number - shall be returned, - or zero shall be returned - with a null pointer returned in memptr, - or - * posix_memalign() shall attempt to allocate some space and, if the allocation succeeds, zero shall be returned and a pointer to the allocated space shall be returned in memptr. The application shall ensure that the pointer is not used to access an object. aligned_alloc() @@ DESCRIPTION, p3 ... If the size of the space requested is 0, -the behavior is implementation-defined: -either a null pointer shall be returned to indicate an error, -or the behavior shall be as if the size were some non-zero value, except that the behavior is undefined if the returned pointer is used to access an object. @@ ERRORS, p2 -The aligned_alloc() function may fail if: - -[EINVAL] - [CX] size is 0 and - the implementation does not support 0 sized allocations. calloc() @@ DESCRIPTION, p3 ... If the size of the space requested is 0, -the behavior is implementation-defined: -either a null pointer shall be returned to indicate an error, -or the behavior shall be as if the size were some non-zero value, except that the behavior is undefined if the returned pointer is used to access an object. @@ ERRORS, p2 -The calloc() function may fail if: - -[EINVAL] - [CX] nelem or elsize is 0 and - the implementation does not support 0 sized allocations. malloc() @@ DESCRIPTION, p3 ... If the size of the space requested is 0, -the behavior is implementation-defined: -either a null pointer shall be returned to indicate an error, -or the behavior shall be as if the size were some non-zero value, except that the behavior is undefined if the returned pointer is used to access an object. @@ ERRORS, p2 -The malloc() function may fail if: - -[EINVAL] - [CX] size is 0 and - the implementation does not support 0 sized allocations. realloc() @@ DESCRIPTION, p2 The realloc() function -shall deallocate the old object pointed to by ptr +shall deallocate the old object pointed to by ptr, +as if by a call to free(), and return a pointer to a new object -that has the size specified by size +that has the size specified by size, +as if by a call to malloc(). The contents of the new object shall be the same as that of the old object prior to deallocation, up to the lesser of the new and old sizes. Any bytes in the new object beyond the size of the old object have indeterminate values. @@ DESCRIPTION, p4 -If ptr is a null pointer, -realloc() [CX] or reallocarray() [/CX] -shall be equivalent to malloc() for the specified size. -Otherwise, -if ptr does not match a pointer earlier returned by -aligned_alloc(), calloc(), malloc(), [ADV] posix_memalign(), [/ADV] -realloc(), [CX] reallocarray(), -or a function in POSIX.1-2024 -that allocates memory as if by malloc(), [/CX] -or if the space has been deallocated by a call to -free(), [CX] reallocarray(), [/CX] or realloc(), -the behavior is undefined. ## We can remove the above text, due to the introduced "as if" ## text that defers to the specifications of free() and malloc() ## in paragraph 2 above. @@ RETURN VALUE, p2 -[OB] If size is 0, [/OB] -[OB CX] or either nelem or elsize is 0, [/OB CX] -either: - [OB] * A null pointer shall be returned [/OB] - [OB CX] and, - if ptr is not a null pointer, - errno shall be set to [EINVAL]. [/OB CX] - [OB] * A pointer to the allocated space shall be returned, - and the memory object pointed to by ptr shall be freed. - The application shall ensure - that the pointer is not used to access an object. [/OB] ## We could keep the second part of the paragraph above, which ## is the behavior mandated by this proposal. However, the ## "as if" rule that defers to malloc() already mandates this. @@ ERRORS, p3 -The realloc() and reallocarray() functions may fail if: - -[EINVAL] - [CX] The requested allocation size is 0 and - the implementation does not support 0 sized allocations. @@ APPLICATION USAGE, p1 -The ISO C standard makes it implementation-defined -whether a call to realloc(p, 0) -frees the space pointed to by p -if it returns a null pointer -because memory for the new object was not allocated. -POSIX.1 instead requires that implementations set errno -if a null pointer is returned -and the space has not been freed, -and POSIX applications should only free the space if errno was changed. +POSIX does not treat realloc(p, 0) specially. +An example of correct use of realloc() is as follows: + + void *new; + + new = realloc(p, n); // n might be 0. + if (new == NULL) { + free(p); + exit(EXIT_FAILURE); + } + p = new; @@ FUTURE DIRECTIONS, p1 -The ISO C standard states that -invoking realloc() with a size argument equal to zero -is an obsolescent feature. -This feature may be removed in a future version of this standard. ## ISO C should fix realloc(p,0) in the same way, as proposed in ## <https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3621.txt>. | ||||
| Tags | No tags attached. | ||||
