Austin Group Defect Tracker

Aardvark Mark IV


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0001797 [1003.1(2024)/Issue8] System Interfaces Objection Error 2024-01-15 23:56 2024-10-11 11:03
Reporter eggert View Status public  
Assigned To
Priority normal Resolution Accepted As Marked  
Status Interpretation Required  
Name Paul Eggert
Organization UCLA Computer Science Dept.
User Reference strftime-%s
Section strftime
Page Number 2136
Line Number 69836-69837
Interp Status Proposed
Final Accepted Text Note: 0006908
Summary 0001797: strftime "%s" should be able to examine tm_gmtoff
Description strftime’s %s conversion specification is intended to be the inverse of localtime. That is, if strftime(buf, sizeof buf, "%s", localtime(&t)) succeeds, buf should contain the decimal representation of t.

Unfortunately when TZ is set to a geographic location, this does not work on many POSIX platforms in some unusual cases. It has even been argued that POSIX 202x/D4 requires cases like these to not work. If that argument is correct, this is a regression from POSIX 2017 and should be corrected. This correction can be made without invalidating existing POSIX platforms.

Here is an example of the problem, set in Caracas:

  #include <stdio.h>
  #include <stdlib.h>
  #include <time.h>

  int
  main ()
  {
    char buf[100];
    time_t t = 1197184500;
    setenv ("TZ", "America/Caracas", 1);
    strftime (buf, sizeof buf, "%s = %Y-%m-%d %H:%M:%S %z (%Z) in Caracas",
          localtime (&t));
    printf ("%lld prints as %s\n", (long long) t, buf);
    if (atoll (buf) != t)
      puts ("time_t mismatch!");
  }

On Ubuntu 23.10, this outputs:

  1197184500 prints as 1197182700 = 2007-12-09 02:45:00 -0430 (-0430) in Caracas
  time_t mismatch!

In response to Dag-Erling Smørgrav’s bug report[1] against TZDB’s strftime implementation, I recently fixed[2] TZDB strftime to use tm_gmtoff when converting %s, so that the Caracas example is guaranteed to output this:

  1197184500 prints as 1197184500 = 2007-12-09 02:45:00 -0430 (-0430) in Caracas

This is what users would invariably expect from strftime.

However, on the TZ mailing list Robert Elz objected[3] that the fix does not conform to POSIX 202x/D4, because POSIX requires that strftime %s must work by calling the equivalent of mktime() and thereby ignoring tm_gmtoff, something that would inevitably result in "time_t mismatch!" output in some situations like this. (Although the original bug report was against gmtime+strftime, a similar bug can occur with localtime+strftime as in the Caracas example.)

If POSIX 202x/D4 actually requires the "time_t mismatch!" output in cases like [1] or like the Caracas example, this is a bug in 202x/D4 that was not present in POSIX 2017. Even if many implementations have this bug, POSIX should not require strftime %s to be incompatible with localtime, and it should allow the TZDB fix.

Here is a bit more explanation about why Ubuntu 23.10 and many other platforms misbehave on the Caracas example. When TZ="America/Caracas", the two timestamps 1197182700 and 1197184500 convert via localtime into struct tm values with identical tm_year, tm_mon, tm_mday, tm_hour, tm_min, tm_sec, and tm_isdst members. This happens because Venezuela changed standard time by moving its clock back 30 minutes on 2007-12-09 at 03:00, and tm_isdst is therefore zero for both timestamps. This can be observed via the following shell transcript run on Ubuntu 23.10, using TZDB’s zdump utility:

  $ zdump -V -c 2007,2008 America/Caracas
  America/Caracas Sun Dec 9 06:59:59 2007 UT = Sun Dec 9 02:59:59 2007 -04 isdst=0 gmtoff=-14400
  America/Caracas Sun Dec 9 07:00:00 2007 UT = Sun Dec 9 02:30:00 2007 -0430 isdst=0 gmtoff=-16200

With the fix[2], TZDB strftime examines tm_gmtoff when converting %s; this lets strftime disambiguate struct tm values in cases like these. However, POSIX 202x/D4 does not list tm_gmtoff as one of the struct tm members that strftime can examine, and that is the basis for the objection[3] that POSIX prohibits the fix.

[1]: https://mm.icann.org/pipermail/tz/2024-January/033488.html [^]
[2]: https://github.com/eggert/tz/commit/a707253f0c3c8cfd7b66fc5ca46cfb0a01e41dee [^]
[3]: https://mm.icann.org/pipermail/tz/2024-January/033549.html [^]
Desired Action In section strftime() DESCRIPTION conversion specifier ‘s’, page 2136 lines 69836-69837, replace this:

Replaced by the number of seconds since the Epoch as a decimal number, calculated as described for mktime(). [tm_year, tm_mon, tm_mday, tm_hour, tm_min, tm_sec, tm_isdst]

with this:

Replaced by the number of seconds since the Epoch as a decimal number, calculated as described for mktime(), except that tm_gmtoff is also available to determine seconds east of UTC. [tm_year, tm_mon, tm_mday, tm_hour, tm_min, tm_sec, tm_isdst, tm_gmtoff]


At the end of section strftime() RATIONALE, after page 2142 line 70105, add the following paragraph:

For the conversion specifiers %s, %z and %Z, it is unspecified whether strftime generates replacement text by examining tm_gmtoff and tm_zone, or by examining tm_isdst along with global state set by tzset(), or by some combination of the two. For example, when using strftime() on the output of gmtime(), it is unspecified whether %z converts to "+0000" or to a string suitable for the local time zone, and it is unspecified whether %s converts to the timestamp passed to gmtime() or to the same timestamp numerically adjusted by local time’s seconds east of UTC.
Tags tc1-2024
Attached Files

- Relationships
related to 0001533Closed 1003.1(2016/18)/Issue7+TC2 struct tm: add tm_gmtoff (and tm_zone) field(s) 
related to 0001816Resolved Issue 8 drafts daylight, timezone, tzname do not work with location-based TZ 
child of 0001612Closed Issue 8 drafts XSH strftime() new (I8) %s conversion requires too much of the struct tm 
child of 0000169Closedajosey 1003.1(2008)/Issue 7 date utility needs ``%s'' 

-  Notes
(0006624)
steffen (reporter)
2024-01-16 01:42

Wait, i see what is meant. I delete my note.
(0006651)
eblake (manager)
2024-02-12 16:11

Feedback from Paul Eggert via email:

On 2024-02-05 08:15, Eric Blake wrote:

> Did you consider the effect of the change on applications that
> populate struct tm directly (and don't currently set tm_gmtoff, except
> perhaps by zeroing the structure)?

Yes. Very few apps do that. (I looked for some in the GNU code I help
maintain, and found none.) They are greatly outnumbered by the applications
that call localtime/localtime_r/mktime/gmtime/gmtime_r/etc. and pass the
result to strftime, which is what this bug report is about.


> Does the latest tzdata code only use tm_gmtoff in the rare cases when
> it is necessary for disambiguation, or is it always used (overriding
> the timezone data)? The bug description implies the former, but the
> desired action would allow the latter.

The former. That is, TZDB 2024a strftime looks only at tm_gmtoff, tm_year,
tm_mon, tm_day, tm_hour, tm_min, and tm_sec to determine %s, because that's
all you need.

The desired action allows either the TZDB behavior, or the glibc behavior
which if I recall consults tm_gmtoff only when tm_isdst is ambiguous. The TZDB
behavior is technically better than the glibc behavior for three reasons: (1)
it removes a multithreading bottleneck, (2) even in a single-threaded platform
it's faster because mktime is slower than using tm_gmtoff, and (3) when user
code mistakenly calls gmtime and then strftime then %s does what the user
expects. The bug report that caused TZDB to behave this way was about (3), but
(1) and (2) also play a part.
(0006677)
kre (reporter)
2024-02-25 06:50
edited on: 2024-02-25 06:54

I do not like this proposed change - as strftime(%s) is specified
in the drafts for Issue 8 is just fine - it and mktime() always
produce the same value, given the same (in range) values in the
struct tm passed to them. Never anything different.

The change that was made to the tzcode implementation to use tm_gmtoff
(which despite what was said in Note: 0006651, is always used if it exists - in
that implementation what TZ is set to is immaterial to strftime(%s)) was
based upon a user hope for what would happen (to get back the time_t
that was used to produce a particular struct tm) completely ignoring
that no such invocation is required to have occurred. strftime()
simply formats as text values from the struct tm which the format
string requests - regardless of how those values were inserted.

The "very few apps" which "do that" must not be broken by a change like
this.

Applications which call gmtime() and then strftime(%s) upon the result,
and expect the (formatted as a string) value that was used as gmtime()'s arg
to be produced are simply broken. (That usage was exactly what led to
the "bug" report that was "fixed" by the tzcode change - nonsense.)

Simply reject this "bug".

I'd also note that while NetBSD has the 2024a version of tzcode (in HEAD
anyway) the change that was made to strftime(%s) in that release was
reverted, we still do things the old way, using TZ, not using tm_gmtoff.

(0006688)
eggert (reporter)
2024-02-26 18:28

Unfortunately draft 4 for Issue 8 has significant problems here, and these problems need to be fixed somehow. The draft does not tell users that strftime %s might inspect tm_gmtoff - something that implementations other than TZDB already do - and it is not entirely clear what the draft means by values "outside the normal range" (line 69773 in draft 4) - for example, is tm_day outside the normal range when it equals 30, and tm_mon happens to be 1?

We agree that applications that call gmtime and then strftime %s are broken (unless localtime happens to be UTC). The desired action doesn't change that. These apps will still not necessarily generate the output that the app authors wanted.

The desired action does need improvement, along the lines of saying that struct tm values are "in range" if they are values that localtime could have set. Expect a revised proposal soon.
(0006689)
eblake (manager)
2024-02-26 19:32

Based on discussions through the 26 Feb 2024 meeting with feedback from Paul Eggert, the following wording is the Austin Group's starting point for an interpretations request to be started after Issue 8 is released. Note that we still need to flesh out the interpretation Rationale (currently a placeholder in the paste below). Among others: the fact that the standard required gmtime_r() (but not gmtime()) to set tm_zone to "UTC" is inconsistent with existing practice where some implementations set it to "GMT". We may also need to add more to the Rationale section of gmtime() and/or other functions on why "UTC" is preferred but why "GMT" is permitted for historical compatibility.

Interpretation response
------------------------
The standard states that the strftime() %s conversion calculates the number of seconds since the Epoch as described for mktime(), and conforming implementations must conform to this. However, concerns have been raised about this which are being referred to the sponsor.

Rationale:
-------------
<Place Rationale, if any, here.>

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
Using draft 4.0 line numbering:

On page 452 lines 15762-15763 section XBD <time.h>, add CX shading to the lines for tm_gmtoff and tm_zone.

On page 1211 line 41381 section gmtime() DESCRIPTION, change (already in CX shading):

    
...where the names in the structure and in the expression correspond.


to:

    
...where the names in the structure and in the expression correspond; additionally, the tm_gmtoff field of the struct tm shall be set to 0, and the tm_zone field shall be set to a pointer to an implementation-defined string set to "UTC" or "GMT", which shall have static storage duration.



On page 1211 line 41397 section gmtime() RETURN VALUE, delete:

    
The structure’s tm_zone member shall be set to a pointer to the string "UTC", which shall have static storage duration.



On page 1428 line 47958 section mktime(), change:

    
shall calculate the time since the Epoch value using either the offset in effect before the change or the offset in effect after the change.


to:

    
shall calculate the time since the Epoch value using either the offset in effect before the change or the offset in effect after the change; mktime( ) may use the value of tm_gmtoff to decide which of these two results is the more appropriate to return.



On page 2135 line 69773 section strftime(), change:

    
If any of the specified values are outside the normal range, the characters stored are unspecified.


to:

    
If any of the specified values are outside the normal range, as if set by localtime( ), the characters stored are unspecified.



On page 2135 line 69777 section strftime(), change:
    
Local timezone information shall be set as though strftime( ) called tzset( ).

to:
    
It shall be implementation-defined whether local timezone information is set as though strftime( ) called tzset( ).


On page 2136 line 69836 section strftime(), change:

    
Replaced by the number of seconds since the Epoch as a decimal number, calculated as described for mktime( ). [tm_year, tm_mon, tm_mday, tm_hour, tm_min, tm_sec, tm_isdst]


to:

    
Replaced by the number of seconds since the Epoch as a decimal number, calculated as described for mktime( ), except that if the tm_gmtoff value does not match the local timezone information, it is implementation-defined whether the number of seconds since the Epoch is calculated using the offset specified by tm_gmtoff instead of the offset derived from the local timezone information. Note that, unlike mktime( ), the tm structure member values passed to strftime( ) need to be within the normal range, as if set by localtime( ), to avoid unspecified output. [tm_year, tm_mon, tm_mday, tm_hour, tm_min, tm_sec, tm_isdst, tm_gmtoff]



After page 2140 line 70007 section strftime(), add a new paragraph to APPLICATION USAGE:

    
The %s conversion should not be used with a tm structure populated solely using strptime( ), since strptime( ) does not set the tm_gmtoff field and the behavior of the strftime( ) %s conversion may depend on this value. Additionally, a tm structure populated by gmtime( ) or gmtime_r( ) is generally not suitable for portable use with the %s conversion of strftime( ); although some implementations of strftime may consult the tm_gmtoff field and reproduce the same value that was provided on input to the gmtime family of functions, this conversion is not required across all implementations.
(0006690)
eblake (manager)
2024-02-26 20:02

We may also need to tweak wording on page 2310 line 75181 tzset() about the relation between tzset() and strftime(), if our intent is to allow implementations that do not update or consult the local timezone information for strftime() (independently of whether we accept 0001816 about obsoleting the 3 global variables aspect of tzset()
(0006691)
geoffclare (manager)
2024-02-29 12:10

The following is an alternative resolution to be considered, which retains the requirement for strftime() %s to give the same result as mktime() (for in-range struct tm field values). It does allow strftime() %s - and mktime() - to make use of tm_gmtoff, but only as a means to choose between two valid results. It also includes fixes for the various other related issues raised during the discussion to far.

Interpretation response
------------------------
The standard clearly states that the strftime() %s conversion calculates the number of seconds since the Epoch as described for mktime(), and conforming implementations must conform to this.

Rationale:
-------------
There is no requirement for applications to set the tm_gmtoff member and therefore implementations cannot rely on it having been set. However, in situations where there are two valid results of the conversion, an implementation may choose to make use of tm_gmtoff to decide between these two values (provided it can do so safely if tm_gmtoff is uninitialized), even though the strftime() %s conversion is not specified as making use of tm_gmtoff, since returning either of the values is allowed by the standard and the method used to choose between them is internal implementation detail that does not affect applications.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
Using draft 4.0 line numbering:

On page 452 lines 15762-15763 section XBD <time.h>, add CX shading to the lines for tm_gmtoff and tm_zone.

On page 1211 line 41381 section gmtime() DESCRIPTION, change (already in CX shading):
    
...where the names in the structure and in the expression correspond.

to:
    
...where the names in the structure and in the expression correspond; additionally, the tm_gmtoff field of the struct tm shall be set to 0, and the tm_zone field shall be set to a pointer to an implementation-defined string set to "UTC" or "GMT", which shall have static storage duration.


On page 1211 line 41397 section gmtime() RETURN VALUE, delete:
    
The structure’s tm_zone member shall be set to a pointer to the string "UTC", which shall have static storage duration.


On page 1428 line 47958 section mktime(), change:
    
shall calculate the time since the Epoch value using either the offset in effect before the change or the offset in effect after the change.

to:
    
shall calculate the time since the Epoch value using either the offset in effect before the change or the offset in effect after the change; mktime() may use the value of tm_gmtoff to decide which of these two results is the more appropriate to return, provided it can do so safely if tm_gmtoff is uninitialized.


On page 2135 line 69773 section strftime(), change:
    
If any of the specified values are outside the normal range, the characters stored are unspecified.

to:
    
If any of the specified values are outside the normal range, as if set by localtime(), the characters stored are unspecified.


On page 2135 line 69777 section strftime(), change:
    
Local timezone information shall be set as though strftime() called tzset().

to:
    
It shall be implementation-defined whether local timezone information is set as though strftime() called tzset().


On page 2136 line 69836 section strftime(), change:
    
Replaced by the number of seconds since the Epoch as a decimal number, calculated as described for mktime(). [tm_year, tm_mon, tm_mday, tm_hour, tm_min, tm_sec, tm_isdst]

to:
    
Replaced by the number of seconds since the Epoch as a decimal number, calculated as described for mktime(). Note that, unlike mktime(), the tm structure member values used by this conversion, other than tm_gmtoff, need to be within the normal range, as if set by localtime(), to avoid unspecified output; the value of tm_gmtoff can be uninitialized (see [xref to mktime()]). [tm_year, tm_mon, tm_mday, tm_hour, tm_min, tm_sec, tm_isdst, tm_gmtoff]


After page 2140 line 70007 section strftime(), add a new paragraph to APPLICATION USAGE:
    
Using a tm structure populated by gmtime() or gmtime_r() with the %s conversion of strftime() should be avoided, as the calculated seconds since the Epoch value will only correspond to the specified broken-down time if the offset from UTC of the local timezone happens to be zero at that time.
(0006692)
geoffclare (manager)
2024-02-29 12:15

I would prefer the resolution in Note: 0006691 over the one in Note: 0006689 (i.e. I think POSIX should not change to allow the new TZDB behaviour).

I agree with the points kre made in Note: 0006677 and in particular with the statement that we must not break apps which use strftime() %s without setting tm_gmtoff. The counter-argument that there are very few such apps does not hold water, for two reasons:

1. It is only possible to survey apps for which we have access to the source. We cannot know how many closed-source apps would be affected.

2. We should not consider only apps which use %s explicitly. There are also apps which use user-specified or configurable time format strings. Although a lot of these would be passing in a struct tm obtained from localtime() (as with, for example, GNU ls -l --time-style=+%s), passing one that is populated a different way is also a reasonable thing for an app to do. For example, it could store broken-down times in a standard format and want to present them to the user in their preferred format; to do this it could use strptime() followed by strftime().

Another point in favour of disallowing "blind" use of tm_gmtoff is to consider what system implementers will do when faced with the choice of how to handle a change in POSIX to allow it. In order to continue to support apps written for POSIX.1-2017 and earlier (which did not have tm_gmtoff and therefore conforming apps cannot set it directly) and also have the TZDB behaviour for apps written for POSIX.1-2024 and later, they would need to provide two different versions of strftime(), selected according to _POSIX_C_SOURCE or _XOPEN_SOURCE. It seems highly unlikely that any system implementers would bother to do that (it has been done in the past, but only when there was no choice); the new TZDB-like behaviour would therefore be rare and become a portability gotcha.
(0006693)
shware_systems (reporter)
2024-02-29 16:19

Additionally, while the text above references how gmtime() should set tm_gmtoff, there is no equivalent text in localtime() for how that field should be set. Do current implementations just set it to zero, leave it as malloc() garbage, compute something if second TZ format is used?

Also, it's questionable whether localtime_r() should behave as if tzset() is called, as the tm struct provided may have value set and unmodified that may conflict with the values in the TZ variable.
(0006698)
geoffclare (manager)
2024-03-01 10:09

> Additionally, while the text above references how gmtime() should set tm_gmtoff, there is no equivalent text in localtime() for how that field should be set.

Good point. My take on this observation is that the addition in gmtime() is unnecessary. Both gmtime() and localtime() are required to set all of the fields in struct tm, and the description of tm_gmtoff in <time.h> ("Seconds east of UTC") is sufficient to require what value it is set to: in the case of gmtime(), the number of seconds east of UTC is zero because gmtime() returns the broken-down time as UTC.

The reason an explicit statement is needed about tm_zone for gmtime() is because its description in <time.h> ("Timezone abbreviation") is not sufficiently clear what string this should be for gmtime().
(0006700)
eggert (reporter)
2024-03-02 09:02

> 1. It is only possible to survey apps for which we have access to the source. We cannot know how many closed-source apps would be affected.

Although that’s true, we can use our best judgment about which programs will benefit. We know that there are programs that benefit from the resolution in Note: 0006689, because we have a bug report to that effect and it’s plausible there are more programs like that. Although these programs may not follow the strict reading of the standard, they’re plausible uses of the standard functions. In contrast, we don’t know of programs that would benefit from the resolution in Note: 0006691, and these programs are less plausible.


> 2.... There are also apps which use user-specified or configurable time format strings. Although a lot of these would be passing in a struct tm obtained from localtime() (as with, for example, GNU ls -l --time-style=+%s), passing one that is populated a different way is also a reasonable thing for an app to do.

Unfortunately it’s not reasonable. These apps are broken regardless of which resolution is taken, because the issue comes up with formats other than %s. Consider the following program, which simulates a user-specified format string containing "%Z" used to format a struct tm set by hand or by strptime:

  #include <stdio.h>
  #include <time.h>
  #include <string.h>
  int
  main (void)
  {
    char buf[1000];
    struct tm tm;

    /* Make TM "uninitialized". */
    memset (&tm, 0x81, sizeof tm);

    tm.tm_isdst = 0;
    strftime (buf, sizeof buf, "%Z", &tm);
    puts (buf);
  }

This program dumps core on GNU/Linux, on FreeBSD, and I expect on pretty much any other implementation that supports TZDB-style struct tm, because it uses a bad pointer in tm.tm_zone. And yet the program conforms to POSIX.1-2017, which says that the call to strftime sets the buffer to some string.

In other words, programmers can’t trust user-specified strftime formats on anything other than completely filled-in struct tm values, and that’s true regardless of what we do about %s.

I suppose we could say that GNU/Linux, FreeBSD, and pretty much everyone else is broken. But it’s better to be realistic and tell users that when using strftime’s %z, %Z and %s formats, tm_zone and tm_gmtoff must be set to in-range values. Insisting on a pedantic reading based on older POSIX would not serve users well because it would give users the wrong impression of what code will actually work.

> consider what system implementers will do

We implementers will figure it out. It’s the user code that matters here anyway.
(0006701)
kre (reporter)
2024-03-03 01:20

Re Note: 0006700

  We know that there are programs that benefit from the resolution in
  Note: 0006689, because we have a bug report to that effect

Incorrect bug reports should not be used to justify changes to anything
(including tzcode). The code in that report was simply wrong, even
with the wording in Note: 0006689 it would still be broken. Attempting
to make broken code work is absurd. If I sent in a bug report claiming
that my code, with a divide by 0, wasn't working as I expected it to, would
you go fix the hardware/OS/something to generate the answer I demanded?

  In contrast, we don’t know of programs that would benefit from the
  resolution in Note: 0006691, and these programs are less plausible.

Here's one, with all of the error handling, #includes, etc, removed to
save space here (they exist in the real thing, naturally):

main(int argc, char **argv)
{
        struct tm tm;
        char *p;
        int i;
        char buf[64];

        p = strptime(argv[1], "%Y-%m-%d %H:%M:%S", &tm);
        tm.tm_isdst = -1;

        for (i = 2; i < argc; i++) {
                setenv("TZ", argv[i], 1);
                tzset();
                strftime(buf, sizeof buf, "%s", &tm);
                printf("%s %s\n", buf, argv[i]);
        }
        exit(0);
}

It can be run like

        prog 2024-03-03T07:23:46 UTC Asia/Bangkok Australia/Melbourne \
               America/New_York Europe/Berlin Africa/Cairo Africa/Windhoek \
               America/St_Johns America/Buenos_Aires America/Anchorage |
                                  sort -n

to discover the east->west order of a set of random zones -- which in
this case gives:

1709411026 Australia/Melbourne
1709425426 Asia/Bangkok
1709443426 Africa/Cairo
1709443426 Africa/Windhoek
1709447026 Europe/Berlin
1709450626 UTC
1709461426 America/Buenos_Aires
1709463226 America/St_Johns
1709468626 America/New_York
1709483026 America/Anchorage

or it can be used to observe differences in summer time behaviour
when used like:

    prog 2024-03-03T07:23:46 Australia/Darwin Australia/Adelaide \
         Australia/Brisbane Australia/Sydney | sort -n
1709411026 Australia/Sydney
1709412826 Australia/Adelaide
1709414626 Australia/Brisbane
1709416426 Australia/Darwin

    prog 2024-06-03T07:23:46 Australia/Darwin Australia/Adelaide \
          Australia/Brisbane Australia/Sydney | sort -n
1717363426 Australia/Brisbane
1717363426 Australia/Sydney
1717365226 Australia/Darwin
1717365226 Australia/Adelaide

from which one observes that when summer time is not in effect, Brisbane
and Sydney are the same, as are Darwin and Adelaide - with the former pair
of zones achieving a wallclock time before the latter, but when summer time
is in effect, all 4 are different, and Adelaide precedes Brisbane, rather
than following it.


There's no compelling reason to break this. I'm not sure I even really
support allowing tm_gmtoff to be used (ever) to handle the very weird cases
where nothing else is available to disambiguate the very odd cases, like
the America/Caracas example given in the description, though if there's any
good, and safe, way an implementation can truly:

    provided it can do so safely if tm_gmtoff is uninitialized.

then it really all that important, provided the code given in this
note always works as it should (where tm_gmtoff is entirely nonsense,
one could init it to ~0 if desired - or leave it to generate a trap
if referenced on hardware that supports that kind of thing).

Also, I don't think that this:

    Note that, unlike mktime(), the tm structure member values used by this
    conversion, other than tm_gmtoff, need to be within the normal range, as
    if set by localtime(), to avoid unspecified output;

is needed either. The reason that strftime() needs in range values in
general, is not because of %s (which did not exist when that requirement
was added) but to avoid problems with conversions like %a and %b if tm_wday
or tm_mon are set to way out of range values (what is the name of the 29th
day of the week?). But for that, all that is needed is that the values are
within the ranges specified by <time.h> - there's no issue at all with

     tm.tm_mon = 1; /* Feb */
     tm.tm_mday = 31;

and then using strftime("%d %b"); The result might not make sense, but
it is perfectly well defined.

For %s we don't really need the "within range ..." restriction at all, because
of the "calcluated as described by mktime()" - but nor do they do any harm
really, as the only reason mktime() permits out of range values is to allow
time arithmetic (which given the POSIX definition of time_t isn't really
required by POSIX apps) and one thing I can't really imagine is anyone using
strftime("%s") for that purpose. But there's no reason to make the ranges
required for the %s conversion any tighter than for any other conversion that
strftime() offers, (and in any case mktime() is a very poor, in fact,
unworkable, interface for that feature, it works OK with gmtime()/timegm(),
but it would actually be far better to create (not here, but by the
implementations) a companion to difftime() (perhaps:
    time_t addtime(time_t, double);
or something like that) to allow values to be added/subtracted from a time_t.
(Then if there are implementations where time_t is not a simple integer
count of seconds, they can do whatever is needed to make that work).

The wording in Note: 0006691 looks OK to me (though I'd prefer the two
changes just mentioned - that is, not to make the two changes referred
to just above in this note).
(0006706)
geoffclare (manager)
2024-03-07 10:44

>> consider what system implementers will do

> We implementers will figure it out. It’s the user code that matters here anyway.

I think you missed my point. I expect almost all system implementers will "figure it out" by taking the easy option and sticking with Issue 7 compatible behaviour. (If they have TZDB as the upstream for this stuff, they will revert the strftime() tm_gmtoff change, as NetBSD has done.) The existence of any system where the strftime() implementation behaves like TZDB then creates a portability problem for user code written on any of the other systems; it will work fine on most systems but misbehave if/when it is ever ported to this unusual and rare system.
(0006707)
geoffclare (manager)
2024-03-07 11:04

> This program dumps core on GNU/Linux, on FreeBSD, and I expect on pretty much any other implementation that supports TZDB-style struct tm, because it uses a bad pointer in tm.tm_zone. And yet the program conforms to POSIX.1-2017, which says that the call to strftime sets the buffer to some string.

The program conforms to POSIX.1-2017, and your results show that GNU/Linux and FreeBSD apparently do not; these implementations must be using tm_zone "blindly" and do not conform because POSIX.1-2017 requires that the only struct tm field used by %Z is tm_isdst. (This doesn't mean implementations can't use non-standard fields such as tm_zone, but if they are used, it must be in such a way that a conforming application can't tell the difference, i.e. it is just internal implementation detail. In the case of tm_zone this would mean checking the pointer is equal to a known valid value for the current timezone before using it.)
(0006718)
geoffclare (manager)
2024-03-11 09:55

> I suppose we could say that GNU/Linux, FreeBSD, and pretty much everyone else is broken. But it’s better to be realistic and tell users that when using strftime’s %z, %Z and %s formats, tm_zone and tm_gmtoff must be set to in-range values.

For %z and %Z, requiring applications to set tm_zone and tm_gmtoff would create a conflict with the C standard. C17 says that the only member used by %z and %Z is tm_isdst.

Of course, if an application explicitly sets tm_zone or tm_gmtoff then it is using an extension to C17 and this could alter the behaviour, and if the structure was populated by localtime() et al then use of tm_zone or tm_gmtoff can be an internal implementation detail, as per my previous note. However, the crucial case is when the application populates the structure in a way that does not set tm_zone or tm_gmtoff; for this case C17 requires that the conversion is done based on tm_isdst.
(0006793)
eggert (reporter)
2024-05-23 08:04

Re Note: 0006701
> The code in that report was simply wrong, even
> with the wording in Note: 0006689 it would still be broken.

That code is unportable, not “wrong” or “broken”. The wording in Note: 0006689 would let implementations do what the code’s author intended. For this use case, that’s better than requiring implementations to not follow the intent. I continue to be skeptical that typical developers would expect strftime(buf, size, "%F %T %Z", gmtime(&t)) to do anything other than output a string representing UTC.


> In contrast, we don’t know of programs that would benefit from the
> resolution in Note: 0006691, and these programs are less plausible.
>
> Here's one, with all of the error handling, #includes, etc, removed to
> save space here (they exist in the real thing, naturally):

That example doesn’t use gmtime, so it is not completely on point.

I accept that one can contrive programs that prefer the other interpretation. However, I continue to maintain that the vast majority of programs assume the interpretations used by TZDB for %z and %Z.

To check this, I did a GitHub code search today with the query “strftime gmtime path:*.c”. Most matches were irrelevant, because the output of gmtime was not passed to strftime, or because the strftime format string did not use %s, %z, or %Z. In a few matches it was hard to see what’s going on, so I ignored them. However, all the relevant matches I found (listed below) preferred the proposed interpretation: that is, they expected strftime to format the gmtime output as UTC, not as local time. In many cases this was because they let the user specify an arbitrary strftime format to be applied to the result of gmtime, and users would naturally expect gmtime to output GMT aka UTC.

Here are the matches I found:

https://github.com/davidmoreno/onion/blob/de8ea938342b36c28024fd8393ebc27b8442a161/src/onion/shortcuts.c#L350 [^]
https://github.com/erlang/otp/blob/945c940f6bc6c0bcb026cdc6ae8f3ce358e859bb/erts/etc/unix/run_erl.c#L583 [^]
https://github.com/esnet/iperf/blob/93c60bf2ea8aa52e98399223ec04a95f65f00f7b/src/iperf_api.c#L948 [^]
https://github.com/jonas/tig/blob/587a59abab3f2ebe7612bc69f44c22df570b2ee3/src/util.c#L208 [^]
https://github.com/openwall/john/blob/9f913a734055aae2606c68851a5a16b0d50a2060/src/logger.c#L324 [^]
https://github.com/RediSearch/RediSearch/blob/10261d2f93e77a3d1118ccded01fb4fa0e1c9cca/src/aggregate/functions/date.c#L36 [^]
https://github.com/RMerl/asuswrt-merlin.ng/blob/466f5aee9e274971480a19497573c4264f865019/release/src/router/aws-iot/src/time_util.c#L56 [^]
https://github.com/RMerl/asuswrt-merlin.ng/blob/466f5aee9e274971480a19497573c4264f865019/release/src/router/rc/conn_diag_log.c#L334 [^]
https://github.com/root-project/root/blob/333f042a16525e746da4ec56c3a5a3ed79ceb677/tutorials/graphs/timeonaxis3.C#L25 [^]
https://github.com/cheat-engine/cheat-engine/blob/54f90a22bfc9d4981f414a7917d70d3d17f190e5/lua/src/loslib.c#L129 [^]


> The reason that strftime() needs in range values in
> general, is not because of %s (which did not exist when that requirement
> was added) but to avoid problems with conversions like %a and %b

It’s true that the in-range requirement was not put in because of %s, because (as you mention) %s was not in the standard back then. However, the in-range requirement is needed for %s regardless of the resolution of this issue. For example, on a host with 32-bit time_t and 32-bit int, a tm_year value of INT_MAX is out of range for localtime/gmtime, and would cause strftime to overflow when calculating %s.

So we cannot wave away the in-range requirement - it’s essential for multiple reasons.


> The existence of any system where the strftime() implementation behaves like TZDB then creates a portability problem

Users will have portability problems no matter what. We already have discrepancies between how strftime behaves for %s/%z/%Z on FreeBSD, z/OS, TZDB, AIX, etc. Our job is to document discrepancies and (if we can get consensus) to suggest a preferred behavior.


> The program conforms to POSIX.1-2017, and your results show that GNU/Linux and FreeBSD apparently do not

It’s not just just GNU/Linux and FreeBSD; it’s also macOS and I expect every other other system that has tm_zone. It would be counterproductive to say that all these systems are wrong and that POSIX is right. This is an area where if POSIX disagrees with common practice, then POSIX needs to be fixed.


> For %z and %Z, requiring applications to set tm_zone and tm_gmtoff would create a conflict with the C standard. C17 says that the only member used by %z and %Z is tm_isdst.

There’s no conflict here. C17 does not specify the method used to deduce the time zone used for %z and %Z. An implementation is free to use whatever screwy method it likes. It can even indulge in undefined behavior, since C17 does not define the method (that’s what “undefined behavior” means, after all...).

C17’s language cannot mean that %z and %Z’s output must be completely determined by tm_isdst and the locale. If that were true, it would prohibit even POSIX.1-2017’s requirement that global state set by tzset() - which is not part of the locale - can also play a role. Luckily it is not true, as C17 gives wide latitude as to how the implementation can determine the timezone. POSIX has already exploited that latitude, and tm_zone and tm_gmtoff are yet another way to exploit that latitude.

While we’re on the topic of %Z, I noticed one other thing: IBM z/OS uses the proposed interpretation for %Z. That is, the IBM documentation <https://www.ibm.com/docs/en/zos/3.1.0?topic=functions-strftime-convert-formatted-time> [^] states that strftime generates an abbreviation appropriate for UTC (not for localtime) if the struct tm was produced by gmtime.

TZDB, GNU/Linux, FreeBSD, macOS and z/OS all disagree one way or another with the idea that strftime should not look at tm_zone or tm_gmtoff when computing %s, %z, and %Z. POSIX should not try to impose a requirement that would invalidate all these systems.

There’s one other potential issue with requiring strftime to do the equivalent of inspecting the tzname and timezone global variables: these variables do not contain well-defined value when TZ is set to something typical like TZ='Europe/London', due to the resolution of Issue 0001816 “daylight, timezone, tzname do not work with location-based TZ”. This suggests that %s, %z, and %Z could expand to anything if one uses a location-based TZ. Although POSIX.1-2017 allows that behavior, the intent of the next POSIX, as I understand it, is to nail this down more precisely.

In my next comment I’ll try to reword the latest proposed resolution to address these issues.
(0006794)
eggert (reporter)
2024-05-23 08:07

Although the resolution proposed in Note: 0006691 has a lot to like, we need to do better with %s, %z, and %Z, as discussed in my previous comment. One thing that’s not clearly nailed down is that the implementation should be consistent in this area. That is, whatever it does for %s should be consistent with what it does for %z, and likewise for %Z. This is a “should” not a “must” because not every implementation is consistent, but intuitively it makes sense that the three formats should expand to consistent output.


Interpretation response
------------------------
The standard states that when the input is in the normal range the strftime() %s conversion calculates the number of seconds since the Epoch as described for mktime(), and conforming implementations must conform to this. However, concerns have been raised about this which are being referred to the sponsor.

Rationale:
-------------
In previous editions of POSIX, struct tm lacked tm_gmtoff and tm_zone members, so strftime needed to inspect global state to process %z and %Z. The current version allows strftime to instead inspect tm_gmtoff for %z, and tm_zone for %Z, as is common practice. For consistency it also allows strftime to inspect tm_gmtoff to process %s, a newly-required format.

A strftime that inspects tm_gmtoff and tm_zone works better with applications that use geographical timezones, or that fill in struct tm members by calling gmtime or equivalent. Conversely, a strftime that inspects global state is more compatible with previous editions of POSIX when programs use the second TZ format in XBD Section 8.3 and fill in struct tm members individually without calling localtime or equivalent. A strftime implementation can do both: that is, it can inspect global state when the second TZ format is used, and use tm_gmtoff and tm_zone when a geographical timezone is used.

This edition of the standard does not say whether the abovementioned global state consists of the contents of the variables tzname and timezone, or some other part of the system state that user code cannot modify directly. Modifying tzname and timezone therefore might (but might not) affect the global state that in turn might affect strftime’s behavior with %s, %z, and %Z, or might not if strftime inspects tm_gmtoff and tm_zone instead.

Regardless of implementation method, implementations should process the conversion specifications %s, %z, and %Z consistently. For example, if strftime uses tm_gmtoff to process %z, it should also use tm_gmtoff to process %s and should use tm_zone to process %Z. Conversely, if strftime uses global state to determine one of the the three conversion specifications, it should use the same global state to determine the other two.


Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
Using draft 4.0 line numbering:

On page 452 lines 15762-15763 section XBD <time.h>, add CX shading to the lines for tm_gmtoff and tm_zone.

On page 1211 line 41381 section gmtime() DESCRIPTION, change (already in CX shading):


    ...where the names in the structure and in the expression correspond.


to:


    ...where the names in the structure and in the expression correspond; additionally, the tm_gmtoff field of the struct tm shall be set to 0, and the tm_zone field shall be set to a pointer to an implementation-defined string set to "UTC" or "GMT", which shall have static storage duration.



On page 1211 line 41397 section gmtime() RETURN VALUE, delete:


    The structure’s tm_zone member shall be set to a pointer to the string "UTC", which shall have static storage duration.



On page 1428 line 47958 section mktime(), change:


    shall calculate the time since the Epoch value using either the offset in effect before the change or the offset in effect after the change.


to:


    shall calculate the time since the Epoch value using either the offset in effect before the change or the offset in effect after the change; mktime() may use the value of tm_gmtoff to decide which of these two results is the more appropriate to return, provided it can do so safely if tm_gmtoff is uninitialized.



On page 2135 line 69773 section strftime(), change:


    If any of the specified values are outside the normal range, the characters stored are unspecified.


to:


    If any of the specified values are outside the normal range, as if set by localtime(), the characters stored are unspecified.



On page 2135 line 69777 section strftime(), change:


    Local timezone information shall be set as though strftime() called tzset().


to:


    It shall be implementation-defined whether local timezone information is set as though strftime() called tzset().



On page 2136 line 69836 section strftime(), change:


    Replaced by the number of seconds since the Epoch as a decimal number, calculated as described for mktime(). [tm_year, tm_mon, tm_mday, tm_hour, tm_min, tm_sec, tm_isdst]


to:


    Replaced by the number of seconds since the Epoch as a decimal number, calculated as described for mktime(). Note that, unlike mktime(), the tm structure member values used by this conversion need to be within the normal range, as if set by localtime(), to avoid unspecified output. [tm_year, tm_mon, tm_mday, tm_hour, tm_min, tm_sec, tm_isdst, tm_gmtoff]



After page 2140 line 70007 section strftime(), add a new paragraph to APPLICATION USAGE:


    Using a tm structure populated by gmtime() or gmtime_r() with the %s, %z, and %Z conversion of strftime() is not portable unless local time is UTC, as the implementation might process the conversions either by inspecting the structure’s tm_gmtoff and tm_zone members, or by inspecting global state.
(0006795)
geoffclare (manager)
2024-05-23 10:47

>> For %z and %Z, requiring applications to set tm_zone and tm_gmtoff would create a conflict with the C standard. C17 says that the only member used by %z and %Z is tm_isdst.

>There’s no conflict here. C17 does not specify the method used to deduce the time zone used for %z and %Z. An implementation is free to use whatever screwy method it likes.

Only the C committee can give a definitive answer on what C17 requires here. We cannot finalise new wording for %z and %Z until we liaise with them. Therefore our next step on this bug should be to prepare a paper for Nick to submit for consideration by the C committee.
(0006797)
kre (reporter)
2024-05-23 16:57

Re Note: 0006793 and Note: 0006794

I really believe that those notes show a lack of understanding of the
issue here.

The following comment in Note: 0006793 demonstrates that:

  > Here's one, with all of the error handling, #includes, etc, removed to
  > save space here (they exist in the real thing, naturally):

  That example doesn’t use gmtime, so it is not completely on point.

But it is exactly on point. How is the implementation of strftime()
intended to learn whether or not its caller used gmtime(), or localtime()
or neither of those.

Let me give that example again, with one more minor addition, to make
the underlying difficulty more obvious:


main(int argc, char **argv)
{
        time_t now;
        struct tm tm;
        char *p;
        int i;
        char buf[64];

        (void)time(&now); /* or fill in "now" however you like */
        tm = *gmtime(&now);
        /* do something with the tm, doesn't matter what */


        p = strptime(argv[1], "%Y-%m-%d %H:%M:%S", &tm);
        tm.tm_isdst = -1;

        for (i = 2; i < argc; i++) {
                setenv("TZ", argv[i], 1);
                tzset();
                strftime(buf, sizeof buf, "%s", &tm);
                printf("%s %s\n", buf, argv[i]);
        }
        exit(0);
}

nb: same caveat about missing error handling, #includes, etc, as when
the earlier version of this was in Note: 0006701

Now the program has called gmtime() - that call just happens to be 100%
irrelevant to the calls of strftime() (using a struct tm which had the
results from gmtime() copied into it, some time in the past). How does
your implementation propose to generate sensible results here.

Even without the call to gmtime() there, how does your implementation
propose to tell whether there was one or not?

It doesn't matter that this might be an unusual thing to do, the fields
that the strptime() call filled in (along with tm_isdst which is set
separately) are all that is required for mktime() to work on the data,
and I think it was previously agreed that strftime(%s) and mktime()
should produce the same result (one as a string, the other as a time_t
but that's immaterial). That's certainly how it has always been
specified. Applications are entitled to operate that way.

Similarly for %z and %Z - both C and POSIX have promised application
writers that all they need do is set tm_isdst, in an otherwise completely
randomly filled (or unfilled) struct tm, and results will be produced.

Certainly the standards do not, and should not, tell the implementers how
to generate the results, but those implementers need to make things work
when the application does the bare minimum required of it by the standards,
regardless of how many other applications do things differently.

We simply cannot ignore what has been promised before, and change the
state of the world, requiring applications to update from what has been
promised as being acceptable, and which has previously worked in conforming
applications. There is simply no way to fix this with the current API.
You'd need entirely new functions, which no existing application can be
calling (obviously) with a different set of rules.

Re Note: 0006795 -- I don't think simply passing this off to the C committee
is really the right thing to do. Certainly, we can let them be the "bad
guys" so we can throw up our hands and just say "we need to conform to the C
standards" and make it look as if it isn't our problem. The right thing
to do here is simply to reject this bug, and not waste everyone's time on it
any more than it already has.
(0006798)
eggert (reporter)
2024-05-24 18:59

Re Note 0006797:
> That example doesn’t use gmtime, so it is not completely on point.
>
> But it is exactly on point.

I wrote “on point” because I thought the example was a program in the wild. But it now appears that it was a test program written to address this issue. Although I’ve cited several real-world programs using gmtime + strftime + %s/%z/%Z, we haven’t seen a similar set of programs using strptime + tm_isdst=-1 + strftime + %s/%z/%Z, so the real-world programs we’ve seen suggest that strftime should consult tm_gmtoff+tm_zone rather than global state.


> How is the implementation of strftime() intended to learn whether or
> not its caller used gmtime(), or localtime() or neither of those.

Note 0006794 suggests that strftime use tm_gmtoff (for %s and %z) and tm_zone (for %Z).


> It doesn't matter that this might be an unusual thing to do

Unfortunately it does matter, because so many real-world implementations and programs disagree with a reading of the standard that requires strftime to ignore tm_gmtoff/tm_zone when processing %s/%z/%Z. When the standard isn’t clear on a matter it’s better to clear it up; and when clearing it up, it’s better to minimize disruption to common practice; and to do that, we need to know what common practice is. Although a test app that we write might be useful for other things, it does not help us discern common practice in apps.


> it was previously agreed that strftime(%s) and mktime()
> should produce the same result

Yes, when the struct tm values are inside the normal range (that is, when they could have been produced by localtime). But we are talking about what strftime should do with struct tm values outside the normal range.


> Similarly for %z and %Z - both C and POSIX have promised application
> writers that all they need do is set tm_isdst, in an otherwise completely
> randomly filled (or unfilled) struct tm, and results will be produced.

I don’t see that written down. The C standard says that strftime can consult tm_isdst for %Z. It does not say that the output of %Z is completely determined by tm_isdst’s value. Implementations can also look elsewhere when generating %Z output, and they can impose additional requirements on the “elsewhere”.

This is true even for POSIX.1-2017, which says that if TZNAME_MAX is 6 and you set TZ='ABCDEFG0' in the environment, strftime with %Z is allowed to dump core because the time zone abbreviation is too long. This provision of POSIX.1-2017 does not violate the C standard, even though it is an additional requirement above and beyond strftime’s caller setting tm_isdst. And requiring tm_zone to have sensible contents is not a violation either: tm_zone is an extension to the C standard and the C standard has nothing to say about what tm_zone’s contents can or should be.
(0006799)
kre (reporter)
2024-05-24 21:32

Re Note: 0006798

  But we are talking about what strftime should do with struct tm values
  outside the normal range.

No, we aren't. strftime() is clear the results are unspecified if the
relevant fields aren't within the normal range(s). You can do what you
like in that case (but only the relevant fields, when converting %a for
example, it isirrelevant what tn_mday or tm_mon or ... are set to, just
tm_wday). The same goes for all of the conversions.

  The C standard says that strftime can consult tm_isdst for %Z.

I don't have that one in front of me, but I doubt it says anything
much different than posix says, which is:

  The appropriate characters are determined using the LC_TIME category of
  the current locale and by the values of zero or more members of the
  broken-down time structure pointed to by timeptr, as specified in
  brackets in the description.

What that means is that the specified fields of the struct tm (the
"broken-down time structure") are all that the application needs to
ensure are set in order to perform a specific conversion. While you're
certainly correct that the implementation can look at other data, that
other data cannot come from the struct, as there's no expectation that
any of that will have been set by the application, and there's no field
in the struct (which would need to be some kind of checksum to detect
changes to other fields made by the application) by which it is possible
to determine that other fields have consistent values with the fields of
interest to the conversion.

But yes, you can certainly use what is in TZ, or anything obtained from
its use, and anything you can get from the LC_TIME locale.

And while:

  tm_zone is an extension to the C standard and the C standard has
  nothing to say about what tm_zone’s contents can or should be.

is certainly true, this conclusion:

  And requiring tm_zone to have sensible contents is not a violation either:

is incorrect. A conforming C application cannot set tm_zone as no such
thing exists. Hence an implementation which is supposed to support
conforming C applications cannot use data from that field, as it has no
way to know if anything is there or not (or anything relevant for sure).
The implementation is certainly allowed to set that field (as in localtime()
for example), a conforming application will never look there, and so will
never care if it is set or not.

POSIX is intended to support conforming C applications.

And finally:

   When the standard isn’t clear on a matter it’s better to clear it up;

That's right, but in this case it is clear. Always has been (even if the
wording was a bit bizarre in some places). Furthermore, since this affects C
programs, it should be fixed, if any fixing is needed, by the C standard first,
not by some end-run performed here.
(0006801)
eggert (reporter)
2024-05-24 22:56

> A conforming C application cannot set tm_zone as no such
thing exists.

No, there are many ways a conforming C application can set tm_zone. The several real-world applications I cited in Note: 0006793 all set tm_zone by calling gmtime, and these apps work fine on GNU/Linux, macOS, z/OS, etc. This is common practice.

Obviously we disagree about how to interpret POSIX. I find the overly-strict interpretation unconvincing because (a) significant real-world implementations have disagreed with that interpretation for decades, (b) it's easy to find real-world applications that rely on these implementations' behavior, and (c) it's hard to find real-world applications that rely on the overly-strict interpretation.

This is not a new issue. What's new is that this lack of clarity or glitch in the standards is being brought to the standardization bodies' attention.
(0006802)
kre (reporter)
2024-05-25 17:12

Re Note: 0006801

    there are many ways a conforming C application can set tm_zone.

There cannot be, as C does not have such a field, any attempt to
reference it, in any way, directly or indirectly, would be non-conforming.

  [...] I cited in Note: 0006793 all set tm_zone by calling gmtime

That's not the application setting tm_zone (etc) it is the implementation.
That's allowed, as an extension to the standard. And while the application
certainly knows it called gmtime() there is no way (assuming it is a conforming
C application) that it can deduce from doing so that tm_zone (or tm_gmtoff
which is more relevant for %s) was set by that call.

Such a program should be able to be compiled and run on a system in which
gmtime() does not set those fields, which is perfectly allowable for
implementations of the C standard (it will not be for POSIX applications
after the new standard appears, at least for implementations claiming
conformance). On such a system the (conforming) program should run and
produce the same results as on a system that happens to set the new fields.

Furthermore, strftime() needs to work correctly when the struct tm passed
to it has never seen the results of any previous implementation provided
function, and also when it has, but that was in some previous incarnation,
whose use has been completed, and the struct has now been repurposed to
be used just for strftime, with values entirely unrelated to the one it
was previously used for, and only the required values for the format
string used assigned (new) values.

Unless you can provide me with text in the C or POSIX standard which says
that gmtime() localtime() or something equivalent must have generated
(or updated, perhaps via mktime()) any struct tm which is to be passed
to strftime() then the implementation of strftime() simply *must* work
when that has not happened. "work" means producing the correct result,
which then is obviously also required to be the result if struct tm
with the same required field values had been produced by localtime() or
gmtime() - two different results cannot both be correct (normally anyway).

It makes no difference how common it might be for strftime() to be called
with a struct tm returned from localtime() or gmtime(). Unless there is
a requirement somewhere making that mandatory for applications, then
we cannot assume that it has happened. Nor can we require that tm_gmtoff
or tm_zone be set by the application, as conforming C applications cannot
do that, and strftime() is a C standard interface (it would be different were
it one added by POSIX.)

It is irrelevant how many non-conforming, extended, or simply buggy,
implementations, or applications which depend upon them, you can find.
And I'd note, that until earlier this year, your implementation (or ado's
or whoever it was who added strftime()), which I suspect is, or is the
basis of, most real world implementation (certainly of ones which include
the tm_gmtoff and tm_zone fields) was not one of the ones which abused
strftime(%s) the way you are now proposing.
(0006908)
geoffclare (manager)
2024-10-10 15:27

Interpretation Response
------------------------

The standard clearly states that when the input is in the normal range the strftime() %s conversion calculates the number of seconds since the Epoch as described for mktime(), and conforming implementations must conform to this.

When the input is not in the normal range, the standard is unclear on this issue, and no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor.

Rationale:
-------------
When the input is in the normal range: there is no requirement for applications to set the tm_gmtoff member and therefore implementations cannot rely on it having been set. However, in situations where there are two valid results of the conversion, an implementation may choose to make use of tm_gmtoff to decide between these two values (provided it can do so safely if tm_gmtoff is uninitialized), even though the strftime() %s conversion is not specified as making use of tm_gmtoff, since returning either of the values is allowed by the standard and the method used to choose between them is internal implementation detail that does not affect applications.

When the input is not in the normal range: the standard states that the strftime() %s conversion calculates the number of seconds since the Epoch as described for mktime(), implying that out of range member values should be handled as for mktime(), but it also states ``If any of the specified values are outside the normal range, the characters stored are unspecified.''

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------

On page 452 lines 15753-15754 section XBD <time.h>, add CX shading to the lines for tm_gmtoff and tm_zone.

On page 1211 line 41369 section gmtime() DESCRIPTION, change (already in CX shading):
...where the names in the structure and in the expression correspond.

to:
...where the names in the structure and in the expression correspond; additionally, the tm_zone field shall be set to a pointer to an implementation-defined string set to "UTC" or "GMT", which shall have static storage duration.


On page 1211 line 41385 section gmtime() RETURN VALUE, delete:
The structure’s tm_zone member shall be set to a pointer to the string "UTC", which shall have static storage duration.


On page 1427 line 47885 section mktime(), change:
The mktime() function shall make use of only the tm_year, tm_mon, tm_mday, tm_hour, tm_min, tm_sec, and tm_isdst members of the structure pointed to by timeptr; the values of these members shall not be restricted to the ranges described in <time.h>.

to:
The mktime() function shall make use of only the tm_year, tm_mon, tm_mday, tm_hour, tm_min, tm_sec, and tm_isdst members of the structure pointed to by timeptr, <CX>except that tm_gmtoff may also be used in order to choose between two valid return values, provided it can be used safely if uninitialized</CX>; the values of these members shall not be restricted to the ranges described in <time.h>.


On page 1428 line 47952 section mktime(), change:
shall calculate the time since the Epoch value using either the offset in effect before the change or the offset in effect after the change.

to:
shall calculate the time since the Epoch value using either the offset in effect before the change or the offset in effect after the change; mktime() may use the value of tm_gmtoff to decide which of these two results is the more appropriate to return, provided it can do so safely if tm_gmtoff is uninitialized.


On page 2134 line 69724 section strftime(), change:
The functionality described on this reference page is aligned with the ISO C standard. Any conflict between the requirements described here and the ISO C standard is unintentional. This volume of POSIX.1-2024 defers to the ISO C standard.

to:
Except for the members of the tm structure used by the <tt>z</tt> and <tt>Z</tt> conversion specifiers (see below), the functionality described on this reference page is aligned with the ISO C standard. Any other conflict between the requirements described here and the ISO C standard is unintentional. This volume of POSIX.1-2024 defers to the ISO C standard for all strftime() functionality except in relation to this aspect of the <tt>z</tt> and <tt>Z</tt> conversion specifiers.


On page 2135 line 69760 section strftime(), change:
If any of the specified values are outside the normal range, the characters stored are unspecified.

to:
If any of the specified values are outside the normal range, as if set by localtime(), the characters stored are unspecified.


On page 2135 line 69764 section strftime(), change:
Local timezone information shall be set as though strftime() called tzset().

to:
It is implementation-defined whether local timezone information is set as though strftime() called tzset().


On page 2136 line 69823 section strftime() (%s), change:
Replaced by the number of seconds since the Epoch as a decimal number, calculated as described for mktime(). [tm_year, tm_mon, tm_mday, tm_hour, tm_min, tm_sec, tm_isdst]

to:
Replaced by the number of seconds since the Epoch as a decimal number, calculated as described for mktime(), except that the tm structure member values used by this conversion need to be within the normal range, as if set by localtime(), to avoid unspecified output. [tm_year, tm_mon, tm_mday, tm_hour, tm_min, tm_sec, tm_isdst, tm_gmtoff]


On page 2137 line 69854 section strftime() (%z), change:
<CX>If tm_isdst is zero, the standard time offset is used. If tm_isdst is greater than zero, the daylight saving time offset is used. If tm_isdst is negative, no characters are returned.</CX> [tm_isdst, <CX>tm_gmtoff</CX>]

to:
[<CX>tm_gmtoff</CX>]


On page 2137 line 69858 section strftime() (%Z), change:
Replaced by the timezone name or abbreviation, or by no bytes if no timezone information exists. [tm_isdst, <CX>tm_zone</CX>]

to:
Replaced by the timezone name or abbreviation, or by no bytes if no timezone information exists. <CX>If the tm_zone value was not set by one of the standard functions that set it, the application shall ensure that it is either a null pointer (indicating that no timezone information exists) or points to a string containing the timezone name or abbreviation. [tm_zone].</CX>


On page 2137 line 69862 section strftime(), change:
If a struct tm broken-down time structure is created by localtime() or localtime_r(), or modified by mktime(), and the value of TZ is subsequently modified, the results of the <tt>%Z</tt> and <tt>%z</tt> strftime() conversion specifiers are undefined, when strftime() is called with such a broken-down time structure.

If a struct tm broken-down time structure is created or modified by gmtime() or gmtime_r(), it is unspecified whether the result of the <tt>%Z</tt> and <tt>%z</tt> conversion specifiers shall refer to UTC or the current local timezone, when strftime() is called with such a broken-down time structure.

to:
If a struct tm broken-down time structure is created by localtime() or localtime_r(), or modified by mktime(), and the value of TZ is subsequently modified, the results of the <tt>Z</tt> strftime() conversion specifier are undefined, when strftime() is called with such a broken-down time structure.


After page 2139 line 69953 section strftime(), add a new paragraph to APPLICATION USAGE:
Using a tm structure populated by gmtime() or gmtime_r() with the %s, %z, and %Z conversion of strftime() is not portable unless local time is UTC, as the implementation might process the conversions either by inspecting the structure’s tm_gmtoff and tm_zone members, or by inspecting global state.


After page 2142 line 70092 section strftime(), add a new paragraph to RATIONALE:
The ISO C standard specifies that the <tt>z</tt> and <tt>Z</tt> conversion specifiers use only the tm_isdst member of the tm structure, whereas POSIX.1 implementations have the additional structure members tm_gmtoff and tm_zone which provide information used in these conversions. Therefore this standard does not defer to the ISO C standard in relation to this aspect of the <tt>z</tt> and <tt>Z</tt> conversion specifiers. A future version of the ISO C standard is expected to allow the use of ``all members, including any non-standard additional members''.


On page 2376 line 77023 section wcsftime(), change:
The functionality described on this reference page is aligned with the ISO C standard. Any conflict between the requirements described here and the ISO C standard is unintentional. This volume of POSIX.1-2024 defers to the ISO C standard.

to:
Except for the members of the tm structure used by the <tt>z</tt> and <tt>Z</tt> conversion specifiers (see [xref to strftime()]), the functionality described on this reference page is aligned with the ISO C standard. Any other conflict between the requirements described here and the ISO C standard is unintentional. This volume of POSIX.1-2024 defers to the ISO C standard for all wcsftime() functionality except in relation to this aspect of the <tt>z</tt> and <tt>Z</tt> conversion specifiers.
(0006912)
agadmin (administrator)
2024-10-11 11:03

Interpretation Proposed: 11 October 2024

- Issue History
Date Modified Username Field Change
2024-01-15 23:56 eggert New Issue
2024-01-15 23:56 eggert Name => Paul Eggert
2024-01-15 23:56 eggert Organization => UCLA Computer Science Dept.
2024-01-15 23:56 eggert User Reference => strftime-%s
2024-01-15 23:56 eggert Section => strftime
2024-01-15 23:56 eggert Page Number => 2136
2024-01-15 23:56 eggert Line Number => 69836-69837
2024-01-16 01:29 steffen Note Added: 0006623
2024-01-16 01:42 steffen Note Added: 0006624
2024-01-16 01:42 steffen Note Deleted: 0006623
2024-02-01 16:42 nick Relationship added related to 0001533
2024-02-12 16:11 eblake Note Added: 0006651
2024-02-25 06:50 kre Note Added: 0006677
2024-02-25 06:54 kre Note Edited: 0006677
2024-02-26 18:28 eggert Note Added: 0006688
2024-02-26 19:23 eblake Relationship added related to 0001816
2024-02-26 19:32 eblake Note Added: 0006689
2024-02-26 19:52 eblake Relationship added child of 0001612
2024-02-26 19:55 eblake Relationship added child of 0000169
2024-02-26 20:02 eblake Note Added: 0006690
2024-02-29 12:10 geoffclare Note Added: 0006691
2024-02-29 12:15 geoffclare Note Added: 0006692
2024-02-29 16:19 shware_systems Note Added: 0006693
2024-03-01 10:09 geoffclare Note Added: 0006698
2024-03-02 09:02 eggert Note Added: 0006700
2024-03-03 01:20 kre Note Added: 0006701
2024-03-07 10:44 geoffclare Note Added: 0006706
2024-03-07 11:04 geoffclare Note Added: 0006707
2024-03-11 09:55 geoffclare Note Added: 0006718
2024-05-23 08:04 eggert Note Added: 0006793
2024-05-23 08:07 eggert Note Added: 0006794
2024-05-23 10:47 geoffclare Note Added: 0006795
2024-05-23 16:57 kre Note Added: 0006797
2024-05-24 18:59 eggert Note Added: 0006798
2024-05-24 21:32 kre Note Added: 0006799
2024-05-24 22:56 eggert Note Added: 0006801
2024-05-25 17:12 kre Note Added: 0006802
2024-10-10 15:25 geoffclare Project Issue 8 drafts => 1003.1(2024)/Issue8
2024-10-10 15:27 geoffclare Note Added: 0006908
2024-10-10 15:29 geoffclare Interp Status => Pending
2024-10-10 15:29 geoffclare Final Accepted Text => Note: 0006908
2024-10-10 15:29 geoffclare Status New => Interpretation Required
2024-10-10 15:29 geoffclare Resolution Open => Accepted As Marked
2024-10-10 15:30 geoffclare Tag Attached: tc1-2024
2024-10-11 11:03 agadmin Interp Status Pending => Proposed
2024-10-11 11:03 agadmin Note Added: 0006912


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker