View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0001616 | Issue 8 drafts | Shell and Utilities | public | 2022-11-08 23:03 | 2025-03-11 17:33 |
| Reporter | illiliti | Assigned To | |||
| Priority | normal | Severity | Editorial | Type | Enhancement Request |
| Status | New | Resolution | Open | ||
| Product Version | Draft 2.1 | ||||
| Name | Mark Lundblad | ||||
| Organization | |||||
| User Reference | |||||
| Section | Shell and Utilities | ||||
| Page Number | - | ||||
| Line Number | - | ||||
| Final Accepted Text | |||||
| Summary | 0001616: Standardize mktemp utility | ||||
| Description | The current standard has no reliable and convenient utility to create temporary file or directory, despite that low-level interfaces, mkstemp/mkdtemp, are available for this use case. The mktemp(1) utility is a perfect candidate to fill this gap. Its interface is fairly simple and mostly consistent across various implementations, however not without some differences. I evaluated GNU, LSB, NetBSD, FreeBSD, OpenBSD, Illumos, Solaris, sbase, toybox, busybox implementations and here is what I found: - FreeBSD, NetBSD accept multiple templates. Others do not. - FreeBSD does not specify -p option. Others do. - LSB does not specify -d and -p options. Others do. - FreeBSD, NetBSD accept parameter for -t option. Others do not. - FreeBSD, NetBSD use -t option to specify prefix. Others use it to explicitly tell mktemp to create temporary file in temporary directory rather than in current directory. - All have different default value for template. - All have different requirements regarding how many Xs template should contain, but it seems they all accept six Xs. - sbase, Illumos, Solaris override -p parameter with TMPDIR value if it is set. Others do not. - GNU overrides -p parameter with TMPDIR value or /tmp if parameter is empty string. Others do not. - OpenBSD, Illumos, Solaris(not sure), toybox do not allow '/' in template if -p option is passed. Others do. Given that, I think mktemp interface is not completely lost and below is my attempt to make it portable. | ||||
| Desired Action | Standardize mktemp utility. Here is the most minimal and portable example: NAME mktemp - create temporary unique file or directory SYNOPSIS mktemp [-dqu] [-p directory] [template] DESCRIPTION mktemp creates a temporary file based on a template as if by calling mkstemp(3). If no template is specified, an implementation-defined value shall be used instead. If template does not contain exactly six Xs at the end, the behavior is unspecified. If template does not contain '/', the file shall be created in the current directory. Otherwise, the file shall be created by its path. The filename of created temporary file shall be written to the standard output. OPTIONS -d Create directory instead of file as if by calling mkdtemp(3). -p directory Change directory where temporary file will be created. With this option, it is unspecified what happens if template contains '/'. If TMPDIR is set, it is unspecified whether its value take precedence over passed parameter or vice versa. If passed parameter is empty string, the behavior is unspecified. -q Fail silently if an error occurs. -u Operate in unsafe mode. A unique name is generated, but the temporary file shall be unlinked before mktemp exits. Use of this option is not encouraged. SEE ALSO mkdtemp(3), mkstemp(3) | ||||
| Tags | No tags attached. | ||||
|
|
While i accept it as natural as it is so widely used, please note that POSIX says for pathchk(1): The pathchk utility was new for the ISO POSIX-2: 1993 standard. It, along with the set −C(noclobber) option added to the shell, replaces the mktemp, validfnam, and create utilities that appeared in early proposals. So what you do is if ( set -C ) >/dev/null 2>&1; then set +C else # For heaven's sake auto-redirect on SunOS/Solaris if [ -f /usr/xpg4/bin/sh ] && [ -x /usr/xpg4/bin/sh ]; then exec /usr/xpg4/bin/sh "${0}" "${@}" else synopsis 1 'sh(1)ell without "set -C" (for safe temporary file creation)' fi fi and then old_umask=`umask` umask 077 i=1 while :; do tmpfile="${tmpdir}/MYNAME-${i}.tmp" ( set -C : > "${tmpfile}" ) >/dev/null 2>&1 && break i=`expr ${i} + 1` if [ ${i} -gt ${max} ]; then echo >&2 'Cannot create a temporary file within '"${tmpdir}" exit ${EX_TEMPFAIL} fi done trap "exit ${EX_TEMPFAIL}" HUP INT QUIT PIPE TERM trap "trap \"\" HUP INT QUIT PIPE TERM EXIT; rm -f ${tmpfile}" EXIT umask ${old_umask} ..after finding a tmpdir and doing some other setup. |
|
|
Yeah, your example should work, but it is not simple. Look at how simple mktemp is: tmpfile=$(mktemp) or directory: tmpdir=$(mktemp -d) mktemp is impossible to misuse unlike shell. Plus it is already more-or-less portable and widespread on all open-source systems. Also unrelated, but it seems that your example does not work with yash when tmpfile is a dangling symlink. Looks like a yash bug. Reported. |
|
|
Here's a few more data points. The uutils mktemp is basically same as coreutils, as expected: (ins)root 5 (7104) 0 ~ # uu-mktemp --help create a temporary file or directory. Usage: uu-mktemp [OPTION]... [TEMPLATE] Arguments: [template] Options: -d, --directory Make a directory instead of a file -u, --dry-run do not create anything; merely print a name (unsafe) -q, --quiet Fail silently if an error occurs. --suffix <SUFFIX> append SUFFIX to TEMPLATE; SUFFIX must not contain a path separator. This option is implied if TEMPLATE does not end with X. -p, --tmpdir [<DIR>] interpret TEMPLATE relative to DIR; if DIR is not specified, use $TMPDIR ($TMP on windows) if set, else /tmp. With this option, TEMPLATE must not be an absolute name; unlike with -t, TEMPLATE may contain slashes, but mktemp creates only the final component -t Generate a template (using the supplied prefix and TMPDIR (TMP on windows) if set) to create a filename template [deprecated] -h, --help Print help information -V, --version Print version information Bash's "example" loadable mktemp: (ins)root 5 (7104) 0 ~ # bash -c 'enable -f mktemp{,}; type mktemp; help mktemp' mktemp is a shell builtin mktemp: mktemp [-d] [-q] [-t prefix] [-u] [-v varname] [template] ... Make unique temporary file name Take each supplied filename template and overwrite a portion of it to create a filename, which is unique and may be used by the calling script. TEMPLATE is a string ending in some number of 'X's. If TEMPLATE is not supplied, shtmp.XXXXXX is used and $TMPDIR is used as the name of the containing directory. Files are created u+rw; directories are created u+rwx. Options, if supplied, have the following meanings: -d Create a directory instead of a file -q Do not print error messages about file creation failure -t PREFIX Use PREFIX as the directory in which to create files -u Do not create anything; simply print a name -v VAR Store the generated name into shell variable VAR Any PREFIX supplied with -t is ignored if TEMPLATE is supplied. The return status is true if the file or directory was created successfully; false if an error occurs or VAR is invalid or readonly. And the AST libcmd builtin under ksh93 (edit: u+m also has this one): (ins)root 5 (7104) 0 ~ # ksh93v -c 'builtin mktemp; type mktemp; mktemp --man' mktemp is a shell builtin NAME mktemp - make temporary file or directory SYNOPSIS mktemp [ options ] [ prefix [ directory ] ] DESCRIPTION mktemp creates a temporary file with optional base name prefix prefix. If prefix is omitted then tmp is used and --tmp is implied. A consecutive string of trailing X's in prefix is replaced by a pseudorandom combination of [0-9a-zA-Z]characters, otherwise the first 5 characters of prefix is catenated with a pseudorandom string to construct a file name component of 14 characters. If directory is specified or if prefix contains a directory prefix then that directory overrides any of the directories described below. A temporary file will have mode rw------- and a temporary directory will have mode rwx------, subject to umask(1). Generated paths have these attributes: * Lower case to avoid clashes on case ignorant filesystems. * Pseudo-random part to deter denial of service attacks. * Default pseudo-random part (no specific X... template) formatted to accomodate 8.3 filesystems. A consecutive trailing sequence of X's in prefix is replaced by the pseudo-random part. If there are no X's then the pseudo-random part is appended to the prefix. OPTIONS -d, --directory Create a directory instead of a regular file. -m, --mode=mode Set the mode of the created temporary to mode. mode is symbolic or octal mode as in chmod(1). Relative modes assume an initial mode of u=rwx. -p, --default=directory Use directory if the TMPDIR environment variable is not defined. Implies --tmp. -q, --quiet Suppress file and directory error diagnostics. -R, --regress=seed The pseudo random generator is seeded with seed instead of process/system specific transient data. Use for testing only. A seed of 0 is silently changed to 1. -t, --tmp|temporary-directory Create a path rooted in a temporary directory. -u, --unsafe|dry-run Check for file/directory existence but do not create. Use this for testing only. SEE ALSO mkdir(1), pathtemp(3), mktemp(3) IMPLEMENTATION version mktemp (AT&T Research) 2012-12-12 author Glenn Fowler <glenn.s.fowler@gmail.com> author David Korn <dgkorn@gmail.com> copyright Copyright (c) 1992-2015 AT&T Intellectual Property license http://www.eclipse.org/org/documents/epl-v10.html In addition ksh93 has the `>;word` redirect which opens a temporary file and renames it to word if the command returned success. It can serve a similar purpose. The canonical method I thought everyone knew is m4's mkstemp ( https://mywiki.wooledge.org/BashFAQ/062#Using_m4 ) Another reasonably widely-available tool - xfs_io from xfsprogs - can be used to open a file with O_TMPFILE. It works on several platforms including with some non-xfs filesystems. |
|
|
Before page 3178 (XCU more), insert new page(s) containing the following: NAME mktemp - create temporary unique file or directory SYNOPSIS mktemp [-dq] [-p directory] [template] DESCRIPTION The mktemp utility shall create a temporary file or |
|
|
This is a copy of 0001616:0007089 with some wording tweaks. Before page 3178 (XCU more), insert new page(s) containing the following: NAME mktemp - create temporary unique regular file or directory SYNOPSIS mktemp [-dq] [-p directory] [template] DESCRIPTION The mktemp utility shall create a temporary regular file or directory based on a template, as if by calling mkstemp( ) or mkdtemp( ), respectively. OPTIONS
|
|
|
rm "$file" Should rather be rm -- "$file"or maybe even rm -f -- "$file"to avoid potential user prompts. No support for mktemp /tmp/XXXXXX.htmlor mktemp -s .htmlthen? |
|
|
No support formktemp /tmp/XXXXXX.htmlormktemp -s .html File extension in the template would mean deviating from current implementations and no longer being an interface to mkstemp() & mkdtemp() as they require the six X's to be the last characters: The application shall ensure that the string provided in template is a pathname ending with at least six trailing 'X' characters. — From mkdtemp(), mkostemp(), mkstemp() specification in issue 8 / POSIX.1-2024 Also out of curiosity, which implementation has -s ? Closest seems to be GNU coreutils with --suffix, but then it seems quite alone in that as [NetBSD, OpenBSD, FreeBSD, illumos, busybox, sbase] doesn't have -s or an equivalent. |
|
|
Several issues were identified the the latest draft proposal. Among other things, several implementations output relative paths if the inputs are relative (as opposed to always outputting an absolute path no matter the inputs). More exploration is needed to compare the different implementations to see what can be specified as portable. |
|
|
With regards to 0001616:0007095, rm "$file" will always work if "$file" is absolute (which is true in the current draft wording, but 0001616:0007098 points out that it is not true of existing implementations); use of rm -- "$file" would only be needed if we relax the wording to allow relative filenames as the output of mktemp in some situations such that "$file" could then begin with '-'. |
|
|
A few notes: First, I support including this utility, including the "-q" option!! Making it easy to *correctly* create a temporary file is a good thing. The -q option is widely used and supported, so it should be included. This utility can counter a particular kind of attack (where the attacker pre-creates a temporary file, and the program "creates" a file but didn't really create it), and makes it easier to "do the right thing". Making it easier to do the secure thing is an *improvement*. Second, I fear that readers won't realize that this *always* creates a new file or directory, because sometimes the term "create" is informally used to mean "create if it doesn't already exist". I suggest modifying the description "The mktemp utility shall create a temporary regular file or directory..." by adding the word "new" before the word "temporary". By adding one word, it's obvious this is a NEW file or directory. You could argue that this is already true, but I think adding one 3-letter word for clarity is worth it. Third, PLEASE change example 1 from rm "$file" to rm -- "$file" ; the addition of -- is ALWAYS allowed under POSIX. Since we know some existing implementations return relative filenames, even if they're not compliant, it would avoid a problem. Also, people often copy examples and then make changes; changing the template to something else that might begin with "-" would invalidate the argument in a way not obvious to skimming readers. Fourth, example 2 has an extraneous </tt> formatting item at its end. That won't execute well :-). Fifth, under application usage, add: "Note that some historical implementations reply with a relative pathname, not an absolute pathname." That way, people are warned about the old behavior. Sixth, it's too bad that a more general templating mechanism isn't widely supported. I suggest adding a FUTURE DIRECTIONS section to encourage it in the future, like this: FUTURE DIRECTIONS Future versions may allow a sequence of at least trailing <tt>'X'</tt> characters anywhere in the template, where the last such sequence is replaced. |
|
|
Responding to 0001616:0007100: I'm ambivalent on whether -q should be standardized, since it is redundant with 2>/dev/null. Even if we decide to exclude it from the next attempted revision of text, it should at least be mentioned in the non-normative text, since it does seem to be present in a number of implementations. Tweaking the wording to emphasize that a NEW file / directory is created is worthwhile. In fact, while we were reviewing the current proposed text, we noted that "OUTPUT FILES None.", although consistent with some other utilities in the standard (mkdir, sh, m4, ...), was misleading, and we will be opening a separate bug to address that more uniformly in existing utilities. I'm still trying to figure out how many implementations output relative names, and if there is ever a case where an implementation outputs a relative name that is NOT relative to the current working directory where mktemp was executed - that would be undesirable. But since more than just GNU Coreutils mktemp (where I'm listed as one of the original authors, although it's been years since I last touched that code) appear to have modes with relative outputs, it is very likely that the next revision will allow relative output, and as such improve the examples to use rm -- "$file" to match. As for the </tt>, it turns out that Mantis recognizes <less-than>pre<greater-than>but not <less-than>tt<greater-than>and the corresponding close tag; I've updated that typo fix in-place. The idea of a FUTURE DIRECTIONS adding support for templating in a suffix is nice; in fact, we may want to standardize mkstemps(3) or mkostemps(3) in XSH (at least GNU libc has those), at which point having mktemp(1) also do suffixing in XCU is an obvious idea. |
|
|
I think "-q" should be standardized. It's widely supported, and most importantly, it's already in use in scripts. In many cases it doesn't matter *why* it failed, it failed. I think it'd be fine if mktemp was also allowed to output relative pathnames IF they are relative to the current directory. I think outputting absolute pathnames is wiser, so that should be an option, but allowing relative to the current directory should be fine. I can't imagine a case where returning a relative pathname from anything other than the current directory would make sense. What you're going to do next with that pathname is use it. If it's a relative pathname not relative to the current directory, the wrong pathname will NECESSARILY be used. I didn't notice the "OUTPUT FILES" thing, good catch. Since the whole point of mktemp is to create a file/directory, that seems fundamentally wrong even if it's "consistent" in some sense. |
|
|
One I noticed today, given the existence of custom generators (like due to -u) in most implementations, I think it would be worth it to explicitly make a newline in the template trigger at least a warning if not an error (I'd lean towards error since mktemp output is described as "a line" in STDOUT section). |
|
|
Re:0001616:0007103 There isn't one implementation where a newline causes trouble. |
|
|
Re:0001616:0007104 It's about consistency with the "Future Directions" section, not whether implementations today forbid newlines, see https://www.austingroupbugs.net/view.php?id=251 applied in issue 8 / POSIX.1-2024. |
|
|
Re:0001616:0007105 The resolution to bug #251 encourages rejecting filenames that include newlines in their names. [T]o explicitly make a newline in the template trigger at least a warning if not an error is not encouragement. |
|
|
Reusing your words might make it more understandable? Effectively I'm saying that due to bug #251, mktemp implementers are encouraged to reject filename [template] argument with newlines. Which for mktemp is specially relevant because it's stdout is newline-terminated with applying the filename [template] argument, or even newline-separated in implementations accepting multiple [template] arguments. |
|
|
What to do in FUTURE DIRECTIONS is tricky. If mktemp had already been in the standard when bug 251 was resolved, it would have been given both the "PARAGRAPH DELIM" and "PARAGRAPH DIRENT" additions because it both creates files and writes a newline-terminated pathname to standard output. However, because mktemp will be added in Issue 9, and Issue 9 could potentially have changes to all the utilities that received "PARAGRAPH DELIM" and/or "PARAGRAPH DIRENT" to enact the future direction, we won't know until nearer the time how to make mktemp consistent with those. Perhaps the best thing would be to include matching FUTURE DIRECTIONS text for now but add a "Note to the editor" about the issue. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2022-11-08 23:03 | illiliti | New Issue | |
| 2022-11-08 23:03 | illiliti | Name | => Mark Lundblad |
| 2022-11-08 23:03 | illiliti | Section | => Shell and Utilities |
| 2022-11-08 23:03 | illiliti | Page Number | => - |
| 2022-11-08 23:03 | illiliti | Line Number | => - |
| 2022-11-08 23:30 | steffen | Note Added: 0006038 | |
| 2022-11-09 13:18 | illiliti | Note Added: 0006041 | |
| 2023-02-22 13:30 | ormaaj | Note Added: 0006166 | |
| 2023-02-22 19:10 | ormaaj | Note Edited: 0006166 | |
| 2025-02-28 18:37 | eblake | Note Added: 0007089 | |
| 2025-03-06 11:09 | geoffclare | Note Added: 0007093 | |
| 2025-03-06 11:10 | geoffclare | Note Edited: 0007093 | |
| 2025-03-06 11:10 | geoffclare | Note Edited: 0007093 | |
| 2025-03-06 11:11 | geoffclare | Note Edited: 0007093 | |
| 2025-03-06 15:15 | eblake | Note Edited: 0007093 | |
| 2025-03-06 15:53 | geoffclare | Note Edited: 0007093 | |
| 2025-03-06 17:19 | stephane | Note Added: 0007095 | |
| 2025-03-06 18:02 | lanodan | Note Added: 0007097 | |
| 2025-03-06 18:04 | lanodan | Note Edited: 0007097 | |
| 2025-03-06 18:05 | lanodan | Note Edited: 0007097 | |
| 2025-03-07 15:11 | eblake | Note Added: 0007098 | |
| 2025-03-07 15:14 | eblake | Note Added: 0007099 | |
| 2025-03-07 20:41 | dwheeler | Note Added: 0007100 | |
| 2025-03-07 21:46 | eblake | Note Edited: 0007093 | |
| 2025-03-07 21:54 | eblake | Note Added: 0007101 | |
| 2025-03-07 21:57 | eblake | Note Edited: 0007101 | |
| 2025-03-07 21:57 | eblake | Note Edited: 0007101 | |
| 2025-03-07 22:32 | dwheeler | Note Added: 0007102 | |
| 2025-03-08 05:15 | lanodan | Note Added: 0007103 | |
| 2025-03-08 06:08 | oguzismailuysal | Note Added: 0007104 | |
| 2025-03-08 06:59 | lanodan | Note Added: 0007105 | |
| 2025-03-08 09:16 | oguzismailuysal | Note Added: 0007106 | |
| 2025-03-08 11:23 | lanodan | Note Added: 0007107 | |
| 2025-03-11 17:33 | geoffclare | Note Added: 0007110 |
