View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000884 | 1003.1(2013)/Issue7+TC1 | Shell and Utilities | public | 2014-10-16 19:46 | 2019-06-10 08:54 |
| Reporter | rhansen | Assigned To | |||
| Priority | normal | Severity | Editorial | Type | Enhancement Request |
| Status | Closed | Resolution | Accepted | ||
| Name | Richard Hansen | ||||
| Organization | BBN | ||||
| User Reference | |||||
| Section | sh | ||||
| Page Number | 3191-3192 | ||||
| Line Number | 106811-106813 | ||||
| Interp Status | Approved | ||||
| Final Accepted Text | See 0000884:0002531. | ||||
| Summary | 0000884: require sh to set IFS to <space><tab><newline> on startup | ||||
| Description | Scripts almost never explicitly set IFS, so permitting the shell to inherit IFS from the environment is a recipe for hard-to-diagnose bugs (including possible security vulnerabilities). While I have not done a thorough investigation, all of the implementations I've looked at already set IFS to <space><tab><newline> on startup. | ||||
| Desired Action | On page 2325 lines 73796-73798, change:Implementations may ignore the value of IFS in the environment, or the absence of IFS from the environment, at the time the shell is invoked, in which case the shell shall set IFS to <space><tab><newline> when it is invoked. to: The shell shall set IFS to <space><tab><newline> when it is invoked. On pages 3191-3192, remove lines 106806-106813 (IFS in sh's Environment Variables section). On page 3203 lines 107331-107334 (sh Rationale), change: The KornShell ignores the contents of IFS upon entry to the script. A conforming application cannot rely on importing IFS. One justification for this, beyond security considerations, is to assist possible future shell compilers. to: One justification for ignoring the contents of IFS upon entry to the script, beyond security considerations, is to assist possible future shell compilers. | ||||
| Tags | tc2-2008 | ||||
|
|
I would think one of the reasons, theoretically, for allowing an import from environment is NLS support, where some locales may commonly use extra separators. Unicode certainly has extra characters it considers white space, ZWNBSP being the most common in use as endian indicator. Resetting IFS in each script created in that environment would be onerous on those developers, I'd think. It's as bad, IMO, as each localized application having to switch from the "C" locale at start up explicitly, rather than use the LC_* and LANG settings. I would rather see it left as a) 2nd level interactive shells (controlling terminals) optionally inherit, and b) possibly as a new requirement, subshells shall inherit the parent shell's setting. It's also plausible to have an ignore_ifs set -o option. As a requirement I can see this being plausible for a sh invocation as the system console session, so there is no reliance of IFS being set by the OS, but that's it. |
|
|
Interpretation response ------------------------ The standard states that implementations may ignore the value of IFS in the environment, or the absence of IFS from the environment, at the time the shell is invoked, in which case the shell shall set IFS to <space><tab><newline> when it is invoked, and conforming applications must be prepared to accept this. However, concerns have been raised about this which are being referred to the sponsor. Rationale: ------------- Some old shells did inherit IFS from the environment, but since most shell scripts do not set IFS as one of the steps in their initialization, this creates a security hole. Most, if not all, recent shells initialize IFS when the shell is invoked and do not change IFS in a subshell environment. This is the desired behavior. Notes to the Editor (not part of this interpretation): ------------------------------------------------------- Make the changes suggested in the Desired Action. |
|
|
Interpretation proposed 20 Feb 2015 |
|
|
Interpretation approved 23rd March 2015 |
|
|
Please change the Desired Action from… | The shell shall set IFS to <space><tab><newline> when it is invoked. … to… | The shell shall set IFS to <space><tab><newline> when it is invoked, if it is set. Reason: mksh currently sets IFS to <space><tab><newline> when it was set in the environment at startup (even if it was set to the empty value), but does not set it if unset. An unset IFS is equivalent to <space><tab><newline> per POSIX, so this is functionally identical but permits existing behaviour. |
|
|
Allowing the shell to leave IFS unset is not quite functionally identical to always setting it to <space><tab><newline>. Consider a script that does the following:In the above case, if IFS was unset before that block of code then it will be set to the empty string after. While I consider that to be a bug in the script, it is a common bug that requires some effort to fix:backup_IFS=$IFS # do stuff here IFS=$backup_IFS That being said, I would not be opposed to your proposed change if others feel that compatibility with existing implementations is more desirable. Because this particular bug report has already been marked as resolved, I would prefer that you submit a new bug report with your suggested change. |
|
|
Hmm, I just realized that this is tagged for TC2, not Issue 8. Given mksh's current behavior, perhaps it would be better to permit implementations to leave IFS unset for TC2, and file a new bug report for Issue 8 to require implementations to set IFS to <space><tab><newline>. |
|
|
@rhansen, actually, backup_IFS (we call it $saveIFS usually) is a good point. I retract my suggestion, thanks for shining light onto that corner; I will change mksh to match, then. (We could use $' \t\n' instead of your construct, but I agree it would be an absolutely unnecessary, and probably not often done, burden on script writers. Although I admit I recently find myself to set IFS explicitly instead of saving/restoring it…) |
|
|
Oops. I was just going to change mksh to match, then I saw that it already *does* set IFS unconditionally. I just made a newbie mistake when checking for existence. So this requirement is just fine and already met. Sorry for the noise… |
|
|
Collateral issues, potentially... Just want to make sure making it a shall instead of may is accounted for in other parts of the standard. If the environment has IFS unset on shell startup, is the variable IFS the above sets the application's responsibility to export or the shell's, when setting up for an exec or subshell of a utility other than sh? Is this left as implementation-defined whether the shell marks it for export automatically, effectively? If a parent script exports as readonly a different IFS value, does the attempt to set this default in a subshell fail as violating the readonly constraint, or is IFS considered 'always writable' now and an attempt to mark it readonly should fail? Does this affect how the make utility is expected to parse the script portions when a target rule is executed? |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2014-10-16 19:46 | rhansen | New Issue | |
| 2014-10-16 19:46 | rhansen | Name | => Richard Hansen |
| 2014-10-16 19:46 | rhansen | Organization | => BBN |
| 2014-10-16 19:46 | rhansen | Section | => sh |
| 2014-10-16 19:46 | rhansen | Page Number | => 3191-3192 |
| 2014-10-16 19:46 | rhansen | Line Number | => 106811-106813 |
| 2014-10-16 19:46 | rhansen | Interp Status | => --- |
| 2014-10-30 15:07 | shware_systems | Note Added: 0002429 | |
| 2015-01-29 17:49 | Don Cragun | Interp Status | --- => Pending |
| 2015-01-29 17:49 | Don Cragun | Note Added: 0002531 | |
| 2015-01-29 17:49 | Don Cragun | Status | New => Interpretation Required |
| 2015-01-29 17:49 | Don Cragun | Resolution | Open => Accepted |
| 2015-01-29 17:50 | Don Cragun | Note Edited: 0002531 | |
| 2015-01-29 17:51 | Don Cragun | Final Accepted Text | => See 0000884:0002531. |
| 2015-01-29 17:51 | Don Cragun | Tag Attached: tc2-2008 | |
| 2015-02-20 12:13 | ajosey | Interp Status | Pending => Proposed |
| 2015-02-20 12:13 | ajosey | Note Added: 0002554 | |
| 2015-03-23 12:09 | ajosey | Interp Status | Proposed => Approved |
| 2015-03-23 12:09 | ajosey | Note Added: 0002599 | |
| 2015-05-27 21:26 | mirabilos | Note Added: 0002678 | |
| 2015-05-27 21:48 | rhansen | Note Added: 0002679 | |
| 2015-05-27 21:50 | rhansen | Note Edited: 0002679 | |
| 2015-05-27 21:50 | rhansen | Note Edited: 0002679 | |
| 2015-05-27 21:50 | rhansen | Note Edited: 0002679 | |
| 2015-05-27 21:51 | rhansen | Note Edited: 0002679 | |
| 2015-05-27 21:52 | rhansen | Note Edited: 0002679 | |
| 2015-05-27 21:57 | rhansen | Note Added: 0002681 | |
| 2015-05-27 22:15 | mirabilos | Note Added: 0002682 | |
| 2015-05-30 21:01 | mirabilos | Note Added: 0002688 | |
| 2015-06-02 15:13 | shware_systems | Note Added: 0002691 | |
| 2019-06-10 08:54 | agadmin | Status | Interpretation Required => Closed |
