View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0001959 | 1003.1(2024)/Issue8 | Shell and Utilities | public | 2025-11-13 23:13 | 2025-11-13 23:13 |
| Reporter | collinfunk | Assigned To | |||
| Priority | normal | Severity | Editorial | Type | Clarification Requested |
| Status | New | Resolution | Open | ||
| Name | Your Name Here | ||||
| Organization | GNU | ||||
| User Reference | |||||
| Section | XCU dd | ||||
| Page Number | 2778 | ||||
| Line Number | 91990 - 91996 | ||||
| Interp Status | |||||
| Final Accepted Text | |||||
| Summary | 0001959: dd conv=lcase and conv=ucase should only translate single byte locales | ||||
| Description | The `conv=lcase` and `conv=ucase` to `dd` require that conversion be done as specified by `LC_CTYPE`. Here is the description of `conv=lcase`, for reference: > Map uppercase characters specified by the LC_CTYPE keyword tolower to the corresponding lowercase character. Characters for which no mapping is specified shall not be modified by this conversion. This requirement means that if `LC_CTYPE` is a multibyte locale, one may have to read across block boundaries to have a full character for case conversion. For example using 512-byte blocks, a UTF-8 character like д, encoded as 0xd0 0xb4, might have 0xd0 at byte 512 of the first block and byte 0xb4 at byte 1 of the second block. This is not a problem if `dd` behaves on single bytes instead of characters. However, introducing case conversion means we we must read entire multibyte characters, even if they extend across a block. Also complicating factor is that case conversion may change the length of the character in Unicode. Take the following example: $ python3 -c 'print(len("ß"))' 1 $ python3 -c 'print(len("ß".upper()))' 2 If we have an input block containing all ASCII characters and `ß` as the last character, using `conv=ucase,sync bs=512` would result in a 512-byte output block followed a second block contains the second byte of uppercase `ß` and 511 NUL bytes. This is probably not what someone expects when using `dd`. | ||||
| Desired Action | I see that there was discussion about this in the past during a 2011 Austin Group meeting. Here is the relevant text for reference [1]: > We concluded that we want to change the wording to limit the lcase and ucase translations to single byte locales as they cannot work on multibyte locales. We will need to have an aardvark filed. We were not happy with limiting the conversion to ASCII. There will need to be some discussion about conv=ascii, ibm and ebcdic overriding the current locale for case conversions. Before seeing this, Pádraig Brady and I discussed this and came to the conclusion that this was the correct behavior. For more complete locale aware case conversion something like `tr '[:lower:]' '[:upper:]' ` is more intuitive. I could not find the original email, but it seems this was never acted upon since there was no bug report. How about making it explicit that `conv=lcase` and `conv=upcase` only operate on single-byte locales? I am not sure the best behavior regarding overriding the current locale. [1] https://www.opengroup.org/austin/docs/austin_539.txt | ||||
| Tags | No tags attached. | ||||
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2025-11-13 23:13 | collinfunk | New Issue |
