Austin Group Defect Tracker

Aardvark Mark IV

Main | My View | View Issues | Change Log | Docs
  

Viewing Issue Simple Details [ Jump to Notes ] [ Issue History ] [ Print ]
ID Category Severity Type Date Submitted Last Update
0000512 [1003.1(2008)/Issue 7] Shell and Utilities Objection Enhancement Request 2011-11-16 22:46 2022-09-12 15:51
Reporter dwheeler View Status public  
Assigned To ajosey
Priority normal Resolution Rejected  
Status Closed  
Name David A. Wheeler
Organization
User Reference
Section make
Page Number 2916
Line Number 95848
Interp Status ---
Final Accepted Text
Summary 0000512: Add macro functions to make
Description Many makefiles depend on having macro functions in macro expansions. This capability exists in GNU make, and possibly others as well. For an example of a makefile approach that depends on this, see: http://miller.emu.id.au/pmiller/books/rmch/ [^] Since it's implemented in GNU make, it is clearly implementable.

This simple mechanism allows makefiles to easily handle far more complex situations, as well as being easy to understand. It also provides a simple and clean extension mechanism.

Macro functions are easily distinguished from simple macro references by including a space, e.g., "$(wildcard *.c)" instead of simple "$(name)".

The proposed action proposes only a subset of GNU make's functions, but for the ones that are listed, they are intended to be compatible with GNU make. I have not included $(foreach...), for example, but this could be easily added.

I have listed functions near each other that are "related"; it would be reasonable to sort them into alphabetical order for the final specification.

In some cases, this capability is more efficient if there is a way to perform immediate evaluation; see bug 330. However, it does not strictly depend on it.
Desired Action Add after the current macro text a new section:

Macro functions

A macro function is a macro reference in the form $(name arguments) or ${name arguments}, where before expansion there is one or more tabs or spaces between the function name and one or more comma-separated arguments. The tabs and spaces between the name and first argument are not considered part of the initial argument's value. The name and arguments are then evaluated, but only the commas (if any) in the original definition separate argument parameters (an argument may expand to a comma, but this does not create an additional argument parameter). Commas that could be interpreted as "separating" arguments for functions that do not accept more than that number of arguments are considered normal commas and have no effect. It is legal to have zero-length arguments.

The following macro functions are defined:
$(subst FROM,TO,TEXT) : Perform simple substitution. Every occurrence of exactly FROM is replaced by the value of TO in TEXT, and the result returned. This is a case-sensitive match.

$(patsubst PATTERN,REPLACEMENT,TEXT) : Perform pattern substitution.
  Every whitespace-separated word in TEXT that matches PATTERN is replaced with REPLACEMENT, and the result is returned. In PATTERN, the first '%' (if any) will match any one or more characters in a word. If REPLACEMENT also includes %, the first % in REPLACEMENT is replaced by the text that matched % in the PATTERN. Any '%' after the first '%' in PATTERN or REPLACEMENT is not considered special. A '%' quoted with a '\' preceding backslash is considered an ordinary % (it only matches a '%' if in PATTERN, and it is a simple '%' if in REPLACEMENT). A pair of backslashes is considered a single slash if a unquoted '%' has not appeared, otherwise it is considered two backslashes. Whitespace between words is replaced with a single space character, and whitespace at the beginning and end of the text is removed (see the macro function strip).

$(strip STRING) : Strip text. Whitespace at the beginning and end of STRING is removed, and all internal sequences of whitespace are replaced with a single whitespace character.

$(findstring FIND, IN): Find text. Look an exact match of IN inside the text FIND; if this occurs, return FIND, otherwise, return the empty string.

$(wildcard GLOB_PATTERN): Get file names using a pattern. Given PATTERN, returns a list of file names that currently match PATTERN as a glob pattern (see section 2.13, "Pattern Matching Notation"). If more than one matches, they must be separated by a single tab or space character.

$(addsuffix SUFFIX,NAMES): Add a filename suffix. For each whitespace-separated value in name, append SUFFIX to it, and apply macro function strip to the result.

$(addprefix PREFIX,NAMES): Add a filename prefix. For each whitespace-separated value in name, prepend PREFIX to it, and apply macro function strip to the result.

$(shell COMMAND): Execute a shell command. Execute COMMAND using the shell and return its result. Since there is only one argument (COMMAND), any commas are considered part of the command.

$(if CONDITION,THEN[,ELSE]): If-then-else.
Remove all whitespace from the beginning and end of CONDITION, then evaluate it. If it expands to a non-empty string, consider it true; evaluate and return THEN (do not evaluate ELSE). If the condition expands to a non-empty string, consider it false; do not evaluate THEN, but instead evaluate and return ELSE (if there is no ELSE, return an empty string). Only THEN or ELSE are evaluated, never both.

$(or CONDITION1[,...]): Short-circuiting logical or. For each condition, evaluate it in turn (left-to-right) by removing all whitespace from its beginning and end, and then evaluating it. The first condition that returns a non-empty string is returned; all later conditions are not evaluated (this is a short-circuiting operation). If all conditions return empty strings, then return an empty string.

$(and CONDITION1[,...]): Short-circuiting logical and. For each condition, evaluate it in turn (left-to-right) by removing all whitespace from its beginning and end, and then evaluating it. The first condition that returns an empty string causes the entire function to return an empty string; all later conditions are not evaluated (this is a short-circuiting operation). If all conditions return non-empty strings, then return the result of the last condition.

Implementations may support other macro functions as extensions.
Tags No tags attached.
Attached Files

- Relationships
related to 0000337Appliedajosey Add standard support for "!=" (macro shell assignment) in make 

-  Notes
(0001022)
dwheeler (reporter)
2011-11-16 23:01

 After I posted this, I realized that http://miller.emu.id.au/pmiller/books/rmch/ [^] uses the "filter" function. So let's add that to the proposal, as well as its opposite filter-out:

$(filter PATTERNS,TEXT): Return matching TEXT words. Returns every whitespace-separated word in TEXT that match any of the whitespace-separated patterns listed in PATTERNS, in the order they appear in TEXT. The returned words are separated by one tab or space character, and all leading and trailing whitespace is removed (see strip). See "patsubst" for the pattern rules.

$(filter-out PATTERNS,TEXT): Return TEXT words that do not match. Returns every whitespace-separated word in TEXT that do not match any of the whitespace-separated patterns listed in PATTERNS, in the order they appear in TEXT. The returned words are separated by one tab or space character, and all leading and trailing whitespace is removed (see strip). See "patsubst" for the pattern rules. This is the opposite of the filter function.

Also, I'd like to make it possible for implementations to use tab separation instead of space separation. So in the patsubst text above, change:
"Whitespace between words is replaced with a single space character" to:
"Whitespace between words is replaced with a single space or tab character".

Similarly, in strip, change:
"all internal sequences of whitespace are replaced with a single whitespace character."
to:
"all internal sequences of whitespace are replaced with a single space or tab character."
(0001027)
joerg (reporter)
2011-11-17 11:19

 I object "addsuffix" and "addprefix" as this is a superfluous additional implementation for functionality that has already been introduced by pattern macro expansions in SunPro make in 1986. Pattern macro expansions has been widely implemented by all major make implementations.

Why do we need "shell" after we did introduce !=
We should not define everything but a useful set of features.

Note that there are plenty of globbing/pattern methods. Introducing globbing and/or pattern matching would need to describe the appling method.
(0001028)
joerg (reporter)
2011-11-17 11:19

 I object "addsuffix" and "addprefix" as this is a superfluous additional implementation for functionality that has already been introduced by pattern macro expansions in SunPro make in 1986. Pattern macro expansions has been widely implemented by all major make implementations.

Why do we need "shell" after we did introduce !=
We should not define everything but a useful set of features.

Note that there are plenty of globbing/pattern methods. Introducing globbing and/or pattern matching would need to describe the applying method.
(0001031)
brkorb (reporter)
2011-11-17 16:41

 $(if CONDITION,THEN[,ELSE]): If-then-else.
  Remove all whitespace from the beginning and end of CONDITION, then
  evaluate it. If it expands to a non-empty string, consider it true;
  evaluate and return THEN (do not evaluate ELSE). If the condition expands
  to a non-empty string, consider it false;
       ^^^^
non-empty string shouldn't be both true and false. :)
(0001054)
dwheeler (reporter)
2011-11-27 21:03

 Regarding comment #1028:
I presume that by the phrase "pattern macro expansions" you mean variable references of the form $(VAR:A=B) where A contains a single '%' character, and with the semantics that if VAR's contents match A, then that content is replaced by B. For example, $(foo:%.o=%.c). (In particular I presume you don't mean pattern rules like %.o : %.c ... if you do, see bug #513).

I agree that pattern macro expansions should be added to the POSIX make specification as well, since they're widely used and implemented. POSIX make only lets you say $(VAR:.c=.o), which if there's a ".c" in the middle of a filename you end up with a disaster.

I think that pattern macro expansions should be added to POSIX as a separate bug report, they're not really the same thing.

I think "addsuffix" and "addprefix" are still useful in the case of functions calling functions, because the pattern macro expansions cannot be applied to function results. But if dropping addsuffix and addprefix is necessary for acceptance, then let's drop them!

Comment #1031:
Whups, my mistake, you're absolutely right. That was a cut-and-paste error, sorry about that. The text you point out there should read "If the condition expands to an empty string, consider it false;"

One function I have NOT listed is $(sort ...), which would be nice primarily for its ability to eliminate duplicates. For example, GNU make has:
$(sort list) which sorts the whitespace-separated words of LIST in lexical order, removing duplicate words. The output is a list of words separated by single spaces. However, GNU make's $(sort) does not respect the current locale's collation order (http://osdir.com/ml/bug-make-gnu/2011-07/msg00015.html). [^] In the context of a make, this is probably reasonable, but it probably makes it impossible to standardize. Which is sad, because in many cases the lexical order is overspecifying things to begin with.

Perhaps a new function could be invented called $(unique ...).
$(unique LIST)
     Remove duplicates in the whitespace-separated words of LIST. The resulting list is returned in some implementation-defined order, which may or may not be the same order as the original LIST.
(0001056)
dwheeler (reporter)
2011-11-28 01:03

 Regarding comment #1027:

> "Why do we need "shell" after we did introduce != We should not define everything but a useful set of features."

Both "!=" and "$(shell)" have their uses:
1. "VAR != command" is a simple way to just run a command: It's easy to read, and you don't have to deal with complications of quoting (e.g., if ")" is in the command).
2. "$(shell ...)" is useful if you want to process the results further with other functions (because it could be a parameter of some other function), or if you want to do things other than just set a traditional macro variable (e.g., set an immediate variable or append a variable).

Perhaps more importantly, supporting both syntaxes helps people move toward a standard syntax, instead of being tied to specific implementations' extensions. The BSDs all support "!=", while GNU make supports $(shell). SunPro supports ":sh=" which I believe is semantically just like "!=", and by supporting "!=", it'd be easy to do a search-and-replace to switch to "!=". In general, Current POSIX make is so feature-impoverished that many people have given up staying with the standard; they just give up and pick a particular make. Which is why I'm proposing so many additions to make, in an attempt to get make up-to-date. Supporting both "!=" and "$(shell ...)" syntaxes is a small step toward helping people move toward the standard instead. Yes, it's a very small step, but small steps can get places.

> Note that there are plenty of globbing/pattern methods. Introducing globbing and/or pattern matching would need to describe the appling method.

I completely agree. I thought I did that though. For $(wildcard), I cross-referenced to section 2.13, "Pattern Matching Notation", which defines globbing patterns - I think it's better to define stuff in one place. The definition of $(patsubst PATTERN,REPLACEMENT,TEXT) does give its definition for its pattern notation. If I missed something, please point it out, it's entirely unintentional on my part.

HOWEVER, there is an ERROR in my proposed definition for $(pathsubst); a "%" should match 0 or more, not 1 or more. So please change the above "match any one or more characters" for "%" into "match zero or more characters". That this is an error is trivially proved by using this makefile with GNU make (which is where I "stole" pathsubst from):
 X=$(patsubst %.c,%.o,.c)
 all:
        @echo $(X)
(0005964)
Don Cragun (manager)
2022-09-12 15:51

     Although many of these functions are useful, the fact that they have been around so long and have not been duplicated in the BSD and SunPro make implementations implies that there hasn't been much user demand to add them. Therefore, this bug is rejected.

- Issue History
Date Modified Username Field Change
2011-11-16 22:46 dwheeler New Issue
2011-11-16 22:46 dwheeler Status New => Under Review
2011-11-16 22:46 dwheeler Assigned To => ajosey
2011-11-16 22:46 dwheeler Name => David A. Wheeler
2011-11-16 22:46 dwheeler Section => make
2011-11-16 22:46 dwheeler Page Number => 2916
2011-11-16 22:46 dwheeler Line Number => 95848
2011-11-16 23:01 dwheeler Note Added: 0001022
2011-11-17 11:19 joerg Note Added: 0001027
2011-11-17 11:19 joerg Note Added: 0001028
2011-11-17 16:41 brkorb Note Added: 0001031
2011-11-27 21:03 dwheeler Note Added: 0001054
2011-11-28 01:03 dwheeler Note Added: 0001056
2022-09-12 15:17 geoffclare Relationship added related to 0000337
2022-09-12 15:51 Don Cragun Interp Status => ---
2022-09-12 15:51 Don Cragun Note Added: 0005964
2022-09-12 15:51 Don Cragun Status Under Review => Closed
2022-09-12 15:51 Don Cragun Resolution Open => Rejected

Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker