Non-recursive Make part 3 – a tool for the fearless

In part 1, I mentioned that it is typically painful to attack a large project that uses recursive make. Nonetheless, if you’re willing to make the effort, here’s a tool to help you pick apart what’s really going on in a project tree of recursive makes.

Have you ever looked at the output of “make -nps”? It shows exactly what variables, rules and actions are relevant to the make target(s) you specify. On the down side, it includes masses of info that is hard for you to parse. On the up side, you don’t have to. The following Perl script analyzes “make -nps …” output, and builds a bog-simple makefile, with relevant variable settings, conditional settings, rules and actions. After that, it’s up to you to follow the rules of NR make: all variables, targets and pathnames must begin with an identifier only used by that project.

#!/usr/bin/perl -w
=cut
NOTE: "rules" output will include some partial extraneous rules
        unrelated to the goal (explicit or default).
=cut
use strict;
use Cwd; my $cwd = getcwd;
my %ignore = map {$_=>1}
                qw(.FEATURES MAKE MAKECMDGOALS MAKEFILE_LIST
            MAKELEVEL MAKE_COMMAND MAKE_VERSION SUFFIXES);

my (%set, %use, %deps, %acts, $skip, $rules, $files, $targ);
open my $ph, '-|', "gmake -nps @ARGV" or die "Cannot run make";
while () {
    $rules = 0 if m{^# Variables};
    $rules = 1 if m{^# Implicit};
    $skip  = 1 if $rules and m{^# Not a target:|^[^#:=\t]*[%(][^=]*:};
    $skip  = 0 if m{^$};
    next if $skip;
    if (my ($var,$imm,$val) = m{^(\S+)\s*(:)?(= ..*)} and !$ignore{$1}) {
        $val =~ s{#}{\\#}g;
        $val =~ s{(\s+)$}{$1#};
        $use{$_} = 1 for $val =~ m{\$[({]([-.\w]+)(?::[^)]*)[})]}g;
        $use{$var} = 1 if $imm ||= '';
        $set{$var} = "$imm$val";
    } elsif (m{^\t.} and $rules) {
        $acts{$targ} .= $_;
        $use{$_} = 1 for m{\$\(([-.\w]+)\)}g;
        $use{$_} = 1 for m{\$\{([-.\w]+)\}}g;
        $use{$targ} = 1 if m{\$\(\$@\)} or m{\$\{\$@\}}
    } elsif (m{^# ((\w+) ([:+])?= .*)}) {
        $deps{$targ} = "$targ: $1\n".($deps{$targ} || '') if $targ;
        $use{$2} = 1 if $3;
    } elsif ($rules and m{^(\S+):\s*(.*)}) {
        $targ = $1;
        $deps{$1} = "$1: $2" if $2;
    } elsif (m{^\s*$}) {
        $targ = '';
    }
}

# Include var values have nested variable expansions
# Recursively determine vars used.
my @curr = keys %use;
while (@curr) {
    my $vals = join ' ', map {$set{$_} || ''} @curr;
    @curr = grep {!$use{$_}++}
                $vals =~ m{\$[({]([-.\w]+)(?::[^)]*)[})]}g;
}

my @vars = sort grep {$set{$_} ||= $ENV{$_}} keys %use;
my $wid = 0;
for (@vars) { $wid = length if $wid < length; }
printf "%-${wid}s %s\n", $_, $set{$_} for @vars;
$acts{$_} ||= '' for keys %deps;
$deps{$_} ||= "$_:" for keys %acts;
print "#----";
s{$cwd/}{}mg for values %deps;
s{//+}{/}mg  for values %deps;
print "\n$deps{$_}\n$acts{$_}"
    for sort grep {$acts{$_} or $deps{$_}} keys %deps;
print "# vim: set nowrap:\n";

Here’s what “rules” can do for the subproject GNUmakefile in Non-recursive make (gmake) part 2:

-include ../rules.mk
export madns    ?= .
# madns imports libtap from util:
util            ?= ../util
PATH            := $(madns):$(PATH)
#---------------- PRIVATE vars:
madns.test      = $(madns)/madns_t
#---------------- PUBLIC TARGETS (see rules.mk):
all             : madns.all
test            : madns.test
install         : madns.install
#---------------- inputs to "install":
madns.bin       = $(madns)/hostip
madns.lib       = $(madns)/madns.a
#---------------- inputs to "clean":
all             += madns
#---------------- PRIVATE TARGETS:
madns.all       : $(madns.bin) $(madns.lib) $(madns.test)
madns.test      : $(madns.test:%=%.pass)

$(madns.bin)    : $(madns.lib)
$(madns.lib)    : $(madns)/madns.o

$(madns.test)   : LDLIBS += -pthread
$(madns.test)   : CFLAGS += -I$(util)
$(madns.test)   : $(madns.lib) $(util)/libtap.a

.INTERMEDIATE   : $(madns)/madns.o $(madns)/madns_t.o

-include $(madns)/*.d

… and the “rules” output, with environment variable $BLD set to debug is:

.DEFAULT_GOAL := all
BLD           = debug
CC            = cc
CFLAGS        = -g -MMD -fdiagnostics-show-option -fstack-protector --param ssp-buffer-size=4 -fno-strict-alias
COMPILE.c     = $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c
CPPFLAGS      = -mtune=native -I$(PREFIX)/include -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE $(CPPFLAGS.$(BLD))
LDLIBS        = $(LDLIBS.$(OSNAME))
LINK.o        = $(CC) $(LDFLAGS) $(TARGET_ARCH)
OSNAME        := Linux
OUTPUT_OPTION = -o $@
PATH          := .:/home/mischas
PREFIX        = /usr/local
all           = madns
#----
../util/libtap.a:
        [ "$^" ] && ar crs $@ $(filter %.o,$^)

.INTERMEDIATE: madns.o madns_t.o

.PHONY: all clean cover debug gccdefs install profile source tags test

all: madns.all
        @echo "$@ done for BLD='$(BLD)'"

clean:
        @rm -rf $(shell $(MAKE) -nps all test cover profile | sed -n '/^# I/,$${/^[^\#\[%.][^ %]*: /s/:.*//p;}') \
        $(clean)  $(foreach A,$(all), $($A)/{gmon.out,tags,*.fail,*.gcda,*.gcno,*.gcov,*.prof}) \
        $(filter %.d,$(MAKEFILE_LIST))

gccdefs:
        @$(CC) $(CPPFLAGS) -E -dM - </dev/null | cut -c8- | sort

hostip: hostip.o madns.a
        $(LINK.o) $^ $(LOADLIBES) $(LDLIBS) -o $@

hostip.o: hostip.c
        $(COMPILE.c) $(OUTPUT_OPTION) $<

install: madns.install

madns.a: madns.o
        [ "$^" ] && ar crs $@ $(filter %.o,$^)

madns.all: hostip madns.a madns_t

madns.o: madns.c
        $(COMPILE.c) $(OUTPUT_OPTION) $<

madns.test: madns_t.pass

madns_t: LDLIBS += -pthread
madns_t: CFLAGS += -I$(util)
madns_t: madns_t.o madns.a ../util/libtap.a
        $(LINK.o) $^ $(LOADLIBES) $(LDLIBS) -o $@

madns_t.o: madns_t.c
        $(COMPILE.c) $(OUTPUT_OPTION) $<

test: madns.test
# vim: set nowrap:

Almost all the above actions are GNUmakefile defaults. Default actions are not expanded unless specified; “rules install” will also show the “madns.install” target’s actions.

Advertisements

About mischasan

I've had the privilege to work in a field where abstract thinking has concrete value. That applies at the macro level --- optimizing actions on terabyte database --- or the micro level --- fast parallel string searches in memory. You can find my documents on production-system radix sort (NOT just for academics!) and some neat little tricks for developers, on my blog https://mischasan.wordpress.com My e-mail sig (since 1976): Engineers think equations approximate reality. Physicists think reality approximates the equations. Mathematicians never make the connection.
This entry was posted in make, non-recursive make. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s