Python distutils builds extensions differently on different machines

I have been working on a Python extension module with lots of files. While building on one machine, python setup.py build will ha开发者_StackOverflow中文版ppily detect changed files, build just those files, and link the whole thing together, just like make. On another machine, however, a single change to any file triggers a recompile of all sources.

Just to be clear. Both machines detect when the package is up to date and won't do anything. It is only when a single file changes that their behavior diverges.

Why is the second machine doing this?

Machine 1 (Does proper per-file dependency check and build.)

Python 2.6.4 (r264:75706, Feb 15 2010, 17:06:03) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-44)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

setuptools-0.6c11-py2.6

LSB Version: :core-3.1-amd64:core-3.1-ia32:core-3.1-noarch:graphics-3.1-amd64:graphics-3.1-ia32:graphics-3.1-noarch
Distributor ID: CentOS
Description: CentOS release 5.4 (Final)
Release: 5.4
Codename: Final

Machine 2 (Rebuilds everything when a single source file changes.)

Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.

setuptools-0.6c11-py2.6

No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 10.04 LTS
Release: 10.04
Codename: lucid

---

I looked into the Mercurial repo and found this change:

Issue #5372: Drop the reuse of .o files in Distutils' ccompiler (since Extension extra options may change the output without changing the .c file).

IOW, it was a simplistic optimization that was removed.

---

I've tracked this down to a change in distutils between Python 2.6.4 and Python 2.6.5. Two methods in distutils.ccompiler.CCompiler, namely, _setup_compile and _prep_compile, have had the same chunk of code removed:

if self.force:
    skip_source = {}            # rebuild everything
    for source in sources:
        skip_source[source] = 0
elif depends is None:
    # If depends is None, figure out which source files we
    # have to recompile according to a simplistic check. We
    # just compare the source and object file, no deep
    # dependency checking involving header files.
    skip_source = {}            # rebuild everything
    for source in sources:      # no wait, rebuild nothing
        skip_source[source] = 1

    n_sources, n_objects = newer_pairwise(sources, objects)
    for source in n_sources:    # no really, only rebuild what's
        skip_source[source] = 0 # out-of-date
else:
    # If depends is a list of files, then do a different
    # simplistic check.  Assume that each object depends on
    # its source and all files in the depends list.
    skip_source = {}
    # L contains all the depends plus a spot at the end for a
    # particular source file
    L = depends[:] + [None]
    for i in range(len(objects)):
        source = sources[i]
        L[-1] = source
        if newer_group(L, objects[i]):
            skip_source[source] = 0
        else:
            skip_source[source] = 1

This code checks each source file against its target object, and marks it to be skipped if it is older than the target. I don't know why it was removed, but it explains the discrepancy. When I put it back in as a test, the compiler reverts to per-source dependency analysis.