MANIFEST.in ignored on "python setup.py install" - no data files installed?_问答_开发者

Here's my stripped-down setup.py script with non-code stuff removed:

#!/usr/bin/env python

from distutils.core import setup
from whyteboard.misc import meta


setup(
    name = 'Whyteboard',
    version = meta.version,

    packages = ['whyteboard', 'whyteboard.gui', 'whyteboard.lib', 'whyteboard.lib.pubsub',
                'whyteboard.lib.pubsub.core', 'whyteboard.lib.pubsub.utils', 'whyteboard.misc'],

    py_modules = ['whyteboard'],
    scripts = ['whyteboard.py'],
)

MANIFEST.in:

include *.txt
include whyteboard-help/*.*
recursive-include locale *.mo
recursive-include images *.png

When I run "python setup.py install sdist" I get a nice .tar.gz with a "whyteboard-0.41" 开发者_开发知识库root folder, with my locale/ images/ and whyteboard-help/ folders inside. This also has my whyteboard.py script that launches my program from inside the whyteboard source package.

So:

whyteboard/
 ├── locale/
 ├── images
 ├── whyteboard-help/
 ├── whyteboard/
 │  ├── __init__.py
 │  └── other packages etc
 ├── whyteboard.py
 ├── README
 ├── setup.py
 └── CHANGELOG

This mirrors the source of my program, is how everything should be, and is correct.

However when I run "python setup.py install" none of my data files are written - only the "whyteboard" source package, and the whyteboard.py is placed in /usr/local/lib/python2.6/dist-packages/.

Ideally, I'd like the same directory structure as what's been generated in the .tar.gz file to be created in dist-packages, as this is how my program expects to look for its resources.

How can I get "install" to create this directory structure? It seems to be ignoring my manifest file, as far as I can tell.

MANIFEST.in tells Distutils what files to include in the source distribution but it does not directly affect what files are installed. For that you need to include the appropriate files in the setup.py file, generally either as package data or as additional files.

I couldn't figure out why my MANIFEST.in file was being ignored when I ran python setup.py install - turns out include_package_data=True solves the problem. The package_data option isn't actually required.

Some notes in addition to Ned's answer (which hits on the core problem):

Distutils does not install Python packages and modules inside a per-project subdirectory within site-packages (or dist-packages on Debian/Ubuntu): they are installed directly into site-packages, as you've seen. So the containing whyteboard-xx directory in your sdist will not exist in the final installed form.

One implication of this is that you should be careful to name your data_files in a way that clarifies what project they belong to, because those files/directories are installed directly into the global site-packages directory, not inside any containing whyteboard directory.

Or you could instead make your data package_data of the whyteboard package (which means it needs to live inside that package, i.e. next to __init__.py), and then this isn't a problem.

Lastly, it doesn't make much sense to have both a whyteboard.py module in py_modules and a whyteboard/__init__.py package in packages. The two are mutually exclusive, and if you have both, the whyteboard.py module will be ignored by imports in favor of the package of the same name.

If whyteboard.py is just a script, and is not intended to be imported, then you should use the scripts option for it, and remove it from py_modules.

You should use setuptools:

#!/usr/bin/env python

from setuptools import setup, find_packages
from whyteboard.misc import meta


setup(
  name = 'Whyteboard',
  version = meta.version,

  packages = find_packages(),
  include_package_data=True,

  py_modules = ['whyteboard'],
  scripts = ['whyteboard.py'],
)

This is not actually using the MANIFEST file to do the job, but it includes all the needed files.

Running python 2.6.1 on Mac OSX, I had absolutely no luck except by using the data_files parameter in setup.py. Everything with MANIFEST.in simply resulted in files being included in the dist package, but never installed. I checked some other packages and they were indeed using data_files to specify additional files.

I created a short function to help enumerate all the files from a directory tree in the

(target_dir, [file list]) format that data_files expects:

def gen_data_files(*dirs):
    results = []

    for src_dir in dirs:
        for root,dirs,files in os.walk(src_dir):
            results.append((root, map(lambda f:root + "/" + f, files)))
    return results

Now I can just call this inside my setup call:

setup(... data_files = gen_data_files("docs", "lib") ...

And everything in those trees gets installed.

Minimal published runnable example

https://github.com/cirosantilli/python-sample-package-with-data
https://pypi.org/project/python-sample-package-with-data

Key takeaway: only MANIFEST.in worked for me, package_data did not.

Tested on Ubuntu 19.10, Python 3.7.5, wheel==0.32.3, setuptools==41.1.0, twine==3.1.1.

How end users use the package from https://pypi.org/project/python-sample-package-with-data/:

python3 -m pip install --user python-sample-package-with-data
python-sample-package-with-data

Expected output:

hello data

How maintainers publish it:

# One time setup.
python3 -m pip install --user setuptools wheel twine

# Every time you want to publish.
python setup.py sdist bdist_wheel
twine upload dist/*
rm -rf build dist *.egg-info

The actual files:

MANIFEST.in

# Or else pip install cannot find README.md on the setup.py under certain conditions.
include README.md

# This actually adds the data file.
include python_sample_package_with_data/mydata.txt

python-sample-package-with-data

#!/usr/bin/env python3

import python_sample_package_with_data

print(python_sample_package_with_data.get_data(), end='')

python_sample_package_with_data/__init__.py

try:
    import importlib.resources as importlib_resources
except ImportError:
    # In PY<3.7 fall-back to backported `importlib_resources`.
    import importlib_resources

def get_data():
    return importlib_resources.read_text(__name__, 'mydata.txt')

python_sample_package_with_data/mydata.txt

hello data

setup.py

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

from setuptools import setup, find_packages

from os import path
this_directory = path.abspath(path.dirname(__file__))
with open(path.join(this_directory, 'README.md')) as f:
    long_description = f.read()

setup(
    name='python-sample-package-with-data',
    version='0.0.3',
    description='My short description',
    long_description=long_description,
    long_description_content_type='text/markdown',
    url='https://github.com/cirosantilli/python-sample-package-with-data',
    author='Ciro Santilli',
    author_email='ciro.santilli.contact@gmail.com',
    packages=find_packages(),
    include_package_data=True,
    scripts=['python-sample-package-with-data'],
)

Bibliography: