I need a way to find the dependencies for each of my Python package's sub-modules at runtime so I can initialize them in a proper order (see my current [EDIT: former] solution here, which doesn't work to well), so at first I used the standard Python module modulefinder, but that was way too slow (~1-2 seconds per module).
My next choice was to analyze all the globals of each module, and find from those globals which sub-module each sub-module depends upon. (This is my current solution EDIT: I have a better solution now - see my answer). Th开发者_Python百科is algorithm is much faster than modulefinder (it takes <200ms per module), but it only works for relative imports, instead of the fully-qualified import style, which is unacceptable.
So, what I need is either:
- A quicker alternative to modulefinder
- An alternative algorithm
NOTE: I call my dependency analyzer at the start of each module, like so:
# File my_package/module3.py
import my_package.module1 # Some misc. module
import my_package.module2 # Some other misc. module
import my_package.dependency_analyzer
my_package.dependency_analyzer.gendeps()
(Just in case it helps you any.)
Thank you!
EDIT: I have a solution now - see my answer.
I think I have a solution to my own question :)
Here's what would go into the dependency_analyzer module talked about above:
import sys
from sys import _getframe as getframe
import atexit
examined_modules = []
def gendeps():
"""Adds the calling module to the initialization queue."""
# Get the calling module's name, and add it to the intialization queue
calling_module_name = getframe(1).f_globals['__name__']
examined_modules.append(calling_module_name)
def init():
"""Initializes all examined modules in the correct order."""
for module in examined_modules:
module = sys.modules[module]
if hasattr(module, 'init'):
module.init()
if hasattr(module, 'deinit'):
# So modules get de-initialized in the correct order,
# as well
atexit.register(module.deinit)
Now, at the start of each module (after all the import statements - this is crucial), a call to gendeps is placed. This algorithm works because each time a module is imported, that call to gendeps is executed. However, since all of the import statements are placed before the call to gendeps in your own module, the least-dependent modules are placed in the initialization queue first, and the most-dependent modules are placed in the initialization queue last.
精彩评论