Bug report
Bug description:
linecache.getlines(filename) and .getline(filename, lineno) return nothing for a module that was imported from a zip archive on sys.path via zipimport, unless the caller also passes module_globals.
For such modules, __file__ and .co_filename look like /path/to/archive.zip/pkg/mod.py. In linecache.updatecache(), the os.stat() on that path fails (NotADirectoryError), the lazy-loader lookup only works when module_globals is supplied, and the final fallback that searches sys.path only handles relative file names. So the lookup gives up and returns [].
Many callers only have the file name: pdb's list command, warnings (when the module globals are gone), doctest, inspect.getsourcefile() fallbacks, and traceback formatting of code objects whose frames have been cleared. For a file on disk these all work; for the same module inside a zip they silently show no source.
import linecache, sys, zipfile
with zipfile.ZipFile('mods.zip', 'w') as zf:
zf.writestr('zipmod.py', 'def f():\n return 1\n')
sys.path.insert(0, 'mods.zip')
import zipmod
print(linecache.getlines(zipmod.__file__)) # [] but should be the two source lines
print(linecache.getline(zipmod.f.__code__.co_filename,
zipmod.f.__code__.co_firstlineno)) # '' but should be 'def f():\n'
The information needed is already available: zipimport.zipimporter implements get_data(pathname), which accepts exactly these absolute paths, and sys.path_importer_cache maps the archive path to that importer.
Proposed fix: after the lazy-loader attempt fails, have updatecache() check whether the file name lies under a sys.path_importer_cache entry whose finder has get_data()...
This also benefits any other path entry finder that provides get_data() for an archive.
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Linked PRs
Bug report
Bug description:
linecache.getlines(filename)and.getline(filename, lineno)return nothing for a module that was imported from a zip archive on sys.path via zipimport, unless the caller also passes module_globals.For such modules,
__file__and.co_filenamelook like /path/to/archive.zip/pkg/mod.py. Inlinecache.updatecache(), theos.stat()on that path fails (NotADirectoryError), the lazy-loader lookup only works when module_globals is supplied, and the final fallback that searches sys.path only handles relative file names. So the lookup gives up and returns [].Many callers only have the file name: pdb's list command, warnings (when the module globals are gone), doctest, inspect.getsourcefile() fallbacks, and traceback formatting of code objects whose frames have been cleared. For a file on disk these all work; for the same module inside a zip they silently show no source.
The information needed is already available: zipimport.zipimporter implements get_data(pathname), which accepts exactly these absolute paths, and sys.path_importer_cache maps the archive path to that importer.
Proposed fix: after the lazy-loader attempt fails, have updatecache() check whether the file name lies under a sys.path_importer_cache entry whose finder has get_data()...
This also benefits any other path entry finder that provides get_data() for an archive.
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Linked PRs