pyomo.common.fileutils

This module provides general utilities for working with the file system

this_file([stack_offset]) Returns the file name for the module that calls this function.
this_file_dir() Returns the directory containing the module that calls this function.
find_path(name, validate[, cwd, mode, ext, ...]) Locate a path, given a set of search parameters
find_file(filename[, cwd, mode, ext, ...]) Locate a file, given a set of search parameters
find_dir(dirname[, cwd, mode, pathlist, ...]) Locate a directory, given a set of search parameters
find_library(libname[, cwd, include_PATH, ...]) Find a dynamic library using find_file to search typical locations.
find_executable(exename[, cwd, ...]) Find an executable using find_file to search typical locations.
import_file(path[, clear_cache, infer_package]) Import a module given the full path/filename of the file.
PathManager(finder, dataClass) The PathManager defines a registry class for path locations
PathData(manager, name) An object for storing and managing a PathManager path
pyomo.common.fileutils.this_file(stack_offset=1)[source]

Returns the file name for the module that calls this function.

This function is more reliable than __file__ on platforms like Windows and in situations where the program has called os.chdir().

pyomo.common.fileutils.this_file_dir()[source]

Returns the directory containing the module that calls this function.

pyomo.common.fileutils.find_path(name, validate, cwd=True, mode=4, ext=None, pathlist=[], allow_pathlist_deep_references=True)[source]

Locate a path, given a set of search parameters

Parameters:
  • name (str) – The name to locate. The name may contain references to a user’s home directory (~user), environment variables (${HOME}/bin), and shell wildcards (? and *); all of which will be expanded.
  • validate (function) – A function to call to validate the path (used by find_file and find_dir to discriminate files and directories)
  • cwd (bool) – Start by looking in the current working directory [default: True]
  • mode (mask) – If not None, only return entries that can be accessed for reading/writing/executing. Valid values are the inclusive OR of {os.R_OK, os.W_OK, os.X_OK} [default: os.R_OK]
  • ext (str or iterable of str) – If not None, also look for name+ext [default: None]
  • pathlist (str or iterable of str) – A list of strings containing paths to search, each string contains a single path. If pathlist is a string, then it is first split using os.pathsep to generate the pathlist [default: []].
  • allow_pathlist_deep_references (bool) – If allow_pathlist_deep_references is True and the name appears to be a relative path, allow deep reference matches relative to directories in the pathlist (e.g., if name is foo/my.exe and /usr/bin is in the pathlist, then find_file() could return /usr/bin/foo/my.exe). If allow_pathlist_deep_references is False and the name appears to be a relative path, then only matches relative to the current directory are allowed (assuming cwd==True). [default: True]

Notes

find_path uses glob, so the path and/or name may contain wildcards. The first matching entry is returned.

pyomo.common.fileutils.find_file(filename, cwd=True, mode=4, ext=None, pathlist=[], allow_pathlist_deep_references=True)[source]

Locate a file, given a set of search parameters

Parameters:
  • filename (str) – The file name to locate. The file name may contain references to a user’s home directory (~user), environment variables (${HOME}/bin), and shell wildcards (? and *); all of which will be expanded.
  • cwd (bool) – Start by looking in the current working directory [default: True]
  • mode (mask) – If not None, only return files that can be accessed for reading/writing/executing. Valid values are the inclusive OR of {os.R_OK, os.W_OK, os.X_OK} [default: os.R_OK]
  • ext (str or iterable of str) – If not None, also look for filename+ext [default: None]
  • pathlist (str or iterable of str) – A list of strings containing paths to search, each string contains a single path. If pathlist is a string, then it is first split using os.pathsep to generate the pathlist [default: []].
  • allow_pathlist_deep_references (bool) – If allow_pathlist_deep_references is True and the filename appears to be a relative path, allow deep reference matches relative to directories in the pathlist (e.g., if filename is foo/my.exe and /usr/bin is in the pathlist, then find_file() could return /usr/bin/foo/my.exe). If allow_pathlist_deep_references is False and the filename appears to be a relative path, then only matches relative to the current directory are allowed (assuming cwd==True). [default: True]

Notes

find_file uses glob, so the path and/or file name may contain wildcards. The first matching file is returned.

pyomo.common.fileutils.find_dir(dirname, cwd=True, mode=4, pathlist=[], allow_pathlist_deep_references=True)[source]

Locate a directory, given a set of search parameters

Parameters:
  • dirname (str) – The directory name to locate. The name may contain references to a user’s home directory (~user), environment variables (${HOME}/bin), and shell wildcards (? and *); all of which will be expanded.
  • cwd (bool) – Start by looking in the current working directory [default: True]
  • mode (mask) – If not None, only return directories that can be accessed for reading/writing/executing. Valid values are the inclusive OR of {os.R_OK, os.W_OK, os.X_OK} [default: os.R_OK]
  • pathlist (str or iterable of str) – A list of strings containing paths to search, each string contains a single path. If pathlist is a string, then it is first split using os.pathsep to generate the pathlist [default: []].
  • allow_pathlist_deep_references (bool) – If allow_pathlist_deep_references is True and the dirname appears to be a relative path, allow deep reference matches relative to directories in the pathlist (e.g., if dirname is foo/bar and /usr/bin is in the pathlist, then find_dir() could return /usr/bin/foo/bar). If allow_pathlist_deep_references is False and the dirname appears to be a relative path, then only matches relative to the current directory are allowed (assuming cwd==True). [default: True]

Notes

find_dir uses glob, so the path and/or directory name may contain wildcards. The first matching directory is returned.

pyomo.common.fileutils.find_library(libname, cwd=True, include_PATH=True, pathlist=None)[source]

Find a dynamic library using find_file to search typical locations.

Finds a specified library (file) by searching a specified set of paths. This routine will look for the specified file name, as well as looking for the filename followed by architecture-specific extensions (e.g., .dll, .so, or .dynlib). Note that as this uses :py:func:find_file(), the filename and search paths may contain wildcards.

If the explicit path search fails to locate a library, then this returns the result from passing the basename (with ‘lib’ and extension removed) to ctypes.util.find_library()

Parameters:
  • libname (str) – The library name to search for
  • cwd (bool) – Start by looking in the current working directory [default: True]
  • include_PATH (bool) – Include the executable search PATH at the end of the list of directories to search. [default: True]
  • pathlist (str or list of str) – List of paths to search for the file. If None, then pathlist will default to the local Pyomo configuration library directory (and the local Pyomo binary directory if include_PATH is set) and the contents of LD_LIBRARY_PATH. If a string, then the string is split using os.pathsep. [default: None]

Notes

find_library() uses find_file() with allow_pathlist_deep_references=True, so libnames containing relative paths will be matched relative to all paths in pathlist.

pyomo.common.fileutils.find_executable(exename, cwd=True, include_PATH=True, pathlist=None)[source]

Find an executable using find_file to search typical locations.

Finds a specified executable by searching a specified set of paths. This routine will look for the specified file name, as well as looking for the filename followed by architecture-specific extensions (e.g., .exe). Note that as this uses find_file(), the filename and search paths may contain wildcards.

Parameters:
  • exename (str) – The executable file name to search for
  • cwd (bool) – Start by looking in the current working directory [default: True]
  • include_PATH (bool) – Include the executable search PATH at the end of the list of directories to search. [default: True]
  • pathlist (str or list of str) – List of paths to search for the file. If None, then pathlist will default to the local Pyomo configuration binary directory. If a string, then the string is split using os.pathsep. [Default: None]

Notes

find_executable() uses find_file() with allow_pathlist_deep_references=False, so search strings containing relative paths will only be matched relative to the current working directory. This prevents confusion in the case where a user called find_executable("./foo") and forgot to copy foo into the local directory, but this function picked up another foo in the user’s PATH that they did not want to use.

pyomo.common.fileutils.import_file(path, clear_cache=False, infer_package=True)[source]

Import a module given the full path/filename of the file. Replaces import_file from pyutilib (Pyomo 6.0.0).

This function returns the module object that is created.

Parameters:
  • path (str) – Full path to .py file.
  • clear_cache (bool) – Remove module if already loaded. The default is False.
class pyomo.common.fileutils.PathData(manager, name)[source]

An object for storing and managing a PathManager path

path()[source]

Return the full, normalized path to the registered path entry.

If the object is not found (or was marked “disabled”), path() returns None.

get_path()[source]

DEPRECATED.

Deprecated since version 5.6.2: get_path() is deprecated; use pyomo.common.Executable(name).path()

disable()[source]

Disable this path entry

This method “disables” this path entry by marking it as “not found”. Disabled entries return False for available() and None for path(). The disabled status will persist until the next call to rehash().

available()[source]

Returns True if the registered path is available.

Entries are available if the object was found found in the search locations and has not been explicitly disabled.

rehash()[source]

Requery the location of this path entry

This method derives its name from the csh command of the same name, which rebuilds the hash table of executables reachable through the PATH.

class pyomo.common.fileutils.ExecutableData(manager, name)[source]

A PathData class specifically for executables.

property executable

Get (or set) the path to the executable

class pyomo.common.fileutils.PathManager(finder, dataClass)[source]

The PathManager defines a registry class for path locations

The PathManager defines a class very similar to the CachedFactory class; however it does not register type constructors. Instead, it registers instances of PathData (or ExecutableData). These contain the resolved path to the directory object under which the PathData object was registered. We do not use the PyUtilib register_executable and registered_executable functions so that we can automatically include Pyomo-specific locations in the search path (namely the PYOMO_CONFIG_DIR).

Users will generally interact with this class through global instances of this class (pyomo.common.Executable and pyomo.common.Library).

Users are not required or expected to register file names with the PathManager; they will be automatically registered upon first use. Generally, users interact through the path() and available() methods:

>>> from pyomo.common import Executable
>>> if Executable('demo_exec_file').available():
...     loc = Executable('demo_exec_file').path()
...     print(os.path.isfile(loc))
True
>>> print(os.access(loc, os.X_OK))
True

For convenience, available() and path() are available by casting the PathData object requrned from Executable or Library to either a bool or str:

>>> if Executable('demo_exec_file'):
...     cmd = "%s --help" % Executable('demo_exec_file')

The PathManager caches the location / existence of the target directory entry. If something in the environment changes (e.g., the PATH) or the file is created or removed after the first time a client queried the location or availability, the PathManager will return incorrect information. You can cause the PathManager to refresh its cache by calling rehash() on either the PathData (for the single file) or the PathManager to refresh the cache for all files:

>>> # refresh the cache for a single file
>>> Executable('demo_exec_file').rehash()
>>> # or all registered files
>>> Executable.rehash()

The Executable singleton looks for executables in the system PATH and in the list of directories specified by the pathlist attribute. Executable.pathlist defaults to a list containing the os.path.join(pyomo.common.envvar.PYOMO_CONFIG_DIR, 'bin').

The Library singleton looks for executables in the system LD_LIBRARY_PATH, PATH and in the list of directories specified by the pathlist attribute. Library.pathlist defaults to a list containing the os.path.join(pyomo.common.envvar.PYOMO_CONFIG_DIR, 'lib').

Users may also override the normal file resolution by explicitly setting the location using set_path():

>>> Executable('demo_exec_file').set_path(os.path.join(
...     pyomo.common.envvar.PYOMO_CONFIG_DIR, 'bin', 'demo_exec_file'))

Explicitly setting the path is an absolute operation and will set the location whether or not that location points to an actual file. Additionally, the explicit location will persist through calls to rehash(). If you wish to remove the explicit executable location, call set_path(None):

>>> Executable('demo_exec_file').set_path(None)

The Executable singleton uses ExecutableData, an extended form of the PathData class, which provides the executable property as an alais for path() and set_path():

>>> loc = Executable('demo_exec_file').executable
>>> print(os.path.isfile(loc))
True
>>> Executable('demo_exec_file').executable = os.path.join(
...     pyomo.common.envvar.PYOMO_CONFIG_DIR, 'bin', 'demo_exec_file')
>>> Executable('demo_exec_file').executable = None
rehash()[source]

Requery the location of all registered executables

This method derives its name from the csh command of the same name, which rebuilds the hash table of executables reachable through the PATH.

pyomo.common.fileutils.register_executable(name, validate=None)[source]

DEPRECATED.

Deprecated since version 5.6.2: pyomo.common.register_executable(name) has been deprecated; explicit registration is no longer necessary

pyomo.common.fileutils.registered_executable(name)[source]

DEPRECATED.

Deprecated since version 5.6.2: pyomo.common.registered_executable(name) has been deprecated; use pyomo.common.Executable(name).path() to get the path or pyomo.common.Executable(name).available() to get a bool indicating file availability. Equivalent results can be obtained by casting Executable(name) to string or bool.

pyomo.common.fileutils.unregister_executable(name)[source]

DEPRECATED.

Deprecated since version 5.6.2: pyomo.common.unregister_executable(name) has been deprecated; use Executable(name).disable()