OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

Python import system: Difference between create_module and exec_module

  • Thread starter Thread starter lupl
  • Start date Start date
L

lupl

Guest
Question regarding the Python import system and the importer (finder, loader) protocol:

In this example on Real Python, a custom importer for importing CSV files is implemented by specializing find_spec and exec_module. What puzzles me is the use of exec_module here, according to my understanding

  • create_module is responsible for creating and returning a module object
  • exec_module is responsible for executing the module's code in its namespace.

Since the goal here is to hook into module creation and not module code execution, to me create_module would be the logical candidate for implementing this logic.

I tried to create_module for a toy JSON importer and it works just fine:

Code:
from importlib.machinery import ModuleSpec
import json
import pathlib
import sys


class JSONImporter:

    def __init__(self, json_path):
        self.json_path = json_path

    @classmethod
    def find_spec(cls, name, path, target=None):
        *_, module_name = name.rpartition(".")
        json_file_name = f"{module_name}.json"

        directories = sys.path if path is None else path

        for directory in directories:
            json_path = pathlib.Path(directory) / json_file_name

            if json_path.exists():
                return ModuleSpec(name, cls(json_path))

    def create_module(self, spec):
        try:
            with open(self.json_path) as f:
                json_object = json.load(f)
        except json.JSONDecodeError:
            return None
        else:
            return json_object

    def exec_module(self, module):
        pass


sys.meta_path.append(JSONImporter)

Obviously the implementation in the Real Python article also does the job, but it seems to me that their implementation using exec_module is a bit awkward. Am I missing something?
<p>Question regarding the Python import system and the importer (finder, loader) protocol:</p>
<p>In this <a href="https://realpython.com/python-import/#example-import-data-files" rel="nofollow noreferrer">example on Real Python</a>, a custom importer for importing CSV files is implemented by specializing <code>find_spec</code> and <code>exec_module</code>. What puzzles me is the use of <code>exec_module</code> here, according to my understanding</p>
<ul>
<li><a href="https://docs.python.org/3/library/importlib.html#importlib.abc.Loader.create_module" rel="nofollow noreferrer">create_module</a> is responsible for creating and returning a module object</li>
<li><a href="https://docs.python.org/3/library/importlib.html#importlib.abc.Loader.exec_module" rel="nofollow noreferrer">exec_module</a> is responsible for executing the module's code in its namespace.</li>
</ul>
<p>Since the goal here is to hook into module creation and not module code execution, to me <code>create_module</code> would be the logical candidate for implementing this logic.</p>
<p>I tried to <code>create_module</code> for a toy JSON importer and it works just fine:</p>
<pre><code>from importlib.machinery import ModuleSpec
import json
import pathlib
import sys


class JSONImporter:

def __init__(self, json_path):
self.json_path = json_path

@classmethod
def find_spec(cls, name, path, target=None):
*_, module_name = name.rpartition(".")
json_file_name = f"{module_name}.json"

directories = sys.path if path is None else path

for directory in directories:
json_path = pathlib.Path(directory) / json_file_name

if json_path.exists():
return ModuleSpec(name, cls(json_path))

def create_module(self, spec):
try:
with open(self.json_path) as f:
json_object = json.load(f)
except json.JSONDecodeError:
return None
else:
return json_object

def exec_module(self, module):
pass


sys.meta_path.append(JSONImporter)
</code></pre>
<p>Obviously the implementation in the Real Python article also does the job, but it seems to me that their implementation using <code>exec_module</code> is a bit awkward. Am I missing something?</p>
 
Top