OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

Best way to package a Python CLI outside of its associated library

  • Thread starter Thread starter Gabriel Belouze
  • Start date Start date
G

Gabriel Belouze

Guest
I'm packaging a library and a CLI associated to it. I'm using flit

The library makes use of long-to-import dependencies (numpy, torch and whatnot), so I would like to split the CLI part so that it doesn't need to do the imports to have, e.g., shell completion. Of course, it will have to do the import to execute some command - but that's fine and expected.

Let me illustrate with an attempt that does not work:

Code:
.
├── pyproject.toml
│   | 
│   | ...
│   |
│   | [project.scripts]
│   | mycli = "mylibrary.cli:main "
│   |
│   | ...
│   |
│
└── src/mylibrary
    ├── __init__.py
    │   | 
    │   | from . import foo, cli
    │   |
    │
    ├── foo.py
    │   | 
    │   | import very_long_import
    │   |
    │
    └── cli.py

Now even something like mycli --help will need to do a import very_long_import, which is unecessary.

On the other hand, I can structure my package as

Code:
.
├── pyproject.toml
│   | 
│   | ...
│   |
│   | [project.scripts]
│   | mycli = "mylibrary.cli:main "
│   |
│   | ...
│   |
│
└── src/mylibrary
    ├── __init__.py
    │   | 
    │   | # empty
    │   |
    │
    ├── foo.py
    │   | 
    │   | import very_long_import
    │   |
    │
    └── cli.py

Now mycli --help is fast, but I can no longer write

Code:
import mylibrary

mylibrary.foo.do_some_stuff()

I have to instead write

Code:
import mylibrary.foo

mylibrary.foo.do_some_stuff()

While before I could do both.

Ideally, I would want something like

Code:
.
├── pyproject.toml
│   | 
│   | ...
│   |
│   | [project.scripts]
│   | mycli = "cli:main "
│   |
│   | ...
│   |
│
└── src
    ├── cli
    │   ├── __init__.py
    │   └── main.py
    │
    └── mylibrary
        ├── __init__.py
        │   | 
        │   | from . import foo, cli
        │   |
        │
        └── foo.py
            | 
            | import very_long_import
            |

where the CLI is shipped as a script on top of the library, not an entry point that's part of the library. But of course here - with flit at least - we get

Code:
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniconda/base/bin/mylibrary", line 5, in <module>
    from cli import main
ModuleNotFoundError: No module named 'cli'

Do you know a way to achieve this library/script separation, either with flit or some other building tools (as long as I only have to write a pyproject.toml and no setup.py junk 😇).
<p>I'm packaging a library and a CLI associated to it. I'm using <a href="https://flit.pypa.io/en/stable/" rel="nofollow noreferrer">flit</a></p>
<p>The library makes use of long-to-import dependencies (<code>numpy</code>, <code>torch</code> and whatnot), so I would like to split the CLI part so that it doesn't need to do the imports to have, e.g., shell completion. Of course, it will have to do the import to execute some command - but that's fine and expected.</p>
<p>Let me illustrate with an attempt that does not work:</p>
<pre><code>.
├── pyproject.toml
│ |
│ | ...
│ |
│ | [project.scripts]
│ | mycli = "mylibrary.cli:main "
│ |
│ | ...
│ |

└── src/mylibrary
├── __init__.py
│ |
│ | from . import foo, cli
│ |

├── foo.py
│ |
│ | import very_long_import
│ |

└── cli.py
</code></pre>
<p>Now even something like <code>mycli --help</code> will need to do a <code>import very_long_import</code>, which is unecessary.</p>
<p>On the other hand, I can structure my package as</p>
<pre><code>.
├── pyproject.toml
│ |
│ | ...
│ |
│ | [project.scripts]
│ | mycli = "mylibrary.cli:main "
│ |
│ | ...
│ |

└── src/mylibrary
├── __init__.py
│ |
│ | # empty
│ |

├── foo.py
│ |
│ | import very_long_import
│ |

└── cli.py
</code></pre>
<p>Now <code>mycli --help</code> is fast, but I can no longer write</p>
<pre class="lang-py prettyprint-override"><code>import mylibrary

mylibrary.foo.do_some_stuff()
</code></pre>
<p>I have to instead write</p>
<pre class="lang-py prettyprint-override"><code>import mylibrary.foo

mylibrary.foo.do_some_stuff()
</code></pre>
<p>While before I could do both.</p>
<p>Ideally, I would want something like</p>
<pre><code>.
├── pyproject.toml
│ |
│ | ...
│ |
│ | [project.scripts]
│ | mycli = "cli:main "
│ |
│ | ...
│ |

└── src
├── cli
│ ├── __init__.py
│ └── main.py

└── mylibrary
├── __init__.py
│ |
│ | from . import foo, cli
│ |

└── foo.py
|
| import very_long_import
|
</code></pre>
<p>where the CLI is shipped as a script on top of the library, not an entry point that's part of the library. But of course here - with <code>flit</code> at least - we get</p>
<pre class="lang-bash prettyprint-override"><code>Traceback (most recent call last):
File "/opt/homebrew/Caskroom/miniconda/base/bin/mylibrary", line 5, in <module>
from cli import main
ModuleNotFoundError: No module named 'cli'
</code></pre>
<p>Do you know a way to achieve this library/script separation, either with <code>flit</code> or some other building tools (as long as I only have to write a <code>pyproject.toml</code> and no <code>setup.py</code> junk 😇).</p>
 
Top