OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

Segmenting 3D Cell Images with Omnipose in Python

  • Thread starter Thread starter six
  • Start date Start date
S

six

Guest
I am trying to segment 3D tiff cell images with Omnipose but am running into a shape mismatch error. I am following the example code on the Omnipose documentation and have even tested with the same file they used. Note that I am not testing 3D Omnipose models, but rather 2D Cellpose models with do_3D=True and omni=True.

This is the start of the guide I am following on the Omnipose docs.
image


Task​


Want to be able to segment 3D tiff files using Omnipose mask building but with Cellpose 2D models.

The Error:​


When I attempt to segment 3D images using Omnipose's 2D cellpose models (by setting do_3D=True and omni=True), I get a shape mismatch error. Particularly, I am forced to configure the model with nclasses=3 even though they did nclasses=2 in the documentation example. I am forced to do this because I get a separate error otherwise (more on that below). As such, I believe this is the root cause of the shape mismatch error I am experiencing. The error occurs when calling model.eval. Here is the last part of the error message:

Code:
ValueError: operands could not be broadcast together with remapped shapes [original->remapped]: (3,2)  and requested shape (4,2)

Here is a link to a txt file with the full error message.

Note that my code is identical to the Omnipose documentation example for 3D segmentation with do_3D aside from two things. First, I have set gpu=True instead of gpu=False in my model configuration. Second, I have nclasses=3 instead of nclasses=2 in my model configuration. This is because when I set nclasses=2, I get a separate error:

Code:
RuntimeError: Error(s) in loading state_dict for CPnet:
    size mismatch for output.2.weight: copying a param with shape torch.Size([3, 32, 1, 1]) from checkpoint, the shape in current model is torch.Size([2, 32, 1, 1]).
    size mismatch for output.2.bias: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([2]).

You can see the full error I get when setting nclasses=2 at this link. Note that this error only occurs when gpu=True is set. When I have gpu=False, I get a similar output but it does not produce an error:

Code:
failed to load model Error(s) in loading state_dict for CPnet:
    size mismatch for output.2.weight: copying a param with shape torch.Size([3, 32, 1, 1]) from checkpoint, the shape in current model is torch.Size([2, 32, 1, 1]).
    size mismatch for output.2.bias: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([2]).

Then my only choice seems to be having nclasses=3, as I want to use my gpu. Here are the lines of code which generate the shape mismatch error when I configure the model when nclasses=3.

Code:
from cellpose_omni import models, core

model_name = "plant_cp"

use_GPU = core.use_gpu()
model = models.CellposeModel(gpu=use_GPU, 
                             model_type=model_name, 
                             net_avg=False, 
                             diam_mean=0, 
                             nclasses=3, 
                             dim=2, 
                             nchan=2)

Code:
masks_cp, flows_cp, _ = model.eval(imgs,
                                   channels=[0,0],
                                   rescale=None,
                                   mask_threshold=0,
                                   net_avg=0,
                                   transparency=True, 
                                   flow_threshold=0.,
                                   verbose=0, 
                                   tile=0,
                                   compute_masks=1, 
                                   do_3D=True, 
                                   omni=1,
                                   flow_factor=10)

You can see the full code for the above snippet here.

I have played around with the parameters for both model configuration and the model.eval function but have not been able to resolve the issue.

Specification​


I am running the script remotely on Narval clusters with Omnipose 0.4.4. I am using CUDA 11.8 and Python 3.10.2. The OS is Gentoo Base System release 2.6 provided the Narval clusters. Here are the GPUs I am using on the cluster:

Code:
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.15              Driver Version: 550.54.15      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA A100-SXM4-40GB          Off |   00000000:01:00.0 Off |                    0 |
| N/A   30C    P0             51W /  400W |       0MiB /  40960MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA A100-SXM4-40GB          Off |   00000000:81:00.0 Off |                    0 |
| N/A   30C    P0             50W /  400W |       0MiB /  40960MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   2  NVIDIA A100-SXM4-40GB          Off |   00000000:C1:00.0 Off |                    0 |
| N/A   29C    P0             49W /  400W |       0MiB /  40960MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

Note that I am using a virtual environment instead of Conda. The compute nodes I am using do not allow Conda. Here is the result of running pip list, showing all my installed dependencies.

Thank you,
<p>I am trying to segment 3D tiff cell images with Omnipose but am running into a shape mismatch error. I am following the example code on the <a href="https://omnipose.readthedocs.io/examples/mono_channel_3D.html" rel="nofollow noreferrer">Omnipose documentation</a> and have even tested with the <a href="https://github.com/kevinjohncutler/omnipose/tree/main/docs/test_files_3D" rel="nofollow noreferrer">same file</a> they used. Note that I am not testing 3D Omnipose models, but rather 2D Cellpose models with <code>do_3D=True</code> and <code>omni=True</code>.</p>
<p>This is the start of the guide I am following on the Omnipose docs.
<img src="https://github.com/kevinjohncutler/omnipose/assets/144740367/187fb142-bd37-4dc9-87c9-8ccaf3689fd5" alt="image" /></p>
<h2>Task</h2>
<p>Want to be able to segment 3D tiff files using Omnipose mask building but with Cellpose 2D models.</p>
<h2>The Error:</h2>
<p>When I attempt to segment 3D images using Omnipose's 2D cellpose models (by setting do_3D=True and omni=True), I get a shape mismatch error. Particularly, I am forced to configure the model with <code>nclasses=3</code> even though they did <code>nclasses=2</code> in the documentation example. I am forced to do this because I get a <a href="https://github.com/mrdandelion6/omnipose_errors/blob/main/shape_mismatch_error/2dim_error.txt" rel="nofollow noreferrer">separate error</a> otherwise (more on that below). As such, I believe this is the root cause of the shape mismatch error I am experiencing. The error occurs when calling <code>model.eval</code>. Here is the last part of the error message:</p>
<pre><code>ValueError: operands could not be broadcast together with remapped shapes [original->remapped]: (3,2) and requested shape (4,2)
</code></pre>
<p>Here is a <a href="https://github.com/mrdandelion6/omn...shape_mismatch_error/shape_mismatch_error.txt" rel="nofollow noreferrer">link</a> to a txt file with the full error message.</p>
<p>Note that my code is identical to the Omnipose documentation example for 3D segmentation with <code>do_3D</code> aside from two things. First, I have set <code>gpu=True</code> instead of <code>gpu=False</code> in my model configuration. Second, I have <code>nclasses=3</code> instead of <code>nclasses=2</code> in my model configuration. This is because when I set <code>nclasses=2</code>, I get a separate error:</p>
<pre><code>RuntimeError: Error(s) in loading state_dict for CPnet:
size mismatch for output.2.weight: copying a param with shape torch.Size([3, 32, 1, 1]) from checkpoint, the shape in current model is torch.Size([2, 32, 1, 1]).
size mismatch for output.2.bias: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([2]).
</code></pre>
<p>You can see the full error I get when setting <code>nclasses=2</code> at this <a href="https://github.com/mrdandelion6/omnipose_errors/blob/main/shape_mismatch_error/2dim_error.txt" rel="nofollow noreferrer">link</a>. Note that this error only occurs when <code>gpu=True</code> is set. When I have <code>gpu=False</code>, I get a similar output but it does not produce an error:</p>
<pre><code>failed to load model Error(s) in loading state_dict for CPnet:
size mismatch for output.2.weight: copying a param with shape torch.Size([3, 32, 1, 1]) from checkpoint, the shape in current model is torch.Size([2, 32, 1, 1]).
size mismatch for output.2.bias: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([2]).
</code></pre>
<p>Then my only choice seems to be having <code>nclasses=3</code>, as I want to use my gpu. Here are the lines of code which generate the shape mismatch error when I configure the model when <code>nclasses=3</code>.</p>
<pre class="lang-py prettyprint-override"><code>from cellpose_omni import models, core

model_name = "plant_cp"

use_GPU = core.use_gpu()
model = models.CellposeModel(gpu=use_GPU,
model_type=model_name,
net_avg=False,
diam_mean=0,
nclasses=3,
dim=2,
nchan=2)
</code></pre>
<pre class="lang-py prettyprint-override"><code>masks_cp, flows_cp, _ = model.eval(imgs,
channels=[0,0],
rescale=None,
mask_threshold=0,
net_avg=0,
transparency=True,
flow_threshold=0.,
verbose=0,
tile=0,
compute_masks=1,
do_3D=True,
omni=1,
flow_factor=10)
</code></pre>
<p>You can see the full code for the above snippet <a href="https://github.com/mrdandelion6/omn...ismatch_error/shape_mismatch_error_code.ipynb" rel="nofollow noreferrer">here</a>.</p>
<p>I have played around with the parameters for both model configuration and the <code>model.eval</code> function but have not been able to resolve the issue.</p>
<h2>Specification</h2>
<p>I am running the script remotely on Narval clusters with Omnipose 0.4.4. I am using CUDA 11.8 and Python 3.10.2. The OS is Gentoo Base System release 2.6 provided the Narval clusters. Here are the GPUs I am using on the cluster:</p>
<pre><code>+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA A100-SXM4-40GB Off | 00000000:01:00.0 Off | 0 |
| N/A 30C P0 51W / 400W | 0MiB / 40960MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA A100-SXM4-40GB Off | 00000000:81:00.0 Off | 0 |
| N/A 30C P0 50W / 400W | 0MiB / 40960MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 2 NVIDIA A100-SXM4-40GB Off | 00000000:C1:00.0 Off | 0 |
| N/A 29C P0 49W / 400W | 0MiB / 40960MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
</code></pre>
<p>Note that I am using a virtual environment instead of Conda. The compute nodes I am using do not allow Conda. Here is the <a href="https://github.com/mrdandelion6/omnipose_errors/blob/main/pip_list.txt" rel="nofollow noreferrer">result</a> of running <code>pip list</code>, showing all my installed dependencies.</p>
<p>Thank you,</p>
 
Top