OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

Python opencv/cv2 video generation - FPS/Frames desync

  • Thread starter Thread starter Neruay
  • Start date Start date
N

Neruay

Guest
I have a weird problem related to fps/sync of frames. Firstly, I have an input .mkv video in 30fps.

Code:
cap: cv2.VideoCapture = cv2.VideoCapture(example_video.mkv)

Then I break it into frames and work with them using while loop, simultaneously building up output video using edited frames (basically I analyze them with ML model and draw bounding boxes around found objects).

So on every iteration of the cycle, one frame being taken out of original video, being edited and put on top of other frames of output video

Code:
out_fps = 30
frame_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
frame_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
out = cv2.VideoWriter('output.mp4', fourcc, out_fps, (frame_width, frame_height))

while cap.isOpened():
    success: bool
    frame: cv2.typing.MatLike
    success, frame = cap.read()
    if success:
        * analyzing and modifying frame *
        out.write(frame)
    else:
        break
out.release()

However, my output video for some reason is always a bit longer than the original one, resulting in desync between video and audio tracks (I also add original audiotrack to output video later, so input and output video should have the exact same duration and this is crucial for me).

For example, my input video can be 41:42 minutes long, but output one will be 41:46, resulting in progressing audio desync, which gets worse with video duration, sometimes I will have ~12 hours long ones, resulting in multiple minutes desync.

Code:
get_video_duration("example_video.mkv") #    selfmade function for returning 
get_video_duration("output.mp4") #           video duration in milliseconds
>>> 2502900
>>> 2506166

When i try to check their fps its stays exactly the same - at 30.0

Code:
get_video_fps("example_video.mkv")
get_video_fps("output.mp4")
>>> 30.0
>>> 30.0

Please, let me know if someone encountered (and potentially solved) similar problem. Thank you!

I've already tried a lot of workarounds, such as artificially changing output fps by a little (for example changing 30.0 to 30.0479616) to compensate this weird behaviour, and while it can reduce desync from 4 sec to like 0.2 sec it is still not ideal and does not work on bigger videos for some reason.

For example, 30.0479616 fps value will work on my 42 min long video, but for another video, lets say, 2 hours long, it won't work, desync still will be there, just by another %diff and I cant afford to process every video 2 times to calculate this "offset" before rendering final version.
<p>I have a weird problem related to fps/sync of frames.
Firstly, I have an input .mkv video in 30fps.</p>
<pre><code>cap: cv2.VideoCapture = cv2.VideoCapture(example_video.mkv)
</code></pre>
<p>Then I break it into frames and work with them using while loop, simultaneously building up output video using edited frames (basically I analyze them with ML model and draw bounding boxes around found objects).</p>
<p>So on every iteration of the cycle, one frame being taken out of original video, being edited and put on top of other frames of output video</p>
<pre><code>out_fps = 30
frame_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
frame_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
out = cv2.VideoWriter('output.mp4', fourcc, out_fps, (frame_width, frame_height))

while cap.isOpened():
success: bool
frame: cv2.typing.MatLike
success, frame = cap.read()
if success:
* analyzing and modifying frame *
out.write(frame)
else:
break
out.release()

</code></pre>
<p>However, my output video for some reason is always a bit longer than the original one, resulting in desync between video and audio tracks (I also add original audiotrack to output video later, so input and output video should have the exact same duration and this is crucial for me).</p>
<p>For example, my input video can be 41:42 minutes long, but output one will be 41:46, resulting in progressing audio desync, which gets worse with video duration, sometimes I will have ~12 hours long ones, resulting in multiple minutes desync.</p>
<pre><code>get_video_duration("example_video.mkv") # selfmade function for returning
get_video_duration("output.mp4") # video duration in milliseconds
>>> 2502900
>>> 2506166
</code></pre>
<p>When i try to check their fps its stays exactly the same - at 30.0</p>
<pre><code>get_video_fps("example_video.mkv")
get_video_fps("output.mp4")
>>> 30.0
>>> 30.0
</code></pre>
<p>Please, let me know if someone encountered (and potentially solved) similar problem. Thank you!</p>
<p>I've already tried a lot of workarounds, such as artificially changing output fps by a little (for example changing 30.0 to 30.0479616) to compensate this weird behaviour, and while it can reduce desync from 4 sec to like 0.2 sec it is still not ideal and does not work on bigger videos for some reason.</p>
<p>For example, 30.0479616 fps value will work on my 42 min long video, but for another video, lets say, 2 hours long, it won't work, desync still will be there, just by another %diff and I cant afford to process every video 2 times to calculate this "offset" before rendering final version.</p>
 
Top