Group Abstract Group Abstract

Message Boards Message Boards

3
|
760 Views
|
12 Replies
|
12 Total Likes
View groups...
Share
Share this post:

Wolfram video processing is too slow for real-life situations

Jaques Secretin
Posted 22 days ago

Wolfram Video Processing is too slow in real-life situations. I love using Wolfram Language for real-life applications, and I appreciate that Wolfram has recently focused on first-class video functionality. However, it’s a pity that while it works fine for demonstrations, it is far too slow for practical use.

For example, when I use CapCut to edit and render a 5-minute video, it takes only a few seconds. But in Wolfram Mathematica, the same process is extremely slow. For a simple 7-minute video (with only text overlay), Wolfram takes 6 minutes just to render. That is unusable! Imagine processing a 30-minute video—it could take 45 minutes, while CapCut would finish in under a minute. enter image description here

I have also tried adjusting encoders, frame rates, and other settings, but it didn’t help. Can someone suggest a solution? Thank you very much.
POSTED BY: Jaques Secretin
12 Replies
Sort By:

Here is the update. It seems that Wolfram is using the command from ffmpeg. For example, the subtitle: I managed to get the speed = 26x. enter image description here

When I run the ffmpeg, I get the 64x. Wow ! So much faster, with the same result ! enter image description here

So in a long run, I need to switch more to the direct approach of using ffmpeg.
POSTED BY: Jaques Secretin

Thank you @Carlo Giacometti , I would like to use Wolfram to edit a longer video, not only add text (subtitle), but also editting the video. The function VideoCombine is the "soft subtitle", it's fast. But I need the "hardcode subtitle", so I need to render the video with the text.

Can you help me answer these questions, so that I would like to improve the rendering speed in Wolfram.

1) As far as I know, Wolfram is using ffmpeg to edit, and render the video. So in general, the rendering speed should be similar, compared to using Python and ffmpeg command? Is it correct?

enter image description here

2) My computer has the NVIDIA GPU, so in general, when I use the NVIDIA Codec, it could be very fast. When I try to use the NVENC Coding, with EncodingSpeed, the render speed is still slow, nothing has improved. Why? How can we use the NVIDIA Codec in Wolfram? I have tested several codecs in Wolfram, and none of these codecs get the similar speed like using Capcut.

enter image description here

enter image description here

3) I have found that several posts also confirm that the configuration in Wolfram, or ffmpeg is very slow (rendering speed). For example this post: Why is ffmpeg so slow to create a video compared to CapCut, for example? If I would like to render the video in Wolfram, by using this option: -c:v h264_videotoolbox -b:v 6000K, how to do that? Or how to apply the preset very fast, to encode in Wolfram: Like this command: ffmpeg -i input.mp4 -c:v h264_nvenc -preset fast -cq 23 -c:a aac -b:a 128k output.mp4

enter image description here
POSTED BY: Jaques Secretin

So to make sure I understand correctly: you can make Mathematica use FFMPEG for video encoding, but it is still slow because it is presumably using its default settings which are slow, and you have also figured out how to make FFMPEG go faster using its command line configuration options, but are asking if there is a way to set these configuration options when calling it from within the Wolfram Language?
POSTED BY: Gareth Russell

I read the Wolfram documentation, it seems that Wolfram is using ffmpeg. But I wonder why is it too slow? compared to another solution (using another software to render the video). I need the confirmation of the Wolfram Developper.

POSTED BY: Jaques Secretin

Mathematica is indeed using ffmpeg to write images to a video file. Using the Nvidia based encoders should be quite fats, but there are three parts to this operation:

1) frames are read from the input file and loaded in the kernel (decoding)

2) the kernel performs the desired operation (rendering)

3) the resulting frame is written to the output video file (encoding)

Only operations 1) and 3) are performed by ffmpeg.
POSTED BY: Carlo Giacometti

Hello, I want to add the logo, and the SRT (Subtitle text) to the video. Well, for the 14 minutes video, in Wolfram, it says that I need 9h to render the video. It is unacceptable !

enter image description here

I have the latest GEFORCE NVIDIA GPU. In Capcut, it takes only 1 minutes to do that. So I hope that you can help me. I have a lot of videos, that's why I need to do this programmatically. Best regards,
POSTED BY: Jaques Secretin

What does VAddCaption call? have you tried any of the suggested solutions?
POSTED BY: Carlo Giacometti

To rasterize subtitles into the video, try

VideoFrameMap[{{"subtitles", "filename" -> "1.srt"}}, Video ["1.mp4"]]
POSTED BY: Carlo Giacometti

Thank you @Carlo Giacometti .

1) I think I have found a bug with VideoFrameMap[{{"drawtext", "text" -> "test"}}, Video["test.mp4"]] I hear the sound "Ding", but no error is shown. And then the memory state of the all the symbol in the notebook is cleared. I have tested 3-4 times, and the same happened.

2) VAddCaption is a function based on your propositions: The text is converted to Rasterize: textImg=Rasterize["Text", Background -> None]; and Then the textImg is added to Video by OverlayVideo[Video["test.mp4"],textImg] However, this approach is super slow. For 15 minutes videos, it renders frame by frame, and it will takes 9h to finish.

3) The fastest approach is: VideoFrameMap[{{"subtitles", "filename" -> "test.srt"}}, Video["test.mp4"]] However, this approach is not explained in the Wolfram Documentation. I can't found it. I think that, this approach is using the directly ffmpeg command. That's why it is so much faster. But I can't change the position, or the text color. I have the default white text.

4) So my question: I would like to change the logo (image), and I also want to change the text color, and modify the text position, using the VideoFrameMap[{{"subtitles}}] approach, how to do that? Thank you very much.
POSTED BY: Jaques Secretin

To change styling, you can either use the force_style argument in the subtitles filter (documented here: https://ffmpeg.org/ffmpeg-filters.html#subtitles-1), eg:

VideoFrameMap[{{"subtitles", 
   "filename" -> FindFile["ExampleData/bullfinch.mkv"], 
   "force_style" -> "PrimaryColour=&HCCFF0000"}}, 
 Video["ExampleData/bullfinch.mkv"]]

Or you can create a new subtitle track based on the existing one using WL functionality. You can specify SubtitleStyle to control the appearance of all subtitles. If you want to specify each subtitle on ints own you can use the Text and Style wrappers to control appearance and positioning of each subtitle.

v = Video["ExampleData/bullfinch.mkv"];
newVid = 
  VideoCombine[{v, VideoExtractTracks[v, {"Subtitle", 1}]}, 
   SubtitleStyle -> {Italic, Orange, Bold, Larger, 
     FontFamily -> "Big Caslon", Background -> LightGreen}];
VideoFrameMap[{{"subtitles", 
   "filename" -> Information[newVid, "ResourcePath"][[1]]}}, newVid]
POSTED BY: Carlo Giacometti

If the goal is only to overlay some text, consider adding subtitles:

In[34]:= VideoCombine[{v, "Text"}]; // AbsoluteTiming

Out[34]= {1.52954, Null}

or use the "drawtext" (https://ffmpeg.org/ffmpeg-filters.html#drawtext) ffmpeg filter:

In[30]:= 
VideoFrameMap[{{"drawtext", "fontsize" -> "56", 
     "text" -> "Hello World"}}, v]; // AbsoluteTiming

Out[30]= {3.3095, Null}
POSTED BY: Carlo Giacometti

The operation in WL will be necessarily slower, since the operation is performed in a kernel evaluation, and the second argument can be an arbitrary expression. We are working on making this faster for the next version. In the meantime, here are some tricks to achieve a speedup in this particular operation:

In[13]:= OverlayVideo[v, "Text"]; // AbsoluteTiming

Out[13]= {10.8377, Null}

In[24]:= t = Rasterize["Text", Background -> None];
OverlayVideo[v, 
   Rasterize["Text", Background -> None]]; // AbsoluteTiming

Out[25]= {8.15988, Null}

In[26]:= 
Parallelize[
   VideoFrameMap[ImageCompose[#, "Text"] &, 
    Video["ExampleData/bullfinch.mkv"]]]; // AbsoluteTiming

Out[26]= {2.0267, Null}

In[27]:= 
Parallelize[
   VideoFrameMap[ImageCompose[#, t] &, 
    Video["ExampleData/bullfinch.mkv"]]]; // AbsoluteTiming

Out[27]= {1.78399, Null}

I'm using a 15 second video of the same dimensions.
POSTED BY: Carlo Giacometti
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard