Autonomous Video Factory: How n8n Wakes Up RTX 3090 via Wake-on-LAN and Renders in ComfyUI
In recent years, I, Ilya Dubovik, have been engaged in automating processes at LancerLab. One of the most interesting tasks was creating an autonomous video factory that efficiently uses a powerful RTX 3090 for rendering videos with ComfyUI. In this article, I will detail how we solved the problem of idle expensive graphics cards and costly cloud rendering.
Why This is Needed
The RTX 3090 is a powerful graphics card for video rendering. However, when it is not in use, it simply sits idle, which is an inefficient use of resources. Additionally, cloud rendering solutions can be very expensive and slow under high load. We decided to create a system that automatically wakes up the server with RTX 3090 via Wake-on-LAN and starts video rendering through ComfyUI.
Pipeline Architecture
Our system consists of several key components that work together to achieve the final goal. Here is the architecture scheme:
Telegram bot -> n8n (Docker on Ubuntu) ->
Gemini/LLM (with cascading fallback) ->
Keenetic RCI API (Wake-on-LAN) ->
SSH/Task Scheduler (StartComfyUI on 192.168.1.63:8188) ->
FFmpeg
Let’s break down each of these components in more detail:
- Telegram bot: Users send commands through the Telegram bot, which forwards them to n8n.
- n8n (Docker on Ubuntu): This is the main orchestrator that processes requests and manages the entire system’s operation.
- Gemini/LLM: Used for processing commands and generating scene descriptions. We use cascading fallback to ensure reliability.
- Keenetic RCI API (Wake-on-LAN): Sends Wake-on-LAN packets to wake up the server with RTX 3090.
- SSH/Task Scheduler: Starts ComfyUI on the server and manages the rendering process.
- FFmpeg: Used for post-processing video, including scene concatenation and frame extraction.
Challenges and How We Solved Them
During development, we encountered several issues. Here’s how we solved them:
Issue 1: Machine with RTX 3090 is sleeping or in hibernation
When the machine is in sleep or hibernation mode, it cannot immediately respond to a Wake-on-LAN packet. To solve this problem, we added authentication to the Keenetic API and a waiting loop for port 8188 readiness.
# Example code for sending WoL packet and checking port availability
import socket
import time
def send_wol_packet(mac_address):
# Sending Wake-on-LAN packet
pass
def check_port_availability(ip, port):
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
return s.connect_ex((ip, port)) == 0
mac_address = 'XX:XX:XX:XX:XX:XX'
ip = '192.168.1.63'
send_wol_packet(mac_address)
while not check_port_availability(ip, 8188):
time.sleep(5) # Wait 5 seconds before the next check
Issue 2: Scene breaks during long video generation
During the generation of a long video, scene breaks may occur. To solve this problem, we used Frame-to-Frame Continuity, where FFmpeg extracts the last frame of the previous scene as the starting point for the next.
# Example FFmpeg command to extract the last frame
ffmpeg -i input.mp4 -vf "select=last_frame" output.png
# Example FFmpeg command to concatenate frames
ffmpeg -i scene1.mp4 -i scene2.mp4 -filter_complex "[0:v][1:v]concat=n=2:v=1:a=0[out]" -map "[out]" final_video.mp4
Issue 3: 429 Rate Limits and cloud LLM failures
Cloud models may encounter rate limit restrictions or simply fail. To solve this problem, we implemented cascading fallback models and a separate telemetry/logging bot.
# Example code for handling errors and fallback
def process_request(model):
try:
response = model.process(request)
except Exception as e:
if isinstance(e, RateLimitExceededError):
fallback_model = get_fallback_model()
response = fallback_model.process(request)
else:
log_error(e)
return response
# Example code for telemetry
def log_telemetry(data):
telemetry_bot.send_message(data)
Specific Example: FFmpeg Command Fragment for Extracting the Last Frame and Concatenating, as well as JSON/Steps Scheme
Here is an example of an FFmpeg command to extract the last frame and concatenate scenes:
# Extracting the last frame
ffmpeg -i input.mp4 -vf "select=last_frame" last_frame.png
# Concatenating frames
ffmpeg -i scene1.mp4 -i scene2.mp4 -filter_complex "[0:v][1:v]concat=n=2:v=1:a=0[out]" -map "[out]" final_video.mp4
The JSON/steps scheme for video rendering might look like this:
{
"steps": [
{
"step_name": "Extract Last Frame",
"command": "ffmpeg -i input.mp4 -vf "select=last_frame" last_frame.png"
},
{
"step_name": "Concatenate Scenes",
"command": "ffmpeg -i scene1.mp4 -i scene2.mp4 -filter_complex "[0:v][1:v]concat=n=2:v=1:a=0[out]" -map "[out]" final_video.mp4"
}
]
}
Summary and Plans
In this article, I detailed the creation of an autonomous video factory that uses n8n to wake up RTX 3090 via Wake-on-LAN and render videos in ComfyUI. We solved several key issues, such as idle graphics cards, scene breaks during video generation, and cloud model limitations.
In the future, we plan to add support for additional video formats and improve error handling. We are also considering integrating with other tools for automating rendering processes. Stay tuned for our updates!
“Automation is the key to efficient resource utilization.” — Ilya Dubovik
Добавить комментарий