Files
Unraid_Intel-ARC_Deployment/README.md
2024-12-11 14:58:21 -05:00

17 KiB
Raw Blame History

Guide to Intel ARC AV1 Encoding on Unraid + Tdarr Node Killer + SAB Speed Control (Bonus)

Want to help? Click the ★ (Star) button in the upper-right corner!

This guide shows you how to optimize your media library with AV1 encoding on Unraid while also managing GPU resources between Plex and Tdarr. Youll learn how to significantly reduce video file sizes, save substantial storage space, and automatically free up your GPU for Plex transcodes when needed. Additionally, youll learn how to pause Tdarr when Plex requires the GPU and then restart Tdarr once Plex is done.

What youll learn:

  • How to configure your Intel ARC GPU on Unraid.
  • How to set up and optimize Tdarr for AV1 encoding.
  • How to manage SABnzbd download speeds based on Plex streaming activity.
  • How to use the Tdarr Node Killer script to give Plex GPU priority over Tdarr.

Whether youre a seasoned Unraid user or just starting out, this step-by-step guide will help you achieve better resource management, substantial storage savings, and an improved streaming experience.


Table of Contents


Data Savings with AV1 Encoding

AV1 encoding can dramatically reduce your file sizes. In practice, using three Intel ARC GPUs to encode just 10-15% of a large library resulted in roughly 37TB of saved space. For a 300TB collection, this could bring the total size down to around 75-100TB with careful AV1 conversion.

image

In other words, AV1 can provide huge storage and cost savings.


AV1 Drawbacks

AV1 isnt flawless. Some devices may not natively support AV1 decoding yet. Additionally, AV1 encoding can be more resource-intensive and may take longer. For more details, check out the AV1 Drawbacks page to understand compatibility issues and potential trade-offs.


Upgrading to Unraid 7.0 and Installing Required Plugins

Before setting up AV1 flows or using the Tdarr Node Killer Script, ensure youre running Unraid 7.0 (or newer) and have the necessary GPU-related plugins.

Installing Intel GPU TOP Plugin

Install Intel GPU TOP by ich777 from the Unraid Community Apps store. This plugin allows you to monitor your Intel ARC GPUs performance directly from the Unraid dashboard.

Intel GPU TOP Plugin

Installing GPU Statistics Plugin

Next, install the GPU Statistics plugin by b3rs3rk. With Intel GPU TOP, youll gain comprehensive insights into your GPUs usage during encoding or transcoding tasks.

GPU Statistics Plugin

Once installed, youll see real-time GPU usage:

(No width specified for these screenshots)
GPU Usage Example 1
GPU Usage Example 2


Deploying Plex with Intel ARC GPU Support

Adding the Intel ARC GPU to the Plex Docker Template

In your Plex Docker template, add the Intel ARC GPU as a device. Without this, Plex wont recognize the GPU for hardware acceleration.

(No width specified originally)
Add Intel ARC GPU to Plex Template

Configuring Plex Settings

Enable hardware transcoding in Plex and HDR tone mapping (if supported). If multiple GPUs are present, select the correct one in Plex settings.

image

Verifying GPU Transcoding

Play a media file that requires transcoding. Check Plexs dashboard and your GPU stats to confirm the GPU is handling the task. You should see minimal CPU usage and smooth playback.

Plex GPU Transcoding


Setting Up Tdarr

What is Tdarr?
Tdarr simplifies media transcoding through a user-friendly interface. It automates batch conversions without requiring manual command-line settings. Although the interface can be confusing at first, once you understand it, Tdarr becomes a powerful tool for media optimization.

If you find this guide helpful, please consider clicking the ★ (Star) button above. It helps others discover this content and shows your support.

Deploying Tdarr Server

When installing Tdarr, you may see an option to deploy both a Tdarr Server and a Tdarr Node in one container. To simplify troubleshooting, its recommended to deploy them separately.

  1. Install the Tdarr Server via the Unraid App Store (ensure its labeled “Tdarr,” not “Tdarr Node”).
  2. Name it something identifiable, like “Tdarr_Server.” (I use - Server)
  3. Ensure the server IP is set correctly, usually your Unraid servers IP.
  4. Set the internal node option to False, so youll deploy a separate node later.
image image image

Tdarr Transcoding Location

Your transcoding location and hardware choice are crucial. For occasional transcoding, using an SSD/NVMe cache drive is fine. For heavy and continuous transcoding (multiple streams, multiple GPUs), consider dedicating a separate NVMe drive to handle the workload. Avoid using HDDs or RAM for transcoding as they cause performance issues and potential errors.

Warning: Bottlenecks & SSD Wear

Heavier workloads can wear out your SSD/NVMe quickly. By using a dedicated, inexpensive NVMe for transcoding, you protect your main drives from excessive wear.

image

Deploying Tdarr Node(s)

After setting up the Tdarr Server, install the Tdarr Node container (listed separately in the Unraid App Store). The node handles the actual transcoding, while the server manages nodes, libraries, and workflows.

image
  1. Install the Tdarr Node via the Unraid App Store (ensure its labeled “Tdarr Node,” not “Tdarr).
  2. Name it something identifiable, like - Node1 or N1
    • Using 2 or more GPU's? Repeat the process and call the next node - Node2 or N2
  3. Ensure the server IP is set correctly, usually your Unraid servers IP. Also ensure the NodeIP is the same IP (trust me on it)
  4. Make sure the configs and logs match the node # for simplicity.
  • Using 2 or more GPU's? Repeat the process and label each one based off the node name - node2 or n2
  1. Ensure the transcode cache matches the server's template path that you created. For this, add the node name at the end such n1 or node1
  2. Ensure to assign the correct GPU to the node. To see your GPUs, type:

ls -la /dev/dri/

WARNING:

One of these numbers will reflect your iGPU, even for AMD processors. Do not assign the iGPU to your Tdarr Node! I have not found a good way to discover which one the iGPU from the CMD line... but if you goto Plex and Transcode and look at the order of the GPUs from the menu, it actually follows that order.

image image image image

[UNDER CONSTRUCTION - DEPLOYING TDARR]


AV1 Tdarr Flow

Change Log:

  • v1: Original AV1 flow.
  • v2: Removed B-frames.
  • v3: Greatly improved quality.
  • v4: Removed images from files, reducing failure rates from ~25% to nearly 0%.

JSON Script: av1_flow_v4.json

image

What is the AV1 Flow?

The AV1 Flow is a predefined workflow that converts your media into the AV1 format. Once applied, Tdarr processes your libraries, delivering significant space savings without you needing to master complex transcoding parameters.

Importing the AV1 Flow in Tdarr

  1. Open Tdarrs flows section.
  2. Scroll down and select the “Import” option.
  3. Paste the AV1 Flow JSON script.
  4. Apply it to your libraries.

Adding a New Flow in Tdarr
Scroll to Import Option
Pasting the JSON Content

Once set, Tdarr will begin transcoding your files to AV1.


Optimizing AV1 Encoding Settings

Experiment with quality (CRF) and bitrate settings in the AV1 Flow to find a balance between file size and image quality. Ensure hardware acceleration is enabled so the GPU handles most of the processing.


SABNZBD Speed Control - Bonus

The SAB Speed Script dynamically adjusts SABnzbd download speeds based on Plex streaming activity. When Plex is active, it reduces download speeds to prevent buffering. During off-peak times, it ramps speeds back up, maximizing your bandwidth efficiency.

Requirements:

Script: sab_speed_control.sh

After saving the script, configure it to run at array startup and choose “Run in Background.”

image image

Tdarr Node Killer Script

Change Log:

  • v1: Original Script
  • v2: Uses Tautulli to monitor Plex, simplifying detection.

Overview

The Tdarr Node Killer script ensures Plex always has priority access to the GPU. If Plex and Tdarr share the same GPU, Tdarrs transcoding tasks can pause whenever Plex starts transcoding, preserving a smooth streaming experience.

Script Behavior

  • When Plex starts transcoding (detected via Tautullis API), the script stops the Tdarr Node.
  • After Plex stops transcoding, the script waits a cooldown (e.g., 3 minutes) before restarting Tdarr. This prevents rapid start/stop cycles if users frequently pause or stop streaming.

Script: tdarr_node_killer.sh

Install the User Scripts plugin, add the script, set it to run at array startup, and run it in the background.

image

Step-by-Step Implementation for Unraid

  1. Tdarr Node Running, No Plex Transcoding:

    Tdarr Node Running

  2. Script Monitoring Plex:

    image
  3. Plex User Starts Transcoding:

    Plex User Starts Transcoding

  4. Script Detects Transcoding & Stops Tdarr Node:

    image

    Tdarr Node Stopped

  5. Tdarr Node Completely Stopped:

    Tdarr Node Dead

Script Behavior After Plex Transcoding Stops

The script waits a set cooldown period after Plex finishes before restarting the Tdarr Node.

  1. Countdown Before Restarting Tdarr Node:

    Countdown

  2. Tdarr Node Restarts After Cooldown:

    image

    Tdarr Node Coming Online

  3. Tdarr Node Fully Online Again:

    Tdarr Node Online

Troubleshooting Common Issues

  • Plex Not Using GPU? Double-check Docker template settings and Plexs hardware transcoding settings.
  • Tdarr Not Restarting? Verify the script and Tautulli API settings. Make sure the script runs in the background.
  • High CPU Usage? If HDR tone mapping is on, ensure your GPU and drivers support it. Update all plugins and drivers.

Experimental: Running the Script on Other Operating Systems

You can run the Tdarr Node Killer script on other Linux distributions or operating systems that support Docker and systemd. The process is similar: place the script, grant proper permissions, and create a systemd service.

Step-by-Step Implementation for Other OSes

  1. Save the Script: Save your Tdarr Node Killer Script as tdarr_node_killer.sh in /usr/local/bin/.

    sudo nano /usr/local/bin/tdarr_node_killer.sh
    
  2. Set the Proper Permissions:

    sudo chmod +x /usr/local/bin/tdarr_node_killer.sh
    sudo chown root:root /usr/local/bin/tdarr_node_killer.sh
    
  3. Create a Service File: Create a service file for the script:

    sudo nano /etc/systemd/system/tdarr_node_killer.service
    
  4. Add the Following Content:

    [Unit]
    Description=Tdarr Node Killer Script
    After=network.target
    
    [Service]
    Type=simple
    ExecStart=/bin/bash /usr/local/bin/tdarr_node_killer.sh
    Restart=on-failure
    
    [Install]
    WantedBy=multi-user.target
    
  5. Reload Systemd:

    sudo systemctl daemon-reload
    
  6. Start and Enable the Service:

    sudo systemctl start tdarr_node_killer.service
    sudo systemctl enable tdarr_node_killer.service
    

This ensures the script runs automatically and manages your GPU resources even if youre not on Unraid.


Backup and Recovery Tips

Before making changes:

  • Backup Plex configs (metadata, watch history, etc.).
  • Backup Docker templates so you can quickly restore containers.
  • Backup your Unraid flash drive so you dont lose your server setup.

Test your backups occasionally to ensure they work when you need them.


Summary

By setting up AV1 encoding with Intel ARC GPUs, you can achieve massive storage savings and still maintain great quality. Adding the optional Tdarr Node Killer Script ensures Plex always has priority access to the GPU when needed. With careful tuning and a bit of experimentation, you can streamline your servers performance, reduce storage costs, and keep everyone happy with smooth, high-quality streams.

Found this useful? Consider clicking the star (★) button at the top!