One of the personal projects I had sitting on the backburner for the past year (or more accurately, many years) was upscaling/enhancing videos from the 2000’s. That was a period when 480p (or 540p) widescreen was more common than 720p (and the rare 1080 was mostly interlaced and not progressive). I would say that this year (2025) is resulting in an intersection of decent hardware plus software (with better LLM models fine tuned for video enhancement).
I previously reviewed Aiarty’s Video Enhancer AI (since they have been offering a free 90-day trial for both Mac and Windows versions). Compared to Topaz Video AI, it’s much simpler to use because it doesn’t give you all of those granular settings (which are great for getting as high quality output as possible). For my use case though, the results are nearly the same (I could eek out a slight bit more quality with Topaz by trial-and-error dialing-in the settings specifically for each video). But I concluded that for what I am doing (which is more personal and not business), it doesn’t make sense at this time (because that does add time to the entire process).
And this is entire workflow is time/energy (in terms of power) consuming. While these Apple Silicon Mac’s have great performance per watt efficiency, that isn’t a concern for me if there is a brute force tool like a desktop system that has a decent GPU to crunch through the ML upscaling work very quickly. Thus all of that (ML upscaling) is now being handled by my mini-tower PC since it’s Nvidia GeForce RTX 4070 Ti SUPER can handle this nearly twice as fast as my M4 Pro mini (I can imagine how much the gap widens with an RTX 4090 or something like the 5080 Ti). It might take an M4 Max (16-core with 40-core GPU) system to start getting close to this 4070 Ti SUPER (and cost a pretty penny as well) whereas it is hard to imagine what Apple would charge for an M4 Ultra if/when they release a system with it (plus the M5 is around the corner where it makes more sense to wait for that).
The AI/ML-based upscaling/enhancing is only a part of this process though. Most of the pro shot source I have is initially upscaled to 720p without AI (far faster/results that look clean but not sharp). At least with current tech, some of the interpolation (4x) that happens with AI/ML yields results that don’t look natural (I think going from 480 to 1920 is a bit too much GIGO while 2x yields a more reasonable 960p, but just a tad short of the 1080p). Thus I utilize ffmpeg to do a more traditional upscaling to 720p and then run that through Video Enhancer AI at 2x and 60fps to get to 1440p60. I then take that, and downscale it to 1080p60 with Digiarty VideoProc Converter AI (it’s the same company as Aiarty by the way). This downscaling makes the resulting output gain sharpness.
On the encoding side, the performance is a different story since most systems hardware nowadays has both an H.264 and HEVC hardware encoders as part of the GPU. I took the same 2560x1440p60 video and converted them to 1080p (high quality encoder setting) with VideoProc Converter on 4 different systems. The following are all real-time (the recording adding some overhead where each system lost maybe 10fps of performance on average). The M3 Pro averaged around 190fps, M4 Pro around 205fps, Mac Pro around 230fps, and PC (using only the RTX 4070 Ti SUPER) around 490fps. All of the Mac’s used both their CPU and GPU.
My 2019 Mac Pro (Intel based) performed the worse with that AI/ML (for upscaling) workload since it’s Radeon Pro W5500X GPU isn’t much of a performer for those of hardware accelerated tasks. The two Apple Silicon based Macs did well with those same AI/ML upscaling tasks by comparison; some of this is helped by the “Neural Engine” tech built into the chips since 2017 with the A11. Both the M3 and M4 Pro have a 16-core Neural Engine. Given how little wattage these Apple Silicon chips use, their performance highlights their efficiency when comparing their size and price versus that Mac Pro). With video encoding tasks though, that older Mac Pro did manage to hold its own (showing where it will still be useful for these specific tasks in this workflow). While it is becoming more outdated, it has that sweet spot for hardware video encoding (it would perform a lot better with the 12-core Xeon; the costs for the 28-core Xeon isn’t worth it however for the end of the line Intel based Mac). However, putting a better GPU into the system was something I decided to do since the largest gains should come from that (I have a Radeon Pro W6800X MPX module on the way). It sucks that these last support 6xxx series Radeon Pro’s don’t have better hardware decoding on the Mac (I guess part of that might be unoptimized drivers for the macOS) since it seems like not even the duo versions of the W6800X and W6900X can playback HEVC 8K video without stutters).
VideoProc Converter is straightforward since it provides a lot of base presets (the parameters can be changed and those changes can be saved as your own custom preset). The software did crash on occasion (where I lost that custom preset). Besides those minor issues, the software was straightforward. I only used the video conversion part; the rest of the functionality I didn’t yet try but will do that one day. The AI assisted ones likely are a pared down version of Video Enhancer AI. Thus I’m not reviewing the software at this time (this post is mainly about the video encoding).
Which brings me to the brute force solution; a decent PC GPU wipes the floor for these two tasks. Both Video Enhancer AI and VideoProc Converter AI only use the GTX 4070 Ti SUPER GPU (unlike the 3 Mac’s where it utilizes both the CPU and GPU). The Tensor core similarly helps with AI/ML tasks like Apple Silicon’s Neural Engine. The 4070 Ti SUPER wasn’t a top line card in the 40 series (that is the 4080 and 4090). And while the newer 50 series are a mixed bag for gaming, Nvidia geared them for AI/ML tasks (this video stuff might make something like a RTX 5080 Ti worthwhile if it is on average, 30% faster). AI/ML upscaling is roughly twice as fast (than my M4 Pro mini) and close to twice as fast in video encoding as my Mac Pro (with the anemic W5500X, it still beat both Apple Silicon systems). Once I receive the W6800X, will see how much better it is at both tasks (I don’t expect too much gains on the AI/ML upscaling but expect to see it increase significantly with encoding). And that will be perfect since this will allow my PC to do just the AI/ML tasks while my Mac Pro handles the encoding.
It’s not a completely flawless process though. These AI/ML upscaling software still has challenges with smaller details (thus a subject that is zoomed out may not be upscaled well (a good example is concert footage where you have multi-camera sequences that are combined into a final footage with transitions that zoom in and zoom out of the performer (those close up shots have more detail to work with). This is why there is a lot of headroom for the technology to increase. But I really liked the results I got (following is one of the re-processed clips from the August 7, 2000 concert at Nakano Sun Plaza).
I did all of those with Video Enhancer AI and VideoProc Converter. I’m pretty sure these type of niche software (the video specific LLM’s for AI/ML upscaling) will be even better an year from now where the detail level and interpolation may have that refinement where you can go to 4x with 480p source and not have it looking like garbage where you need to either shrink the player window and/or downscale the video. This still isn’t a solution to upscale to 4K where you can play it on a native 4K display like a home theater setup. Even some disc players that have a hardware upscaling chip in them have trouble getting that actual sharp looking picture from lower resolution videos. That leap in technology is still a few years away (we’re closer to it based on the above).
One Comment
Comments are closed.