Duplicate logic for Dolby Vision and HDR10+

[theimmortal]

Hello,

I am looking for a way to remove duplicates that prioritizes the HDR Dynamic Range. My situation is I have some movie files in both 4k and 4k Dolby Vision and the fn:duplicates seems to only care about resolution, bit rate, and fireside. This does not always equate to it choosing the Dolby Vision version. Is there a way to include this in the command?

[rednoah]

You can configure the duplicate script to allow both versions. That might help.

Adding support for comparing non-numerical properties (not comparable by nature, subject to interpretation) is not planned.

One could write a custom script though, for custom deduplication. Just selecting only Dolby Vision or only HDR10+ can be done with simple filebot commands.

[theimmortal]

How would I write the command to allow both versions?

Currently I use the following command to remove lower quality duplicates:

Code: Select all

filebot -script fn:duplicates /Movies --action delete

This works perfectly when wanting to select the 4K version between the following:

Avatar - 1080p.mkv
Avatar - 2160p.mkv

But what I want to do is make sure it selects the Dolby Vision version when choosing between the following:

Avatar - 1080p.mkv
Avatar - 2160p.mkv
Avatar - 2160p Dolby Vision.mkv

From your response, you mentioned that choosing between Dolby Vision and HDR10+ is beyond the scope of the script, which is okay since this scenario would be extremely rare. My goal is simply to ensure that if a situation occurs where there are two 4K movies versions in the folder, that the one which has the dynamic metadata wins out.

[rednoah]

I see. The duplicates script actually no longer supports custom deduplication formats. This feature has been stripped in favor of simplicity and conflicts with other use cases.


I have now added {hdr} to the internal deduplication key, in addition to {vf}:
https://github.com/filebot/scripts/comm ... afe21a8e5d


Please add -non-strict to your command to deduplicate by Video Format, which now also includes the HDR type:

Code: Select all

filebot -script dev:duplicates /path/to/files -non-strict

You will also need to use -script dev:duplicates for the time being to grab the latest script revision directly from GitHub.

From your response, you mentioned that choosing between Dolby Vision and HDR10+ is beyond the scope of the script, which is okay since this scenario would be extremely rare. My goal is simply to ensure that if a situation occurs where there are two 4K movies versions in the folder, that the one which has the dynamic metadata wins out.

If default strict mode is used, then the one with the higher resolution / bit rate will win out. Dolby Vision / HDR10+ file are typically larger (same resolution, higher bit rate, because more information) and would thus typically win out. That's unfortunately not a guarantee though, even if it probably most works. I don't have enough 4K / Dolby Vision / HDR10+ sample files though, so really, I have no idea.

[rednoah]

Code: Select all

Avatar - 1080p.mkv
Avatar - 2160p.mkv
Avatar - 2160p Dolby Vision.mkv

You could use filebot -find to find files that don't have a sibling file with Dolby Vision in the file name and then use -exec to run arbitrary commands on them:

Code: Select all

$ filebot -find . --filter '{ folder.listFiles{ it != f && it.name =~ /Dolby.Vision/ } }' -exec ls -l {f} +
...
-rw-r-----@ ... Avatar - 1080p.mp4
-rw-r-----@ ... Avatar - 2160p.mp4

Replace ls -l with rm -v to delete files.



EDIT:


This code is simple and tested only for the use case above. If you add another Avatar - 1080p Dolby Vision file then the code might just delete all your files because the code is simple and doesn't account for that. So make sure to test for corner cases before actually deleting files, and adjust the code according to your needs.

e.g.

Code: Select all

Avatar - 1080p Dolby Vision.mp4
Avatar - 1080p.mp4
Avatar - 2160p Dolby Vision.mp4
Avatar - 2160p.mp4

Code: Select all

$ filebot -find . --file-filter 'f.video && !(f.name =~ /Dolby.Vision/)' --filter '{ folder.listFiles{ it != f && it.name =~ /Dolby.Vision/ } }' -exec ls -l {f} +
...
-rw-r-----@ ... Avatar - 1080p.mp4
-rw-r-----@ ... Avatar - 2160p.mp4