Flag to include specials from fn:miss script

[dokuro]

I understand this is very much an edge case but could a flag be added or is there a way to run filebot -script fn:miss /path/to/media so that it will include specials ? I'm pretty sure we don't want this by default but it would be nice if someone wanted to find missing specials that they could.

BTW... For context and my use case I'm less interested in actually finding missing special episodes (although I'd still find this useful) but I'm more interested in the detection of changes for specials episode information.

Thanks as always.

[rednoah]

BTW... For context and my use case I'm less interested in actually finding missing special episodes (although I'd still find this useful) but I'm more interested in the detection of changes for specials episode information.

Please try Find files where online metadata has changed for the task at hand.



EDIT:

Note that the "print episode that has changed" feature we added last month already works for specials:
https://github.com/filebot/scripts/blob ... groovy#L48

Presumably, something doesn't work for you? Can you provide xattr metadata for the test file that doesn't work?

[dokuro]

Unfortunately as discussed in this first part of this comment viewtopic.php?p=61035#p61035 it does not work unless I run it on individual shows themselves rather than root directory. I guess I could script it to run on each show but the miss script runs better.

[rednoah]

it does not work unless I run it on individual shows themselves rather than root directory

There is no such limitation. Both approaches are doing the same thing based on the same data, so they'll work the same, unless there's a bug. If there's a bug, then we can look into that:
viewtopic.php?p=61043#p61043



EDIT:

e.g. find-episode-metadata-changes.sh works with subfolders, so if it doesn't work for you, then you could do some trial and error testing to see what makes your use case different from my test case:

Console Output: Select all

$ find-episode-metadata-changes.sh 'TV Shows'
Alias - 1x01 - X | Alias - 1x01 - Truth Be Told | TV Shows/Alias/Season 01/Alias - S01E01 - Truth Be Told.mp4
Firefly - Special 7 - X | Firefly - Special 7 - Adam Baldwin Sings Hero of Canton | TV Shows/Firefly/Specials/Firefly - S00E07 - Adam Baldwin Sings Hero of Canton.mp4

e.g. miss script is doing what you're asking for by default, so if it doesn't work for you, then you could do some trial and error testing to see what makes your use case different from my test case:

Console Output: Select all

$ filebot -script fn:miss 'TV Shows'
TheMovieDB::TV::2046 | Alias - 1x01 - X | TV Shows/Alias/Season 01/Alias - S01E01 - Truth Be Told.mp4
TheMovieDB::TV::1437 | Firefly - Special 7 - X | TV Shows/Firefly/Specials/Firefly - S00E07 - Adam Baldwin Sings Hero of Canton.mp4
* Episode #141453 has changed: Alias - 1x01 - X -> Alias - 1x01 - Truth Be Told
* Episode #2235074 has changed: Firefly - Special 7 - X -> Firefly - Special 7 - Adam Baldwin Sings Hero of Canton
...

I'd start by comparing the xattr metadata you have with the xattr metadata you expect based on online episode information:

I'd start by checking if we're working with the same episode metadata:

Shell: Select all

filebot -mediainfo -r /input --filter 'episode' --format '{json}'

Shell: Select all

filebot -list --db TheMovieDB::TV --q 114461 --format '{json}'

[dokuro]

I failed to test the miss script myself for changed specials and you are correct, it does indeed detect changes so I'm all good with this. Thanks.

I am however still at a loss why find-episode-metadata-changes.sh does not work when using a higher level folder path. So, for example:

Shell: Select all

sh find-episode-metadata-changes.sh '/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}'

works perfectly. It detects ONLY changes and is working as expected. However ...

Shell: Select all

sh find-episode-metadata-changes.sh '/data/MediaServer/8a-TV/Anime'

... does not as list everything for every under the folder, example below.

Console Output: Select all

Sword Art Online Alternative: Gun Gale Online - 1x01 - Squad Jam |  | /data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E01 - Squad Jam.mp4
Sword Art Online Alternative: Gun Gale Online - 1x02 - GGO |  | /data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E02 - GGO.mp4
Sword Art Online Alternative: Gun Gale Online - 1x03 - Fan Letter |  | /data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E03 - Fan Letter.mp4
Sword Art Online Alternative: Gun Gale Online - 1x04 - Death Game |  | /data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E04 - Death Game.mp4
Sword Art Online Alternative: Gun Gale Online - 1x05 - Leave the Last Battle to Me |  | /data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E05 - Leave the Last Battle to Me.mp4
Sword Art Online Alternative: Gun Gale Online - 1x06 - SAO Loser |  | /data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E06 - SAO Loser.mp4
Sword Art Online Alternative: Gun Gale Online - 1x07 - Second Squad Jam |  | /data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E07 - Second Squad Jam.mp4
Sword Art Online Alternative: Gun Gale Online - 1x08 - Booby Trap |  | /data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E08 - Booby Trap.mp4
Sword Art Online Alternative: Gun Gale Online - 1x09 - Ten-Minute Massacre |  | /data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E09 - Ten-Minute Massacre.mp4
Sword Art Online Alternative: Gun Gale Online - 1x10 - The Devil's Comeback |  | /data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E10 - The Devil's Comeback.mp4
Sword Art Online Alternative: Gun Gale Online - 1x11 - Psycho LLENN |  | /data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E11 - Psycho LLENN.mp4
Sword Art Online Alternative: Gun Gale Online - 1x12 - Applause |  | /data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E12 - Applause.mp4
Sword Art Online Alternative: Gun Gale Online - Special 1 - Re frain |  | /data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Specials/Sword Art Online Alternative: Gun Gale Online - S00E01 - Refrain.mp4

Maybe the path is causing a problem somehow. `/data/MediaServer/8a-TV` is a mount point to my NAS. I've reviewed the metadata both local and online and they appear the same.

EDIT:
Okay, I think the fact my media is located via network path somehow is causing the issue I'm reporting. I copied 'Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}' to a local folder, '/home/plex/Downloads/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}'. Then when I run both ..

Shell: Select all

sh find-episode-metadata-changes.sh '/home/plex/Downloads/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}'
sh find-episode-metadata-changes.sh '/home/plex/Downloads/Anime'

... completed without any problem only detecting the changes. Does any of this give you any clue to what could be causing the problem ?

[rednoah]

Please run the following commands and check if the output matches.


Check if filebot sees the same files:

Shell: Select all

filebot -mediainfo -r '/data/MediaServer/8a-TV/Anime' --filter 'f.video' --format '{f}'

Shell: Select all

filebot -mediainfo -r '/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}' --filter 'f.video' --format '{f}'

Check if filebot sees the same xattr metadata:

Shell: Select all

filebot -mediainfo -r '/data/MediaServer/8a-TV/Anime' --filter 'f.video' --format '{json}'

Shell: Select all

filebot -mediainfo -r '/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}' --filter 'f.video' --format '{json}'

[dokuro]

Here you go... (they look identical to me):

Console Output: Select all

plex@dokuro:/data/Personal/Scripts$ filebot -mediainfo -r '/data/MediaServer/8a-TV/Anime' --filter 'f.video' --format '{f}' | grep tmdb-78204
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E01 - Squad Jam.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E02 - GGO.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E03 - Fan Letter.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E04 - Death Game.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E05 - Leave the Last Battle to Me.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E06 - SAO Loser.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E07 - Second Squad Jam.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E08 - Booby Trap.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E09 - Ten-Minute Massacre.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E10 - The Devil's Comeback.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E11 - Psycho LLENN.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E12 - Applause.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Specials/Sword Art Online Alternative: Gun Gale Online - S00E01 - Refrain.mp4
plex@dokuro:/data/Personal/Scripts$ filebot -mediainfo -r '/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}' --filter 'f.video' --format '{f}' | grep tmdb-78204
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E01 - Squad Jam.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E02 - GGO.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E03 - Fan Letter.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E04 - Death Game.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E05 - Leave the Last Battle to Me.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E06 - SAO Loser.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E07 - Second Squad Jam.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E08 - Booby Trap.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E09 - Ten-Minute Massacre.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E10 - The Devil's Comeback.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E11 - Psycho LLENN.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Season 01/Sword Art Online Alternative: Gun Gale Online - S01E12 - Applause.mp4
/data/MediaServer/8a-TV/Anime/Sword Art Online Alternative: Gun Gale Online (2018) {tmdb-78204}/Specials/Sword Art Online Alternative: Gun Gale Online - S00E01 - Refrain.mp4

Regarding the second set of commands, they look the same also, sample below:
First:

Console Output: Select all

{"@type":"Episode","seriesName":"Sword Art Online Alternative: Gun Gale Online","season":1,"episode":1,"title":"Squad Jam","absolute":1,"airdate":{"year":2018,"month":4,"day":8},"id":1457646,"group":"Season 1","seriesInfo":{"database":"TheMovieDB::TV","order":"Airdate","language":"en-US","type":"Scripted","id":78204,"name":"Sword Art Online Alternative: Gun Gale Online","aliasNames":["ソードアート・オンライン オルタナティブ ガンゲイル・オンライン","刀剑神域外传 Gun Gale Online","刀剑神域外传 GGO","소드 아트 온라인 얼터너티브: 건 게일 온라인","МАСТЕРА МЕЧА ОНЛАЙН: АЛЬТЕРНАТИВНАЯ «ПРИЗРАЧНАЯ ПУЛЯ»","刀劍神域外傳 Gun Gale Online","刀劍神域外傳 GGO","SAO Alternative Gun Gale Online","SAO GGO","SAO Alternative: GGO","SAOAGGO","Sword Art Online Alternative GGO","Đao Kiếm Thần Vực: Gun Gale Online","Sword Art Online AlternativeGun Gale Online","Đao Kiếm Thần Vực – Câu chuyện bên lề Gun Gale Online","Альтернативний Сворд Арт Онлайн: Ґан Ґейл Онлайн","Sword Art Online Alternative - Gun Gale Online","SAO Alternative - GGO"],"startDate":{"year":2018,"month":4,"day":8},"genres":["Animation","Action & Adventure","Sci-Fi & Fantasy"],"spokenLanguages":["ja"],"network":"BS11","rating":7.912,"ratingCount":222,"runtime":24,"status":"Returning Series"}}

Second:

Console Output: Select all

{"@type":"Episode","seriesName":"Sword Art Online Alternative: Gun Gale Online","season":1,"episode":1,"title":"Squad Jam","absolute":1,"airdate":{"year":2018,"month":4,"day":8},"id":1457646,"group":"Season 1","seriesInfo":{"database":"TheMovieDB::TV","order":"Airdate","language":"en-US","type":"Scripted","id":78204,"name":"Sword Art Online Alternative: Gun Gale Online","aliasNames":["ソードアート・オンライン オルタナティブ ガンゲイル・オンライン","刀剑神域外传 Gun Gale Online","刀剑神域外传 GGO","소드 아트 온라인 얼터너티브: 건 게일 온라인","МАСТЕРА МЕЧА ОНЛАЙН: АЛЬТЕРНАТИВНАЯ «ПРИЗРАЧНАЯ ПУЛЯ»","刀劍神域外傳 Gun Gale Online","刀劍神域外傳 GGO","SAO Alternative Gun Gale Online","SAO GGO","SAO Alternative: GGO","SAOAGGO","Sword Art Online Alternative GGO","Đao Kiếm Thần Vực: Gun Gale Online","Sword Art Online AlternativeGun Gale Online","Đao Kiếm Thần Vực – Câu chuyện bên lề Gun Gale Online","Альтернативний Сворд Арт Онлайн: Ґан Ґейл Онлайн","Sword Art Online Alternative - Gun Gale Online","SAO Alternative - GGO"],"startDate":{"year":2018,"month":4,"day":8},"genres":["Animation","Action & Adventure","Sci-Fi & Fantasy"],"spokenLanguages":["ja"],"network":"BS11","rating":7.912,"ratingCount":222,"runtime":24,"status":"Returning Series"}}

BTW... did you see my edit above regarding the network path ? You might have missed it if our comment updates crossed each other.

[rednoah]

Based on the output above, the network path is not an issue. File listing is working. Xattr is working.


The next step is to debug things step by step... checking what each binding values returns... because at some point something has to be different, otherwise how can the result be different?




EDIT:

{episodelist} is broken with FileBot 5.1.0 / FileBot 5.1.1 due to incorrect caching in certain non-rename contexts, possibly this context as well, making {episodelist} always yield the episode list of whatever series is encountered first, and not the series at hand. This would explain why it works when we're only testing with one single series.


You can confirm this by checking {episodelist} for all files:

Shell: Select all

filebot -mediainfo -r '/data/MediaServer/8a-TV/Anime' --filter 'f.video' --format '{fn} => {episodelist.episode}'

The issue has been fixed a few weeks ago. Please try again with the latest revision to confirm that it works with newer revisions.

[dokuro]

Okay, bingo! I had `FileBot_5.1.1_amd64.deb` so I went and grabbed beta `FileBot_5.1.2_amd64.deb` from 10th Oct and ...

Shell: Select all

filebot -mediainfo -r "/data/MediaServer/8a-TV/Anime" --file-filter 'f.video' --filter 'episode && none{ episode as String == episodelist.find{ it.episode == episode }.episode as String }' --format '{episode} | {episodelist.find{ it.episode == episode }.episode} | {f}'

... is now working as expected. So it was a bug all along. Thanks for spending the time to debug this with me. Is 5.1.2 likely to go public anytime soon and any drawbacks in staying on the version I now have installed ?

[rednoah]

Mystery solved! Thanks for running tests! The next release is scheduled for this week, pending some more testing and community feedback.