ROM Shepherd Forum

All your ROM collecting needs fulfilled! => New Datfiles => Topic started by: Landcross on January 12, 2018 - 19:45:53

Title: Redump parent/clone
Post by: Landcross on January 12, 2018 - 19:45:53
Hey all,

Tiny backstory: I'm using Launchbox as a front-end for my roms. One of the great features (imo) is the fact that it can combine multiple roms into a single game, which makes it possible to keep the competitionist in me happy while not cluttering up the front-end with a bazillion versions of a game. Launchbox uses its own database to automatically merge those games, but the database is not complete. I've been adding 'alternate titles' to their database every now and then, and today I realized that the same information might also be useful for something like a redump parent/clone datfile.

Now, here's the problem: I have no experience at all with parent/clone datfiles. I only know of their existence at no-intro and that's about it. So, I have 3 questions:

Regarding question 3: I made a quick proof of concept to see if it was doable. I used the existing parent/clone files from no-intro as a guideline, but as I said I have no knowledge of those files so...

Here's the example output:
Code:
Only registered users can see contents. Please click here to Register or Login.
Any thoughts/opinions?  :)

BTW: maybe a 4th question; I have no idea if there is any behind-the-scenes work on this kind of stuff at redump itself. Would be a bit of a waste of time if people start doing the same thing twice. Any info on that?
Title: Re: Redump parent/clone
Post by: disc on January 13, 2018 - 00:35:28
It would be better to automate this process. If you could read in a folder of discs and group them by similar files, you would have a pretty good shot at grouping cross-region releases. The "parent" vs "clone" part of it could be a simple text match: first USA, then Europe, then Japan, for example.

Making such a program might be hard, but as you said, creating those parent/clone dats is also hard.
Title: Re: Redump parent/clone
Post by: Connie on January 13, 2018 - 02:29:44
A parent/clone relation dat offers two uses.
It acts like the option of MAME to give split or merged sets. So all related games are in one archive - usefull for ROM dumps, not so much for large ISO's.

Using 7z on a merged set will offer huge disc space savings over individual files.

If you have 6 language variants of a game using their native language for the title, then I guess a merged parent/clone set would help with possible 'undumped' versions. ?
Title: Re: Redump parent/clone
Post by: Landcross on January 13, 2018 - 08:49:15
[You are not allowed to view links] Register or Login
It would be better to automate this process. If you could read in a folder of discs and group them by similar files, you would have a pretty good shot at grouping cross-region releases. The "parent" vs "clone" part of it could be a simple text match: first USA, then Europe, then Japan, for example.

Making such a program might be hard, but as you said, creating those parent/clone dats is also hard.

It's already semi-automated (for a POC at least :P). It's indeed pretty simple to match roms that just have the same name (except the region). Where things get tricky though (and where automation isn't really possible, aside from the dat-generation of course) is matching games with different titles. A simple example in my example-dat above would be 'Fantastic 4' which is called 'Fantastici 4, I' in Italian. And it gets more 'difficult' with Japanese titles and titles which have just completely different names in different regions.



[You are not allowed to view links] Register or Login
A parent/clone relation dat offers two uses.
It acts like the option of MAME to give split or merged sets. So all related games are in one archive - usefull for ROM dumps, not so much for large ISO's.

Using 7z on a merged set will offer huge disc space savings over individual files.

If you have 6 language variants of a game using their native language for the title, then I guess a merged parent/clone set would help with possible 'undumped' versions. ?

Ah, I see. I hadn't though of that. Sounds pretty logical indeed, merging different languages in a single zip to save compression. But, as you said, might not gain that much for isos.

I don't quite see how parent/clone information could help with possible undumped versions though? A game could've been released in so many languages yet are often released in very few, the fact that you're missing a language doesn't mean it's missing right?
Title: Re: Redump parent/clone
Post by: Connie on January 13, 2018 - 23:19:39
[You are not allowed to view links] Register or Login
......the fact that you're missing a language doesn't mean it's missing right?
Of course not. But...

If you have
English, French [ABCP12]
German, Spanish [ABCY12]
USA [ABCE12]
Japanese [ABCJ12]

sitting in a merged archive, then it makes it easier to see that you 'could' be missing

Italian, Netherlands [ABCX12]
Title: Re: Redump parent/clone
Post by: Chaos_Therum on March 05, 2018 - 20:01:09
Just gonna throw in my two cents. From the testing I've done with playstation games in particular they would see a huge space savings with p/c dats. I saw on average a 70% decrease in size when I merged multiple regions even bigger when I used gpack but it's been abandoned so I don't want to archive my entire collection into a dying compression program even though it is all open source.

Wish I knew how to program gpack gave me a nearly 80% decrease in size for psx I merged all of the # games and went from 60 GB to 8 GB but I digress I would definitely use your dat if you made one.
Title: Re: Redump parent/clone
Post by: dxzdxz1 on March 27, 2018 - 03:36:41
Don't know if you found a solution for your question, but you could be enable deduplication or transparent compression on your PSX/DC/Saturn/etc drive. It will give you great hdd space savings.

For some games there's also differences on soundtrack and or some graphics or even different endings for each region release, isn't a good idea to keep only a single version for every game after all.
Title: Re: Redump parent/clone
Post by: an0rmal on April 03, 2018 - 21:09:35
You could check Mobygames for alternates, example for Fantastic 4:

Alternate Titles
"神奇四侠" -- Chinese spelling (simplified)
"ארבעת המופלאים" -- Hebrew spelling
"Les 4 Fantastiques" -- French title

This is for GC. They don't have the italian title, but i guess this will very very helpful for the project.
As it could be scrapped (they offer an api) per game and then merged with "main" title.