Author Topic: Scanning v4 CHDs with CMPro 4.03/0.145u1  (Read 11483 times)

0 Members and 1 Guest are viewing this topic.

Offline Cassiel

  • Hero Member
  • *****
  • Posts: 653
Re: Scanning v4 CHDs with CMPro 4.03/0.145u1
« Reply #30 on: February 23, 2012 - 00:44:56 »
Some more background/updates (apologies if these lengthy quotes are bugging anyone, but I know not everyone here having issues follows these blogs/boards)

Quote
etabeta
February 22, 2012 at 08:08

just to sum up the current situation about v5 conversions: the main problem reported so far is chdman randomly hanging forever (but without corrupting any file) and requiring you to kill the process.

concerning SHA1s
1- CHD containing HD images: no problem reported with the conversions
2- CHD containing CD images: all CHDs update with no issues, but some CHDs were created by dumpers using very old chdman and as a result they did not take into account the pregap/postgap data; after being updated these CHD will have a different SHA1, but it’s not a mistake because they need to be redumped (next MAME version will have updated SHA1 and a BAD_DUMP flag)
3- CHD containing GD-ROM images: also in this case, all CHDs update with no issues, but some CHDs were created by dumpers using very old chdman; after being updated these CHD will have a different SHA1, but it’s not a mistake (next MAME version will have updated SHA1)
4- CHD containing LaserDisk images: a few CHDs get converted with wrong SHA1 (but contained data remain perfectly fine, it’s just a metadata issue) and few others are reported to be converted to a broken image (but without any damage to the original file)

for CHDs which got diff SHA1s in cases 2 & 3, you will have anyway to extract and recompress the CHDs either with any-post 2009 chdman or with the latest to get proper metadata and to have the right SHA1s that will required in next MAME updates. in this case, you can well use latest chdman to get a better compression

summing up, there are no big risk converting CHDs, but I’d strongly suggest to skip laserdisks for the moment or at least not to delete the original files for them :)
concerning fixing chdman, Jurgen has reported a few memory issues spotted by valgrind which might be the reason for all the problems with LD metadata and hangs (+ Apple GCC miscompiling chdman with default OPTIMIZE values)


Haze
February 22, 2012 at 12:49

Converting now you’d still be converting to non-optimal settings tho.

I find the fact that some MAME dumper(s) used a well out of date chdman binary to dump CD-ROM/GD-ROMs (that they then became the sole source for, and were never cross verified/checked) rather troubling. Slaps all round there I think......   :o

[You are not allowed to view links] Register or Login
Maybe it'll be possible to hack the metadata in the headers of the CHD rather than extract/create the problem dumps.

Really wouldn't go about it like that...

[You are not allowed to view links] Register or Login
Well I'm open to dos scripts that'll sort it all.

Exactly... will end up (the 'best', if not the quickest way) being a BAT script that calls (the latest, fixed and optimised) chdman to re-compress all the MAME CHDs. Not the quickest, but certainly cleanest. Any of the newly discovered bad CHDs (as in the raw data/original dump incomplete) will be removed and replaced in subsequent MAME releases anyway....

Offline sdragon

  • Sr. Member
  • ****
  • Posts: 475
Re: Scanning v4 CHDs with CMPro 4.03/0.145u1
« Reply #31 on: February 23, 2012 - 01:29:32 »
here is a bat by oak2003

[attachment deleted by admin]

Offline niallquinn

  • Hero Member
  • *****
  • Posts: 2151
  • FTM.
Re: Scanning v4 CHDs with CMPro 4.03/0.145u1
« Reply #32 on: February 23, 2012 - 19:28:03 »
[You are not allowed to view links] Register or Login
here is a bat by oak2003

Thanks.

Can you post a link of where you got that from?

Just in case it get's updated.

NQ.
A rom is for life, not just for Christmas.

Offline niallquinn

  • Hero Member
  • *****
  • Posts: 2151
  • FTM.
Re: Scanning v4 CHDs with CMPro 4.03/0.145u1
« Reply #33 on: February 23, 2012 - 20:04:17 »
Linkage - [You are not allowed to view links] Register or Login

Worth keeping an eye on, updates chds like a dream, even scans subfolders, only thing is, it's only dos 8.3 filenames, so won't do a fair bit of the MESS ones.

But certainly interesting.

NQ.
A rom is for life, not just for Christmas.

Offline niallquinn

  • Hero Member
  • *****
  • Posts: 2151
  • FTM.
Re: Scanning v4 CHDs with CMPro 4.03/0.145u1
« Reply #34 on: February 26, 2012 - 16:59:58 »
u2 out, let's see if we can sort this out now.

NQ.
A rom is for life, not just for Christmas.

Offline niallquinn

  • Hero Member
  • *****
  • Posts: 2151
  • FTM.
Re: Scanning v4 CHDs with CMPro 4.03/0.145u1
« Reply #35 on: February 26, 2012 - 20:38:04 »
Good grief, u3 is out now.  Fixes are the xmllist and another CHDman fix.

NQ.
A rom is for life, not just for Christmas.

Offline Connie

  • Hero Member
  • *****
  • Posts: 2156
Re: Scanning v4 CHDs with CMPro 4.03/0.145u1
« Reply #36 on: February 26, 2012 - 22:05:48 »
I did similar to oxy and downgraded the 2 latest v5 and upgraded the 2 v3 upto v4. Now CM is happy.
"Get busy living or get busy dying" - Shawshank Redemption (Stephen King)

My DAT Files - [You are not allowed to view links] Register or Login
My Shared Files - [You are not allowed to view links] Register or Login
My GOG.com Files - [You are not allowed to view links] Register or Login

Offline Cassiel

  • Hero Member
  • *****
  • Posts: 653
Re: Scanning v4 CHDs with CMPro 4.03/0.145u1
« Reply #37 on: February 27, 2012 - 00:39:34 »
The saga continues....

Quote
etabeta
February 26, 2012 at 07:57

optimal setting + improvements to use FLAC for HDs too just hit the repository

as usual, it’s just a matter of waiting for cool things to happen, rather being too impatient ;)


Haze
February 26, 2012 at 11:12

when there’s absolutely no acknowledgment of what you’ve sent I wouldn’t call it impatient, just frustrating because it leaves me in a position of not knowing if a) the submission has been seen, b) if there was something ‘wrong’ with the submission, c) what I should do next.

[...]

anyway, yes, a reworked version of the submission is in, and also of note the way Aaron implemented FLAC on the HDDs means you *might* see some benefits on other CHDs too, not just the Beatmania ones (although I doubt many have raw audio in a suitable LE format, but we’ll see)


Gromasen
February 26, 2012 at 13:39

It seems a “chdman copy” still defaults to a 9.792 hunk size when converting v4 CHDs, while a “chdman createcd” uses the optimized 19.584.


Haze
February 26, 2012 at 14:14

I believe that’s expected behavior, the copy command gets it’s hunk size from the source CHD

// process hunk size
UINT32 hunk_size = input_chd.hunk_bytes();

You could make a case that it’s not ideal behavior, but it does look intentional.

For CDs extract + create is probably the way to go. Hopefully none (currently) have data which isn’t representable when extracted to cue/bin (I know some were toc/dat sourced, so it’s possible) CHDMAN should probably warn you if you’re trying to extract data which can’t be represented, but I have a strange feeling it doesn’t ;-)

(ok, confirmed, extracting to cue/bin can be lossy, without warning, don’t do it, I’m not sure the files produced are even valid.

Anything with

TRACK:xx TYPE:AUDIO SUBTYPE:RW_RAW FRAMES:2086 PREGAP:0 PGTY

MUST be extracted to toc/dat (because it would have almost certainly been sourced from such)
)

Offline niallquinn

  • Hero Member
  • *****
  • Posts: 2151
  • FTM.
Re: Scanning v4 CHDs with CMPro 4.03/0.145u1
« Reply #38 on: February 27, 2012 - 18:02:42 »
[You are not allowed to view links] Register or Login
I did similar to oxy and downgraded the 2 latest v5 and upgraded the 2 v3 upto v4. Now CM is happy.

Whoaa, how did you downgrade 2 of the v5's to v4?

This could be the answer to my monumental fuckup.

Thanks,

NQ.
A rom is for life, not just for Christmas.

Offline indigital

  • Full Member
  • ***
  • Posts: 134
Re: Scanning v4 CHDs with CMPro 4.03/0.145u1
« Reply #39 on: February 27, 2012 - 18:15:55 »
Is it generally speaking a bad idea to update the CHDs to v5 now?

Currently I'm doing a simple

for %i in (*.chd) do chdman.exe copy -i "%i" -o "d:\0\%i"

and wait on how long it will take to finish the converting.

I'll leave the original CHDs in the background of course.

Offline Cassiel

  • Hero Member
  • *****
  • Posts: 653
Re: Scanning v4 CHDs with CMPro 4.03/0.145u1
« Reply #40 on: February 27, 2012 - 18:32:51 »
YUP!   ;)

I'd have another read of the entire thread....

Offline indigital

  • Full Member
  • ***
  • Posts: 134
Re: Scanning v4 CHDs with CMPro 4.03/0.145u1
« Reply #41 on: February 27, 2012 - 18:48:04 »
Yes, Sir. I'll do right from the spot.  :P



Offline RCade

  • Hero Member
  • *****
  • Posts: 1793
Re: Scanning v4 CHDs with CMPro 4.03/0.145u1
« Reply #42 on: February 27, 2012 - 19:56:27 »
[You are not allowed to view links] Register or Login
[You are not allowed to view links] Register or Login
I did similar to oxy and downgraded the 2 latest v5 and upgraded the 2 v3 upto v4. Now CM is happy.

Whoaa, how did you downgrade 2 of the v5's to v4?

This could be the answer to my monumental fuckup.

Thanks,

NQ.


Don't quote me on this NQ but I'd hazard a guess at extracting the chd to raw and using pre u1 chdman to redo.

Lets see how wrong i am lol

RCade

Offline niallquinn

  • Hero Member
  • *****
  • Posts: 2151
  • FTM.
Re: Scanning v4 CHDs with CMPro 4.03/0.145u1
« Reply #43 on: February 27, 2012 - 20:09:20 »
[You are not allowed to view links] Register or Login
[You are not allowed to view links] Register or Login
[You are not allowed to view links] Register or Login
I did similar to oxy and downgraded the 2 latest v5 and upgraded the 2 v3 upto v4. Now CM is happy.

Whoaa, how did you downgrade 2 of the v5's to v4?

This could be the answer to my monumental fuckup.

Thanks,

NQ.


Don't quote me on this NQ but I'd hazard a guess at extracting the chd to raw and using pre u1 chdman to redo.

Lets see how wrong i am lol

RCade

Okay, can anyone create a dos bat, that once placed in the chds directory along with 0145chdman and 0145u3chdman, will go into each directory, extract the data using chdman0145, then create the chd with 0145u3, and delete the used files, leaving only the chd in each directory?

And for a brucie bonus, do the same for the ones that need to be v4, and maybe create those in chdv4?

I need a lie down.  I'm knackered just thinking about it.

:)

NQ.


A rom is for life, not just for Christmas.

Offline Connie

  • Hero Member
  • *****
  • Posts: 2156
Re: Scanning v4 CHDs with CMPro 4.03/0.145u1
« Reply #44 on: February 27, 2012 - 21:43:01 »
I put these together to manually fix and test 1 by 1.

Note:
Only have these tools and ONE CHD in a folder at a time.
The 'CHD Convert v5 to v4' is for CD based CHDs - for HD you'd have to change the options/switches. (For future downgrades??)
In the 'CHD Convert v4 to v5', change the number after the -np switch to how many processors (cores) you wish to use. I've a quad core but 4 would mean an unresponsive PC.

I'm sure someone with 'real' batch skills could work up something far better, but for my 4 conversions, they did the job. :)

Edit: Forgot to say that to upgrade v3 to v4 (who has less than v4 now anyway?) use:
Code:
Only registered users can see contents. Please click here to Register or Login.

[attachment deleted by admin]
« Last Edit: February 27, 2012 - 21:51:15 by Connie »
"Get busy living or get busy dying" - Shawshank Redemption (Stephen King)

My DAT Files - [You are not allowed to view links] Register or Login
My Shared Files - [You are not allowed to view links] Register or Login
My GOG.com Files - [You are not allowed to view links] Register or Login