Author Topic: GoodSet merge batchscripts  (Read 11788 times)

0 Members and 1 Guest are viewing this topic.

Offline attractivo

  • Hero Member
  • *****
  • Posts: 572
GoodSet merge batchscripts
« on: May 14, 2016 - 17:57:11 »
as you Zandro mentioned xmdb, anyone has something newer than the official v3.14 files?


and Zandro instead of sharing the full sets, a move.bat file for your further merged sets (for only the filenames without extensions) would be nice.
i like how you further merge things (like the spc set)

the move.bat would make a lot of sense for n64 because i want to keep the files in v64 byteswapped (like no-intro)
also a batchfile has no hashes stored, so noone can forbid these ;)

peace

Offline Zandro

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3483
  • Bézier Extraordinaire
Re: GoodSet merge batchscripts
« Reply #1 on: May 14, 2016 - 18:34:06 »
@cld75 Thanks for listing these. I've added a new folder with new copies.

@@tractivo: Attached, hope it works for you.
« Last Edit: May 15, 2016 - 02:45:36 by Zandro »

Offline cld75

  • Sr. Member
  • ****
  • Posts: 424
Re: GoodSet merge batchscripts
« Reply #2 on: May 14, 2016 - 18:47:57 »
@Zandro
thanks for the reupload...the aforementioned affected files downloaded fine from the reupload directory


FYI...I think you're missing 10 ROMs....

There are 436 files in the Mega Folder (Right Click -> Info after selecting all the files not including the reupload directory)
2 of the files are Have/Miss txt files
1 of them is a duplicate upload of 1080 Snowboarding
that's 433 total 7z files

I have 433 7z files that I downloaded...Ran 7z Archive test
Results:
Archives: 433
Folders: 0
Files: 5255
Size: 107657728328
Compressed Size: 6511466627


It states there were 5255 files scanned....but from the Have.txt file: You have 5265 of 5282 known Nintendo 64 ROMS (V3.26)

so, is it just me or are there 10 ROMs missing?
« Last Edit: May 14, 2016 - 18:51:50 by cld75 »

Offline attractivo

  • Hero Member
  • *****
  • Posts: 572
Re: GoodSet merge batchscripts
« Reply #3 on: May 14, 2016 - 18:54:29 »
i just tried the batch on Super Mario 64 (as it determines on that if the roms are existing) plus some various other roms to see if it works. also i didnt run the batch on the whole set atm, the ones i tested on got moved to the correct folders. thanks a lot.

do have further merged some of the other sets too? especially gba comes to mind. i am a fan of merging and compressing stuff :)
also this could be a new thing. sharing batchfiles like that.

Offline hikaricore

  • Newbie
  • *
  • Posts: 13
Re: GoodSet merge batchscripts
« Reply #4 on: May 14, 2016 - 22:05:32 »
[You are not allowed to view links] Register or Login
N64 can have better merging. I'll upload my 6GB, miss 17.
[You are not allowed to view links] Register or Login

Thanks for this, nearly completed my set with one exception.  :)
Chou Snobow Kids (J) [!] was being detected by GoodN64 as Chou Snobow Kids (J) [b1].
It's possible I mixed up the files on my end, but you may wish to check the CRC of this rom to confirm.  ^_^

Offline Zandro

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3483
  • Bézier Extraordinaire
Re: GoodSet merge batchscripts
« Reply #5 on: May 15, 2016 - 00:07:49 »
Quote
FYI...I think you're missing 10 ROMs....

I think you're right... doing a rescan now.  I'll have to update the batchfile after getting my absent roms back.

*Edit:  I had lost the ED OS64 v2.xx's, and see that a couple CBFD V_ECTS_Encrypted roms equal others (at least in name) after unswapping. Its archive size grows substantially because I have to preserve their original formats to be accepted by GoodN64.

All fixed, and I moved the archives into the main share.  Browsing might be slow, so sort by date created.  The sort batchfile above is also updated, now properly escaping for 0% in filenames.

Speaking of escaping.

*Edit 2: Dangling Mario Kart hack merged.
« Last Edit: May 15, 2016 - 02:45:17 by Zandro »

Offline cld75

  • Sr. Member
  • ****
  • Posts: 424
Re: GoodSet merge batchscripts
« Reply #6 on: May 15, 2016 - 14:35:46 »
thank you for the fixes Zandro....


It seems you're still missing 3 ROMs after the updates however...

The following archives were updated (date modified):
Mario Kart 64
Conker's Bad Fur Day
Everdrive OS64
Deadly Arts
Virtual Chess 64
Road Rash 64
Parlor! Pro 64 - Pachinko Jikki Simulation Game
Doctor V64 BIOS

After a re-download of the above and a 7z re-scan/archive test, here are the Results:
Archives: 433
Folders: 0
Files: 5262
Size: 107676903208
Compressed Size: 6586463650

It's still not 5265....It's very strange though...your batch file doesn't "eat" ROMs...so if they are not moved to their respective directories,
you should still end up with the leftovers in the main N64Ren directory.  Not sure where those ROMs disappear to...

Offline attractivo

  • Hero Member
  • *****
  • Posts: 572
Re: GoodSet merge batchscripts
« Reply #7 on: May 15, 2016 - 17:28:24 »
i used the batchfile on the complete set and all 5265 roms got sorted to the 433 folders. thanks Zandro  8)
cant say anything about the archive, as i haven't downloaded it

@Zandro do have other sets "better" merged aswell? if so, could you consider sharing batchfiles for these too please. awesome work on the n64 and spc (Tentatively Sorted) sets ;)

also you dont need to archive them in original format. you can make 80% of the set byteswapped v64 (besides the bios and ndd files + some various files), which also match no-intro dat this way. i used tool64 for that. those byteswapped get recognized by goodn64.
« Last Edit: May 15, 2016 - 17:46:32 by @tractivo »

Offline Zandro

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3483
  • Bézier Extraordinaire
Re: GoodSet merge batchscripts
« Reply #8 on: May 15, 2016 - 18:14:50 »
Fan of strong compression, but want byteswapping.  You can't have it both ways. ;)

I will look at my GBA and SNES sets, but once you get past Pokemon and SMW there are diminishing returns.  Also, I don't collect GBA video.

Offline attractivo

  • Hero Member
  • *****
  • Posts: 572
Re: GoodSet merge batchscripts
« Reply #9 on: May 15, 2016 - 18:36:55 »
the compression comes when i merge sources with sabretools, that way goodn64 byteswapped [!] flagged roms get mostly matched with the no-intro entries. which means only one rom per entry. also the source hierarchy while merging determines the goodsets as highest. which means the set-structure (in that case you custom merged set) is not broken. this results in lesser sets in the final collection.

i can make use of a batch for gba without the video roms, as they are well defined with the [v] flag. also goodmerge has already taken care of most of the video roms. so i am interested in a batchfile for the mentioned sets :)

for snes i also build your smw dats under one set only for under the setname. this i do with sabretools again. i merge all your dats to one. then i trims all entries and merge them into one single game. after that i call the set "SMW Hacks (Zandro)". and now i merge it with all other snes dats (good, nointro, tosec etc.). but first i make sure that Zandro as a source comes highest in source-ID hierarchy. when a duplicate to your SMW dats is found in another datfile, it is always taking the name from your dats and puts it under the Zandro set.

so thats what i call a fan of merging and compressing  8)
« Last Edit: May 15, 2016 - 20:32:40 by @tractivo »

Offline Zandro

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3483
  • Bézier Extraordinaire
Re: GoodSet merge batchscripts
« Reply #10 on: May 15, 2016 - 19:01:02 »
I have not sorted GBA and SNES yet, and it will be more than a Sunday project.

Offline attractivo

  • Hero Member
  • *****
  • Posts: 572
Re: GoodSet merge batchscripts
« Reply #11 on: May 15, 2016 - 19:26:23 »
in that case i prefer the gba set, as it is the biggest out of the goodsets

Offline Zandro

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3483
  • Bézier Extraordinaire
Re: GoodSet merge batchscripts
« Reply #12 on: May 19, 2016 - 20:29:02 »
GoodGBA sorter attached. SNES will take much longer to produce, but I should find a bunch of improvements to GoodSNES while doing so, as I had with GoodGBA.

One of the more interesting fixes I had to make was to decry the existence of Our Breeding Series - My Amoeba.  It's supposed to be My Stag Beetle. ;D

Offline attractivo

  • Hero Member
  • *****
  • Posts: 572
Re: GoodSet merge batchscripts
« Reply #13 on: May 19, 2016 - 23:49:20 »
without counting the 1174 video-roms:

goodmerge:
35951 roms merged to 2043 sets

Zandro:
35951 roms merged to 1649 sets

thats a pretty big improvement. also all roms look in place where they belong.
incredible work Zandro, thank you very very much.

though i cant imagine how you spotted that title error for our breeding series :D

by the way, this custom merge variant could easily deserve an own thread. those batchfiles might be interesting to more people than have eyes on this thread here ;) anyway thank you~

Offline Zandro

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3483
  • Bézier Extraordinaire
Re: GoodSet merge batchscripts
« Reply #14 on: May 20, 2016 - 01:41:51 »
Topic split, stemming from your idea.