After compressing music0001.vgm (838kb) with vgm_cmp the file is music0001_optimized.vgm (781kb).
then running it through vgm_facc makes it music0001_optimized_frame.vgm (760kb).
That's still 760kb for a 10 second tune that uses a few channels....
This way there is no point continueing... not sure why the filesize remains this huge??
Moonblaster FM also supports a few PCM tracks, doesn't it? Maybe the generated VGM is including some unneeded PCM data?
Ah, yes, you’re right. The VGM produced by the vgm_rec script in openMSX contains a dump of the entire sample RAM (640K in openMSX). It’s intended to be processed by the tools afterwards.
You can use the vgm_sro tool to automatically strip all sample RAM data that is not actually used.
vgm_sro won't work.
the readme file says:
VGM Sample-ROM Optimizer (vgm_sro)
The following chips need a Sample-ROM:
- SegaPCM
- YM2608 (DELTA-T)
- YM2610 (ADPCM and DELTA-T)
- Y8950 (DELTA-T)
- YMZ280B (Sample ROM)
- RF5C68 (if not streamed)
- RF5C164 (if not streamed)
- YMF271
Moonsound = Yamaha YMF278B OPL4 sound chip
When trying to use this tool, I get message: "no chips with Sample-Rom used!"
It must be an outdated version. Current vgm_sro should support the OPL4.
Hum, the compiled package must be old then, support for OPL4 RAM optimisation was added on Feb 2, 2018.
Are you using the download link that points to the latest VGMTools_2021-11-30.7z?
Thanks, I did have an older version. The lastest version worked, I managed to use vgm_sro.
Alright, my 10 second test tune went from 838KB to 120KB after using vgm_sro, vgm_cmp and vgm_facc.
hmmmm.... that's still way too big for a 10 second tune... That will be 1MB for a 1-2 minute track I guess ?
There are other games that take this approach. Ghost and Goblins does (I made the data converter & playback routine), and so does Bengalack’s new game
How did they manage the huge file sizes ?
Are the files compressed and frame by frame uncompressed ?
Are the file sizes smaller ? Do they play the tunes directly from rom or or they put them in ram ?
And what does your data converter do and how does your playback routine work ?
So the whole point is that we are working on a new game, and MBWAVE replayer (haven't got it to work yet) will probably be too slow. I need to save up cpu time, therefor I'm considering this approach, but I'm starting to think that it might not be worth it or that it might not work if each song is >500kb (uncompressed).
Alright, my 10 second test tune went from 838KB to 120KB after using vgm_sro, vgm_cmp and vgm_facc.
hmmmm.... that's still way too big for a 10 second tune... That will be 1MB for a 1-2 minute track I guess ?
Well I don’t expect the files to be super small, but 120 kB for 10 seconds seems a bit over the top. Maybe MoonBlaster is doing some excessive I/O that is not filtered out by vgm_cmp.
You can use the vgm2txt tool to convert the VGM to a big text file and see what it’s recorded, if there’s anything unusual.
Can you also send that 120 kB test VGM file to me? (Mail in my profile.) I’m curious about it as well.
There are other games that take this approach. Ghost and Goblins does (I made the data converter & playback routine), and so does Bengalack’s new game
How did they manage the huge file sizes ?
The file sizes for the GnG tracks are quite small actually. I don’t think you can get them that small with OPL4 music (with so many channels), but I think there may be more to the large file size you’re getting that’s worth looking in to.
Are the files compressed and frame by frame uncompressed ?
No, they’re uncompressed.
Using a standard LZ-type decompression would be quite slow, it would negate the performance benefit. Although in the past I have been looking into a special type of compression for this kind of data that is more performant (pulling out repeated sequences, similar to subroutines in code). But that’s a whole different topic.
Are the file sizes smaller ?
The largest track is 7.5 kB, but of course none of the Ghosts ’n Goblins tracks exceeds 40 seconds.
One reason why the sizes are this small in comparison is because the composed music is quite simplistic. It does very few instrument changes and most of the events in the data consist of just key-on and key-off, setting frequencies and waiting.
Other than that the custom format also gives a fair reduction. The original VGM file for that track was 18 kB, so this one in particular got shrunk by 59%, to 41% of its original size.
Do they play the tunes directly from rom or or they put them in ram ?
It plays directly from ROM.
And what does your data converter do and how does your playback routine work ?
If you want to take a peek, source code is here, there’s also some info about the format in the README. I’ve also posted about it here.
Note that this is for the Yamaha YM2151 (OPM), YM2810 (OPNB) and YM2608 (OPNA) chips so doesn’t directly apply to OPL4. But the essence is similar to what I described before for the sound effects earlier.
So the whole point is that we are working on a new game, and MBWAVE replayer (haven't got it to work yet) will probably be too slow. I need to save up cpu time, therefor I'm considering this approach, but I'm starting to think that it might not be worth it or that it might not work if each song is >500kb (uncompressed).
Well I can’t make any promises, but it seems that >500 kB is a bit excessive.
Maybe a good case to look at is the size of the OPL4 music for Sonyc after uncompressing it and passing it through vgm_facc:
Track Length Original vgm_facc 01_Title.vgm 1:55 287 kB 259 kB 02_Map.vgm 0:14 26 kB 24 kB 03_Palm.vgm 1:38 187 kB 172 kB 04_Battle.vgm 1:33 309 kB 278 kB 05_Death.vgm 0:04 8 kB 7 kB 06_Over.vgm 1:22 110 kB 100 kB 07_Bonus.vgm 0:49 96 kB 87 kB 08_Ice.vgm 2:15 454 kB 406 kB 09_Ruins.vgm 1:59 373 kB 336 kB 10_Forest.vgm 1:46 339 kB 307 kB 11_Complete.vgm 0:06 19 kB 17 kB 12_Credits.vgm 0:18 46 kB 42 kB Total 13:59 2.2 MB 2.0 MB
And then I would expect with a customized format you can probably reduce the size further down to about 50% to 30% of the VGM size. So at a ratio of 50% you would be looking at 1 MB total for the music data of the Sonyc soundtrack.
In the VGM format the command to do an OPL4 I/O write is 4 bytes: 0xD0 pp aa dd, where 0xD0 means it is a command for the OPL4, pp indicates whether to write to FM bank 1 (0), FM bank 2 (1) or a wave register (2), and aa and dd at last are the register number and value. This means that for a track that only uses the OPL4 wave part, you really need just 2 bytes per command: aa dd. So even with a simple optimisation like this you can get a 50% size reduction.
I made a quick test for this encoding based on modified GNGPlay source code and Sonyc VGMs:
Track Length Original vgm_facc Custom 01_Title.vgm 1:55 287 kB 259 kB 132 kB 02_Map.vgm 0:14 26 kB 24 kB 12 kB 03_Palm.vgm 1:38 187 kB 172 kB 87 kB 04_Battle.vgm 1:33 309 kB 278 kB 141 kB 05_Death.vgm 0:04 8 kB 7 kB 4 kB 06_Over.vgm 1:22 110 kB 100 kB 51 kB 07_Bonus.vgm 0:49 96 kB 87 kB 44 kB 08_Ice.vgm 2:15 454 kB 406 kB 206 kB 09_Ruins.vgm 1:59 373 kB 336 kB 170 kB 10_Forest.vgm 1:46 339 kB 307 kB 156 kB 11_Complete.vgm 0:06 19 kB 17 kB 9 kB 12_Credits.vgm 0:18 46 kB 42 kB 21 kB Total 13:59 2.2 MB 2.0 MB 1.0 MB
I also checked with vgm2txt what the main contributor was to the file size of the 01_Title.vgm track, and most of it is due to vibratos, tremolos and panning modulation. Due to this there are a lot of changes to frequency and other settings every frame. If it would not use these, or if it had used the LFO and VIB parameters (may be not supported by MBWAVE?), file sizes could be easily cut in half again.
That goes to show that the contents of the music really play a big factor in the file sizes.
I also checked with vgm2txt what the main contributor was to the file size of the 01_Title.vgm track, and most of it is due to vibratos, tremolos and panning modulation. Due to this there are a lot of changes to frequency and other settings every frame. If it would not use these, or if it had used the LFO and VIB parameters (may be not supported by MBWAVE?), file sizes could be easily cut in half again.
But this also means that since frequency changes are made a lot, if we can optimise them it would reduce the data size significantly.
As frequencies are currently always set as two register writes (LSB and MSB, 4 bytes: aa dd aa dd), if we would encode them as one combined 16-bit write (3 bytes: aa dd dd) they would get 25% smaller, and the total file sizes would probably shrink by some further 20%.
But I also notice a simpler optimisation than that. The VGM always sets both LSB and MSB if either one changes, even if it already had the same value previously. For vibratos it will often be the case that the MSB does not change, so it can save a lot of bytes if we omit these. (For some reason vgm_cmp did not do this optimisation, probably because they thought it was necessary to do both like OPN chips?)
This optimisation gives another good file size reduction:
Track Length Original vgm_facc Custom Freq-opt 01_Title.vgm 1:55 287 kB 259 kB 132 kB 99 kB 02_Map.vgm 0:14 26 kB 24 kB 12 kB 10 kB 03_Palm.vgm 1:38 187 kB 172 kB 87 kB 72 kB 04_Battle.vgm 1:33 309 kB 278 kB 141 kB 99 kB 05_Death.vgm 0:04 8 kB 7 kB 4 kB 3 kB 06_Over.vgm 1:22 110 kB 100 kB 51 kB 38 kB 07_Bonus.vgm 0:49 96 kB 87 kB 44 kB 35 kB 08_Ice.vgm 2:15 454 kB 406 kB 206 kB 132 kB 09_Ruins.vgm 1:59 373 kB 336 kB 170 kB 126 kB 10_Forest.vgm 1:46 339 kB 307 kB 156 kB 120 kB 11_Complete.vgm 0:06 19 kB 17 kB 9 kB 6 kB 12_Credits.vgm 0:18 46 kB 42 kB 21 kB 18 kB Total 13:59 2.2 MB 2.0 MB 1.0 MB 755 kB
