How to deal with roms >32KB's < 64KB's outside the 0x4000..0xBFFF typical area (without using megarom mappers)

Master (188)

19-02-2020, 13:03

Hi all...

In the last weeks I've been messing around with some MSX experiments on my own. Somewhat unexpectedly I've come across a path I've always wanted to avoid: I reached (and passed) the 32 KB's limit of a "normal" MSX (non mapped) rom. Suddenly my static code/payload inside the .rom file (data that most probably will be eventually transferred onto RAM to use it) is bigger than 32 KB and I'm forced to create an additional page inside the ROM space, that is: my .rom file will be now 48 KB's.
As of now I don't want to feel I need to use/create a megarom file, I want my experiments to fit inside a 64 KB's real EEPROM IC, so I can actually burn my .rom file into a specially designed physical cartridge I have around...

So, after some careful research mainly inside MSX RC (there are some threads speaking about it) I've come across the following conclusion:

1.- I have no idea about the degree of standardness of dealing with "raw/flat" rom files/eeproms bigger than 32 KB's, and up to 64 KB's, without using a rom mapper.

I've seen people participating in MSX DEV's in the past delivering their creations inside a 48 KB's rom file. Are these .rom files intended then to be burnt into a real EEPROM/cartridge afterwards? Or were they just .rom files expected to be loaded inside a real MSX via some random "loadrom" utility?

More importantly: How does this >32KB phyisical rom content get laid out onto the MSX/Z80 slot/pages space? is there an standard to do it? Does OpenMSX do a proper job with it, in case there's a standard (apparently it assumes many proper things on its own, that's fine)?

My main fear is basicly "how to deal with pages 0 and 3" of the MSX/Z80 typical memory visibility space, specially in the SLOT/ROM area.
Is it a valid assumption to expect that parts of my ROM file will appear on page 0? How can I easily use the data living inside this page 0 of slot 1 data, specially if I want the typical BIOS support at the same time?
Same thing with page 3: Should we expect a 48 KB rom last 16 KB's to appear mapped onto the 0xC000..0xFFFF space? What should we do to access this data "easily" without losing the typical RAM page 3 in the process?
Obviously as I'm in charge of the .asm file and/or of the EEPROM programmer I have all the power to "setup" the ORG to whatever address I want (in the .asm file) and/or to fill the 64KB's free space of my EEPROM.For my 48 KB's I'll probably write onto pages 0, 1,2 and fill page 3 with 0xFF, or I also can fill page 0 with 0xFF's and then pages 1, 2 and 3 with my actutal rom data...

So my main doubt arising from this is: How should I use/manage the additional page(s) appearing outside the typical 0x4000..0xBFFF slot memory zone, for roms bigger than 32KB's?
Again: I'm trying not to mess (as far as I can) with megarom way of dealing with big rom files.

BTW, I'm using Grauw's assembler and apparently I'm somewhat managing to create >32 KB's .rom files... Padding is a bit wonky (can't see how to align to 48 KB's, not a priority right now), but otherwise I've just created a 40 KB's rom file without problem.
I will test this rom with OpenMSX afterwards to see how it does assume things when loading it onto its own ROM slot area.

Greetings.

Por geijoenr

Champion (358)

19-02-2020, 14:05

You need to set the Page 2 to ROM in same slot as Page 1. Then you need to switch back and forth Page 0 between BIOS and ROM.

Sample code https://github.com/retrodeluxe/rlengine-msx1/blob/master/boo...

Por gdx

Enlighted (5984)

19-02-2020, 14:57

Developing:
http://www.msx.org/wiki/Develop_a_program_in_cartridge_ROM

You have to juggle the pages. To avoid too much juggling you can use pages 0 and 3 to put the graphic data. You cannot call the routines of Bios from page 0.

Making:
http://www.ebsoft.fr/shop/en/home/28-msx-cartridge-kit-16-64...

Por Grauw

Ascended (10676)

19-02-2020, 15:03

If you go to 48K, use the extra 16K in page 0, not in page 3. If you go 64K, well, really just don’t if you can avoid it at all. Using page 3 for anything but system memory is pretty much incompatible with the OS. Additionally, try to use the memory outside the 32K as much as possible just for data storage, and for routines which are infrequently called, slow and don’t need to rely on the BIOS.

E.g. put all your graphic and level data compressed in page 0, as well as a decompression routine, and you can just interslot call the decompression routine between levels which unpacks the desired level data to RAM and graphic data to VRAM.

Por santiontanon

Paragon (1726)

19-02-2020, 17:55

Agree with the previous answers. 64KB is a mess. In order to access the data in page 3 you'll have to kick out the RAM from there (with all the issues that implies, since the BIOS expects to write system variables there, the stack, etc.). Page 0 is more manageable, but still a bit of a hassle. As they told you above, in order to access page 0 of your ROM, you have to kick the BIOS out. So, while accessing page 0, you do not have access to the BIOS. In practice, this is what I do:
- Store only data that I need sporadically in page 0 (e.g., level maps, or graphics that are read out from there into RAM once, and that's it).
1 - During normal game execution just have BIOS-ROM-ROM-RAM layout
2 - When you want to access page 0, temporarily set ROM-ROM-ROM-RAM
3 - copy whatever you want from page 0 on to RAM
4 - switch back as quickly as possible to BIOS-ROM-ROM-RAM

Steps 1, 2, 3 are "easy", as you can ask the BIOS to do step 2. But step 4 is the problem, as you will have to do the switch manually, without access to the BIOS routines. There are lots of examples of games doing this as people have pointed to you above. Another one, just in case, can be seen here (look at the "pletter_unpack_from_page0" function, which does exactly the 4 steps I told you above): https://github.com/santiontanon/xracing/blob/40ff8bd43137845...

Por Grauw

Ascended (10676)

19-02-2020, 18:08

Afaik BIOS interslot calls to page 0 work, so you can just put a little copy or decompression routine in front of your data and avoid those steps 2 & 4 where you are switching out slot 0 with a manual routine.

Por sd_snatcher

Prophet (3608)

19-02-2020, 18:41

Tip: RDSLT can be used to easily read data from the frame-0 without any other worries.

As Grauw suggested, for higher volumes of data, CALSLT might be a better option as long as you keep the interrupts disabled. Do not try to implement your own interrupt handler there as it would break compatibility with a lot of devices: some disk interfaces, MSX-Audio BIOS, RS-232, MIDI3, MA-20 etc.

Por santiontanon

Paragon (1726)

19-02-2020, 19:33

oh! wow, that's very interesting, I had no idea, then even easier! Will note that down for future reference!

Por friguron

Master (188)

20-02-2020, 00:42

Oh, what an interesting selection of useful info altogether!

As I expected, a "64 KB's raw ROM" seemed more a Proof of Concept than an useful project. I was almost discarding it from the beginning, and you all confirmed it, great.

Now as for the 48 KB sized ROMs: I already noticed that "page 3" should be kept away and as untouchable as it can get, mainly because of RAM, so I had already started to address my efforts to use page 0 instead. This second approach seems not that messy, and is certainly doable as some of you just confirmed.

Besides Grauw shed one of the most useful pieces of knowledge: RDSLT works even when dealing with page 0 data. That's just perfect for my current needs. The main goal of my routines as of now is somewhat "expand" some static data onto different RAM zones, to work from there afterwards.

OTOH, I don't have any special desire or needs to store core coded routines in page 0, quite on the contrary. My project as of now is 99% storage, 1% data expansion. Keeping parts of my static data on rom page 0 and accessing it via RDSLT to blit it somewhere else is what I was really looking for.

Part of my concern also was about the phyiscal rendition of all this, I mean, I want to confirm that data stored on the pysical area 0x0000..0x3FFF inside my EEPROM IC will have a 1:1 reflection on the 0x0000..0x3FFF ROM area when running it inside a real MSX. Apparently, and as you're telling me, it will work that way, so I'm ok with it.

In case dealing with 48 KB's ROM files hadn't exactly been a standard practice in real life, I had also started to create some silly compression schemes to store my data inside my original 0x4000..0xBFFF rom file. So, whatever path I chose (the 48KB's rom file, the compressed data inside 32 KB's, or both of them), it seems I will finally manage to get the goal I was going after in the first place

As I said, thanks for the info. There are many useful micro gems of knowledge in all the answers you all have posted, and my doubts have been solved !

Por gdx

Enlighted (5984)

20-02-2020, 02:19

The only one problem with 64 KB's raw ROM is with MSX 8/16kB because the RAM is switched when using the Rom in page 3 so you must preserve the RAM slot number in VRAM or CPU register to replace the RAM. Other than that, it's no more complicated than using page 0 and doesn't break the compatibility more with devices if you work with interrupts disabled. Page 3 can be very useful for putting graphic data (Pictures and many texts)

Por DamageX

Master (217)

20-02-2020, 05:59

Once upon a time I made a 48KB ROM. It used 0-$BFFF and some RAM at $E000-$F3FF. I didn't use the BIOS at all. However, my program was also launchable as a .COM file from MSX DOS. When started in this manner, it would jump to a small routine at the end which shifted the whole thing back from the $0100 base address to $0000.