Assembler Optimizer

Page 26/57
19 | 20 | 21 | 22 | 23 | 24 | 25 | | 27 | 28 | 29 | 30 | 31

By pgimeno

Champion (318)

pgimeno's picture

10-11-2020, 23:21

Grauw wrote:

Also I didn’t expect Pletter’s dictionary window to be 64 kB, I would’ve thought it was smaller to reduce the size of dictionary references (deflate’s is 32 kB).

Actually, I was wrong. After a closer look, it seems that Pletter's maximum window size is 8KB. So, theoretically LZEXE can "see farther".

By santiontanon

Paragon (1639)

santiontanon's picture

30-01-2021, 19:32

After a long pause, I finally got around to continue working on MDL! I just uploaded a new release (v1.3) finishing up the last details that I wanted to finish before moving on to the next optimizer module. Latest version always in GitHub:

This is what MDL can do as of v1.3

  1. It can read assembler files from lots of different other assemblers (asmsx, jasm, glass, pasmo, sdcc, and tniasm for now)
  2. It can then write them back to disk translating them to standard Zilog syntax (so, you can use MDL to translate a file in one assembler dialect to a standard notation that should be loadable by most other assemblers)
  3. It can generate statistics and annotations to your files (bytes used per file and per line, as well as timing information, etc.)
  4. It can run a pattern-based optimizer and:
    • Display potential optimizations (so you can do them manually) -> this functionality can be intergrated into IDEs, and currently there's instructions to integrate it into sublime, VSCode and vim
    • Automatically apply those optimizations and write optimized files back to disk
  5. (new in v1.3) It can also act as a regular assembler, and generate binaries directly (before and after optimization, as desired)

The new stuff in v1.3 compared to the last version I advertised here (I think v1.1?) is that there is much better support for asMSX (thanks to fubukimaru for his testing/feedback/help with this), sdcc (thanks to salutte for helping with the creation of a makefile to integrate MDL into the SDCC compilation pipeline, which we got to work on Anchors Aweigh, if anyone is interested, I could write a readme on how to do it). And also being able to generate binaries (given all the machinery that was in place this was at most like 100-200 lines of code or so, hehe, so, I do't know why I had not done this before :))

My next immediate step is to add a new optimizer (quite simple, but one that I've always wanted to have automated). I don't have a good name for it yet, but I'm currently calling it a "code reorganizer". Basically, the idea is to identify pairs of blocks of code C1 and C2 that are separate in the source code, but where the last instruction of C1 is a jump to C2, and no other piece of code jumps to C2. If this is detected (and a few other constraints, such as being in the same page in the rom, and some other dialect-specific constraints), then we can move C1 right before C2, and save the jump (saving bytes and cycles). I've always wanted to have something like this for my code, since I always notice these things too late, and I'm too lazy to fix them haha. Hopefully this is the beginning of a few more optimizer, which together can start pushing beyond what MDL can do now! :)

btw, any updates on your sequence optimization Grauw?

By jltursan

Prophet (2619)

jltursan's picture

30-01-2021, 21:19

Seems that the list of features has no end! Big smile Superb!

By Bengalack

Hero (590)

Bengalack's picture

31-01-2021, 11:51

santiontanon wrote:

After a long pause, I finally got around to continue working on MDL!

Awesome @santiontanon! I've been wanting to try this one out on SDCC-code Smile I tried it on my two main files. One generated from sdcc, and one asm-file. The (only) output I get is as follows:

C:\source\repos\noname\project>java -jar mdl.jar allsorts.s -dialect sdcc -po -asm allsorts-mdl.s
WARNING: Redefining symbol .macro
WARNING: First defined in allsorts.s, 101 as $: .macro:    ld____d_iyh
WARNING: Redefined in allsorts.s#105: .macro ld____e_iyl
WARNING: Redefining symbol .endm
WARNING: First defined in allsorts.s, 103 as $: .endm:
WARNING: Redefined in allsorts.s#107: .endm
C:\source\repos\noname\project>java -jar mdl.jar Debug\\objs\\game.asm -dialect sdcc -po
ERROR: parseRestofTheLine: Cannot parse line Debug\\objs\\game.asm#780:         CONOUT  .equ 0x02  left over tokens: [0x02]

Let me know if there is something I do wrong.

By Grauw

Ascended (10602)

Grauw's picture

31-01-2021, 13:14

Cool stuff!

santiontanon wrote:

btw, any updates on your sequence optimization Grauw?

Not much news, it works and compression is decent, but I still need to wrap it up nicely, e.g. it doesn’t run in O(n) yet atm. Also I want to have replayer code for different sound chips. However a lot of work went into getting the compression implementation to work so I got a bit of fatigue, and the project’s on pause at the moment.

By santiontanon

Paragon (1639)

santiontanon's picture

31-01-2021, 16:31

yeah, I hear that! A couple of months ago I was also tired of doing fairly continuous development on Triton/MDL for almost a year and needed a break. It definitively helps to stop for a couple of months, to then retake it again with renewed energy Smile

@Bengalek I think I know what is the problem! I remember reading a previous thread where you mentioned defining macros for "ld d, iyh"-type of instructions (which I see referred above in the output). I had not seen SDCC code with macro before, and I just realized it's not yet supported. Can you show me an example macro definition to I see the syntax? I can add support for that right away and update the release Smile (and I also see the problem in the second output sample). This is great though, I think every dev uses SDCC in a different way, and I was using only one project as the testbed for my code, which didn't use any of those two directives. So, this is a chance to expand coverage Smile

Also, the best way to use MDL with sdcc is to use the "-poapply" flag instead of the "-asm" flag. This is to do only the minimum necessary changes to the input .asm file for optimization. So, for example, your first call would better be like this:

java -jar mdl.jar allsorts.s -dialect sdcc -poapply -po

This will generate a "allsorts.s.mdl.asm" file in the same folder where "allsorts.s" is with just the optimizations applied.

By santiontanon

Paragon (1639)

santiontanon's picture

31-01-2021, 17:27

btw, I just updated the release with some initial support for macros in the asz80 assembler (used by SDCC), hopefully that resolves the problems Smile

By Bengalack

Hero (590)

Bengalack's picture

31-01-2021, 19:16

Great that was quick. Now I ran into some other issues.

First it is binary notation:

C:\source\repos\noname\project>java -jar mdl.jar allsorts.s -dialect sdcc -poapply -po
ERROR: expression failed to parse with token list: [0b10011000, ;LMMM command. TIMP OP 62400 cycles pr line as well]
ERROR: parseCPUOp: Cannot parse line allsorts.s#2110:   ld              a, #0b10011000 ;LMMM command. TIMP OP 62400 cycles pr line as well

The line is like this:
ld a, #0b10011000 ;LMMM command. TIMP OP 62400 cycles pr line as well

Working on the SDCC-generated asm-file takes you into the real wierdness of asz80. "inclusion of A" in a bunch of op-codes which normally does not have this:

C:\source\repos\noname\project>java -jar mdl.jar Debug\\objs\\game.asm -dialect sdcc -poapply -po
ERROR: No op spec matches with operator in line Debug\\objs\\game.asm#1867:     or      a, (hl)

For the last part, one needs to be able to parse some wierd opcode-notation as discussed with Néstor Sancho here:

I don't have the full list at hand, but it is like:

xor n becomes xor a, n
sub n becomes sub a, n
and n becomes and a, n
or n becomes or a, n
cp n becomes cp a, n

The z80-asm-meter-plugin in VS-code works with SDCC-generated files after Néstor fixed those wierd opcodes.

By santiontanon

Paragon (1639)

santiontanon's picture

31-01-2021, 19:41

Oh, true! I don't yet have support for binary constants in that format! Will add right away (there's a snowstorm today in Philadelphia anyway, so, I'm using the day to just work on MDL and a couple of other projects, so I can iterate fast hehe).

And thanks for the additional asz80 instructions!! I had added a few that I had encountered during testing, but not all the ones you mentioned above. So, will add those as well!

I'll update the release once those two things are added Smile

By theNestruo

Champion (351)

theNestruo's picture

31-01-2021, 19:57

Bengalack wrote:

(...) weird opcodes (...)

Oh yes! I remember that "arguably sick" syntax!! At first I though you were mocking me :-D

Page 26/57
19 | 20 | 21 | 22 | 23 | 24 | 25 | | 27 | 28 | 29 | 30 | 31