'last_level_series' performance anomaly

[filbo]

I noticed something rather peculiar - which may only affect old versions, but since I don't understand it I can't be sure.

I was rerunning some tests and noticed that they were running much slower than expected (about 18x slower). This is a sample command which ran 18x slower (using the binaries I sent you earlier):

Code: Select all

$ HOME=$PWD time bin/rocksndiamonds-4.0.0.2-20170411 --mytapes -e "autotest emc_adventures_of_mr_b_2 32"

After a lot of poking around, I learned that it ran fast if ~/.rocksndiamonds/levelsetup.conf contained:

Code: Select all

last_level_series: emc_exception_1

and slow if it contained:

Code: Select all

last_level_series: rnd_tutorial_niko_boehm

Now, I suspect that the following are true, but it would be a bunch of testing to convince myself, and I think you'll know right away what's wrong, so I haven't tested any of these tentative assertions:

1. It's probably governed by whether 'last_level_series' was an EMC or RnD levelset
2. It probably only affects playback of EMC levels
3. It was probably fixed at some point (this I did crudely test by doing the same 2 single-level tests on 4.2.0.1 and observing no significant difference in performance)

But it may be more complex than that. You have, I believe, 4 engines -- what if there is a performance penalty for any levelset being run when 'last_level_series' was from a different engine?

Since part of what I'm doing is timing the different versions against each other, I want it to be fair; so if there was an old bug that contaminated performance depending on which engine was mentioned in levelsetup.conf, I need to know how to handle that. What fake contents to put in levelsetup.conf for each tested levelset. Maybe it would be safest to put its own name? But even testing that would be a big deal, I'd rather hear from you :)

(and I don't think there's any particular importance or significance to testing the performance of the different versions; it's just bulldoggish curiosity... :)

[Holger]

Wow, that really sounds very strange!

I will definitely investigate this very unexpected behaviour!