[4.2.0.0] old tape playback issues

Found a bug in R'n'D? Report it here!

Moderators: Flumminator, Zomis

Post Reply
filbo
Posts: 647
Joined: Fri Jun 20, 2014 10:06 am

[4.2.0.0] old tape playback issues

Post by filbo »

Holger, as we were discussing in PM:

Some 60 tapes out of my collection went from successful playback under 4.1.4.2 to failure under 4.2.0.0. I told you that some were failing because of the addition of wraparound: in the case I saw, the player had been pushing against the steel wall at the edge of the level, so wrapped around in the new playback. I suggested you recognize tapes from older RnD EMC engine and disable wrap when playing them back. You reported that this fixed 22 of the problem tapes.

Since then I've watched playback of the other 38 problem tapes and characterized the problems as follows:

Code: Select all

Tapes that succeed with 4.1.4.1 but fail with 4.2.0.0 + selective disable of wrap
=================================================================================

emc_amiga_mine_18       22: ACM ??
emc_bond_mine_16        41: IRM
emc_diamond_mine        28: ACM ??
emc_elvis_mine_4        14: ACM ??
emc_elvis_mine_5        19, 37: RND
emc_elvis_mine_6        18: ACM ??
emc_elvis_mine_6        56, 71: RND
emc_elvis_mine_7        3, 35: ACM -> RND
emc_elvis_mine_7        30: ACM???
emc_elvis_mine_8        3, 11, 23, 33, 73: ACM??? -> RND
emc_elvis_mine_8        34, 48: ISM (48 not solvable because spring blocks passage?)
emc_elvis_mine_8        55: UNK
emc_emc_mine_3          60: ISM or ACM -> RND
emc_emc_mine_4          52: ISM + MBD
emc_emc_mine_4          75: ACM
emc_emc_mine_5          28, 32, 64: ACM?
emc_emc_mine_5          54: RND
emc_emc_mine_5          66: IBM
emc_emerald_dash_32     0: ACM -> RND
emc_emerald_dash_34     44: ISM
emc_emerald_eater_2     34: FAM / ACM?
emc_emerald_mine_03     42: ACM -> RND
emc_emerald_mine_08     74: YMC
emc_emerald_mine_20     77: YMC
emc_exception_1         25: BXP
emc_down_under_mine_17  68: ACM
emc_emc_mine_2          17: ACM ??

ACM: acid mobility differences (difference in timing or order things fall in?)
BXP: packed-bug explosion differences (pattern of emeralds vs diamonds)
FAM: fake acid mobility differences
IBM: initial bomb movement   \ these things move into a space when they're created, even if
IRM: initial rock movement   | player's there (player dies).  also other differences in when
ISM: initial spring movement / or whether they are allowed to move at game start or creation
MBD: magic ball destruction
RND: something caused random sequence to differ
YMC: difference in interpretation of yam contents out of level file?
For most levels these are more like guesses than definitive statements of what went wrong.

The ones about 'initial movement' and MBD look like probable bugs in the new engine. When something like a spring is suddenly created (e.g. by magic ball or yam), it starts out moving; this is normal. But that initial move is allowed to go through things it wouldn't normally be able to traverse. This kills the player in some of the tape problems. In one (emc_elvis_mine_8 48), created springs penetrate into a passage and block access, making the level unplayable at all -- strongly implying this behavior did not exist in this exact form back on the real EMC engine.

And on emc_emc_mine_4 52, springs being created by magic balls actually end up destroying those magic balls, surely not correct operation.

The one about yams looks like a bug in the old EMC engine. The new one 'sees' various contents in yams (visible in the level creator) which just aren't present in the old engine. For correct playback of old-RnD-EMC tapes, emulation of that bug is necessary.

'RND' is sort of a cop-out, all I can see is that random actors like yams are operating off a different random sequence. Sometimes the cause is apparent, other times I shrug... Hopefully, once the actual bugs are fixed + backwards-compatible playback hacks added, those will just mysteriously start working...
User avatar
Holger
Site Admin
Posts: 4073
Joined: Fri Jun 18, 2004 4:13 pm
Location: Germany
Contact:

Re: [4.2.0.0] old tape playback issues

Post by Holger »

Thanks a lot for this analysis -- this saves me a lot of time!

I already investigated some of these broken tapes, and already found some causes for some of these failing tapes.
And on emc_emc_mine_4 52, springs being created by magic balls actually end up destroying those magic balls, surely not correct operation.
Definitely not. I also found this issue, which was working correctly with the old EM engine, but is simply wrong with the new one, which confuses "initially rolling springs" and "initially pushed springs" -- the former being the correct variant in most cases that usually should be used. However, the latter case may also exists in old EMC V6 style levels, but the new EM engine maps both cases to "pushed springs", which is the one that destroys everything in the direction the spring is moving with the first move (even steel wall), which is definitely wrong if only "spring that starts moving left or right" was intended.

I've already fixed this one, and this fixes already about half a dozen of the listed broken tapes.

More fixes will probably follow -- please stand by! :-)
filbo
Posts: 647
Joined: Fri Jun 20, 2014 10:06 am

Re: [4.2.0.0] old tape playback issues

Post by filbo »

Meanwhile, I've done a lot of work on that regression suite and will send you a new version soon...
filbo
Posts: 647
Joined: Fri Jun 20, 2014 10:06 am

Re: [4.2.0.0] old tape playback issues

Post by filbo »

Teaser: the end of its current output:

Code: Select all

==== SUMMARY OF ERRORS & DEBUG OUTPUT ====

    240 DEBUG: DrawPlayerField(): This should never happen!
    240 DEBUG: DrawPlayerField(): x = %d, y = %d
    260 warning: unknown EM level '/path/to/level' -- using empty level
    260 warning: unknown magic byte 0x%x at position 0x%x
    832 DEBUG: AmoebeUmwandeln(): ax = %d, ay = %d
    832 DEBUG: AmoebeUmwandeln(): This should never happen!
   2288 DEBUG: AmoebeAbleger(): newax = %d, neway = %d
   2288 DEBUG: AmoebeAbleger(): This should never happen!

==== SUMMARY OF SOLVING REGRESSIONS ====

   182 tapes were most recently solved by version 4.0.0.2
   868 tapes were most recently solved by version 4.0.1.0
    38 tapes were most recently solved by version 4.1.4.2
   779 tapes were not solved by any version
 20080 tapes were solvable by the current version 4.2.0.1
User avatar
Holger
Site Admin
Posts: 4073
Joined: Fri Jun 18, 2004 4:13 pm
Location: Germany
Contact:

Re: [4.2.0.0] old tape playback issues

Post by Holger »

868 tapes were most recently solved by version 4.0.1.0
These broken tapes are most likely caused by that 64-bit bugfix in the EM engine in version 4.0.1.2 (although that would mean that they should have been OK in version 4.0.1.1)...
User avatar
Holger
Site Admin
Posts: 4073
Joined: Fri Jun 18, 2004 4:13 pm
Location: Germany
Contact:

Re: [4.2.0.0] old tape playback issues

Post by Holger »

BTW, to find and fix engine regression bugs, it is probably useful to only summarize tape results for the same game engine.

But as your tape archive contains results per level set, it's easy to get this information... :)
filbo
Posts: 647
Joined: Fri Jun 20, 2014 10:06 am

Re: [4.2.0.0] old tape playback issues

Post by filbo »

If the -e autotest results included a statement of the game engine used, I'd probably have indexed by that :)
filbo
Posts: 647
Joined: Fri Jun 20, 2014 10:06 am

Re: [4.2.0.0] old tape playback issues

Post by filbo »

BTW 'engine to use' is really a property of the individual level data file, right? Does anything prevent a levelset from containing levels which invoke different engines?
User avatar
Holger
Site Admin
Posts: 4073
Joined: Fri Jun 18, 2004 4:13 pm
Location: Germany
Contact:

Re: [4.2.0.0] old tape playback issues

Post by Holger »

If the -e autotest results included a statement of the game engine used, I'd probably have indexed by that :)
That could indeed be useful...
BTW 'engine to use' is really a property of the individual level data file, right?
That's right!
Does anything prevent a levelset from containing levels which invoke different engines?
Absolutely nothing! It would probably be a bit unusual, also for the player of such a level set (because the different engines "feel" differently, especially the EMC engine where you can only move every eight frames), so playing such a level set may feel a bit inconsistently, but technically there's nothing that would prevent one from creating such a level set.
filbo
Posts: 647
Joined: Fri Jun 20, 2014 10:06 am

Re: [4.2.0.0] old tape playback issues

Post by filbo »

Somewhere in the code I think I've seen something that reports not just what engine, but what set of version-specific quirks are applied. Or not?

It would be interesting to include those in the autotest run logs. But since it can all vary per-level it gets pretty ugly...

Suppose it were to scan all the levels to be used and report them in clumps (all levels use EMC engine with xyz quirks; or levels 1-70 and 85 use RnD engine with abc quirks, 71-84 use EMC with def quirks...) ?

Except ... with playback it's worse, it's not only what engine + age/version-of-file-format quirks of the level, it's additionally what age/version-of-RnD-recorded-under of the tape. Right? ugh.
User avatar
Holger
Site Admin
Posts: 4073
Joined: Fri Jun 18, 2004 4:13 pm
Location: Germany
Contact:

Re: [4.2.0.0] old tape playback issues

Post by Holger »

I absolutely agree that the whole concept of backwards compatibility and quirks emulation is indeed totally ugly from a technical and programming point of view! :cry:

However, breaking lots of existing tapes with every little engine change or bugfix is also not a very good solution, both from a user's perspective (who may want to replay old tapes for the fun of watching how a difficult level was successfully solved), but also for the programmer, for which a set of working solution tapes is a valuable resource for use as unit tests. This can be very useful to check if critical changes or bugfixes in the game engine code lead to unsolvable levels, especially in parts of the engine code where it was not expected or intended.
Except ... with playback it's worse, it's not only what engine + age/version-of-file-format quirks of the level, it's additionally what age/version-of-RnD-recorded-under of the tape. Right? ugh.
Fortunately, it's not *that* complicated; it mainly boils down to which game engine version should be used for recording or replaying a level or tape. When recording, the engine version to use is determined by the level file (and possibly by the file "levelinfo.conf", which can be used to force using the latest engine version for all levels, but this is usually not needed and probably not very useful in most caeses). When replaying, the engine version to use is determined by the tape file, where the engine version used at the time of recording is stored.

Then the engine version is used to apply the corresponding "quirks" at game engine initialization stage, which are usually translated to switches to see what should be done. (This is important to be able to distinguish different quirks from each others. For example, having three conditions checked for "version < 4.2.0.0" could be the same quirk or three different quirks, so I try to use something like "game.use_old_behaviour_for_this" and "game.use_old_behaviour_for_that" to clearly see what's going on.)

Yes, when looking at the code, this is all ugly nevertheless. :-(

Other ways to deal with this: Simply never change the game engine, regardless of existing bugs, quirks and "funny" or unexpected behaviour, but have lots of helpful tapes or "demos" for it (this is the case with the Supaplex game engine, which was never changed or fixed in its various incarnations, but which has tons of demo files to test the game engine against or simply replay solved levels for fun). Or do changes to the game engine every now and then, not adding compatibility code, but keep the engine code clean and tidy (which is the case with the original code of "Emerald Mine for X11" (which is used as the base for the EMC game engine in R'n'D), which consequently does not offer recording and replaying tapes at all).
Post Reply