score / tape / level server futures...

Got a cool idea that should be in R'n'D? Let's hear it!

Moderators: Flumminator, Zomis

Post Reply
filbo
Posts: 647
Joined: Fri Jun 20, 2014 10:06 am

score / tape / level server futures...

Post by filbo »

I understand the server validates, or attempts to validate, score / tape files. Some questions, suggestions, and ideas about that:

If a level is solved, a tape is uploaded. The server then tests whether it solves a level; but this is asynchronous.

If it turns out to be a levelset the server knows of; a level number it has a copy of; and the tape actually solves that level -- all good. But there are a number of things that can go wrong! Here are some ideas about handling some of them.

Scenario: server knows of this levelset:level, but the uploaded tape doesn't solve that level: alert the player by marking it somehow in the 'hall of fame' part of the game UI; and possibly by some sort of notification (because it's async -- by the time the local game learns of the discrepancy, he's playing the next level; or maybe he quit and a month has gone by.)

Why would the local game & server disagree about whether a tape solves a level?

- bugs in compiler or CPU on client or server
- different RnD versions
- non-identical level definitions
- some sort of cheating by the player
- ?????

Server could request a copy of the client's level definition. If it's identical (or if it's different but still doesn't solve with the provided tape), it knows something is weird, and should notify the player.

If it's different and does solve, maybe the server saves it as a 'variant' of that levelset:level (meaning the server's namespace for levels is now actually levelset:level:variant, though most won't have :variant).

This potentially also 'solves', in a sort of baroque way, cases where two unrelated or more-or-less disjoint levelsets have the same name: I noticed that I have two 'BD4' levelsets: levels/DX-Boulderdash/dx_bd4 'BD4' by 'ECI' with 100 levels, and levels/Emerald_Mine_Club/emc_bd4 'BD4' by 'Paul Williams' with 81 single-player levels and 21 2-player. (But RnD has no trouble distinguishing these locally -- obviously the levelset paths are different, but also the tapes are stored in ~/.rocksndiamonds/tapes/{dx_bd4,emc_bd4}; whereas the server seems to index by the 'name:' field from levelinfo.conf, not the directory name. I couldn't find the DX BD4 levelset in the server UI, which makes sense if they're indexed by name with no 'variant' system. It would be better if a 'variant' system only needed to be used for actual variants, not entirely different levelsets. Though, of course, my example levelsets aren't actually 'entirely different', just ports of the same levels to different historic gaming systems...) (I haven't tested whether tapes from one of the BD4 levelsets solve the other -- I suspect not. In fact, my saved dx_bd4 tapes don't even reliably solve their own levels, possibly because they were recorded under RnD 3.2.6.1 ...)

The same procedures could collect copies of levelsets completely unknown to the server.

Unlike tapes, it seems like levelsets should not be collected entirely automatically. There are copyright and ethical issues. I would want the game to dialog with the player at least once per new levelset, asking things like:

- did you create this levelset?
- [ if not: ] where / how did you get it?
- do you have permission to upload it to a public repository?
- do we have permission to upload it?

Things could get quite hairy if the levelset.conf refers to custom graphics, sound, music. Not too bad if they're specifically for this levelset, but if it refers out to another levelset's artwork, two new problems arise. It might be another levelset unknown to the server, so now the collection has become recursive. Or it might be known, but the player's copy actually has different artwork contents than the server (so now we have artwork variants...)

Dealing with that stuff seems like it might be too gnarly for automated handling. However, as far as I know, whether a tape solves a level definition is never affected by the artwork files -- so I suppose the server could be a repository of bare level definitions, decoupled from their artwork.

Of course if you can upload levelsets then players should be able to download levelsets. (I'm pretty sure you already had that in mind...) So, while browsing the score server, they should be able to point at a levelset and say 'put this on my machine'.

=====

In sum:

- feed back to the player if their tape doesn't solve on the server
- this involves async notifications which might come after the next level was played, or next time the game is run
- collect, retain, and handle as (effectively) different levels, variants of known levelset:levels
- collect unknown levelsets; possibly with artwork, but likely not
- try to be careful about copyright stuff in collecting levelsets
- also provide levelset downloading

=====

wheeee! :)
User avatar
Holger
Site Admin
Posts: 4073
Joined: Fri Jun 18, 2004 4:13 pm
Location: Germany
Contact:

Re: score / tape / level server futures...

Post by Holger »

Scenario: server knows of this levelset:level, but the uploaded tape doesn't solve that level: alert the player by marking it somehow in the 'hall of fame' part of the game UI; and possibly by some sort of notification (because it's async -- by the time the local game learns of the discrepancy, he's playing the next level; or maybe he quit and a month has gone by.)
Active notifications are a bit tricky, as the server cannot contact the client directly, but it could be possible to create some kind of "pending notifications" on the server for a given player UUID, so requests could check for them and return whatever it was about. This would of course be a very advanced feature that may not be worth the effort, maybe...
Why would the local game & server disagree about whether a tape solves a level?

- bugs in compiler or CPU on client or server
- different RnD versions
- non-identical level definitions
- some sort of cheating by the player
- ?????
Different R'n'D versions should theoretically not be the case, as long as the server always runs the latest version of the game with full compatibility garbage. ;-)

Cheating of the player is of course always possible, for example if the player just edits a level to make it easier. This is quite fine if it improves playing fun, but would of course result in a score tape that would not be approved (and therefore would not show up on anyone's Hall of Fame screen).

In many cases there will be another reason: The player using F1 and F2 (quick save and load) to play, which results in those notorious out-of-sync tapes of death. Just had this twice a few days ago by myself. :-(
This potentially also 'solves', in a sort of baroque way, cases where two unrelated or more-or-less disjoint levelsets have the same name: I noticed that I have two 'BD4' levelsets: levels/DX-Boulderdash/dx_bd4 'BD4' by 'ECI' with 100 levels, and levels/Emerald_Mine_Club/emc_bd4 'BD4' by 'Paul Williams' with 81 single-player levels and 21 2-player. (But RnD has no trouble distinguishing these locally -- obviously the levelset paths are different, but also the tapes are stored in ~/.rocksndiamonds/tapes/{dx_bd4,emc_bd4}; whereas the server seems to index by the 'name:' field from levelinfo.conf, not the directory name.
The "primary key" for looking up level sets is the "identifier", which is usually identical to the level set's sub-directory name. So this would be "dx_bd4" and "emc_bd4" here, which would be fine to distinguish these two sets.

Using that identifier, the server first tries to use a configured mapping from a mapping table (mapping the identifier to an already existing level set on the server). If this does not exist (which is normally the case), a database lookup for that identifier in the level sets table is made. If this still does not match (which was the case with "dx_bd4", which did not exist on the server when this check was done), another try is made, this time with the level set name (as defined with the field "name" in the level set's file "levelinfo.conf", which is also stored in the database). If this still does not match, a new level set is defined in the database using the identifier and name from the upload meta data.

In this case, a score tape for "emc_bd4" with the name "BD4" was first uploaded. Subsequent uploads of tapes for "dx_bd4" were not found by identifier, but by name, and were therefore put into this level set, which created quite a mess (mixture of level meta data of both level sets, whichever was uploaded first to define a new level in that set).

I think I will have to fix this manually. Fortunately, such overlapping level set names should not happen too often. But it might be worth pre-defining all level sets and levels stored in the server, regardless of score tapes uploaded for them. AT the moment, the whole process is probably a bit too dynamic.
Unlike tapes, it seems like levelsets should not be collected entirely automatically.
Exactly. Thought about this first, but quickly dismissed it due to the same reasons.
Dealing with that stuff seems like it might be too gnarly for automated handling.
Agreed, that's why I skipped it entirely for now.
Of course if you can upload levelsets then players should be able to download levelsets. (I'm pretty sure you already had that in mind...)
You're right here. ;-)
So, while browsing the score server, they should be able to point at a levelset and say 'put this on my machine'.
Yes, something like that. I thought of a virtual filesystem that is made up of level set meta data that is downloaded, cached and updated inside R'n'D, and that automatically downloads any selected level set that does not exist locally yet. This would really be a cool feature, but definitely do not expect this any time soon. ;-)
filbo
Posts: 647
Joined: Fri Jun 20, 2014 10:06 am

Re: score / tape / level server futures...

Post by filbo »

Active notifications are a bit tricky, as the server cannot contact the client directly, but it could be possible to create some kind of "pending notifications" on the server for a given player UUID, so requests could check for them and return whatever it was about. This would of course be a very advanced feature that may not be worth the effort, maybe...
Hmmm, seems worthwhile to me :)

I don't think it needs to be complex. Queue notifications by UUID. On client startup, send a notifications query (ignore errors so it's less annoying to play offline). Add a 'pending notifications' flag to the reply structure for all other queries (upload tape, query high scores -- are there others?) -- telling the client to issue a notifications query, so they get picked up between plays and not only at startup.
Why would the local game & server disagree about whether a tape solves a level?

- bugs in compiler or CPU on client or server
- different RnD versions
- non-identical level definitions
- some sort of cheating by the player
- ?????
Different R'n'D versions should theoretically not be the case, as long as the server always runs the latest version of the game with full compatibility garbage. ;-)
Sure, but some of the 'compatibility garbage' is code added after playback difficulties are discovered after-the-fact. Which means that 'different Rnd versions' actually means 'as-yet undiscovered backwards compatibility bugs in the current RnD'. So tapes which don't play back correctly are probably an interesting signal. Even if you don't upload unknown level definitions (copyright etc.), you can send up a hash; then the server knows if the tape is supposed to work. If you see things like multiple broken tapes from the same range of RnD versions and different players, it might be something to investigate. (Did I interpret correctly something that I took to mean that the server is keeping un-solvable tapes, just not showing them to clients?)
In many cases there will be another reason: The player using F1 and F2 (quick save and load) to play, which results in those notorious out-of-sync tapes of death. Just had this twice a few days ago by myself. :-(
Hard to believe this is still a bug something like a decade later!

Puts me in mind of my friend who ran a 4-user BBS on a Victor 9000 (8088 not-quite-IBM-compatible MS-DOS machine). It was written in Turbo Pascal and was more or less a complete operating system, only using MS-DOS as a bundle of buggy device drivers. The whole thing was awesome -- but would crash a few times a week, and we spent years trying fruitlessly to debug it. He was convinced it was an uninitialized variable problem. I eventually owned a Sperry IT (8MHz 80186 PC clone); it came with a tech manual that mentioned a port which controlled parity generation. I wrote code to pre-initialize memory with bad parity (turn parity generation on, write 1 to a location; turn parity off, write 0 -- now the parity bit is wrong; reading that memory will raise an NMI). We tried to run the BBS under that and quickly realized ... 80186 has a 16-bit memory bus, while the processor instruction set allows 8-bit access. Any attempt to write an 8-bit value is implemented by the hardware as: read 16 bits, change 8 bits, write back 16 bits. So natural operation constantly reads from uninitialized memory. While we were trying to figure out how to fix that (maybe get the CPU to run in an 8-bit-bus mode?), the motherboard suddenly died. I got it replaced under warranty and we gave up, figuring it probably wasn't meant to have the parity generator toggled thousands of times. And he never did figure out the bug, kept developing the BBS for years and living with weekly crashes. [ /ramble ]
So, while browsing the score server, they should be able to point at a levelset and say 'put this on my machine'.
Yes, something like that. I thought of a virtual filesystem that is made up of level set meta data that is downloaded, cached and updated inside R'n'D, and that automatically downloads any selected level set that does not exist locally yet. This would really be a cool feature, but definitely do not expect this any time soon. ;-)
If you do that, make sure there's a 'download it all now' action (about to get on an airplane and be offline for many hours...)

BTW, with max compression I could manage, my current levels collection equals:

- EMC: 45MB
- Eizzoux: 196MB (or: 4MB + 192MB of *.mp3, *.wav)
- all else: 48MB

Total: 290MB (or: 98MB of complete levelsets, omitting the audio for 21 Eizzoux '*leo*' levelsets)

(Max compression of Eizzoux achieved with `7z -mx9 -md500m`, which takes a couple gigabytes of RAM to compress, and half a gig to uncompress. Final size is 49% of plain `7z` output.)
Post Reply