lack of version info in tapes?

[filbo]

I played a couple of levels just now, and the hall of fame entries for them have 'UNKNOWN' for the PLATFORM, VERSION, COUNTRY fields.

I always play with own-built binaries, out of `git`, not necessarily at any specific version number. 3 builds may be relevant here. Looking in ~/.rocksndiamonds/tapes/emc_haunted_mine_4, I see these files (of interest):

Code: Select all

$ ls -l tape.0{69,70}
-rw-r--r-- 1 4095 Sep 13 18:47 069.tape
-rw-r--r-- 1 1099 Sep 13 19:40 070.tape
$ file tape.0{69,70}
069.tape: Rocks'n'Diamonds data, file format 3.0, tape file, game 4.3.7.0, engine 4.3.7.0, date 20240813, level set "emc_haunted_mine_4", level 69, actions 2000, player 1, [game_speed], [pause], [single_step], [replayed], [small_graphics]
070.tape: Rocks'n'Diamonds data, file format 3.0, tape file, game 4.3.8.2, engine 4.3.8.2, date 20240813, level set "emc_haunted_mine_4", level 70, actions 502, player 1, [game_speed], [pause], [single_step], [replayed], [small_graphics]

I would look at only 69 & 70, but I'm not sure which of 3 recorded on the hall of fame is equivalent to my locally saved file; RnD uploads to the server even if I locally tell the game 'no, do not save that'.

I think the 3 level 69 uploads correspond to this sequence, but I'm not sure:

1. under 4.3.7.0, played and finished the level; this was saved locally as well as uploaded
2. under (not sure which version), played back the tape until almost-end, then took control and finished the level; told it not to save (but it was uploaded anyway)
3. under 4.3.8.2, played back, took control, finished, told it not to save (but it was uploaded anyway)

... I'm probably overthinking it :)

In the hall of fame, this produced 3 nearly-identical entries for me; 1 has version info (4.3.7.0) while 2 don't. Which strongly implies that the 2nd one was also under 4.3.8.2. The locally saved file is still my first run, since the other two times I told it not to save. (I would prefer if 'don't save' would also imply 'don't push to score server'; these are generally things where I'm experimenting with something, might do it multiple times, and do not want to be littering the server with them...)

Level 70 is less complicated, I just played it under 4.3.8.2. So there is a single clean upload of one save file -- which doesn't show version info.

According to file(1), both my local level 69 & 70 files have version info. It doesn't display the other bits.

I'm wondering if there might be 2 or even 3 separate things going on here. Version info should be extracted from the uploaded save file, so is my local 4.3.8.2 build generating that in the wrong format? file(1) can get it.

But I think OS info is communicated by the running RnD when uploading the file; and location info is -- I think -- tied to some sort of magic cookie that relates a particular upload to a particular 'user' on the server; unrelated to the 'player name' associated at upload time. I think.

So:

- version ?wrongly encoded in save file?
- OS ?wrongly transmitted by running binary?
- country ?I got 'logged out' of the server / lost my local copy of the user-identity token that ties all of my uploaded save files together to the country I think I must once have told the server?

[filbo]

file(1) output is from magic entries I made, but I'm pretty sure have sent to Holger in the past. Except I see my copy has changelog entries from only 2y ago, and I'm pretty sure I haven't updated that. So, attached here.

[filbo]

(nevermind, I did update it in 2022, viewtopic.php?t=588)

[Holger]

I played a couple of levels just now, and the hall of fame entries for them have 'UNKNOWN' for the PLATFORM, VERSION, COUNTRY fields.

This is the case for score entries with local tapes (that is, where score and tape were not fetched from the high score server).

For these local scores/tapes, the three fields above are indeed set to "UNKNOWN". This is because the game cannot guess the country in which the tape was recorded, while the server knows it (via the IP address used for uploading the score and tape file).

It also cannot guess the platform the tape was recorded on (although I could store it into the tape file, which is currently not the case). (Simply using the platform of the currently running game would be possible, but would be wrong if you copy a tape file over to your second computer running a different operating system.)

However, it does know the version of the game the tape was recorded on (and also the version of the game engine used to record the game, which indeed can be different if playing older levels), and I'm currently not sure why this is also set to "unknown" for local tapes. I will check this. In any case, this is indeed always correctly stored in each and every tape file created by R'n'D.

In the hall of fame, this produced 3 nearly-identical entries for me; 1 has version info (4.3.7.0) while 2 don't.

Yes, that's by intention. It works as follows:

You have recorded three tapes that solved the level and are therefore uploaded to the server. (Hint: This also happens if you decide not to save the tape file on your system! You can only disable uploading tapes to the server "globally", that is, independently of your decision to save a tape after a solved game or not.)

When entering the hall of fame, the game tries to fetch all score entries of successfully solved games for this level from the server. However, the server only returns the best score for each individual player. This is done to prevent flooding of the hall of fame with 100 tapes of a single player that happen to be better than all other scores for this level of every other player, instead showing the best 100 players, each with their individually best score.

So, the game now has a single score from you fetched from the server, plus three locally stored scores/tapes. It detects that one of these three scores is the same as the one it got from the server, and therefore does not display it when showing your three score entries (if you have selected "showing local and server scores" in the setup).

As described, version, platform and country can only displayed for server scores, so the other two get those "unknown" texts.

- version ?wrongly encoded in save file?

No, it's just not used in the hall of fame for whatever reason. I'll check this.

- OS ?wrongly transmitted by running binary?

No, it's correctly transmitted. But not displayed by intention, as described above, as it is not stored in the tape file itself.

This could easily be fixed by also storing the platform into the local tape file, and showing it in the hall of fame (which obviously would also work well if that tape file was copied over to a different computer with different platform).

- country ? I got 'logged out' of the server / lost my local copy of the user-identity token that ties all of my uploaded save files together to the country I think I must once have told the server?

This could also be solved relatively easily by just asking the server at some point "which country is my IP address associated to?". (For example, at program start, or when uploading a score/tape.)

This is currently not done because I try to limit both any kind of network activity and also any kind of "user profiling" to the absolute necessary. So the game does not do requests like "tell me my country" to the server (but indeed records them once a score/tape upload is done).

Just to prevent "unknown" entries, I could add such a request at program start if it was never done before, and save it in some setup file. (As always, this would not prevent playing R'n'D offline, which does and will always work, by intention.)

[Holger]

Small addition: Yes, I really wanted to add your extended "magic" file to the R'n'D package! I really should do that at next occasion!

[filbo]

add your extended "magic" file to the R'n'D package

And/or figure out how to submit it to the maintainers of the `file` command...

Regarding your excellent explanation of the results I got: if it uploads all my tapes, and downloads all the best scores from the server, it should be able to byte-compare the downloaded ones, discard at least one of the local ones, and show full server-provided info on that one. No? Of course the other local ones get no country value, for reasons you explained.

[Holger]

Regarding your excellent explanation of the results I got: if it uploads all my tapes, and downloads all the best scores from the server, it should be able to byte-compare the downloaded ones, discard at least one of the local ones, and show full server-provided info on that one. No? Of course the other local ones get no country value, for reasons you explained.

Well, that's exactly what R'n'D does, as described above:

So, the game now has a single score from you fetched from the server, plus three locally stored scores/tapes. It detects that one of these three scores is the same as the one it got from the server, and therefore does not display it when showing your three score entries (if you have selected "showing local and server scores" in the setup).

That is, one tape (the best one that it got back from the server) from the three local tapes is replaced by the (identical) tape from the server (but with full version/platform/country info).

Or did I misunderstand you here, maybe?

[filbo]

I think I had confused myself. Plus there is this:

Level 70 is less complicated, I just played it under 4.3.8.2. So there is a single clean upload of one save file -- which doesn't show version info.

I am now not able to reproduce this; my one play of level 70 does show version/etc in the hall of fame.

But that suggests to me a possible fault that I might have run into then, but not now. I probably looked at that play right after having finished the level. Is it possible that either the server hadn't finished processing it at that point; or there is a logic / sequencing error in the code such that server-provided info about the just-completed level might not be visible? (i.e.: immediately after finishing a level, without moving to a different level or levelset or really doing anything else, immediately look in Hall Of Fame)

[Holger]

I think you exactly guessed it right: After you have solved a level, your score and tape is sent to the server, and is stored into the database (meta data) and into the file system (tape file). But the server still does not know for sure if this is really a valid solution tape! For example, it could have been "infected" by the notorious snapshot bug (and should therefore not be listed as a solution tape for others to replay). Or, even worse, it could (well, theoretically) be a malicious request done using curl, sending some garbage instead of a tape file and using some made-up data with top best score etc.

So there's a cron job running every five minutes, validating newly submitted tape files by replaying them using the "autotest" command on the server, updating the score data in the database from "new" to either "failed" or "succeeded".

So, until your tape has been confirmed, it is simply not available from the score server, and your local tape is used. As soon as it has been confirmed, it will be loaded from the server, replacing your local score/tape entry in the hall of fame (and now also using the additional data like country and version from the server).

[filbo]

Ok, that behavior is a bit confusing to the alert user :)

They look in Hall Of Fame, see those things unpopulated ... look back a little while later, and suddenly it's there! Weird!

This could be papered over, as follows:

- the local code already knows the platform & game version, as they're encoded in the file
- when uploading a file to the server, the server should give back to the client its idea of the country (based on the IP address, same one it's going to use to label the file on the server)
- the client then knows what country to label that file as, when displaying it, even though it hasn't been written to disk

This would immediately fix the client's display, failing only if the server couldn't be contacted (but in that case, failure is inevitable, and will eventually be repaired if/when contact is made in the future).

With that heuristic, it might still be possible to get a display of 'unknown', but only by a deliberate set of steps:

1. finish a level; tape gets uploaded to server (and client is informed of country name)
2. quit the game (client forgets country name)
3. immediately restart the game
4. look in hall of fame for that level
5. if the server hasn't yet finished processing that tape, it shows as 'unknown'

(and this could be fixed by caching the 'what country I supposedly am' value in levelsetup.conf or serversetup.conf; but I don't really think the tiny outage is worth that effort :)

[Holger]

Ok, that behavior is a bit confusing to the alert user

Yeah, it's indeed confusing if you pay attention to those details!

This could be papered over, as follows:

Sounds reasonable, and should be easy to implement -- good idea!