Rocks'n'Diamonds 4.3.0.0 Release Candidate 1 released!

R'n'D is always evolving. Check here to see if a new version is out.

Moderators: Flumminator, Zomis

Post Reply
User avatar
Holger
Site Admin
Posts: 4073
Joined: Fri Jun 18, 2004 4:13 pm
Location: Germany
Contact:

Rocks'n'Diamonds 4.3.0.0 Release Candidate 1 released!

Post by Holger »

The development of the new Rock'n'Diamonds high score server already took way too long, so here's something for you to play around with: The first release candidate of the upcoming new version 4.3.0.0 of R'n'D with support for uploading (after you won a game) and downloading (when entering the Hall of Fame) high scores (with tapes) for R'n'D levels played by you and others!

To download the test version, select one of the download links below (depending on your system):

https://www.artsoft.org/RELEASES/linux/ ... rc1.tar.gz (Linux)
https://www.artsoft.org/RELEASES/mac/ro ... .0-rc1.dmg (Mac)
https://www.artsoft.org/RELEASES/window ... -win32.zip (Windows / 32 bit)
https://www.artsoft.org/RELEASES/window ... -win64.zip (Windows / 64 bit)
https://www.artsoft.org/RELEASES/androi ... .0-rc1.apk (Android)

You can also play and test it directly in your web browser:

https://staging.artsoft.org/rocksndiamonds/play

Upon first start of the release candidate version, and if you already have some R'n'D tapes stored on your system, the game will ask you if you want to upload them to the high score server. If you don't want to do that, or if you want to do it later, just say "no" here -- you can still upload your tapes from the setup menu as long as you haven't done it yet.

In addition, there are two new options related to the high score server in the setup/game menu to enable/disable using the score server, and to only show your own, local scores in the Hall of Fame (just like before).

If the score server should be unavailable for whatever reason (no network, server down etc.), you probably won't notice this immediately, as there will be no "timeout" delays (communication to the server is done in the background), and because Hall of Fame entries from the server will be cached locally -- the game will just silently disable any further attempts to access the server until next start (well, you will see an error message in the log file or on the command line).

To be able to rename your player name as shown in the Hall of Fame at a later time also for already uploaded scores, a randomly created unique, but anonymous player ID is generated. Just rename your player name in R'n'D, and the name shown for your score entries will be renamed, too.

To not start with an empty high score server, I have already uploaded the solution tapes from my tape test suite -- that's more than 25'000 score tapes from over 370 level sets, played by over 70 players. So you will already see some high scores for many level sets.

To have a look "behind the scenes", you can also take a look at the web interface of the high score server:

https://api-test.artsoft.org/

The site will show all approved scores (but not those that were rejected, most probably either because the level set is not yet known to the score server, or simply because the uploaded tape was a duplicate of an already existing tape, was not valid or was not solvable for whatever reason).

If everything works as expected, the final version 4.3.0.0 will hopefully be officially released soon, but I still expect a few hiccups here and there, as every attempt to release such a test version in the last few weeks resulted in stumbling upon yet another problem. But it seems to start running a bit more stable and reliably now (fingers crossed ... the last problems were fixed a few minutes ago, and my "issues" list is still not empty yet).

Future versions of the game and server (but not version 4.3.0.0 yet) may contain a way to replay score tapes that were uploaded and approved on the server from the Hall of Fame (directly or as some sort of cheat mode; not sure yet).

I hope you have fun testing the new pre-release version, and please let me know about everything good or bad that you should experience while testing the new version! :-)
filbo
Posts: 647
Joined: Fri Jun 20, 2014 10:06 am

Re: Rocks'n'Diamonds 4.3.0.0 Release Candidate 1 released!

Post by filbo »

Two quick impressions:

1. `git pull` gets nothing -- please commit 4.3.0.0RC1 into the repository!

2. on the API site: every tape is dated between 2021-09-25 13:29:49 and 2021-09-27 21:25:04! Can you reprocess those with proper timestamps? I know the tape format has a date but not a time, so at least stamp each one to 00:00 on the embedded date? (I think there's an older format without date stamps: for those, I guess you could set them all to the day before the first RnD version was published which generated tapes with date stamps ... not very accurate, but still vastly more accurate than 2021-09-25!)

For files which you have yourself -- or are processing from a user's tapes/... hierarchy -- if the tape contains a date stamp, and the file in which it is contained has a filesystem timestamp which is compatible with(*) that date, take that as its full precise origin time. e.g.:

$ file .rocksndiamonds/tapes/emc_ace_mine_1/043.tape
.rocksndiamonds/tapes/emc_ace_mine_1/043.tape: Rocks'n'Diamonds data, tape file, version 3.3.1-2, engine 3.3.1-2, date 20140624, level set "emc_ace_mine_1", level 43

$ ls -log --fu .rocksndiamonds/tapes/emc_ace_mine_1/043.tape
-rw-r--r-- 1 175 2014-07-24 04:57:05.000000000 -0700 .rocksndiamonds/tapes/emc_ace_mine_1/043.tape

==> full time stamp is 2014-07-24 04:57:05 -0700 (convert to time_t...)

(*)'compatible with' probably means 'same YYYYMMDD +/- 1, to account for various oddities and bugs which can occur with timezone processing'...
filbo
Posts: 647
Joined: Fri Jun 20, 2014 10:06 am

Re: Rocks'n'Diamonds 4.3.0.0 Release Candidate 1 released!

Post by filbo »

libgame/http.c DoHttpRequestExt():

- leaks socket_set always (missing SDLNet_FreeSocketSet())
- leaks OS open socket on all error paths after (socket != NULL) (never reaches SDLNet_TCP_Close())

Possibly handle by making another nested function which does the work; middle function allocates and frees those two items, passing them in to the 3rd function. Otherwise the error path handling inside this function gets awful...
filbo
Posts: 647
Joined: Fri Jun 20, 2014 10:06 am

Re: Rocks'n'Diamonds 4.3.0.0 Release Candidate 1 released!

Post by filbo »

Some sources are missing from both git repo and Linux source bundle:

- tests/... (Makefile:MAKE_ENGINETEST)
- Scripts/... (Makefile:MAKE_{LEVELSKETCH,DIST})
- Special/... (src/Makefile:ICON_BASEPATH)
User avatar
Holger
Site Admin
Posts: 4073
Joined: Fri Jun 18, 2004 4:13 pm
Location: Germany
Contact:

Re: Rocks'n'Diamonds 4.3.0.0 Release Candidate 1 released!

Post by Holger »

Hi filbo, thanks for your prompt responses! :-)
1. `git pull` gets nothing -- please commit 4.3.0.0RC1 into the repository!
Sorry, missed that one -- done now!

Branch "master-next-major-release" contains the current pre-release "master" branch for the upcoming new release, while branch "master-next-major-release-test-rc1" contains two additional commits to force using the test API server and to set the temporary "RC1" version.
2. on the API site: every tape is dated between 2021-09-25 13:29:49 and 2021-09-27 21:25:04! Can you reprocess those with proper timestamps?
Including the "real" tape date (in contrast to the upload date) was one of the issues mentioned above -- in fact, I've thought about it and added it to my list just while typing that announcement, but I didn't want to postpone the pre-release again. :-)

Besides some minor tweaks, the other thing on the list is that it might be useful to tell the player if his chosen name already exists on the high score server. Not enforcing uniqueness of player names might be controversial, but I think it makes sense for more than one reason: First of all, I don't want to add "player accounts" with mail addresses and passwords etc., but want to keep things strictly (semi-)anonymous, just like it was when typing in your three-letter-name to the high score list on those old arcade machines in your local pub. ;-) So it will be technically possible to choose an already existing name by intention and submit scores for it, which cannot be distinguished from the scores of other players with the same player name in the Hall of Fame (but can be distinguished on the server by their different player IDs). The only visible hint will be that there is more than one entry in the Hall of Fame with the same player name (as the server will only return the best score for each player for a given level). But it might be a good thing to at least let a player know that somebody else is already playing with the same nickname before choosing "John" as your player name. :-) Allowing use of the same name also makes it possible to play on several different devices with the same player name without having to use central player accounts (and if you know how to edit "setup.conf", you can even copy the player ID to really play as the same player.)

Not using central player accounts was a decision to be independent of a working server, to make things as easy as possible to use, and to use as less user tracking as possible. (It could be argued that using a unique player ID is a violation of these goals.)

Please let me know what you think about it!

Back to the tape date: Thanks for your ideas and inspiration to get a tape date (and even time) that is as accurate as possible, especially regarding old tapes. Fortunately, the "HEAD" chunk that contains the tape date is already used by the very first implementation of tape files, so we will always have a valid date. Your proposed ways to refine it regarding time would then be the icing on the cake. :-)
- leaks socket_set always (missing SDLNet_FreeSocketSet())
- leaks OS open socket on all error paths after (socket != NULL) (never reaches SDLNet_TCP_Close())
Oops! I'll have a look that this! :-o
Some sources are missing from both git repo and Linux source bundle:

- tests/... (Makefile:MAKE_ENGINETEST)
- Scripts/... (Makefile:MAKE_{LEVELSKETCH,DIST})
- Special/... (src/Makefile:ICON_BASEPATH)
Yes, indeed -- these have always been missing (mainly for the reason of very dirty scripting that I do not want to release, some of them containing server and user names I don't want to put into public repositories).

But as you might have noticed with the new "tests" directory, I'm already about to clean things up, so one of the next versions will hopefully finally also contain the scripts for the game engine tests (in "tests") and the packaging process (currently in "Scripts", but will probably then go to the already public "build-scripts"). Things in the "Special" directory are mainly additional files like icons and Mac resource files that should go into new sub-directories under "build-projects".
User avatar
armanelgtron
Posts: 29
Joined: Sat Jul 25, 2015 8:25 pm

Re: Rocks'n'Diamonds 4.3.0.0 Release Candidate 1 released!

Post by armanelgtron »

Well, I was going to complain about my scores not showing up, but it turns out it does work and I just wasn't patient enough, heh.

However, going to older versions of rocksndiamonds (example 4.0.1.0 included with 18.04) wipes the UUID when changing a setting or your name, whoops...
Image
filbo
Posts: 647
Joined: Fri Jun 20, 2014 10:06 am

Re: Rocks'n'Diamonds 4.3.0.0 Release Candidate 1 released!

Post by filbo »

> going to older versions of rocksndiamonds ... wipes the UUID

Hmmm, yeah. Please store in a new file not known to older releases.

=====

I'm OK with unique user ID (well, I think I suggested it in the first place... :)

It's not tracking for tracking's sake, it's allowing users to (in some sense) 'own' their files on the server.

(Oh, in fact, in that regard, can you communicate to me the unique ID associated with my many uploaded score files? I haven't actually built & run it yet, but I'm assuming it's going to go out and find copies of most of my tapes, but with a different ID, so might end up uploading doubles.

=====

On ugly scripting ... dude. Have you ever looked at the install and init scripts from typical *ix packages -- particularly distro / vendor work? I promise nothing in your scripts can possibly be that ugly!

I know I've bugged you about missing files before. Maybe I can look at them outside of git & forum, possibly do some cleanup?
filbo
Posts: 647
Joined: Fri Jun 20, 2014 10:06 am

Re: Rocks'n'Diamonds 4.3.0.0 Release Candidate 1 released!

Post by filbo »

I was using Debian file layout:

RO_GAME_DIR=/usr/share/games/rocksndiamonds
RW_GAME_DIR=/var/games/rocksndiamonds

My build script passes those things to the RnD Makefile, so I didn't notice I needed to change to BASE_PATH. After building and running RC1, it comes up with Niko Böhm's tutorial, and levelsetup.conf is trashed -- all history erased!

I think the history in levelsetup.conf is meant to protect against this sort of thing. But, apparently it is vetted against the list of known levelsets; so if your build points to levelsets in the wrong place, it wipes out the protective history!

Ah, I see that history is available in the GUI. Please don't edit out missing levelsets; instead, let them be seen in the GUI, but if the user tries to select a missing one, tell them it's missing. Maybe also mark missing levelsets (e.g. a different color?) in the GUI menu? ... and then, eventually, it might offer to download the missing levelsets. Dialog with the server, see if any missing ones are known to the server, offer to download those?

=====

Meanwhile: there seems to be only BASE_PATH now. I see only score files were saved in the 'RW' path previously; is it that score files are now completely networked? Or maybe will be written to a ~/.rocksndiamonds/scores dir?
User avatar
Holger
Site Admin
Posts: 4073
Joined: Fri Jun 18, 2004 4:13 pm
Location: Germany
Contact:

Re: Rocks'n'Diamonds 4.3.0.0 Release Candidate 1 released!

Post by Holger »

Well, I was going to complain about my scores not showing up, but it turns out it does work and I just wasn't patient enough, heh.
The cron job starting the check script currently runs once per hour ... I might change this to every five minutes (as the script already takes care about not running more than once).

The "average user" won't notice anyway, as personal scores are also stored locally, but if you also check on the API web site, new scores won't show up until they were approved by the check script. Which of course also means that you won't be able to see why your scores don't show up there, if anything went wrong with the approval process (like missing level sets on the server, or duplicate tape files, or whatever). I'm not sure yet how to handle this; I want to prevent listing all those failed tapes and scores that inevitably will be stored on the server -- especially all those private tapes which will be uploaded together with the names of private levels and level sets, which might be seen as a privacy issue.

Potential solutions: Manually create and give out read-only user accounts to interested parties, or, maybe even better, allow people to "log in" with their player name and player ID, then being able to view all personal score entries, regardless of them being approved or not.
However, going to older versions of rocksndiamonds (example 4.0.1.0 included with 18.04) wipes the UUID when changing a setting or your name, whoops...
Ouch, that's a real show stopper bug! :-o
Hmmm, yeah. Please store in a new file not known to older releases.
Yep, I think that's the way to go. As the player UUID cannot be recovered by R'n'D if it is lost (it's not computed from the player name, or the like), having it deleted simply by playing an earlier version of the game once really should not happen. (I encountered the same behavior during my tests, but did not think any further about it.) The main problem caused by this behavior will be duplicated high score list entries in R'n'D, as the server assumes that these are different players which happen to have chosen the same name (which is generally considered OK).
I'm OK with unique user ID (well, I think I suggested it in the first place... :)
Yes, you did, and it was a very good idea! :-)
It's not tracking for tracking's sake, it's allowing users to (in some sense) 'own' their files on the server.
Before, I only used the bare player name as "ID", but the argument of "being able to claim ownership of one's own scores/tapes" seemed very reasonable to me.
(Oh, in fact, in that regard, can you communicate to me the unique ID associated with my many uploaded score files? I haven't actually built & run it yet, but I'm assuming it's going to go out and find copies of most of my tapes, but with a different ID, so might end up uploading doubles.
That's right (see above), all your solution tape would then exist twice on the server then (but won't be approved, as the identical tape files already exist then). So it won't cause any visual trouble.

All existing solution tapes were pre-uploaded with an empty player ID, so if you send me your current ID via mail or forum message, I'll replace it with your own ID (so you can change the player name, if you like).
On ugly scripting ... dude. Have you ever looked at the install and init scripts from typical *ix packages -- particularly distro / vendor work? I promise nothing in your scripts can possibly be that ugly!
You're probably right! :D

I'm more concerned about the build server names and (especially) user account details I've hard-coded into that script, but I'm already about moving server, accounts and file path details into a separate configuration file that the script will read, and for which I will add a template to easily build one's own configuration file.
I know I've bugged you about missing files before. Maybe I can look at them outside of git & forum, possibly do some cleanup?
Yes, sure ... I'll add something to play around with to a temporary branch soon. :-)
I was using Debian file layout:

RO_GAME_DIR=/usr/share/games/rocksndiamonds
RW_GAME_DIR=/var/games/rocksndiamonds

My build script passes those things to the RnD Makefile, so I didn't notice I needed to change to BASE_PATH.
Meanwhile: there seems to be only BASE_PATH now. I see only score files were saved in the 'RW' path previously; is it that score files are now completely networked? Or maybe will be written to a ~/.rocksndiamonds/scores dir?
Yes. Some background about it: I've dropped support for local multi-player support regarding shared score files in favor of the new score server, and only write personal scores to ~/.rocksndiamonds/scores now. So there's no need to distinguish between read-only and readable/writable shared game data anymore, because it's all read-only now (or goes into the user's personal game data folder).
After building and running RC1, it comes up with Niko Böhm's tutorial, and levelsetup.conf is trashed -- all history erased!

I think the history in levelsetup.conf is meant to protect against this sort of thing. But, apparently it is vetted against the list of known levelsets; so if your build points to levelsets in the wrong place, it wipes out the protective history!
Yes, all level set entries in the history for which no existing level set can found are removed.
Ah, I see that history is available in the GUI. Please don't edit out missing levelsets; instead, let them be seen in the GUI, but if the user tries to select a missing one, tell them it's missing. Maybe also mark missing levelsets (e.g. a different color?) in the GUI menu?
In fact, I stumbled upon the same effect (when switching between different level directories for testing). But I'm not sure if such a complicated approach is worth the hassle. Maybe I can find a simpler solution...
... and then, eventually, it might offer to download the missing levelsets. Dialog with the server, see if any missing ones are known to the server, offer to download those?
Displaying and downloading level sets from the server may be part two (or three) of the story. ;-)

For the problem above, it would only help for level sets known by the server then. So far, I think that losing ten entries of level set history in a test environment is acceptable. :-)
filbo
Posts: 647
Joined: Fri Jun 20, 2014 10:06 am

Re: Rocks'n'Diamonds 4.3.0.0 Release Candidate 1 released!

Post by filbo »

(Oh, in fact, in that regard, can you communicate to me the unique ID associated with my many uploaded score files?
All existing solution tapes were pre-uploaded with an empty player ID, so if you send me your current ID via mail or forum message, I'll replace it with your own ID (so you can change the player name, if you like).
Maybe this could be automatic? If the server contains a tape for a particular [levelset,level,username,UUID=blank], and a user's RnD uploads the same exact [levelset,level,username,UUID=something], same bytes -- then just set the existing file's UUID?

I suppose 'rogue users' could 'claim' already uploaded sets of tapes by changing their username to match, and having the corresponding tapes in their local set before telling RnD to upload. But this seems like a small and more or less unimportant risk.

More risky is that users whose tapes are already uploaded by you might today be using a different in-game name; thus their initial session of tape uploads would fail to match, and fail to assign UUIDs to their tapes. Not sure what to do about that.

In any case, I'll forum-PM you my now assigned UUID.
more than 25'000 score tapes from over 370 level sets, played by over 70 players
I started to ask: are you in current contact with the players representing the 10 largest subsets of this, and if so, how many remaining 'unclaimed' tapes would there be if you got UUIDs from those top players.

But then I looked ... my most recent handoff to you had >22K tapes, so that answers most of the question right there :)
I think that losing ten entries of level set history in a test environment is acceptable. :-)
I really don't care about the history per se. I care about suddenly landing in Niko Böhm and having to dig back through the levelset hierarchy and remember where I was. The 10 history entries are irrelevant as long as I don't lose my current place.
Post Reply