An idea for the next generation files archive

Discussion about Rocks'n'Diamonds, Boulder Dash, Supaplex, Emerald Mine and any other BD hybrid.

Moderators: Zomis, Flumminator

Post Reply
User avatar
Zomis
Posts: 1501
Joined: Mon Jun 21, 2004 1:27 pm
Location: Sweden
Contact:

An idea for the next generation files archive

Post by Zomis » Tue Jun 10, 2014 11:56 am

As you know, the Files Archive is dead. I have the files from the files archive, but I am missing the database that maps the files to an author/title/filename.

I have however an idea for a system that can replace the Files Archive. I'd like to call this "Rocks'n'Diamonds Database" (RnDDB).
I might have told some of you of a previous RnDDB project, but that never got officially released as far as I can remember. This idea is a bit different though.

The idea is to make a distributed system. A bit similar to Peer-to-peer File Sharing systems, but limited to RnD files.
As I develop mostly in Java, I'm thinking Java. Java is cross-platform so it would work on any computer, no matter if you're running Linux, Mac, or Windows.

RnDDB can scan paths on your computer for RnD files, it can detect which one exists in the shared system already, and it can let you upload/share files to the system that does not already exist in the system.
RnDDB can also scan the system and detect files / levelsets that exists "in the cloud" but that does not exist on your computer.
RnDDB would also include a search feature so that you can search for level/levelset titles, a specific author, or possibly even specific aspects of the level (levels with custom artwork, levels with > 100 custom elements being used, etc.)

"The cloud" can either be on a specific server (I have one in my kitchen hosting other things for me, and that server is not going anywhere any time soon). But it can also be constructed so that each computer using the system has it's own local database for the RnD files stored on that computer (that will not require you to install any additional software, the database components can be embedded within the Java application).

The biggest concern I have at the moment about this is the security aspect. Should the system require a login, to detect who have shared what file? Also of course, it needs to be protected from malware.

If I would make this system, I plan on making it open source and hosting it on GitHub.

What do you think about this idea? Ideas, comments, etc?
And the most important question: Would you use it? Because if this system would not have any users, then it'd be a useless system.

User avatar
Holger
Site Admin
Posts: 2929
Joined: Fri Jun 18, 2004 4:13 pm
Location: Germany
Contact:

Re: An idea for the next generation files archive

Post by Holger » Wed Jun 11, 2014 7:17 pm

Hi Zomis,

I had some thoughts and ideas recently for something that may (or may not) be similar to the old files archive, at least regarding some of its functionality. It would be a combination of a "submit" button in R'n'D for your own level set(s) to upload them to a central server, and a way to access all sets on that server inside R'n'D (more or less transparently, downloading sets and levels if needed). This could optionally also be combined with a central high score and tape repository.

I especially thought about this regarding the upcoming Android version of R'n'D, with no easy way (for the "normal" user) to add new level sets to the game. (This is relatively easy for R'n'D running on a normal desktop system, but an integrated solution would be much more user friendly than dropping unzipped archives into the levels folder, of course.)

As always, the main challenge for such an automated system would be to find a way to distinguish the "good" contributions from the "bad" ones. A possible solution I could think of could be, for example, a combination of a voting/ranking system (let the player give one to five stars for a certain level or set somehow) with an editorial system (put certain level sets into some sort of "editor's choice" category).

Implemeting something like that is not the next big thing in R'n'D (which is Android support and therefore SDL2 support), but may be one of the next milestones.

User avatar
Zomis
Posts: 1501
Joined: Mon Jun 21, 2004 1:27 pm
Location: Sweden
Contact:

Re: An idea for the next generation files archive

Post by Zomis » Fri Jun 13, 2014 11:58 am

I had some thoughts and ideas recently for something that may (or may not) be similar to the old files archive, at least regarding some of its functionality. It would be a combination of a "submit" button in R'n'D for your own level set(s) to upload them to a central server, and a way to access all sets on that server inside R'n'D (more or less transparently, downloading sets and levels if needed). This could optionally also be combined with a central high score and tape repository.
I especially thought about this regarding the upcoming Android version of R'n'D, with no easy way (for the "normal" user) to add new level sets to the game. (This is relatively easy for R'n'D running on a normal desktop system, but an integrated solution would be much more user friendly than dropping unzipped archives into the levels folder, of course.)
Yes! I think that are the main features that RnD is currently lacking the most right now! It would be a lot more user-friendly indeed, on both Android and Desktop versions!
As always, the main challenge for such an automated system would be to find a way to distinguish the "good" contributions from the "bad" ones. A possible solution I could think of could be, for example, a combination of a voting/ranking system (let the player give one to five stars for a certain level or set somehow) with an editorial system (put certain level sets into some sort of "editor's choice" category).
That is one of the main challenges indeed. My experience from the old Files Archive is that the voting system works, as long as there are votes. A problem could be the lack of voting (in the Android version, perhaps have an option for quickly ranking the currently-played, or just-finished level / levelset?). Some analysis can possibly be done of the levels to determine the ranking, for example: Big levels with lots and lots of different elements shattered in a seemingly random order is most likely a bad level (at least from my point of view). Levels that has an associated tape is probably a good level (assuming the tape provides an actual solution to the level).

Implemeting something like that is not the next big thing in R'n'D (which is Android support and therefore SDL2 support), but may be one of the next milestones.
What if you didn't have to take care of the Server-part but only needed to take care of the Client-part and perform some HTTP requests? I hope that would speed up the progress a bit? As stated, the Server will also be open-source of course (assuming it's done with the R'n'D Database idea mentioned above). Automatic backups of both files and the database can also be taken regularly and provided for download to make sure it doesn't disappear.

filbo
Posts: 292
Joined: Fri Jun 20, 2014 10:06 am

Re: An idea for the next generation files archive

Post by filbo » Sun Jun 22, 2014 10:03 am

This all sounds pretty cool!

As a user & player, here's what I'd hope to get out of it. Some of this is repeating (affirming) what you've already said, some is expanding or refining it:

- ability to add levels / levelsets / larger collections to my local set
- hierarchical organization and controls making it adequately easy for me to collect all levels known to the repository
- including a way to confirm / crosscheck that I really have it all
- easy way to upload my own created levels, creating my own personal levelset or collection of levelsets which other users can access
- if I change or add a level to my local personal levelset(s), easy way to sync my changes up to the repository
- if I change a level created by someone else, a way to upload it as a "proposed change" such that the repository will inform the original author (or designated maintainer), and also make it available to other users, but not as the default downloaded version of that level
- when I successfully play a level, be able to upload my solution
- (I would hope that the upload receiver would have a way to invoke the game engine and verify that my tape is actually a valid solution of the level I'm tagging it to -- I realize there are potentially some fairly difficult issues to solve for this to work well...)
- be able to configure RND to automatically upload such solutions whenever I finish a level (again, this has difficulties -- but "OMG that'd be a lot of space" actually probably isn't one, since it would only happen for users who went to the trouble of registering with the repository and turning on the option, and tapes are quite small anyway)
- be able to browse other people's solutions to a level
- hopefully the site would be able to evaluate solutions to determine which was the "best", though it's not immediately clear to me what "best" might really mean. Shortest amount of time? Shortest path of motion? Highest "score"? But the scoring in RND levels seems almost completely superfluous to me...
- if there's a "best" function then of course the easiest / default way of getting a solution for the current level would grab the one ID'd as "best"
- some sort of "leader board" showing, for each registered player, how many levels they've solved; how many "best" solutions they were responsible for; average "goodness" if the evaluation function can actually produce a sensible number for this (e.g. "best solution time / user's solution time" -- if I took twice as long, mine's only worth 0.5 "goodness" point)
- obviously the game shouldn't offer access to an online solution until you've solved it locally; equally obviously, that'd be very easy to cheat, so the "prevention" shouldn't be any stronger than the current prevention of skipping levels. That is, (1) an option you need to turn on to say "yes, let me request tapes for levels I haven't personally solved" plus (2) an interactive "are you sure you want to see a spoiler for this level?" question
- user ratings of collections, levelsets and levels

This is not a demand to see all this implemented! This would be a huge amount of work, and for what audience -- perhaps not a lot?

It's a "gee, this all would be nice, in an ideal world".

It *also* could potentially become a way to monetize mobile versions. I have no idea if Holger has any thought to do such a thing, but I think it would be cool if he could make a living on it. (I can also imagine that he might be offended at the very suggestion. If so, sorry! It's a thought and I'm speaking it -- if you'd like to stop it right here, reply and say so.)

Imagine if Android and iOS versions of the game were released, tied into such a repository, and "went viral" -- got really popular. Things like that do happen.

How to monetize? Jeeze, I don't know. Most of the levels belong to other people or companies. Can't charge for them. Charge for uploading tapes? Then nobody would. In fact I don't really see how to *charge* for anything. The only thing I see is some sort of "unobtrusive ads" while browsing the levels / solutions repository.

User avatar
Zomis
Posts: 1501
Joined: Mon Jun 21, 2004 1:27 pm
Location: Sweden
Contact:

Re: An idea for the next generation files archive

Post by Zomis » Wed Jun 25, 2014 6:20 pm

In an ideal world, I'd more or less also like to see those features, yes. However, as this is not an ideal world (yet), somethings need to be prioritized.

I'd prioritize the levels, the tapes would be secondary for me.

Of course, I personally can't do the R'n'D integration, but I could provide a system which can be integrated into R'n'D using an API (using socket requests, HTTP requests, or whatever).

I am not sure how the version-control stuff would work, if you modify someone else's level, or even modify your own. One simple solution would be to associate a newly created level with a "parent-level", so that a tree branch structure can be formed, so if I check the system for information about a level, I could see that the level has 3 levels that has been branched from it, and one of those have another 2 branches, etc.

I'd be willing to create the Java software for this (server + a desktop application that can be used until it's integrated into R'n'D), I would however like to see what the demand is for it (Or hear Holger say: "Yes, that be extremely helpful, start working on it!")

User avatar
Holger
Site Admin
Posts: 2929
Joined: Fri Jun 18, 2004 4:13 pm
Location: Germany
Contact:

Re: An idea for the next generation files archive

Post by Holger » Mon Jun 30, 2014 7:51 pm

Hi filbo,

just some quick comments before answering in more detail (not enough time at the moment):

First of all: I am not offended by anthing you have written, don't worry. :-)

Indeed I'm not interested in making money of R'n'D. But not because I'm rich and don't need the money (I'm not), but because R'n'D is a hobby project for me and should never be anything other than that. Trying to earn some money with it may ruin that. :-)

filbo
Posts: 292
Joined: Fri Jun 20, 2014 10:06 am

Re: An idea for the next generation files archive

Post by filbo » Wed Jul 02, 2014 10:50 am

Ok, that seems like a healthy attitude -- and it helps guide things like this. With no need to consider how to monetize, that makes it easier to do some parts of an online interactive levels / solutions repository.

User avatar
Zomis
Posts: 1501
Joined: Mon Jun 21, 2004 1:27 pm
Location: Sweden
Contact:

Re: An idea for the next generation files archive

Post by Zomis » Wed Jul 23, 2014 10:58 am

Here is how the application looks right now:

Image

The left view shows the local levels in my user Rocks'n'Diamonds directory, the right view shows the levels existing on the server.

There are a few bugs to fix when uploading/downloading big levelsets, but it is partially working at least. I need to fix those bugs before a release though.

The source code is already available at GitHub: https://github.com/Zomis/RocksAndDatabases

Let me know what you think of the design and also let me know what the interest in this is right now. Are there enough users to make this useful?

Post Reply