Rocks'n'Diamonds on Apple Silicon

[Flumminator]

Hi Holger and everybody!

So I've just got myself a new and shiny MacBook and it comes with the "Apple Silicon" M2 processor. While checking all my installed apps whether there's a native build, I found that for Rocks'n'Diamonds there's not yet an Apple Silicon build.

So I just went ahead, cloned the git repo and built it myself and it runs like a charm. (just had to set MAC_TARGET_VERSION_MIN = 13.0 to avoid version conflicts with my SDL libs)

Now the question is: I would like to share it like a typical MacOS-App in a .dmg and everything. I see in the Repo there's an App-skeleton in the build-projects/mac directory, but I'm missing some installation instructions how to populate it (especially how to add the SDL-framework and link the binary against this).

Can you help me with this?

[Flumminator]

Btw, I did a little performance comparision by running a few tapes.

Here's the Intel-Release of R'n'D 4.3.5.1 run using the "Rosetta 2" emulation:

Code: Select all

===============================================================================
Number of levels played: 177
Number of levels solved: 177 (100%)
-------------------------------------------------------------------------------
Summary (for automatic parsing by scripts):
LEVELDIR [ OK ] 'nbbugmine', SOLVED 177/177 (100%)
===============================================================================

real	1m32.928s
user	1m31.157s
sys	0m1.741s

And here the native build, playing the same levelset:

Code: Select all

===============================================================================
Number of levels played: 177
Number of levels solved: 177 (100%)
-------------------------------------------------------------------------------
Summary (for automatic parsing by scripts):
LEVELDIR [ OK ] 'nbbugmine', SOLVED 177/177 (100%)
===============================================================================

real	1m18.197s
user	1m15.522s
sys	0m2.258s

That's about 12% improvement

[Holger]

Hi Flummi, great to see you in the forum again!

Yes, you're right, the new M1/M2 Mac processors are there since some time, and R'n'D probably needs a native build!

In fact, this is on my todo list for the near future (whatever this may be), and I think it would be best to build a universal binary (or how it is called) that contains both the code for intel and Apple processors, so we wouldn't have two separate packages like for the win32 and win64 platforms. As far as I can tell, the SDL library is also already prepared for this (although I haven't tried that yet).

As my own MacBook is really old now (from mid-2015, but still fully supported, currently running Monterey, and still with a surprisingly good battery life for its age), I also plan to move to a new MacBook, so I then could test things by myself.

As a side note, I am surprised how fast the processor emulation runs using Rosetta 2! I'm sure that there will be no noticeable speed difference when running R'n'D, or will it? But yes, we do need native code nevertheless.

Regarding packaging a proper .dmg file: Yes, the skeleton for the Mac package is already part of the repo, but the main build script is still missing, mainly because it is a big and dirty shell script with lots of hardcoded stuff. Even though it works great for me (I can package, upload and deploy a new game version of both R'n'D and also of spin-offs like Mirror Magic for all five platforms and architectures like win32/win64 with one single shell command), it really needs a lot of cleanup before I can release that beast to the public (and before it is even remotely useful for somebody else).

For building all required SDL libraries (SDL, SDL_image, SDL_mixer with sub-libraries for MOD and MP3 support, and SDL_net) in Mac framework format, I have another dirty, non-generic script with hardcoded rubbish, which I also hesitate to release at its current state.

I really should clean up all that stuff and add it all to the repo, especially as it would enable people to easily release their own custom R'n'D based games with everything that's required, for all platforms. (OK, that does not happen all too often.)

Until then, I could send the two scripts (for building SDL and for packaging/releasing) to your private mail address. But even then I would prefer to do some cleanup first, else I may have to answer too many questions on them...

[Flumminator]

If you could send me the scripts I would be willing to have a look

On the other hand - as you noticed - the Rosetta emulation is really good, so there's absolutely no urgency in providing a native build. For comparison: Even with the emulation the tapes run nearly twice as fast than on my previous MacBook Pro (mid 2017) and for just playing the game this is more than enough "horsepower"

Regarding the universal binary: Yes, that sounds nice as well. I'm however not sure what the oldest MacOS-version is you can compile for. Maybe some would like to keep an Intel-build that can be run on older versions of macOS? Most open-source projects I've checked in the last days seem to prefer dedicated builds for Intel and ARM (that includes also big ones like Libreoffice and GIMP).

I might also have to take a look at the SDL version I'm using, because when "warping" tapes on my native build I see some flickering which is not present with your release build. Maybe I have to experiment here a bit. So far I haven't put any thought into it, I just took whatever Homebrew has offered me.

[filbo]

That's more like a 16-20% speedup (depending how you count sys & real time), which is better than 12%, but still astonishingly low.

It makes me think that (1) Rosetta is remarkably good, but also (2) the compiler optimization for ARM & especially Apple Silicon is apparently pretty weak so far...

[Flumminator]

It makes me think that (1) Rosetta is remarkably good, but also (2) the compiler optimization for ARM & especially Apple Silicon is apparently pretty weak so far...

I would vote for the first one. I also remember Apple bragging about that some Intel Apps ran faster on the M1-chip than on their previous generation Intel-based machines. As far as I know, Rosetta also does just-in-time compilation, so especially in a scenario like this where the same code is translated once and then run over and over again I think that "near-native performance" is certainly possible.

Regarding the compiler optimizations: Since arm64 is in use for years in probably billions of iOS and Android mobile devices, I think some effort has been put in the compiler optimizations, so my personal expectation is they should be on par with the x86 optimizations.

Maybe running Rocks'n'Diamonds tapes is not exactly a representative benchmark

I've also compiled again with -O3 instead of -O2, which seems to give another slight bit of performance gain. Here's again the measurements for my - now 180 - tapes:

Rosetta:

Code: Select all

real	1m36.794s
user	1m34.825s
sys	0m1.956s

native, -O2:

Code: Select all

real	1m21.571s
user	1m18.838s
sys	0m2.486s

native, -O3:

Code: Select all

real	1m16.836s
user	1m14.250s
sys	0m2.372s

[Holger]

Regarding packaging a proper .dmg file: Yes, the skeleton for the Mac package is already part of the repo, but the main build script is still missing, mainly because it is a big and dirty shell script with lots of hardcoded stuff. Even though it works great for me (I can package, upload and deploy a new game version of both R'n'D and also of spin-offs like Mirror Magic for all five platforms and architectures like win32/win64 with one single shell command), it really needs a lot of cleanup before I can release that beast to the public (and before it is even remotely useful for somebody else).

I had a look again, and even though it's ugly, it might work well also for others. So I will attach my current version here.

It is not as bad as I thought, and it even has a config file, so you shouldn't have to change too much in the main script, but just edit the config file.

You mainly have to specify a Git repo to fetch the code from (can be a local one, or the official one, if you don't have code changes, but just want to build the latest version for a new platform, for example). Then you have to specify a hostname and username to build the code (can also be the local host). The following is still hardcoded in the script:

Code: Select all

PROJECTS_SUBDIR="projects"
RELEASE_SUBDIR="$PROJECTS_SUBDIR/RELEASES/incoming"
PACKAGE_SUBDIR="Packages"
SHARED_LIBS_SUBDIR="$PROJECTS_SUBDIR/libs/SDL2"

That is, you should probably have a directory hierarchy like this in the home directory of the user used to build the game on the host used to build, and on the host where the packaging is initiated:

~/projects/rocksndiamonds/BUILD/ <-- game will be build here on the remote host
~/projects/libs/SDL2/ <-- SDL libraries will be copied from this location when packaging on the remote host
~/projects/RELEASES/incoming/ <-- packages will be downloaded to this local directory

For building all required SDL libraries (SDL, SDL_image, SDL_mixer with sub-libraries for MOD and MP3 support, and SDL_net) in Mac framework format, I have another dirty, non-generic script with hardcoded rubbish, which I also hesitate to release at its current state.

Same here. You will have to have a look at it to make it work (especially how to name directories and such), but then it should do.

I use "SDL2.clean" with all cleanly extracted libraries and "SDL2.build" to build them, so I don't have to extract again for a clean rebuild. Should look like this:

Code: Select all

holger@eriador:~/src/SDL2/SDL2.BUILD$ ll SDL2.clean/
total 0
drwxr-xr-x  12 holger  staff   384 10 Feb  2022 .
drwxr-xr-x   8 holger  staff   256 10 Feb  2022 ..
lrwxr-xr-x   1 holger  staff    11 10 Feb  2022 SDL2 -> SDL2-2.0.20
drwxr-xr-x@ 44 holger  staff  1408 10 Jan  2022 SDL2-2.0.20
lrwxr-xr-x   1 holger  staff    16  1 Jul  2020 SDL2_image -> SDL2_image-2.0.5
drwxr-xr-x@ 57 holger  staff  1824 30 Jun  2019 SDL2_image-2.0.5
lrwxr-xr-x   1 holger  staff    16  1 Jul  2020 SDL2_mixer -> SDL2_mixer-2.0.4
drwxr-xr-x@ 64 holger  staff  2048 31 Okt  2018 SDL2_mixer-2.0.4
lrwxr-xr-x   1 holger  staff    44  1 Jul  2020 SDL2_mixer_mod -> SDL2_mixer-2.0.4/external/libmodplug-0.8.9.0
lrwxr-xr-x   1 holger  staff    39  1 Jul  2020 SDL2_mixer_mp3 -> SDL2_mixer-2.0.4/external/mpg123-1.25.6
lrwxr-xr-x   1 holger  staff    14  1 Jul  2020 SDL2_net -> SDL2_net-2.0.1
drwxr-xr-x@ 41 holger  staff  1312  3 Jan  2016 SDL2_net-2.0.1
holger@eriador:~/src/SDL2/SDL2.BUILD$ 

I think it's best to post all that stuff here rather than sending and explaining in private messages, so others can see it, too, and questions are answered in public.

I think I should add all that stuff to the official repo soon...

Please let me know of any major show stoppers when using these scripts! And if you should have any questions, of course!

[Flumminator]

I think it's best to post all that stuff here rather than sending and explaining in private messages, so others can see it, too, and questions are answered in public.

Thank you, Holger!

I've had a short look into the scripts and my first reaction is just: Wow! That's really sophisticated and incredibly well-documented! When I look at some scripts we use at work (and a great deal of those has been written by me, I have to admit), they may not be as big, but could certainly do with more clear documentation like that.

I'll likely give it a try over the weekend, then I'll come back with more feedback or maybe even an ARM64-binary .

[Holger]

Thanks for the praising!

Just wanted to add that if you put the two files "make_dist.sh" and "make_dist.conf" (which you should copy from "make_dist.conf.tmpl" and replace some of the template values like "__HOSTNAME__", "__USERNAME__" and "__PATH__") into a sub-directoy "Scripts" inside the R'n'D directory (like "rocksndiamonds-x.x.x.x"), you can simply use the top level Makefile to invoke the distribution package script. Just have a look at all targets after "# distribution targets". For example, "make dist-package-mac" would then invoke the packaging script to build and package the Mac version of the game. (Calling "make release-all" would build, package, copy and upload the game for all platforms (Linux, Windows 32 and 64 bit, Mac, Android and Emscripten), but for this to work, all build environments for all platforms would have to be properly set up.)