There are about 24-25000 tapes in the collection.
It uploaded some, I'm not sure how many (maybe stdout.txt & stderr.txt could be rotated through a few .1-.9 names?)
... anyway, something like 6000(?) of them. Then died with a timeout error.
I restarted and ... it's re-uploading everything from the beginning. Which means I'll need to be lucky enough to have 25000 uploads succeed with zero errors. (If it happens again, I suppose I'll be manually moving already-uploaded directories out of ~/.rocksndiamonds/tapes, and hoping it doesn't confuse the process...)
Suggestions for improvement:
- if a single upload fails, retry it at least a couple of times
- keep a log or something, so that next session can start from where it left off
- when the automatically prompted initial upload session fails, serversetup.conf 'ask_for_uploading_tapes' should not be turned off
- when any upload session fails (including things like sudden power off), the game should prompt to continue next time
- this implies something like a commit log (starting with a flag committed to disk: 'I have an upload in progress, here are the parameters')
- commit log doesn't need to be aggressive; this isn't a filesystem. If the upload just leaves itself a note once a minute, it'll only have to re-upload up to a minute's worth next time
- actually, could just log upload-completed levelset names; next upload can walk the entire namespace again, quickly skipping through ones already in the log. ('complete' directories could be scanned for tape files with newer timestamps than the start of the log, to interact properly with a player who might still be adding tapes during a series of partial uploads. I'm 100% ok with using filesystem timestamps for this purpose; if someone wants to play shenanigans, there isn't really anything they can do. Marking files as 'new' makes them re-upload, but so does restarting the process. Marking them 'old' makes them not upload, which might be useful in some case, but certainly doesn't harm the server...)
- keep stdout.txt, stderr.txt from recent runs; this could be a series of backup files .1 through .9; or append rather than overwrite, starting each run with a header in those files ('2021-11-07 01:04 -0700 Starting Rocks'n'Diamonds 4.3.0.0') (if appending, should still probably rotate the files when they get too big -- I'm thinking maybe 10MB -- but in that case only a single .old is needed for each one)