slowdowns and parsing and hashes, oh my

Found a bug in R'n'D? Report it here!

Moderators: Flumminator, Zomis

Post Reply
Posts: 500
Joined: Fri Jun 20, 2014 10:06 am

slowdowns and parsing and hashes, oh my

Post by filbo »

Hi Holger,

This is a summary of research I did months ago and never finished until now. I was trying to figure out why startup / overhead times for different RnD releases changed as they did. This doesn't really report a 'bug' per se...

As it turned out, I tested behavior between commits cbcc8ab3 (2017-01-14) and 13c30dea (2017-10-03). Within this span, one commit caused a measurable speedup of my test runs: ff84e204 (2017-02-01 'headless mode ... for autotest'); and a later one caused a measurable slowdown, coincidentally of about the same magnitude: b270fc80 (2017-05-06 'max global animations 8 => 32').

Now, the speedup is obviously caused by exactly the thing you were doing, which was intended to speed up exactly the thing I was doing :)

The slowdown ... well, that commit is quite a bit more complex than the summary would suggest. Yes you changed the values of a couple of #defines from 8 to 32, but also added +24 strings to parse in several different places, stuff like that.

Which reminded me of running RnD under `ltrace` and observing a near-infinite number of string compares. Which made me wonder if the whole thing ought to have some sort of string hashing or ... I don't know what. Something so that when parsing a config file, instead of calling a bunch of routines which then compare the passed string to all 57 strings that routine knows about, instead says 'I have this here string' and is automatically routed to the handler for that string.

Anyway, I'm sure the slowdown here is due to additional flailing in the conf file parser. So it's organic, can't be helped unless the entire parsing machine is revamped more than you'd probably want to mess around with :) [Update: I poked around in the code and was reminded that you do have a hashing scheme. But it appears that out of the entries in the gigantic image_config[] structure, only the ones that are intended to store filenames participate in the hashing. The even larger set of integer, boolean, and non-filename string values are doomed to be looked up the hard way. Why?]

(Neither of those was the slowdown I was initially looking for, and then I got distracted off onto other things and didn't get any further than that.)
Post Reply