FLHerne has a patch which substantially reduces output time with pypy in Iron Horse and FIRS compiles http://www.flherne.uk/files/nicer-quick-output.diff

I ran some tests against http://www.flherne.uk/files/nicer-quick-output.diff using Iron Horse and nml at rev 660722b

Timings are purely for 'Writing output' step, tested over a couple of runs, with primed caches.

Python 3.8

Without patch: ~1.5s
With patch: ~1.7s - but also note there's an ignored Exception "FileNotFoundError: [Errno 2] No such file or directory: './generated/iron-horse.nfo.e5kf2jss.nmltemp'"

13% slower.

**PyPy"

Without patch: ~6.0s
With patch: ~1.0s

85% faster.

The specific nml rev used is chosen because the patch conflicts with the next rev f71103b

The 5s improvement for PyPy with this patch reduces a total compile time from 30s to 25s.

I am +/-0 on whether we should include this; it's a massive boost for PyPy, but a performance regression for Python 3.8.

For context though:

• when developing newgrf, I compile frequently, 10s of times per hour, and reduced cycle times are really beneficial for staying in flow
• the ideal compile is around 10 seconds
• I've previously achieved ~30% performance increases by aggregating reductions of a couple of seconds here and there
• improvements of less than a couple of seconds aren't usually worth the candle, and tend to just be noise
• it's rare that a single large intervention be made to improve newgrf compile time; notable exceptions are:
  • the caching strategy that frosch added several years ago
  • using nmlc with PyPy instead of Python 3 for (around 50-60% faster)
  • aggressively substituting grfcodec for nml where only graphics have changed

The slow performance of StringIO is fixed in PyPy 7.3.1.