More data:

[22:04]RurouniDonut:Deleting old vessals just made my save go from 252 mb to 6 mb

https://drive.google.com/open?id=15bLvwZiRsIiOOg602r4tfFR7fna_4v4P

162MB sfs, of which ~155MB is from 2 vessels ~1500d old.

I have managed to read the save and produce a .wl file for the vessel trajectories. The code to do this is in https://github.com/pleroy/Principia/tree/2400 (some of that is throw-away, but some will ultimately be checked in).

The output file (776 MiB...) is on Google Drive and shared with @eggrobin. Good luck trying to read it in Mathematica.

https://drive.google.com/open?id=1RDNwA2KRJU1miBiLIhgaaRehjXtOVdVf
133MB sfs, reduces to ~25MB after deleting ~15 vessels aged 0.5y-3y in various orbits around Earth, Mars & Venus.

Notable (in both linked saves): simple zipping reduces the size by a good third. So if nothing else, different compression algorithms could be worth investigating (though higher compression might come at the cost of slower saves & loads).

Notable (in both linked saves): simple zipping reduces the size by a good third. So if nothing else, different compression algorithms could be worth investigating (though higher compression might come at the cost of slower saves & loads).

I noticed the same, but it mostly comes from the fact that the save uses base64, i.e., 6 bits per bytes. It stands to reason that zipping would save 25%. A better compression algorithm (zstd?) would probably not save more than ~5%.

hypervelocity from KSP forums here, providing a hefty persistent file for your analysis. I tried uploading it to gist but website kept crashing before finishing upload. Let me know if anything else is needed or if there's any other way I can help.

https://drive.google.com/file/d/1pxOinC2T4251Q8no2SX1sds2FfulqMsU/view?usp=sharing

@damianreds: Thanks, I might try to see how the code in #2485 helps with your save.

Running #2485 on the save dated Jan 31 above. Looking only at the Principia part, i.e., the part that's encoded in base64.

Before (gipfeli, base64): 156'075'134 bytes.
After (zfp+gipfeli, base64): 42'282'108 bytes.
Gain: 3.7x.

Running #2485 on the save provided by @damianreds. Looking only at the Principia part, i.e., the part that's encoded in base64.

Before (gipfeli, base64): 83'633'693 bytes.
After (zfp+gipfeli, base64): 20'934'426 bytes.
Gain: 4.0x.

Curiously this save is very slow to deserialize (135 s), irrespective of zfp. To be continued...

@damianreds: Your pain comes from the fact that your game is in the year 2023 (I suspect that you started by warping all the way to 2020 or some such). On every scene change, Principia recomputes the history of the solar system from 1951 to 2023. That takes about 2 minutes.

Obviously that's not very smart of us, but then the alternative of explicitly storing 72 years of history has its own drawbacks (it runs out of memory on my machine, for instance).

This will need to be addressed in a future release, but it will take time. In the meantime, I recommend playing closer to the origin date of 1951 if possible.

@pleroy thank you Pascal! I very much appreciate your insights. You are correct in your assumption, I did start by warping all the way through 2020.
Don't worry about it, the fantastic experience Principia provides significantly outweighs a few mins of wait time. I will be looking forward for a future fix but this will not prevent me from enjoying your work.
Thank you & the team for your awesome work!

EDIT: @pleroy now that I think about it, is this issue first noticed now? I have been using Principia for a while now and always fast forwarded to roughly our present time and this is the first time I have experienced such significant delays.

@damianreds: The "problem" was introduced in פרנקל. In that version we started to plot the trajectories of celestials. At the same time, we dropped the capability to forget the past histories; instead the setting of the history length hides the histories but doesn't delete them. Overall this seems useful because, well, all these trajectories look nice.

We were aware that it would make saves bigger/slower, but the situation is not all that bad if you are a few years after 1951. We just didn't anticipate your use case, where 70 years would need to be integrated on each scene change.

got it, thank you, very much appreciated! :)

游戏加载的时候能直接从.sfs加载当前时间飞船的位置，而不是重新计算一遍吗？还有，能不能说让这些历史记录不放到内存里（如果必须的话放到页面文件里）来节约一些内存？我只有8G内存，加载RP-1就很耗时间、很占内存了，我不希望这个也占用我的内存，让我的游戏更卡了。

Is it possible to provide an option to forget the history over 2 weeks of some specified vehicle? I have some long-running vehicles, such as communication satellites around bodies. I don't necessarily care about their trajectories. Maybe making the history of them transient could make the size more manageable. (I would appreciate even a crappy one like making the history transient if the vehicles have some prefix in their names.)

Slowing down scene changes is actually extremely harmful for the game play, as KSP does require a lot of scene changes to do even the most basic things. Having more memory usage instead of slow down is preferable. 32 / 64 GB machines are normal this days. 2-minute scene change waits quickly add-up to the time when it becomes more money efficient to simply buy more RAM (lost time is money).

How about the following idea: have a configurable number of GB to allocate to the in-memory persistence of system state. If that number is exceeded, recompute the missing portion and drop a message somewhere "tune that particular number to get faster scene changes".

This would allow people with sufficiently powerful machines(64 - 128 GB) allocate as much as they want and not worry about slowdowns, while smaller machines (16 gb or so) still both don't crash and people know how to address the problem (get more RAM).

If you come here to comment on this issue, please provide facts: on my computer which has characteristics X, the attached save takes Y seconds to load and Z seconds to write. And please give us your save.

Unless you are an astronomer or a Squad developer, your suggestions are not going to tell us anything that we didn't know about the problem.

https://drive.google.com/file/d/1GB6LWSBCE67M0YhpufxOs5Gi44uhzTbK/view?usp=sharing

This save is around 16 years into the game. It has a mix of interplanetary probes, earth sats, lunar sats, and lunar surface sats.