On seccond thoght, having a "don't care" value for the affinity calculation makes no sense when it's not an average but just a sum. Either the affinity should be average of the tiles (in which case "don't care" means the tile doesn't count in the calculation, as opposed to a zero value which draws the average towards zero), or there shouldn't be any "don't care" value.

Perhaps the treetypes should also have a bitflag set of tree types they can mix with on the same tile. I don't know (yet) how trees present on a tile are actually handled when there's more than one, but gameplay behavior around it seems to be that it's completely random what any second and third tree are.

It does make sense to use a "don't care value". But it is needed to use a threshold which is equal or higher than the sum of the "don't care" values.

Just use or interpret it as a (signed) afinity of a tree towards the landscape types. Then calculate the sum over a field (say 3x3). Then a negative afinity will push trees away from borders with a landscape they don't like - and 0 is a nice value for "don't care", too

My own idea for diameter has been 5 or 7 tiles, maybe circular instead of square pattern, with a weighting pattern tuned towards Euclidian distances rather than Manhattan distances.

Such calculation has to be run for every tile in the tile loop. I fear that computationally expensive averaging or summation over larger areas will have quite a performance impact... as such I'd start low. Same reason to prefer manhatten over euclidean distance scale - especially as the impact on the distance is minor.
(Yes premature optimization is not good :P )

The summation is only performed when a new tree is planted, which isn't too often. The distance handling may as well be done with a static tile of weights, where inner tiles are multiplied by a larger value. I don't think the performance impact would be significant.

One more random idea: Also allow weights to be given to "tile slopes in $cardinal_direction", to allow certain trees to be more/less prevalent on slopes, or even on only hillsides facing certain directions.

Although this would be nice to have, it isn't something we expect to fulfill in the next year, and on that basis I'm closing it. We do this to keep the project manageable, productive and fun. We hope you do understand. Thanks for contributing though! Here you can find more about how we handle feature requests.