I think it is a very personal decision, based on what types of specific bacterial and/or fungi someone is dealing with. As a general rule, I regard yeasts like Candida albicans as likely suspects in negative reactions to sugar. Reducing starchy foods can help, as reducing refined carbohydrates. That might give a little more room for a sugar treat, but ultimately I believe it’s important to pursue a balance of gut bacteria that counterbalances the yeast. For example, skipping the rolls with dinner might reduce the reaction to dessert but it’s likely to catch right back up to where it was. Skipping desserts/sugar, refined carbs, and starchy things for a few weeks while eating lots of fiber and maybe some quality probiotics might help address the underlying imbalance that caused issues in the first place.