Use EntityViewBuilder #pre_render and #post_render callbacsk to track recursion.

core/lib/Drupal/Core/Entity/EntityViewBuilder.php

...n/Field/FieldFormatter/EntityReferenceEntityFormatter.php

core/lib/Drupal/Core/Render/Renderer.php

core/modules/comment/src/CommentViewBuilder.php

...c/Kernel/EntityReference/EntityReferenceFormatterTest.php

core/tests/Drupal/Tests/EntityViewTrait.php