Do not run ValidReferenceConstraint on removed items.

This solve the issue with Entityqueue Dragtable Widget triggering a validation error on removing references that lost their acceptance criteria in the queue.

For example if a published node was added to a queue and was un-published it will not be removable from the queue and will prevent the whole queue from being edited/saved, even if you are trying to remove it to fix the issue.

This solution was inspired from entity_browser module Entity Reference Browser Widget [Drupal\entity_browser\Plugin\Field\FieldWidget\EntityReferenceBrowserWidget.php]

Closes #2823942