In AggregateReferenceDeserializerModifier we now look up the original property name on the Jackson metadata before trying to resolve the property target type. We also explicitly guard against null values returned from that lookup and opt out of the customization in those cases.