It would be nice to have the asserting party's full entity descriptor available at runtime.
One way to achieve this is to extend AssertingPartyDetails and have the resulting OpenSamlAssertingPartyDetails contain the associated EntityDescriptor as a member variable.
This implementation would be used by OpenSamlAssertingPartyMetadataConverter when constructing the RelyingPartyRegistration.Builder.