With CloudHSM, I’m assuming the hardware is a single point of failure? If lost, or damaged I’m assuming a new device must be issued and new keys generated, or does the fact that the hardware maintaining the private key, once lost, means you’re unable to decrypt data in order to make use of a new CloudHSM?
It is advised that you create CloudHSM in a highly available design to avoid having a single point of failure. You can do so by creating a CloudHSM cluster which spreads your CloudHSM instances in more than availability zones with your VPC. Here is the link to AWS documentation with the explanations https://aws.amazon.com/cloudhsm/faqs/