The course just mentions failover, but not recovery.
Route53 continues to do health checks on the failed primary. Once the it passes a pre-configured number of consecutive health checks the DNS will revert to the primary. See Q. When my failed endpoint becomes healthy again, how is the DNS failover reversed?