Should You Ever Stop Following Up?