I agree; he's just trying to rattle you because his feelings are hurt.I don't lose sleep worrying about it. Little to no chance anyone's going to take you to court on a truthful review. Plus my attorney's license gives me some comfort.
Did he send the same message to everyone else that gave him a bad review?
The court system is already packed with bogus law suits, if they added in suing people for bad reviews, your case wouldn't be heard for about 50+ years.