“There’s a huge loyalty factor, but it’s not as strong as it was,” billionaire investor Warren Buffett says.