The clubs are just 10 miles apart but Hughes said: "The game is only a derby because of the distance between us.
"But it doesn't compare in terms of intensity to other derby games.
"There will obviously be an edge because you get that in every Premiership game but I don't sense any great feeling between the two clubs that make it a big derby."
But Allardyce insists Hughes will soon be eating his words.
He said: "I'm surprised Mark doesn't think this game is a real derby.
"He only played big games for Manchester United and, although he had a spell as a player at Blackburn, he doesn't know about this derby.
"Blackburn are our closest Premiership rivals and it certainly matters to the two sets of supporters."