Tuncay Korkmaz was left disappointed as Deeping Rangers’ positive performance failed to produce a point.
Jordan Avis finished off a flowing move to level before the break but Liam Dolman’s double won it for top-of-the-table AFC Rushden & Diamonds.
Deeping boss Korkmaz – who won January’s manager-of-the-month award – said: “I’m never pleased to lose and they were two very cheap goals to give away.
“We created chances and had better possession. To be honest, anyone watching the game for 70 minutes would have picked us as the team at the top of the league.
“We played tremendous football and scored a great goal. But we got hit by two set-pieces where we have got to do better.
“We should have learned the lesson from the first goal when he had a free header.
“The second goal was the same again as he didn’t even have to jump but he put in another free header.
“That’s very disappointing after all the effort we’ve put into the game.There were a lot of positives.
“I thought the sending-off could have gone either wait with a yellow or red card. But the referee was very poor in both halves for both sides.”
Matt Carter is suspended for Tuesday night’s Knockout Cup tie at Yaxley.