Graham Potter admitted Albion were hard done by after their six-match unbeaten run was ended by Crystal Palace's 2-1 win at the Amex.
Joel Veltman scored his first goal for the club to equalise ten minutes into the second half but Christian Benteke stunned Albion with a winner in the fifth minute of stoppage time.
“It's difficult to take, a very sore one,” said Potter. “We did everything except get points and that's the most important thing so we're very disappointed.
“I think they were in our box three times and scored twice. That's the beauty of football although it doesn't feel very beautiful at the moment.
“It was quite a one-sided game in terms of pressure and chances but we didn't do enough and they scored their two – that's life.
“We got our goal in the second half and pushed and pushed and looked like the team who were going to win, so to get done at the end like that is very painful."
Albion are four points above the relegation zone ahead of Saturday's trip to West Bromwich Albion.
“The players are distraught but we have to deal with it. We'll have to keep fighting and prepare ourselves for another big game on Saturday. We've been on an unbeaten run but we have to respond now.”