The Seagulls are now nine points off top spot, as their unbeaten home record came to an agonising end against the South Londoners.
Ciara Sherwood’s strike was enough to give the Lionesses the points, despite Jay Lovett’s side creating a host of opportunities to win the game; particularly in the second half.
Albion started the game brightly, passing the ball around confidently and creating a number of early chances; with Lily Agg’s 25-yard strike almost catching out Millwall keeper Francesca Gibbs.
But the away side began to look dangerous on the counter-attack; and they took the lead in the 22nd minute when Emma Whitter’s surging run towards the Albion box resulted in a ball being laid off for Sherwood, who slotted homel into the bottom corner.
The hosts were then largely restricted to half-chances for the remainder of the first 45 minutes, as Catherine O’Hagan’s curled shot went wide, before Charlotte Gurr’s looping effort sailed over the bar.
Lovett’s team came out with renewed vigour after the break; Cherelle Khassal’s close-range header went narrowly over the target, before a powerful strike by Agg almost skimmed the top of the goal.
The remaining half-hour of the match was a backs-to-the-wall job for Millwall, with Albion piling on the pressure in search of that elusive equaliser.
The Seagulls were awarded a free-kick just outside the area, but Khassal’s strike went inches wide of the post.
Faye Rabson was then brought on for the final 20 minutes and almost had an immediate impact; her superb run ended with a ball being laid off for Agg, before the visitors cleared the threat.
Agg then had another chance to score, but her fierce low drive from distance was superbly tipped away by Gibbs with two minutes of normal time left.
Then deep into stoppage time, Agg found O’Hagan in space, whose shot fired into the side netting in nearly the last kick of a highly-entertaining game.
Manager Lovett was very disappointed with the result, but could not fault the overall effort of his players.
He told Seagulls.co.uk, “We can take a lot of positives from the game. The scoreline doesn't reflect our performance. We absolutely dominated the second half; it's ultimately a disappointing result, but the girls worked their socks off.
“I’m sure the goals will come back, as the quality we have in our squad is very good.”
Albion: Mangan (GK); Cheshire, Ryan (c), James (Thompson 88), Boswell (Rabson 72); Barton, Agg, Gurr (Somes 83); O’Hagan, Stenning, Khassal. Subs not used: Sole (GK), Whitcombe.