Championship high-flyers Newcastle were left to concentrate on their promotion bid after losing 3-0 at League One Oxford.
Kane Hemmings fired the U's in front two minutes into the second half.
Aleksandar Mitrovic had the chance to level from a penalty on 67 minutes, but the Serbian's spot-kick was saved by Simon Eastwood.
With nine minutes left, Curtis Nelson headed home from a corner and Antonio Martinez nodded in a late third.
Vanarama National League side Lincoln pulled off another FA Cup shock as they knocked Sky Bet Championship leaders Brighton out in the fourth round with a 3-1 win at Sincil Bank.
Premier League Tottenham needed two late goals to beat League Two Wycombe 4-3 at White Hart Lane, while Liverpool lost at home to Wolves.
Lincoln, who beat Ipswich after a replay in round three, had to come from behind after Richie Towell put the Championship side ahead on 24 minutes.
The Imps, currently top of the National League, were handed a way back into the tie on 57 minutes. Glenn Murray grappled with Robinson, and Alan Power converted the spot-kick past substitute goalkeeper Casper Ankergren, who replaced the injured Niki Maenpaa.
An own-goal from Seagulls defender Fikayo Tomori, who is on loan from Chelsea, put the non-league side in front on 62 minutes.
Robinson added a third on the break to secure Lincoln's place in the FA Cup fifth round for only the second time since losing to Glasgow Rangers in 1886-87.
Wycombe went ahead at Spurs on 23 minutes when captain Paul Hayes fired a volley past Michel Vorm.
Spurs, who showed nine changes from the starting XI which drew at Manchester City, were stunned again when Hayes converted a penalty in the 36th minute.
Son Heung-min pulled the Premier League side back into the tie on the hour and Vincent Janssen levelled when the Dutchman slotted in a 65th-minute penalty.
However, the Chairboys - fifth in League Two - silenced the White Hart Lane faithful once again as Garry Thompson headed home with just seven minutes left.
England playmaker Alli, though, dashed dreams of a massive FA Cup upset before Son completed a dramatic victory in stoppage time.
An afternoon of unexpected results had started with Wolves' win over Liverpool at Anfield in the lunchtime kick-off.
The Sky Bet Championship side were ahead after just 52 seconds when Richard Stearman glanced in a free-kick at the far post.
The Reds, who were beaten in their EFL Cup semi-final second leg by Southampton on Wednesday night, fell further behind just before the break when Andreas Weimann, on loan from Derby, slotted in after a swift counter attack.
Divock Origi pulled a goal back for Jurgen Klopp's much-changed side with five minutes left, but Wolves held out for the closing stages to claim another Premier League scalp after knocking out Stoke in round three.
Theo Walcott's hat-trick helped Arsenal coast past Southampton 5-0 in the late game, with Danny Welbeck also scoring his first goals in eight months.
Welbeck netted twice in just his fourth outing since a second bout of serious knee trouble, before Walcott stole a treble as the Gunners thumped the lacklustre EFL Cup finalists at St Mary's.
Welbeck's double strike offered hope for a future resurgence, and the end to two years dogged by knee problems.
The former Manchester United forward last troubled the scorers in Arsenal's 1-0 Premier League win over Norwich on April 30. But on the south coast he appeared close to full throttle.
Arsene Wenger watched Arsenal's cakewalk from the directors' box, serving the first of a four-match touchline ban for pushing fourth official Anthony Taylor during the frenzied 2-1 win over Burnley last weekend.
There was, though, no such trouble for Premier League leaders Chelsea, who saw off Brentford 4-0 at Stamford Bridge.
Willian opened the scoring on 14 minutes with a fine free-kick, before Spanish forward Pedro made it 2-0 before the break.
Substitute Branislav Ivanovic - who is reportedly set for a move away from Stamford Bridge before the end of the transfer window - added a third on 69 minutes and Michy Batshuayi scored a late penalty.
Manchester City ran out 3-0 winners at Premier League strugglers Crystal Palace.
Winger Raheem Sterling broke the deadlock just before half-time, with Leroy Sane and Yaya Toure, in the last minute, completing a comfortable afternoon in south London for Pep Guardiola's men.
Middlesbrough saw off League Two Accrington Stanley at the Riverside Stadium with a 25-yard effort from Stewart Downing in the second half.
Burnley beat Bristol City 2-0 at Turf Moor, where Wales forward Sam Vokes and Belgian Steven Defour were on target.
Huddersfield won 4-0 at Rochdale. German forward Collin Quaner, signed from Union Berlin, scored on his debut before a penalty from Isaiah Brown and a late double for Michael Hefele secured the Terriers' place in the last 16.
Blackburn beat Blackpool 2-0, as the visitors finished with 10 men at Ewood Park.
Sam Gallagher put Rovers ahead on nine minutes, with winger Elliott Bennett doubling the lead in the 22nd minute before Blackpool midfielder Kelvin Mellor collected a second caution midway through the second half.