Manchester United 2 Newcastle 1
West Brom 1 Liverpool 2
QPR 2 Reading 3
Arsenal 6 Coventry 1
Carlisle 0 Tottenham 3
Norwich 1 Doncaster 0
Portuguese star Anderson lashed home a superb 44th minute opener and when Tom Cleverley added the home side's second in the 58th minute, Ferguson's side seemed to be home and dry.
But Magpies substitute Papiss Cisse came off the bench to reduce the deficit in the 62nd minute and almost grabbed a stunning equaliser when his overhead kick struck the bar.
Arsenal heaped more misery on npower League One crisis club Coventry with a rampant 6-1 win at the Emirates Stadium.
The Sky Blues kept the Gunners at bay for 39 minutes until new boy Oliver Giroud netted his first goal for the club to give Arsene Wenger's men a slim half-time lead.
Goals from Alex Oxlade-Chamberlain, Andrei Arshavin, Ignasi Miquel and a double from Theo Walcott sealed Coventry's fate after the break, while Callum Ball's goal gave them some consolation at 4-0.
Jerome Sinclair became Liverpool's youngest ever player after coming off the bench for the last nine minutes of his side's 2-1 win at West Bromwich Albion.
Nuri Sahin sent Brendan Rodgers' men through to round four with an 82nd minute winner after the Turk had earlier equalised Gabriel Tamas' third minute opener for the hosts.
But it was Sinclair - aged 16 years and six days - who stole the headlines when Rodgers opted to send him on moments before Sahin's winner in place of German Samed Yesil.
Tottenham avoided any mishaps in Cumbria with a 3-0 win over Carlisle, the goals coming from Jan Vertonghen in the 37th minute, a stunning Andros Townsend strike on 53, and a late third from Gylfi Sigurdsson.
Reading enjoyed some relief from their tough start in the Barclays Premier League as they twice battled back from behind to win 3-2 at QPR.
David Hoilett's early strike for the hosts was cancelled out by Kaspars Gorkss, only for Djibril Cisse's 71st minute penalty put Rangers back in front.
But the Royals responded in style with Nicky Shorey equalising five minutes later and Pavel Pogrebnyak - who also missed a late penalty - netting the winner nine minutes from time.
Chris Hughton's youthful Norwich side edged past League One Doncaster 1-0 at Carrow Road thanks to a 26th minute goal from full debutant Alexander Tettey.