A penalty from Mikel Arteta broke the deadlock before Marouane Fellaini hit the second to double a deserved and comfortable lead, but a goal from Steven Taylor just before the break threw the Magpies a lifeline.
A second-half finish by winger Damien Duff brought the scores level in a game of two clear halves.
Desperate to win their first home game of the season Everton looked to have found the battling qualities of last seasons fifth-placed finish as David Moyes' side flew from the kick off.
Efforts from Steven Pienaar, Arteta and Yakubu had the visitors pinned on the back foot and a goal looked almost a certainty.
The breakthrough came on 16 minutes when Leon Osman played through a ball to on-rushing Leighton Baines inside the box.
The wing-back raced to the by-line before being hacked down by a late Nicky Butt sliding tackle and referee Howard Webb had no hesitation in pointing to the spot.
Despite offside protests for the Osman pass the decision stood and Everton captain Arteta calmly stepped up and sent Newcastle keeper Shay Given the wrong way.
The visitors felt they should have equalised moments later when a long-range effort from Danny Guthrie pinballed around inside the box and fell at the feet of Taylor who unleashed from ten yards, but a well-placed Baines on the goal-line took the full force of the ball on his chest.
Wave after wave of Everton attack tested the shaky looking Newcastle backline and the lead could have been doubled just after the half-hour mark when Pienaar whipped in a cross to the near post, but a lunging Yakubu couldn't wrap his foot around his volley to steer it goalbound.
The lead was deservedly doubled a minute later after a clever backheel from Pienaar found Baines on the touchline, who quickly curled in a cross to the front post where an on-rushing Fellaini got in front of his man to volley his first goal for Everton.
Against the run of play and deep into added time the Everton lead was cut in half after Geremi was given time to flight in a high cross to the back post, where Taylor had ghosted in to head the ball against the bar and over the line taking his side in at 2-1 down.
The half-time team-talk from Magpies boss Kinnear must have still been ringing in the ears of his players as the black and whites equalised within a minute of the restart.
A baffled Everton backline stood with arms in the air for offside as Taylor chased down a through ball, but he carried on to pick out an unmarked Duff inside the box, who sidefooted past a stranded Tim Howard to level the scores.
Newcastle keeper Given pulled off the save of the game on 65 minutes after a battling Osman won the ball deep in the visitors half and after a one-two with Arteta, the ball fell kindly to Osman in the box and his snap shot was heading for the top corner before the keeper acrobatically tipped it over the bar.
A tactical change from the Newcastle bench saw Shola Ameobi replace Geremi and the change nearly paid immediate dividends as the striker found himself one on one with Howard.
The keeper could only parry the ball to the feet of Charles N'Zogbia who looked certain to score, but Everton once again had Baines to thank for a goal-line block to keep the scores level.
Efforts from Ameobi and two late strikes from Blues substitute Victor Anichebe could not find the net and the home side left the field disappointed at having squandered a two-goal lead.