Begovic's error gifted Papiss Cisse the chance to give Alan Pardew's side the lead, but Jonathan Walters headed a leveller with 81 minutes gone before subs Kenwyne Jones and Cameron Jerome combined for a winner.
Bosnia and Herzegovina international Asmir Begovic: "I would like to have done a little bit better with that. But thankfully the guys bailed me out in the end, and we were able to get three points.
"We have been playing really well at home, so it would have been a shame to lose our run to that. After going a goal down, we really stuck together.
"The guys who came off the bench really wanted to make an impact and were fantastic.
"When they came on, they really put us on top. The guys have been chomping at the bit to get their chance, and they proved they want to be in the team. They weren't sulking. They were just helping us get the result.
"Cameron has been fantastic. He has always been an impact player ever since he came here. It's nice for him to have a chance.
"We are getting the results now that our performances deserve. It's been nice to be able to push up that table and look up rather than down."