The 48-year-old has paid for a run of two points from four games which means the Scots - bottom of Group A - have little chance of reaching their first finals in 16 years.
A seven-man Scottish Football Association panel has been considering Levein's future since late last week, and the former Hearts and Dundee United manager will not now be in charge for this month's game against Luxembourg.
Ex-Middlesbrough manager Gordon Strachan is a clear early favourite with bookmakers, while former bosses Walter Smith and Alex McLeish are also prominent in the betting.