After former Toon midfielder Lee Clark was sacked, Grayson led the club to promotion via the npower League One play-offs last May and his side had hovered in and around the top six during the first four months of the season.
But they have failed to win any of their last 12 league matches and are now seven points clear of the relegation zone in 18th place.
Huddersfield chairman Dean Hoyle: "We brought Simon in with the immediate aim to lead the club to promotion and he did that at the first attempt. It is an achievement for which we are grateful.
"We knew the transition to Championship football would be difficult and our targets for this season have been realistic, with retention of our status the priority.
"However, having worked so hard to get back into the second tier we have to do everything we can to consolidate that position.
"After exceeding all our expectations in the early part of the season, our results and performances as a team and from individual players have tailed off alarmingly.
"Worsening displays have accompanied a run of just six points and no wins from the last 12 league games, including some very heavy defeats, culminating at Watford last Saturday.
"While we would all have hoped for even some modest signs of improvement to begin to emerge as the weeks have passed that has not been the case. This has made the position untenable.
"We will now seek to recruit a new first-team manager who can, in the short term, lead and reinvigorate the team and players for the remainder of this season and also drive us forward towards our long-term aims."