"I think the most serious mistake the US could make... would be to have a calendar-based withdrawal," McCain told reporters at the NATO headquarters in Kabul on Saturday, according to AFP.
The Arizona Republican marked the Fourth of July holiday – the US Independence Day - with US soldiers stationed in the Afghan capital.
McCain, the chairman of the Senate Armed Services Committee, said the complete pullout of troops would be "an opening for the Taliban to gain great success in Afghanistan."
NATO's combat mission in Afghanistan formally ended in December last year, 13 years after it started with the stated goal of removing the Taliban from power.
US President Barack Obama has backpedalled on plans to reduce the troop size in Afghanistan by nearly half this year, agreeing to keep 9,800 soldiers until the end of 2015.
However, the Obama administration has indicated that it would remove all its personnel over the next 15 months.
McCain said that plan required a "reassessment" as Afghan forces incurred heavy losses with major cities under threat of a resurgent Taliban.
"The threat environment continues to evolve in ways that clearly, in my view, demands a reassessment of the administration's current calendar-driven drawdown of US forces with a plan that must be based on conditions on the ground," he stated.
Afghan President Ashraf Ghani, whose nominee for defense minister was rejected by lawmakers for the second time on Saturday, met with McCain in the capital.