The extremists, including Afghans, Pakistanis, Arabs and men from Central Asia, quickly imposed a reign of terror.
In one instance, villagers were summoned to a beheading where some fainted while others froze as they watched in horror.
The Taliban, eager to expand their domestic political power, pledged to the Trump administration last year they would prevent any attacks on the West from Afghan soil after foreign troops leave.
Recent success in containing IS is central to the calculus of President Joe Biden, who decided earlier this month to pull all remaining US troops out of Afghanistan by the summer.
Mr Biden argues that threats to the West, whether by IS or remnants of the al-Qaida network, can be defused from a distance.
Yet there are concerns that in the potential chaos of a post-withdrawal Afghanistan, IS “will be able to find additional space to operate,” said Seth Jones, senior vice-president at the Centre for Strategic and International Studies in Washington.
Some note that it took more than three years to dislodge and degrade IS fighters, many of them ethnic Pashtuns from Pakistan’s tribal regions and Afghans from the northeastern Nangarhar and Kunar provinces.
The retreating militants left behind mined roads and fields.
Khan, the tribal leader, fled his village of Pananzai with his six brothers and their families at the height of the battles against IS.
They’re not rushing home, even though the family of 63 people is crammed into nine small rooms in Nangarhar’s provincial capital of Jalalabad.
“We are afraid they will return,” Khan, a father of 12, said of IS fighters.
The US invaded Afghanistan 20 years ago after al-Qaida militants, hosted by the Taliban, staged the September 11 terrorist attacks.
In recent years, Washington has come to see the Taliban as a national force, with no ambitions beyond their borders, according to a US defence official who spoke on condition of anonymity in line with regulations.
The Taliban, familiar with mountain caves and dirt paths in remote terrain, are a useful ally against IS, which is viewed by the US as the greatest threat emanating from Afghanistan, the official said.
In justifying his withdrawal decision, Mr Biden noted that terror threats are “metastasising around the globe” and that “keeping thousands of troops grounded and concentrated in just one country, at the cost of billions each year, makes little sense to me and our leaders.”
The withdrawal is under way, with the final phase starting on Saturday.
By September 11, America will have withdrawn its last 2,500 to 3,500 troops, and about 7,000 allied forces from NATO are following the same timetable.
But there are concerns about IS re-emerging, particularly if the Taliban and the Afghan government can’t reach a power-sharing deal.
Intra-Afghan peace talks remain stalled, despite US efforts to jump start them.
Ongoing fighting between the Taliban and the government could further erode the morale of Afghanistan’s 300,000-plus security forces who sustain heavy casualties daily and are plagued by widespread corruption. It’s unclear how the troops can be a bulwark against new terrorist threats.
At the same time, IS continues to recruit among radicalised university students and disgruntled Taliban, said a former Afghan security official who spoke on condition of anonymity because he was not authorized to talk to reporters.
IS has also resumed a campaign of targeted killings of minority Shiite Muslims, many of them ethnic Hazaras, as well as women’s rights activists and media workers. They claimed attacks last year on two educational facilities, including Kabul University, that killed more than 50 students.
Washington blamed IS for a brutal assault last year on a maternity hospital in a largely Hazara neighbourhood of Kabul. Infants and pregnant women were killed.
In March, seven Hazaras who worked in a stucco factory in the eastern city of Jalalabad were killed in an attack claimed by IS.
The assailants tied their victims’ hands behind their backs and shot each with a single bullet to the back of the head.
Some residents there are afraid to point the finger at IS, fearing they might be targeted next.
IS operatives are said to occupy an entire neighbourhood near the central Talashi roundabout. They have infiltrated the motorised rickshaw business and use the vehicles for targeted killings, said taxi driver Saida Jan.
Evan Kohlmann, a terrorism consultant, said for a while it appeared the IS presence in Afghanistan and surrounding regions “was all but dead,” but the group’s operations “have since resumed in earnest.”
“They represent a significant terrorist threat, but their tactics remain in the realm of assassination and sabotage,” said Kohlmann, who has worked with the FBI and the Nine Eleven Finding Answers Foundation that emerged following the assaults on America.
“They don’t seem to be in a strong position of conquering and holding territory,” or of threatening the US, he said.
The Taliban say they have made good on promises to the US by ordering fighters to keep non-Afghans from their ranks, and telling al-Qaida to leave the region. Some analysts say they’re not convinced the Taliban have distanced themselves from groups like al-Qaida.
US officials, meanwhile, acknowledge the withdrawal will reduce Washington’s intelligence gathering capacities, even if IS and al-Qaida aren’t in a position to attack US targets from Afghanistan.
Asfandyar Mir at Stanford University’s Centre for International Security and Cooperation said the US will be able to continue technical eaves-dropping from a distance, while on-the-ground intelligence gathering will weaken further.
“The US campaign in Afghanistan has been notoriously poor at getting good information and being played by rent-seeking actors, the cost of which is borne by innocent civilians in raids and strikes gone wrong,” said Mr Mir.
“With US forces out, and unable to provide security to potential informers, existing sources will dwindle and opportunities for bad actors to dupe the US will grow,” he said.
This content first appear on 9news