When the Taliban emerged in 1994 and took control of Kabul in 1996, the US developed a cautiously positive view of them. During most of the 90s, at least until 1998, it considered Iran (and Shias), but not Al-Qaeda (and Sunnis), as the threat. Moreover, the US regarded the Taliban as relatively safer “traditional Islamists”, as opposed to “pan-Islamists” who were considered a greater threat because of their global aspirations. Although it categorized Al-Qaeda under pan-Islamists also, the US considered Shias supported by Iran to be much more dangerous.
Thus, it placed its hopes in the Taliban to help it against Iran. The fact that the Taliban were mostly Pashtuns added to their appeal, due to the historic Persian-Afghan rivalry. This perception was further strengthened by the initial negative reaction of pan-Islamist mujahideen groups, toward the Taliban.
However, when in August 1998 Al-Qaeda carried out lethal attacks against US embassies in Kenya and Tanzania, the US started taking Taliban as an imminent and serious threat. The positive (or at least acceptable) view of the Taliban that had existed in the US so far started eroding, as they saw that Al-Qaeda and its leader were using Afghan territory as a safe haven under Taliban’s protection. Around the same time, concerns about human rights, especially the treatment of women by the Taliban, started getting more visibility in the US and the West. Finally, the October 2000 suicide attack by Al-Qaeda against USS Cole, a destroyer of the United States Navy in Yemen's Aden harbor, ended any hopes still held by the US of using the Taliban for its purposes and accepted their loss to Al-Qaeda.
However, the US continued to engage with the Taliban in negotiations via Pakistan, to stop them from permitting the use of Afghan territory by Al-Qaeda and other global jihadists, but nothing came of it. Then 9/11 happened, and after the Taliban refused US demands to hand over Osama Bin Laden (OBL), the US led an international coalition to attack and remove the Taliban from the government.
The main goal of US in Afghanistan has been to ensure that Afghan territory is not used by terrorists against the US and to eliminate Al-Qaeda hideouts along with its leadership there. In line with its policy of liberal hegemony, the US, with military action, helped the creation of the liberal Afghan state.
Lately, from 2019, the US started negotiating with Taliban, after it changed its foreign policy from liberal hegemony to ‘America First’. Under this new policy, it has become irrelevant for the US whether there is a democracy or not in another country. Its only reason to intervene outside its borders has become the security of US interests. Liberal hegemony had tied US security with liberal democracy. ‘America First’ untied them. The US considered Taliban minus Al-Qaeda was not its problem. Taliban found no problem in accepting severance of ties with an almost non-existent organization. The result was the US-Taliban deal.
The US expects Taliban, sans Al-Qaeda, to return to being what they were originally, that is, traditional Islamists with no global agenda. They also hope for them to be a force against the pan-Islamist group IS, and against Iran, which has regained its status as the top threat to US interests in the Middle East.
The big question is how sure can one be that Taliban will not get close to the IS, as they did earlier with Al-Qaeda? How much can be counted on the second and third tiers of Taliban command to not join IS even if there is no truce/alliance between their organizations.
The IS-Khorasan, as the IS in Afghanistan is known, is mostly composed of former Taliban members. Moreover, there has been no violent conflict between Taliban and IS in the post-deal time, while there are reports that the two groups carry out low-level collaboration when either of them attacks Afghan Government or civilian targets. It is also known that the IS founder had stayed in Afghanistan before moving to Iraq, after the US invasion of Iraq in 2003, to form ISIS (Islamic State in Iraq and Syria).
Following a faulty negotiation process leading to a faulty peace deal, and considering post-deal statements/actions, there is a likelihood of US leaving Afghanistan, on schedule or earlier, and for Afghanistan to then enter in even more violent conflict and have pockets which will serve as safe havens for global jihadists. Alternatively, there is also a real chance of US getting bogged down in Afghanistan with even more forces being deployed there.