A policy of regime change is one where a country through a wide variety of policy mechanisms, both covert and overt, seeks to remove the leadership or political power structure from another nation-state. This is a clear departure from the presumption of Westphalian sovereignty whereby what happens within the borders of a nation-state is the business of that nation-state alone.
The argument for regime change typically is that the detested government does harm to its own people and that if it were replaced with a legitimate democratic government, those harms would cease. While this is true of many governments, it is not the case in every situation. For example, the government of Patrice Lumumba of Congo-Leopoldville was corrupt and committed human rights abuses but it did not harm the people of the country and he did not lead them towards communism. In this instance a pressure campaign that focused on economic and diplomatic means could have achieved the desired result.
In cases like this, overthrowing the foreign leader is more likely to create chaos than it is to benefit American interests. This is because the imposed leaders are beholden to both their domestic audience and their external patrons who want different things. This drives a wedge between patron and protege, and it can also cause the imposed leadership to become disengaged from its own citizenry.
In addition, attempting covert regime change is rarely quick or cheap and always comes with serious consequences in the long run. The most significant of these are that it breeds mistrust in the United States among its peers and undermines its own national security interests by causing countries to be less willing to engage in nuclear arms control talks.