As you mentioned, defining “anarchism” is already a challenge that can make discussions even amongst seasoned anarchists a grueling uphill voyage. But I think in this conversation a larger semantic issue must be addressed before your more specific rationale can be addressed. The semantic question here is, “What is the State?”
Now it seems to me like your approach to answering this question is to look at what states do. States tax people, they establish a judicial order, they administer public services, and specifically regarding your response, states incarcerate people. I can understand why you, along with many anarchists themselves, would critique the State based on what states do. After-all, if anarchists are going to fight for the abolition of the State, it would be useful to catalogue all the things that would be abolished along the way.
However, as one of many diverse anarchist thinkers, this is not how I approach the State myself. When I think about the State, I am thinking less about what states do and more about the history of states and how those histories describe what the State is. From this historical perspective, one of the more common themes is that states come into existence through conquest. This isn’t the only way that states are formed, as has been discussed by Peter Gelderloos in “Worshiping Power”; but, it’s the example I’ll use here.
As a historical entity, the State can be a Monarchy, a People’s State, a Republic, a Theocracy, etc. without fundamentally changing what makes the State distinct. And that distinction is that the State is always the first and only absolute power: the Sovereign. Before the land and other property rights that it grants its subjects, the particular taxes it imposes, the forms of law that it adopts, etc., the State establishes itself as the sovereign power.
So above all else, what we anarchists (at least some of us) seek to abolish is this sovereignty itself. This is important because following from this conception of the State, one can understand how an anarchist may be against the State’s prisons, but not necessarily against prisons created by …say, an autonomous commune. If the State is imagined as the sum of its functions, then it becomes quite difficult to imagine how all of those functions it currently positions itself to manage would be carried out without it. Fortunately, that isn’t the case!
I could elaborate on how this different conceptualization of the State changes the way that the State’s “withering away” is then imagined. There are also many other issues (such as the issue of Capitalism) that this different conceptualization impacts. But I think it’s sufficient to begin here, where our perspectives seem to initially diverge and save the rest for some other time.