Fuck AI

Maybe this will help: https://theanarchistlibrary.org/library/the-anarchist-faq-editorial-collective-an-anarchist-faq-full#text-amuse-label-seci58

I haven't internalized all of it yet.

I think this is the most direct answer:

Simply put, if people impose themselves by force on others then “they will be the government” and “we will oppose them with force” for “if today we want to make a revolution against the government, it is not in order to submit ourselves supinely to new oppressors.” [Malatesta, Op. Cit., p. 99] This applies to defending a free society against organised counter-revolution and against those within it conducting anti-social (“criminal”) activities. The principle is the same, it is just the scale which is different.

The "bunch of vigilantes" will be accountable to the people because they will be the people. They will act to reject anti-social activities, re-educate people that chose to do anti-social acts (so they have a pro-social understanding), and remediate any harms due anti-social activities.