If offering the same flexibility to everybody is viable: yes. sure. don't restrict people just for shits and giggles.
But some businesses need people to be available at all times, and those businesses can handle letting some of their staff take time off on short notice or work during very specific hours only, but they can not handle giving the same consideration to everybody.
In those cases, preferential treatment for those who need it most is the right choice imo.
(Just abolishing work is, at this point in time, not viable.)