Fair use, in the US anyway, is based on four factors (source):
(1) the purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes;
(2) the nature of the copyrighted work;
(3) the amount and substantiality of the portion used in relation to the copyrighted work as a whole; and
(4) the effect of the use upon the potential market for or value of the copyrighted work.
Not selling fan art helps to bolster factor #1, though that alone isn't enough. Fair use itself would need to be determined by a judge, but ensuring the work is transformative enough and doesn't disincentivize someone from purchasing the original work is probably enough commercial or not, but noncommercial theoretically should help.
(Not a lawyer, but I've followed this a bit)
Edit: note that fan art could be fair use but violate a trademark or other similar protected mark.
This is not generally true. The fan art also usually needs to be sufficiently transformative, and could still be violating, for example, if a character is widely licensed.
Fair use is really complicated. Usually it's better to see if the copyright holder has any public policies on community creations, like WOTC's fan content policy.