Many virtual reality experiences create a sense of embodiment and self-presence through a self-avatar, allowing users to interact with virtual environments and others. %One part of the avatar that is 
At a minimum, avatar hands are consistently visible from the first person perspective. When it is not desirable for users to customize their avatar, how should such hands appear? We describe three mixed-method, exploratory studies: two online  (N = 515, N = 322) and one in-person (N = 34), investigating user attitudes toward avatar hands that were consistent or discrepant with their physical appearance. Participants were randomly assigned to view avatar hands with different skin colors. Across studies, we increased the level of embodiment. Participants' attitudes towards avatar appearance varied by context, but experience with avatars was the most influential factor in accepting discrepant avatars. Additionally, we highlight criteria delineating different contexts and how social attitudes and VR perceptions affect comfort with self-discrepant or self-consistent avatars.