As I have studied philosophy, a consistent theme at least among modern philosophy is the denial of something fundamental. This used to cause me great anxiety because I needed to defend these fundamental things. However, after awhile it came to a point, where these are not things that need to be defended from theories, but the things that need to be explained by theories. If a theory cannot explain the primary data of experience and indeed demands we recast it as illusion, the theory has failed, false and more than likely self-contradictory.
Consider the following argument: Everything is made of atoms. Atoms are mostly emptiness. Therefore, everything is made of mostly emptiness. This is taken as an argument against the solidity of things. Things are not solid because they are composed of things lacking solidity. The first response is that things are solid and that the theory is deficient if it cannot explain solidity. In fact if I cannot reliably depend upon my perception of solidity, then I cannot reliably depend upon my perception of some measurement device.
Solipsism falls into the depths of the self because the other cannot be known for certain. All sorts of arguments are proposed in its favor. All of these hold no weight on the first count that I indeed experience as primary data an interaction with other minds. So too with the denial of free will. My primary experience is my free will. This is something that needs explained, not dismissed.
In contrast my primary experience of the stars is lights in a moving sphere around earth. This seems at first strange to us, but looking into the sky this is my primary experience. However, the explanation that is given about the true nature of the stars is also capable of explaining my experience of them. My primary data is not illusory and positively erroneous, rather it is too coarse and limited.