The only person you are being unfair to is yourself by second guessing your right to control your own body. Just because you have had sex with this person in the past does not mean you owe them sex in the future. Just because your relationship started out as a sexual one does not mean it has to continue to play out that way. Sex should never feel like an obligation or debt, it should only ever be something you really want to do. That said, are you sure that this guy expects sex or is it possible that he is in the same position as you are, that is are you sure that he doesn't think YOU expect sex? Have you tried talking to him about your feelings? You may be surprised to learn that he feels exactly the same way.
All too often sex confuses the issue between people and when you have sex too soon it is much too easy to feel close to somebody who you are not really close to at all. Sex taints a new relationship and makes it feel more serious than it may truly be, this can cause feelings of anxiety and regret much like the feelings you are expressing. In this situation you absolutely must slow things down. If either one of you values the relationship at all you need to stop having sex until BOTH parties are comfortable with it. If one of you doesn't really want to be doing it or if one of you is pressuring the other it will impact the relationship in a negative way. The only time sex has any place in a relationship is when both people want it, if only one person wants to be having sex the door for resentment is left wide open. The person wanting sex may come to resent the fact that their partner doesn't and the person not wanting sex may resent the person whom they feel is pressuring them. Resentment is poison to any relationship, to a new relationship it is lethal.
If you want this relationship to last you should stop having sex and tell the guy how you feel. You should be open and honest without being mean. Don't use the word "mistake" when describing how you feel about having had sex too soon, instead choose your words carefully. Describe things as "feeling rushed" or say that "sex is clouding your getting to know eachother", but do not make it out to seem that the sex you shared was a bad thing. Make it clear it is the fact that it happened too soon that makes you feel regretful, not that it happened at all. Then tell him what you want, how you see things going between the two of you, and reassure him that you really do like him. By staying true to yourself you are being fair to everyone!