Office Friend?

I'm working at InterContinental Hotels Group one day and I'm out on a break or something. I get on the elevator to go to the lobby and I start having a conversation with a guy on the elevator. He's someone I vaguely recognize from Worldspan, but I'm not quite able to place him. I'm not sure what we were talking about, but it was an interseting conversation. I continue to talk with him after we reach the lobby and we end up walking over to another elevator and getting in. I notice that this elevator doesn't go up to the 24th floor, but stops at the 8th floor. I tell the guy I'm talking with that I'll just go up with him and then head back down. We continue talking and when we reach his floor he asks if I'm going to the meeting with him. I find that somewhat odd, but I say that I'm just going to go back down. Later that day I get an email from the guy. He basically asks if he had fun today following him around and that I can't do that anymore. He says he works for Hilton and it's just not appropriate to be going around with me. I'm offended by the email because I had no intention of following him to whatever meeting he was going to. I was just enjoying the conversation. I send him back a note saying basically that. I write the whole thing off and prepare to leave for the day. On my way out, I see the guy. I ignore him and start heading out to my car. As I'm going out, another guy yells out to me that he wants to play some racquetball sometime soon and he wonders if I'll join him. I say sure. At that point, the guy I was talking to in the elevator comes over and appologizes. Apparently the fact that I'm doing something with someone else doesn't make me a desperate fool looking for a single friend. That makes it okay for us to be friends. I say okay, but the whole thing leaves a bad taste in my mouth.