Alright, this one was an adventure for me. So this info will be here for the next time I need to set up WCF in a virtual directory. This tutorial will start from the beginning, which is installing IIS. This is assumed that Visual Studio 2010 is already installed. So let's get started with the list:
1. Install IIS through Programs and Features. If you're feeling lazy, just try to install as much of the IIS options you can check. Otherwise, you just need ASP.NET under World Wide Web Services. This will auto-check dependencies as well. And IIS Metabase under IIS 6 Management Compatibilty. Also, you will need to go under the Microsoft .NET Framework 3.5.1 section, which is separated from the IIS section btw, and check both boxes under that. They should have the description with Windows Communication Foundation within it.
2. Make a new wcf service project or open an existing one. Go to it's project properties. Under the Web tab, select he use local IIS option. When you save, it will create the virtual directory for you. There are other ways to deal with the virtual directory like publishing, or adding it directory in the IIS manager. It is up to you.
If you get an error when you try to save the project properties, check if you have all the IIS components installed. VS2010 does a good job telling you what you need anyways.
Hmmm, that seemed to be a short list. But here are some problems + fixes that you may have to encounter.
Errors you may have encountered:
1. "Something about having a duplicate handler in your web.config". What happened here was that I tried to add a handler myself before installing the WCF stuff under Microsoft .NET Framework 3.5.1 section. Therefore, IIS couldn't even populate it's integrated handlers, let alone, let me add any more handlers. So I had to edit the web.config manually by copying it outside it's original directory, removing the handler, and then moving it back into the original directory. It will not let you edit it if you open it within the original directory.
2. 404.17: The requested content appears to be script and will not be served by the static file handler.
Check if you installed all the necessary components for IIS and WCF. You may have to run aspnet_regiis.exe -i under the command line if you have ASP.NET 4.0. After doing this, WCF should run smooth and you should be able to see the .svc file in your browser as intended. This solution may be different if you are running an older version of ASP.NET.
3. Make sure the application pools between the website and your virtual directory are ASP.NET 4.0. I haven't thoroughly tested this one out yet. I changed the website to ASP.NET anyways just to be safe.
Local Host Stuff
If you tried to access your IIS or .svc from another computer, you may find that it will not connect by timing out or what not. This may be caused by the firewall. If you are using Windows Firewall, find and enable the inbound rule for World Wide Web Services. The port should be 80.
When on another computer, you may not want to type in 192.168.1.23 just to get to connect to your service. You may want to type in http://MySite instead. This will be controlled in a DNS Server. The DNS settings should be located in your router firmware.