There can be a few different issues here. Lets try to briefly cover them.

1. If you open up messages in the badmail directory and you see something like:

Diagnostic-Code: smtp;554 Service unavailable; Client host [xxx.xxx.xxx.xxx] blocked using dnsbl.njabl.org (or some other Open Relay Database )

you are being labeled as a spammer by an Open Relay Database. Contact the Open Relay database that has blocked you for more information.

2. If you see a message like:

Unable to connect to transport….

Your mailserver cannot get out to the network, or else it cannot resolve DNS names. Using nslookup, verify you can correctly resolve DNS names, and perform MX record lookups on the sever. If that works, you may want to either try forwarding your mail to a smart host, or sending directly (depending upon which setting you have checked in the SMTP Service properties).

A smart host, is a mail sever that can forward mail on behalf of your server. If you have an additional mail server available, enter that mail server's IP address, or DNS name in the smart host text box. To get to the smart host text box:

• Open the IIS Admin MMC
• Right-click the Virtual SMTP Server, and select Properties
• On the Delivery tab, click the Advanced button.
• Under the Smart Host textbox, enter your smart host.
• Either check/or uncheck The "Attempt direct delivery before sending to your smart host.".
• Close all dialogs.

You may need to stop and restart the services after changing these settings.