First off, when we talk about "bad" email relaying, we are generally talking about what is called "open email relaying." This already begins to describe the problem, doesn't it? An open email relay is an SMTP (outgoing mail) server that allows ANY person with access to the Internet to send their email through it. In other words, the mail server doesn't verify that it is authorized to send mail from the email address that a user is trying to send from. This, of course, can attract spammers, which leads to the blacklisting of your IP address by many of the major blocking organizations, if you are the party with the open relay.
It is illegal to send spam through an open relay as of the Can Spam Act of 2003.
If you are not sure if your email server allows open relay, or wish to ensure that it doesn't, check out this application note.
Example:
Let's say that you just set up an email server, maybe exim that typically comes with servers with the popular cpanel control software, or qmail that typically comes with the Plesk control panel. Your server has some domains created. Port 25 is the default port to receive mail from other hosts.
Then, some other server on the Internet connects to your server on port 25. This might be another mail server running exim, qmail, etc., or a user, who wants to send mail from his/her desktop mail client, such as Outlook. In this qmail example:
The remote host identifies itself: HELO example.server.com.
Your server replies with: 250 mail.yourdomain.com.
The remote host sends the "From" part of the mailer envelope: MAIL FROM: <person@server.com>
Your server responds with: 250 ok
The remote server now sends one or more RCPT TO commands, which show the recipients of the message: RCPT TO:<person@someotherdomain.com>
But someotherdomain .com is not one of the domains that your server hosts.
If your server accepts the message and attempts to deliver it (250 ok), you've got an open relay!
If your server rejects the email (553 sorry, that domain isn't in my list of allowed rcpthosts) then your server is NOT acting as an open relay.
Click here for more articles on blacklists, bulk email, and spam issues