You’re a smart guy, figure it out!

Mike Grushin’s thoughts on everything tech-related and more…

Archive for the ‘Web Development’ Category

SMTP Relays – a way to avoid SPAM/Bulk email folder

Posted by mikeg on December 3, 2009

Every web application sends emails: registration, forgot password, reminders, etc. How do you make sure that these emails get into your user’s Inbox? There are 2 ways: “free” do-it-yourself approach is described at the end of this post. The easier, but not free, approach uses an SMTP Relay Provider.

These providers usually have relationships with major web-based email solutions (gmail, yahoo, hotmail, etc) and also follow the latest standards and best practices for making sure that your email is delivered. Some of the things these companies do is outlined in the DYI portion.

SMTP Relays should not be confused with “email marketing” offerings. Email marketing offerings usually specialize in List Management, opt-in/opt-out, tracking, etc. Most of SMTP Relays do not offer these features.

Here is a list of the providers that I found while researching this:

SMTP Relay Providers

www.unifiedemail.net

  • We implemented this for a customer (their choice) and had no issues with it. Implementation was easy, pricing seems reasonable.
  • Site is a bit sparse on information but customer reported that phone support was prompt and helpful
  • You can find pricing here, but to give an example: with 5K daily limit/75K monthly limit you will pay $99.95/month

www.sailthru.com

  • NYC-based, recommended by a few people on nextNY list
  • Pricing: $6 per thousand of emails, $60 monthly minimum, discounts after 500K
  • Seems to offer a few additional features on top of regular relay: “Triggermail — Our flagship service monitors delivery of transactional email into the user Inbox. If it gets sent we’ll tell you when it was delivered and what happened when it was opened.”

www.authsmtp.com

  • Another one that was recommended by a few people on nextNY list
  • When looking at pricing keep in mind that listed limits are monthly, but the price is yearly. Here are monthly prices to make it easier to compare to other services:
    • $2/month for 1K emails/month
    • $14/month for 10K emails/month
    • $134/month for 100K emails/month
    • $667/month for 500K emails/month
    • goes up from there

www.socketlabs.com/od

  • Pricing:
    • $79/month for 25K emails/month
    • $249/month for 100K emails/month
    • $449/month for 500K emails/month
  • You do not get a dedicated ip with the first 2 plans. Not sure how big of an issue as long as SocketLabs keeps their IP range clean

www.smtp2go.com

  • Looks like they target consumers as their highest plan allows for up to 300 emails a day
  • They offer “corporate” product (https://smtpcorp.com/signup/), but it still doesn’t look like an STMP Relay product

www.port25.com

  • Pricing is not listed on the site – you have to contact them. Hard to believe that companies still play “personalized quote” game

www.sendgrid.com

  • If you use this link you can also get 25% off the regular price.  So check them out and give SendGrid a try.

DYI Approach

Very good post on this topic can be found here: How to ensure your email gets delivered

This covers email marketing techniques, but applies to this discussion as well: How can I prevent my emails being marked as spam or from being blacklisted

Note: This checklist below is a modified version of a post from Oct 2007 by Tobin Schwaiger-Hastanan on Business of Software List.

First steps:

  • Make sure your mail host is a valid A record
  • Make sure your MX record is set to the full host name of your mail host (which is set up as a A record)
  • Make sure you can do a reverse lookup of your mail hosts IP address.  (This will be important for setting up SPF/Sender ID records)
  • Set up SPF/SenderID information as a TXT record. (www.zytrax.com/books/dns/ch9/spf.html)

Once above is configured

  • Verify DNS/MX records using http://www.dnsstuff.com/. Make sure there are no warnings or errors before contacting support for any of the email services. Yahoo has a policy of not reviewing changes for 6 months after the last request is made
  • Test your emails with Spam Assassin.  Send yourself an email from your web application and copy the raw message (including headers) to a text file and run it against spam assassin’s command line test. The lower the score the better.  A score of 5 or more means that your email will most likely get filtered out by any spam filter.  I’ve also been told that anything 3 or higher is bad.
  • Once you’ve tested out your emails in spam assassin, try testing it out across other services.  I initially was using Email Reach (www.emailreach.com). I think it’s a great service, but it’s still a young/small company. So if you find issues/bugs with their product, you will need to have patience with their customer service.  It’s a cheap product for what it does.  It will test delivery of your email to a large list of email providers, ISPs, and even email clients.

Tips

  • If you are sending plain text emails, make sure you have the charset in the Content-Type header set to ISO-8859-1. When I changed the charset to ISO-8859-1, gmail was no longer delivering my emails to the spam folder.  However hotmail and yahoo still were.
  • Yahoo! Mail allows you to view the headers of incoming emails.  You can see items they’ve added to the header.  Review them
  • Visit http://postmaster.hotmail.com and review their policies and troubleshooting guides. If you adhere to their rules, try to contact them
  • Implement DomainKeys on your mail server.  This is basically a crypto key that is stored in a DNS TXT record, and some work done on your mail server to get it to sign outgoing mail.

Posted in Development, Web Development | Tagged: , , , , | 21 Comments »

IE6 “tax” — interesting point of view

Posted by mikeg on November 6, 2008

http://pingv.com/blog/laura/2008/internet-explorer-6-tax

…when you’re spending 35-50% of your theming time just trying to get a cool new design to work on one rather archaic browser, it’s no small matter.

It’s a ≈40% tax on web theming

Posted in Development, Web Development | Tagged: | Leave a Comment »

ip2Location — easy Geo Location based on IP address

Posted by mikeg on April 22, 2008

Was working on a project where a client wanted to track users’ physical locations as their register on theirs site. Looked around for possible solutions and found 2 options: hosting ip2location database yourself or access via a web service.

The choice really depends on your needs:

  • If performance is a concern or internet connection is not always available, hosting database is the way to go
    • The downside is that you will need to updated that database rather frequently to keep it up to date
  • Based on my requirements of under 500 lookups a month, I decided to go with a web service solution. One big plus is that neither my client nor I have to worry about updating the database.

The same vendor (IP2LOCATION) provides both solutions and it looks like they have all the bases covered by providing examples and API across all popular languages. Web Service itself is extremely simple, so integration was absolutely painless.

 

Prices for web service are rather reasonable as well: $199/year for up to 500 requests/month, $399 gets you 2000 and it goes up from there.

The good news is that you can also get a free license for up to 90 requests per month.

Posted in Development, Web Development | Tagged: | 2 Comments »

IIS6 Compression: other tidbits

Posted by mikeg on April 22, 2008

A few other useful tidbits about IIS6 and compression:

 

  • You can enable compression per site using the following commands:

disable compression:

cscript C:\Inetpub\AdminScripts\adsutil.vbs set w3svc/{sitenumber}/root/DoStaticCompression False
cscript C:\Inetpub\AdminScripts\adsutil.vbs set w3svc/{sitenumber}/root/DoDynamicCompression False

enable compression:

cscript C:\Inetpub\AdminScripts\adsutil.vbs set w3svc/{sitenumber}/root/DoStaticCompression True
cscript C:\Inetpub\AdminScripts\adsutil.vbs set w3svc/{sitenumber}/root/DoDynamicCompression True

One way to find the {sitenumber} is to search for “header” within metabase. I searched for “staging.reviewbasics.com”. That’s the header binded to the website I am looking for

image

 

  • Disable compression on a specific file (from here):

First, to create a node for the page in the metabase, I right click on the file and go to properties. Make a change of some sort, apply, then change it back.
Then, from the command prompt enter:
“cscript C:\Inetpub\AdminScripts\adsutil.vbs set W3SVC/{siteID}/Root/{subfolder}{page.asp}/DoDynamicCompression False”

 

 

  • Scott mentions that in his experience compression introduces less than 5% overhead

 

Posted in Development, Web Development | Tagged: , | Leave a Comment »

IIS6 Compression: file extensions and testing

Posted by mikeg on April 21, 2008

See my previous post on setting up compression in IIS6 and reasons why you would want to do this

As I was doing research and thinking about what type of “extensions” I should compress, I considered the following extensions:

  • aspx
  • js
  • css
  • xml
    • above extensions were an easy decision
  • ashx
    • this was a concern because we use ASHX to generate XLS and PDF reports. I wanted to make sure that compression doesn’t break anything
    • Turns out both XLS and PDF reports were delivered successfully in the browsers I tested. The interesting thing was that they were also compressed on the server and decompressed by the browser. Fiddler reported that 552KB PDF was downloaded as 150KB. Similar compression was reported by XLS download
    • Before turning this one make sure to test all your handlers across browsers
  • asmx
    • this one was a concern because we expose our web services to a number of partners. I still have to do some more testing, but during my research I was reminded that IIS is pretty smart and it will not compress content unless the client sends “zip” header.
    • We utilize ASMX from our Flex Reporting interface by returning an XML file. The results were very impressive: 5.9MB XML file was compressed down to 454KB.
  • axd
    • These are responsible for bundled JS/CSS for ASP.NET/Ajax.ASP.NET/Telerik/etc
    • We use both Telerik and AjaxControlToolkit and when I was investigating our site in Fiddler prior to enabling compression, I noticed that some AXD were reporting as already being compressed.
    • I didn’t notice any issues/errors during testing after applying compression, so my conclusion was that IIS is smart in determining that content is already compressed and not applying compression again.
  • svc
    • this is WCF (Windows Communication Foundation) way of exposing web services
    • I decided not to include these to minimize integration issues
  • swf
    • swf is already a compressed format. you can try Zipping SWF and you will see that they don’t compress well
    • No reason to include

Above testing was done on Windows XP using IE7, FF2 and Safari3.1

This is what my IIsCompressionScheme in IIS metabase looks like:

image

Posted in Development, Web Development | Tagged: , | 1 Comment »

IIS6 Compression

Posted by mikeg on April 21, 2008

Compression is a very “cheap and easy” way to improve responsiveness of your site. Here is an excerpt from an excellent white paper: High Performance Web Sites: The Importance of Front-End Performance

There are three main reasons why front-end performance is the place to start.

  1. There is more potential for improvement by focusing on the front-end. Cutting it in half reduces response times by 40% or more, whereas cutting back-end performance in half results in less than a 10% reduction.
  2. Front-end improvements typically require less time and resources than back-end projects (redesigning application architecture and code, finding and optimizing critical code paths, adding or modifying hardware, distributing databases, etc.).
  3. Front-end performance tuning has been proven to work. Over fifty teams at Yahoo! have reduced their end-user response times by following our performance best practices, often by 25% or more.

Here is the link to a post that contains PDF and PPT: http://nate.koechley.com/blog/2007/06/12/high-performance-web-sites/, or slideshare link: http://www.slideshare.net/techdude/high-performance-web-sites/

Compression is number 4 out of 14 Rules.

————————————–

Steps below are for enabling compression in IIS6.

I found very good instructions around this subject here: Scott Forsyth’s post, but I am a visual person and decided to add some screenshots to make this process easier when I have to repeat it in the future.

Please refer to Scott’s post for background information about compression and compression in IIS6 specifically.

Steps below are required because IIS6 doesn’t have an easy to use interface, but provides all the necessary functionality through its metabase. You can think of metabase as registry for IIS6. All the settings that you see in IIS Manager and all those that you do not see can be confugred using metabase. The good news is that in IIS6 metabase is an XML file that can be modified with a Notepad. And just like with registry, you have to be very careful when making changes — and definitely BACKUP before starting.

  • Backup the metabase:
    1. Open up IIS Manager
    2. Right-click on the Server node
    3. Choose All Tasks -> Backup/Restore Configuration
    4. Follow the screens
    5. The good news it looks like IIS automatically backups the metabase

imageimage

  • Before enabling compression on your site, take some baseline measurements to see how much bandwidth you will be saving (you and your users)
    • If you haven’t played with Fiddler, you should spend some time with it.
    • It is an excellent tool for seeing exactly what your application (in most cases Browser) requests, amount of information coming back, time it takes, etc
    • I provide stats towards the bottom of this post
  • Enable Compression within IIS Manager
    • Unfortunately setting only these settings is not enough but required nonetheless
    • IIS Manager, right click on “Web Sites” node and choose properties
    • “Service” Tab, check
      • “Compress application files”
      • “Compress static fiels”
    • You should set max size to make sure you don’t run out of space. IIS6 allows a max of 1024MB
    • Default location of compressed files is “%windir%\IIS Temporary Compressed Files”. Scott mentions that IUSR_{machinename} needs to have Write rights to this directory. I didn’t give those rights to IUSR and it compression works for me. Be default that directory seems to have IIS_WPG group with full control.

image image

  • Next step is “Add new Web Service Extension for gzip.dll”
    • IIS Manager
    • Click on Web Service Extensions
    • Right click in empty space and choose “Add a new Web service extension..”
    • Call it HTTP Compression
    • Point it to c:\windows\system32\inetsrv\gzip.dll
    • Check the “Set extension status to Allowed” checkbox

image clip_image002

  • Enable Direct Metabase Edit
    • IIS Manager, right click on Server node, choose properites
    • Set “enable direct metabase edit”
    • Below is excerpt from Scott’s post

One of many large improvement with IIS 6 is that the metabase isn’t in binary format anymore and can be edited directly using Notepad or any other tools that allows editing an XML file. Personally I prefer to enable Direct Metabase Edit so that I can edit it and the change takes affect immediately. If this isn’t enabled, you will need to stop and start the web services for any changes to take affect. Of course, like editing the windows registry there is always the chance of something going wrong so be careful. Unlike the windows registry though, if you make a mistake and the metabase is saved and doesn’t conform to the proper XML scheme, it won’t take affect, so thanks to the IIS team it’s quite difficult to completely mess up the metabase. To enable this, right-click on the server (top level) in the IIS snap-in. There is a single checkbox that needs to be checked. This part couldn’t get easier.

image

  • Metabase edits
    • MAKE A BACKUP (see steps above)
    • Open the metabase located at C:\Windows\system32\inetsrv\metabase.xml in Notepad
    • Search for <IIsCompressionScheme
    • There should be two of them, one for deflate and one for gzip. Basically they are two means of compression that IIS supports.
    • Add your extensions (ASPX, JS, CSS, etc) to the list extensions in HcScriptFileExtensions. Make sure to follow the existing format carefully, an extra space will keep this from working correctly. Do this for both deflate and gzip.
    • HcDynamicCompressionLevel=”9″ (from 0)
      • see Scott’s post about the reasons why it should be 9 and not 10
    • Please see my next post about which extensions I added and some surprising results.
  • Last step is to IISRESET
  • Here are some statistics for the home page of www.reviewbasics.com. Images are not included in those numbers
    • Before compression:

351,210
text/javascript: 99,693
application/x-javascript: 143,287
text/xml: 649
application/x-shockwave-flash: 50,092
text/x-component: 1,832
text/html: 24,272
~headers: 6,681
text/css: 24,704

    • After compression:

223,629
text/javascript: 38,878
application/x-javascript: 108,231
text/xml: 649
application/x-shockwave-flash: 50,092
text/x-component: 3,664
text/html: 7,258
~headers: 7,621
text/css: 7,236

This is ~37% saving. I think that’s pretty impressive for about 30 minutes of work. See my next post for some other interesting and impressive results

Posted in Development, Web Development | Tagged: , , | 4 Comments »

How to easily create "test users" using Gmail

Posted by mikeg on January 11, 2008

Whenever you release a new version of any application that requires login, you need to test account creation functionality. There are a few possible ways:

  • Most likely you already have a few accounts with your gmail, yahoo, etc personal emails. You can certainly delete those from your system, but sometimes you have some test data in there that you want to keep
  • You can continue creating email addresses at any of the web email providers — that requires a rather significant amount of time as they ask you for a lot of information

This is where Gmail and automatic labels come to to the rescue:

  • Let’s say my email is mike@gmail.com
  • Gmail has a very useful feature where you can send an email to mike+anythingYouWantHere@gmail.com and that email will end up in your mailbox
  • “+” helps mimic “email alias” functionality. You can then apply filters within Gmail
  • That pretty much provides you with unlimited number of email addresses without all the hassle of setting them up

This technique is useful to automatically file emails and can also help you figure out which of the million websites that you signed up for actually sold your email address. It will not help you stop spam, but at least you will know who is responsible for it — sign up to websites using some system like this: mike+websiteName@gmail.com.

Here is help page on this subject from Gmail: http://mail.google.com/support/bin/answer.py?answer=12096&topic=13271

Posted in Development, QA, Testing, Web Development | Tagged: | 2 Comments »

How to easily create "test users" using Gmail labels

Posted by mikeg on January 11, 2008

Whenever you release a new version of any application that requires login, you need to test account creation functionality. There are a few possible ways:

  • Most likely you already have a few accounts with your gmail, yahoo, etc personal emails. You can certainly delete those from your system, but sometimes you have some test data in there that you want to keep
  • You can continue creating email addresses at any of the web email providers — that requires a rather significant amount of time as they ask you for a lot of information

This is where Gmail and automatic labels come to to the rescue:

  • Let’s say my email is mike@gmail.com
  • Gmail has a very useful feature where you can send an email to mike+anythingYouWantHere@gmail.com and that email will end up in your mailbox
  • “+” after your real address tells Gmail to “label” that email
  • That pretty much provides you with unlimited number of email addresses without all the hassle of setting them up

This technique is useful to automatically file emails and can also help you figure out which of the million websites that you signed up for actually sold your email address. It will not help you stop spam, but at least you will know who is responsible for it — sign up to websites using some system like this: mike+websiteName@gmail.com.

Here is help page on this subject from Gmail: http://mail.google.com/support/bin/answer.py?answer=12096&topic=13271

Posted in Development, QA, Testing, Web Development | Tagged: , | 2 Comments »

 
Follow

Get every new post delivered to your Inbox.

Join 183 other followers