How to Designate an IP Address as Permitted Sender

If you have your own domain handling email, then chances are you may run into email issues.  It is extremely important that you properly designate your IP Address as a permitted sender.

First, let’s test to see if your domain is a permitted sender.

Log into your domain’s email account, [email protected] let’s say.  Send an email to another email account that you have access to.  Just to name a few… gmail, Yahoo!, or Hotmail.

The important part is for you to view headers of this email, often referred to as “Show Full Headers”.

You will see something that looks like this, but note the SPF softfail error:

—- Original message —–

X-Received: by 10.224.25.8 with SMTP id x8mr31561868qab.77.1382465370255;
Tue, 22 Oct 2013 11:09:30 -0700 (PDT)
Return-Path: [email protected]
Received: from my3.example.com (my3.example.com. [64.131.70.223])
by mx.google.com with ESMTPS id k5si10380639qen.50.2013.10.22.11.09.30
for [email protected]
(version=TLSv1 cipher=RC4-SHA bits=128/128);
Tue, 22 Oct 2013 11:09:30 -0700 (PDT)
Received-SPF: softfail (google.com: domain of transitioning  [email protected] does not designate 64.131.70.223 as permitted sender) client-ip=64.131.70.223;
Authentication-Results: mx.google.com;
spf=softfail (google.com: domain of transitioning [email protected] does not designate 64.131.70.223 as permitted sender) [email protected]

 

If you see Received-SPF: pass in the header then you have nothing to worry about.  However, as you can see from above, I have a softfail issue.  You may also see some with “fail”.

The fix is actually pretty easy.  Login to your registrar where you registered the domain.  What we need to do is create a new TXT record and add that specific IP (64.131.70.223) as a permitted sender.

The record will look something like this:

TXT    example.com    “v=spf1 ip4:64.131.70.223 ~all”

Add this entry to your registrar and wait for the record to update across the web.  You can test your TXT / SPF entry with this handy tool.

A few things to note here:

  • Make sure you use the quotation marks around the whole thing.  (Do not just copy and paste from here because WordPress sometimes uses a different characters for quotes.)
  • The last part ~all designates a softfail for any non permitted senders.  Softfail basically means that you haven’t finalized your email settings yet.  These emails have a chance of getting through, even though they fail.
  • Once you send another test email and confirm that you PASS, change the ~all to -all.  This means that any emails not from your designated senders should fail.

Sounds great, but what happens when you use another email provider, like mailjet to send email on your behalf?

Your TXT entry will then look like this:

TXT    example.com    “v=spf1 ip4:64.131.70.223 include:spf.mailjet.com -all”

Setting up email can be a bit frustrating, so hopefully this helps speed things along.

Still confused?  You may also find Eric’s article on setting up SPF records useful.

 

Apache mod_fcgi (FastCGI) vs DSO (mod_php)

My experiment with FastCGI has come to an end.

You may remember my last post about upgrading my server to the latest Apache, PHP 5.4.x, and switching to FastCGI.

The biggest issue with this new setup was the fact that APC was not shared amongst all the PHP processes, which really made things more difficult.  Did I really want to recode my caching solution?  An alternative would be to use Redis, but that is assuming FastCGI was faster than DSO and worth using.

Caching aside, FastCGI is slower than DSO based on testing in a production environment.  In some high traffic situations FastCGI performed significantly slower.

On the other hand, FastCGI was great with resource management compared to DSO.  It managed to keep memory usage low during both real world testing and benchmarks that I threw at it.

In the end, I guess it depends on what you are looking for.  Speed or better resource management?

 

 

 

Initial Impressions of Apache 2.4.x, PHP 5.4.x, and FastCGI (mod_fcgid)

Over the past two weeks my main server for FantasySP has undergone a massive software update.  I wanted to make sure things will be rock solid for burstable traffic.  Sunday mornings during football season can be pretty rough unless you are prepared.

My previous configuration was Apache 2.2.x, PHP 5.3.x, and DSO ( mod_php).

The new configuration is  Apache 2.4.x, PHP 5.4.x, and FastCGI (mod_fcgid).

Let’s start off by showing you guys some performance graphs…

NewRelic Data
NewRelic Data

In order for you to see the correct data, I removed everything except php and httpd to get a fair comparison.  MySQL and other services are irrelevant.

You will notice a few trends:

  1. Because I moved away from mod_php to mod_fcgid the memory allocation is completely different.  The drop in httpd memory is now passed to separate php processes.
  2. CPU Usage is also different due to mod_fcgid and is now shown in the php process rather than httpd.
  3. Overall CPU% is lower under the new configuration.
  4. Overall Memory Usage is also lower under the new configuration.

What accounts for the lower CPU% and Memory usage?  Apache, PHP, mod_fcgid, or a combination?  The newest versions of Apache and PHP are supposed to have better memory utilization.  If you want to learn more about mod_fcgid in comparison to mod_php then this is a must read.

There are a few drawbacks to using mod_fcgid, most notably is that APC is no longer a shared cache to all processes.  Instead there are many instances of APC running, which can use more memory and cause higher loads to MySQL due to more misses than before.

I honestly did not anticipate this and had to react accordingly.  Load times to the site were noticeably affected.  I had no choice but to alter some of my caching to be saved to a separate MySQL table, rather than being stored in APC.

Huge pain in the ass, but something I considered doing previously for historical data purposes.

I’ve seen evidence from the PHP site that there will be speed increases just by upgrading to PHP 5.4.x from 5.3.x.  However, the jury is still out with this for me.  It appears that it could be true based on early New Relic data.  However, I’m still making caching changes due to APC and it has skewed the data a bit.

It appears that I’ve successfully offloaded slow load times to the odd hours of the day (12AM – 5AM), and by the time 8AM hits things are speedy. I won’t know for sure until a month goes by and things normalize.

The biggest question is if Apache 2.4.x is faster than 2.2.x.  Early benchmarks are mixed, but it is entirely way too early to make a definitive call.  I know for a fact that memory and cpu usage have decreased due to the new setup based on the NewRelic graphs above.  However, I cannot confirm that performance has improved.

Unfortunately I could not find my previous Apache Bench numbers based on my older configuration.  That would have likely given me all I needed to know.

I will continue to tweak Apache in the coming weeks and see what happens.  By the end of September I should know for sure if my current Apache configuration is noticeably better or not.  Though I do plan to stick with mod_fcgid for now and see where things go.

Stay tuned for a follow-up post down the line.

UPDATE: Check out the follow-up post here.

Facebook’s First Premium Feature: Profile Wiki’s

There has been a lot of talk about what Facebook should do next to boost revenue.  Recently Biz Stone mentioned something called “Facebook Premium” and suggested to lose the ads and “maybe some special features” to collect $10 per month.

His idea is just skipping the surface of the water.  Although it did get me thinking…  What type of feature would entice users to pay for it?

I propose Profile Wikis.

I know what you’re thinking, they already have Facebook Timeline. Yes, but that is automatically generated in chronological order.  It’s a great free feature to have, but this builds upon that and could be called something like Facebook Stories.

Of course Facebook may have to think of a better name for this, but essentially it would work very similar to how Wikipedia functions with a Facebook twist.

Let’s say I enable my Wiki page. My friends get a notice that it has been enabled and they are free to apply for editor access if they wish.  These approved friends would be allowed to use any of my posted photos, videos, location check-ins, or status updates.

I would have to elect a moderator to approve new edit changes or additions.  The wiki page can be set to public, private, or friends only.

The entire Wiki is open ended with no set structure.  The best part is that I have no say in what gets written.  It is entirely created and approved by my friends, so it is extremely important who I approve.  I would get notified of any changes or edits and they could be posted on my timeline if I so choose.

If my friends decide to highlight and approve my addiction with Farmville, then so be it. 

I think the Wiki feature itself would be extremely addicting to Facebook users both young and old.  Younger kids will fight to gain editor or moderator status.  Those of us connecting with old friends can easily catch up to see what we missed.  It would also be a great way to remember a friend or loved one who has passed away.

The approval and writing process has to be super clean and slick.  

It could be part GitHub and part Wikipedia.  Merge the “Wedding” wiki with the “master” wiki once it’s ready for primetime. (Of course Facebook wouldn’t use that type of language, but you get the idea)  Create a mockup layout with sections that need to be assigned to specific friends to take on.  Who’s in charge of the section on senior year of college?

See a graph of contributions detailing various parts of the Wiki so users get credit they are due. For example, Sally wrote 70% of the wedding section.

The reading experience has to be cutting edge and interactive, perhaps a take on this Rolling Stone article.  Let the reader get immersed in the Wiki.  Responsive layout for all devices and easily readable. Order a hardcover book if you’d like.

Once this platform is mature, Facebook could even release this software on its own to make collaborative writing an easier process for newspapers and magazines.  The possibilities are endless.

This is just ONE feature that Facebook could create under their “Facebook Premium” program.

 

Setting up 64bit WAMP Server under Windows 8 Using Latest Builds

Setting up WAMP Server can be tricky depending on your configuration.  This post is going to walk you through some of the tougher steps to get your local machine up and running so you can get back to development.

Keep in mind this is not a newbies guide to WAMP.  Look elsewhere if you can’t get the basics working.  What we are trying to tackle here are more advanced problems that you may experience.

A couple of snags I hit were WAMP running slowly, APC refusing to install, and getting cURL working.

At the time of writing this, the current bleeding edge WAMP Bundle has the following specs:

Apache 2.4.2 – Mysql 5.5.24 – PHP 5.4.3 XDebug 2.1.2 XDC 1.5 PhpMyadmin 3.4.10.1 SQLBuddy 1.3.3 webGrind 1.0

In my particular case, I opted to run 64bit code but I do not see any benefits other than making things more difficult. You may want to stick with 32bit.

Initial Steps

Installing WAMP at the start is easy and I am going to assume you can handle clicking next on the install screen and get the basics working.  Since you are using Windows 7/8 it is important that you install the latest runtimes for your setup:

http://www.microsoft.com/download/en/details.aspx?id=8328 x86 (32-bit)

http://www.microsoft.com/download/en/details.aspx?id=13523 x64 (64-bit)

It is also extremely important to make sure IIS is not running on your computer.  To do that go to: “Turn Windows Feature On and Off” and disable IIS, otherwise it will take up Port 80 and waste resources.

After a few reboots you should be at the point where you can configure PHP/Apache and add additional extensions.

Apache Configuration

This is where the fun begins.  To get to the Apache configuration file Left Click on WAMP Server systray and browse to Apache -> httpd.conf.  I am not going to go through the entire process, but will add some useful code snippets.

If you have any subdomains then this is the time to set those up and it looks something like this:

##########################
NameVirtualHost *:80

<VirtualHost *:80>
DocumentRoot “G:\wamp\www\FantasySP”
ServerName localhost
</VirtualHost>

<VirtualHost *:80>
DocumentRoot “G:\wamp\www\reddit-ama”
ServerName rlocalhost
</VirtualHost>

<VirtualHost *:80>
DocumentRoot “G:\wamp\www\FantasySP\m”
ServerName m.localhost
ErrorLog logs/subdomain_error.log
</VirtualHost>

As you can see, I have two separate projects FantasySP and Top IAmA, one running on localhost and the other on rlocalhost.  Most of you will have your DocumentRoot as \www\ and can probably skip this bit.  (Also, don’t forget to update your Windows HOSTS file.)

One of the snags you may encounter at this point might be getting Apache to recognize .htaccess files.  I modified my rules to look like this:

<Files “.ht*”>
Order allow,deny
Allow from all
Satisfy All
</Files>

Another snag might be the fact you can’t access the server at all.  By default it may “Order Deny,Allow”, then specify a specific IP that has access.  You can either update your localhost IP or just use “Allow from all”.

By default Apache has a few modules disabled that should be enabled.  Left Click on WAMP Server systray and browse to Apache -> Apache Modules.  You will want to enable deflate_module and rewrite_module.  Without the rewrite module your clean URLs won’t load.  Without deflate enabling gzip compression will cause a configuration error.  If you have expires rules in your htaccess file then include the expires_module.

If Apache is still having errors then chances are you need another missing module or you screwed up your httpd.conf file.  Make sure you back this up before you start fiddling around.

Configuring PHP

Now that Apache is up to snuff, it’s time to add missing PHP extensions and configurations.

Immediately you may want to enable the PHP short open tag.  Left Click LAMP Systray -> PHP -> PHP Settings.  Otherwise things like <? and ?> will cause errors.

(However, as septor in the comments pointed out, the short open tags will affect XML documents.)

Next up is making sure the cURL is working (assuming you want this).  In my case, cURL would not work with the included extension so I had to find an updated version that would work on my PC.

Grab the latest cURL extension that fits your OS.  Overwrite php_curl.dll in your WAMP folder similar to this: C:\wamp\bin\php\php5.4.3\ext

Restart Apache and run <? phpinfo(); ?> to see if cURL shows up this time.  If it doesn’t then try a different DLL file until it does show up.

The biggest pain in the ass will be to get APC working.  After a bit of frustration I finally figured out the easiest way to get it working.

In your php.ini file add the following line at the end to first DISABLE APC.  apc.enabled=0.  It is important APC is disabled first because it will make debugging a lot easier.

Find the correct php_apc.dll file that will work for your setup. You can find that on this handy page full of modules.

You can either try each one and restart apache if you’re lazy.  Or you can look at your phpinfo output to see what version you need.  TS stands for thread safe.  VC9 stands for the runtime library it was compiled with. (Remember when you installed those runtimes a bit earlier?).

Copy the .dll into the ext folder and restart apache.  If you picked the wrong one then Apache will crash.  Otherwise the systray icon will turn Green.  Once it appears in phpinfo output then you’ve found the right one.

Go back to your php.ini configuration and change apc.enabled=1.  There is a good chance Apache will crash once APC is enabled.  I think this has to do with the fact Serialization Support shows as Disabled (or broken).  That value should show up as “php”.

After dealing with this problem for a day or so I read that you had to copy the php_apc.dll file into one of your Windows System folders.

If you are running 64bit WAMP then copy php_apc.dll into C:\Windows\SysWOW64. For 32bit copy into C:\Windows\System32.

Only do that last step if Apache errors when APC is enabled.  I have no idea why these files need to be copied there, so if you know the reason then feel free to post in the comments.

At this point you can specify your own advanced APC configurations if you want.

Fixing Slow Performance

Right after I started testing FantasySP on localhost I quickly realized that it was much slower under this configuration than my last configuration.

Last time I was running PHP 5.3 with Apache 2.2.x.  So either the new Apache was slow or PHP.

As of this writing WAMP comes with Apache 2.4.2.  The latest version right now is Apache 2.4.4.   Apparently Apache 2.4.4 uses VC10 runtimes instead of VC9 to enhance performance under Windows.

I figured it might be best to update Apache to see if that would improve the speeds.  I got the latest Apache Build and created a new directory in wamp/bin/apache/apache2.4.4.

I coped over wampserver.conf from the apache2.4.2 directory and my httpd config file.  I also re-enabled my PHP extensions.  You can now specify which version of Apache you would like by Left Clicking the WAMP systray -> Apache -> Version.

Once WAMP Restarted using the newer version of Apache it was much faster. Though there was still something slowing it down.

I watched task manager in Windows 8 to see which applications were using a lot of CPU.  (A great reason to upgrade to Windows 7+ is the improved task manager).  Anyways, I noticed that Windows Defender was using a lot of CPU when httpd.exe or mysqld.exe would run.

As it turns out, real-time spyware protection was slowing things down!  Open up Windows Defender (or whatever you use for spyware/virus protection) and exclude real-time protection for httpd.exe and mysqld.exe.

It also makes sense to disable Windows Indexer from indexing C:/Wamp/*.

Final Thoughts

Phew.  After all of those configurations and changes, localhost runs as fast as my previous setup under Apache 2.2 and PHP 5.3.  As long as you take the time to set things up properly then things should run smooth under Windows 7 or Windows 8.

Even though you will experience growing pains by updating your WAMP environment I highly suggest taking a weekend to do so to make sure your applications run well on an updated stack.

Hopefully this post has proved useful to easing your pain during the WAMP setup process.  I have a strong feeling that I will be consulting this very blog post 3 years down the road when I have to do it all over again…