For example, if you are logged out, Washington Post’s newsletter page 302 redirects you to a login page that then serves a 200. This means a Google search for newsletters for washingtonpost.com gets indexed with the text from the login page. If they were to serve a 401 “Unauthorized” on the pages that require logins, these login pages would not be indexed. Use caution with that scenario though. It might be better to handle search engines differently than normal users so the content can be indexed.

What if sometimes you need to take a feature offline (or even, godz forbid, the entire site) for maintenance/upgrades? I assume you provide some sort of messaging to the users. If that page serves a 200 status and Google happens to try to index during that time you get screwed by having all of that feature’s pages get indexed as a single page. It is better to have the offline message page return a 503 status so the search engine knows that the service is temporarily unavailable and to try again later.

It is becoming more common for sites to capture a 404 and try to do a site search, or similar processing, for a possible match. Frequently when this happens it is handled within the app code rather than having the web server deliver the default 404 page. At GameSpot.com, the programmer who first set this up forgot to serve the 404 status code and was returning a 200. This meant that when a duplicate game was deleted from the database, the links to it were still showing up on Google. Once we made sure the code returned a status of 404 they went away next time Google indexed the site.

Finally, don’t forget about the 400 status code. If you try to load phpBB’s forums with an invalid forum id it will return a 200 status code even though it is not a valid request. This is a good situation to serve a 400 Bad Request. Keep in mind that that you probably wouldn’t want to return a 400 if the forum id just doesn’t exist (e.g., http://www.phpbb.com/community/viewforum.php?f=123456) because it might exist at some point. At the very least, return a 404.

As a side note, with HTTP/1.1 there is a new 307 status code that is very similar to how you’re used to using 302s. It will probably be a few more years before HTTP/1.1 is the default for browsers & the 307 will be truly usable.

Join the SF MySQL Meetup for our April meetup where Bret Taylor from FriendFeed talks about how they store their data in a "schema-less" system on top of MySQL. FriendFeed stores over 250 million entries and a bunch of other data, from comments and "likes" to friend lists.

As the database has grown, they have tried to iteratively deal with the scaling issues that come with rapid growth. They did the typical things, like using read slaves and memcache to increase read throughput and sharding our database to improve write throughput. However, as FriendFeed grew, scaling their existing features to accommodate more traffic turned out to be much less of an issue than adding new features.

In particular, making schema changes or adding indexes to a database with more than 10 – 20 million rows completely locks the database for hours at a time. After some deliberation, FriendFeed decided to implement a "schema-less" storage system on top of MySQL rather than use a completely new storage system. This talk attempts to describe the high-level details of the system.

Bret Tayler is one of the founders of FriendFeed.com. Before starting FriendFeed, he was a Group Product Manager at Google, responsible for Google’s developer products. Before that, he launched Google Maps and Google Local, and before that he worked on Google’s crawling and indexing infrastructure.

• Overview
  • What is the Yahoo! Open Stack?
    • Developer Tools – YUI, Tutorials
    • YOS SDK for PHP
  • Data APIs – YQL, BOSS …
  • Social APIs – Profiles, Connections, Updates, …
  • Application Platform
    • OpenSocial Support
• How does it all work?
  • Development Workflow
• Building an open application with PHP and YOS
• Common Problems & Solutions
  • CAJA
  • OpenSocial

Before we keep going, if you don’t know what svn hooks are you can read up on them at http://svnbook.red-bean.com/en/1.1/ch05s02.html.  svnbook.red-bean.com is my reference site of choice whenever I need to look up a svn command.

My #1, all-time, greatest in the world, bestest favorite is a svn pre-commit hook to syntax check my PHP code.  There is nothing I hate more than a developer checking in code with syntax errors and breaking qa or stage or even, god-forbid, production.  It’s as easy as setting up the hook to run a script that calls “/path/to/php -l /path/to/file.php” and on failure reject the commit.  This way code that has syntax errors will never make it into SVN.  Now if only open-source projects like ZendFramework did something like this before releasing their code.  To be fair, the syntax errors are in their documentation not in the /library/ files but that’s still pretty dumb & annoying.

Another nice way to use a post-commit hook is to run something like PHP_CodeSniffer against the commit.  CodeSniffer is an awesome PEAR package that scans your file and flags coding standards violations.  It comes pre-packaged with sniffers for the PHPCS, PEAR, Squiz and  Zend.  Set it up to run and if it finds coding standard violations you can have the hook send out nasty little blame messages.    This isn’t something that I’d want to run on pre-commit just in case you really need to get something into production NOW.  But if I hosted an open source project, I’d consider it.

Having minified JavaScript & CSS has become the standard way of serving JS & CSS.  But who really wants to take the time to minify or combine them?  Once again, enter svn hooks.  One of the developers set this up for GameSpot and it has been working great!  Basically it checks the commits against a defined naming standard, file.src.js or file.src.css, and then runs the YUI! Compressor on it and commits in file.min.(js|css).  He also made it check a global.conf in the file’s directory so it came mash up smaller js files into a global.min.js so we serve one file instead of ten.

Of course, let’s not forget about what is probably the most common svn hook; the post-commit email hook.  I like svnmailer better than mailer.py that comes with svn by default.  I think it’s easier to configure/change and to setup rules for how you want the emails to look or where to go.

If you’re using a ticket tracking system like Bugzilla or Trac that doesn’t have its own way of peeking into svn (like JIRA can with this plugin) you can write a post-commit hook to update the ticket when something is committed.  You can see how Trac recommends you do it at http://trac.edgewall.org/browser/trunk/contrib/trac-post-commit-hook.

To help you on the way to creating your own svn hooks, tigris has a few hooks that are distributed with svn.  They can be quite helpful so you should take a look at them.

For the main presentation David Gomel, Senior Program Manager leading SEO within CBS Interactive, will speak to us about SEO in a Web 2.0 world. The session will cover many of the important SEO issues that involve front and back-end engineers on a daily basis. Many of the decisions that tech folks make have the potential to materially impact search engine traffic. Even if you have been to a similar talk in the past, you should consider attending as we will be discussing topics that are frequently evolving. Please bring questions!

Topics will include:

• HTML fundamentals
• CSS Usage
• JavaScript/Flash/Ajax
• Graceful Degradation
• Multimedia Content
• Duplicate Content
• Redirection
• URLs
• Bot Traffic
• Robots.txt

…And More!

David Gomel is the Senior Program Manager leading SEO within CBS Interactive. He is responsible for SEO knowledge sharing and training within the company.

When I’m dealing with user inputted data where I want to allow HTML I have two concerns. First, I don’t want to allow XSS (some xml parsers think <p kkk=”></p>” closes the <p> tag). Second, the user frequently enters invalid html (e.g., doesn’t close the <a> tag). Fortunately tidy can easily deal with both. The second issue is the easiest to solve by running tidy->cleanRepair() on the html. The first is taken care of by looping through the tidy nodes and rebuilding the html using a whitelist. More about how to do this after the break.

To start, you’ll need to setup your tidy options. I tend to use the following options. The word-2000 option is great because it strips out all of the annoying html, css, etc. that Microsoft Word inserts for its formatting.

Add/Remove these options as it works best for you. I’ve found that these work best for the cleaning & repairing of accidentally invalid HTML. Read the tidy docs for what each of these mean and how they affect what tidy does. Please note that in the above options I include “force-output” so that I can display back to the user what the results of trying to fix the HTML is so that they can fix their input.

If you’re parsing user generated content, usually they are just entering the tags and HTML for the specific content they want posted. This means you’ll need to wrap that input in the <!DOCTYPE>, <HTML> and <BODY> tags before processing it through tidy.

So how you setup the array for tag validation is pretty much up to you. The format I used looks a lot like this:

The auth_level is a filter that I have in place to only allow users with a certain amount of access, such as internal staff, to use that attribute. As you probably already know, attributes like onmouseover are an easy gateway to XSS exploits. As you add new elements to the tag rule definition you just need to update the validate function in order to validate against it.

You may have noticed that I’m using the deprecated HTML attributes instead of putting all of this in a “style” attribute. I’ve found that most users don’t know CSS styles and are familiar with the old-style HTML attributes. If you force your users to use styles you’ll have whole other validation routine to ensure exploits are not in the CSS.

Note: Please don’t just copy/paste this code into your code and use it. I’m quite positive there are bugs and typos all throughout. Oh, and no, this is not the code used by the sites I work on. It is similar but definitely not the same.

Peter Zaitsev will be giving his excellent presentation “Scaling MySQL powered Web Sites by Sharding and Replication” to the upcoming SF MySQL 6:00pm on November 3rd @ the CBS Interactive (formerly CNET Networks) building in SOMA San Francisco (235 2nd St).

RSVP here: http://mysql.meetup.com/30/calendar/8912109/

Description from the 2008 Velocity conference:
When your application grows beyond capacity of a single MySQL server there are few ways to scale, with most typical being Replication and Sharding.

In this presentation you will learn how, depending on your application performance scalability and high availability goals, you can choose what proper scaling strategy will be good for you.

We also will speak about scalability limitations of replication of Replication and Sharding, implementation complexities, and aspects of operation.

Peter Zaitsev’s bio:
MySQL Performance Blog

Peter helped many companies ranging from one man startups to Fortune 500 companies with their Performance and Scaling problems on various stages starting from architecture design and down to queries and schema optimization.

Over cause of the years Peter has spoken on many International conferences focused on MySQL, Open Source, Databases or High Performance Web applications.

Before co-founding Percona Peter worked for MySQL AB as head of High Performance Group being involved in Support, Consulting, Development and working with vendors helping to optimize their software or hardware to perform well with MySQL.

Peter also was involved in number of Web Startups in the roles ranging from CTO to Consultant and has a lot of experience in MySQL and Web Applications Operations, Deployment, Quality Assurance and Development.
Please join us!

Map to CNET Networks, Inc.
When you arrive, please look for the PHP/MySQL Meetup sign pointing you to the conference room.

Look forward to seeing you there!

But what if the query that’s going to hit the database is pretty intensive and you don’t want more than one user to hit the db at a time? That’s easily handling via a semaphore lock.

More below the break –>

Another good use of the semaphore locking is to set the expire time as part of the cached data and have an extended expire time set in memcache. This allows you to have more control over what happens when the cached data becomes stale. You can make it so that one user repopulates the cache while other users continue to get the existing cache until the first user has finished.

When you mash these functions together, you end up with a system where only one user every freshens the cache and/or hits the database with a specific query at a time. There are a lot of other things that suddenly become available once you start thinking of memcached beyond just saving your db from hits. You have things like session handling, smarty template caching, flags for per-server processing (e.g., clearing local file cache), and even as a temporary database.