How FriendFeed uses MySQL to store schema-less data – SF MySQL Meetup April 6th, 2009

When: 6:30pm Monday April 6th, 2009
Where: CBS Interactive – 235 2nd Street San Francisco, CA
RSVP: How FriendFeed uses MySQL to store schema-less data


Join the SF MySQL Meetup for our April meetup where Bret Taylor from FriendFeed talks about how they store their data in a "schema-less" system on top of MySQL. FriendFeed stores over 250 million entries and a bunch of other data, from comments and "likes" to friend lists.

As the database has grown, they have tried to iteratively deal with the scaling issues that come with rapid growth. They did the typical things, like using read slaves and memcache to increase read throughput and sharding our database to improve write throughput. However, as FriendFeed grew, scaling their existing features to accommodate more traffic turned out to be much less of an issue than adding new features.

In particular, making schema changes or adding indexes to a database with more than 10 – 20 million rows completely locks the database for hours at a time. After some deliberation, FriendFeed decided to implement a "schema-less" storage system on top of MySQL rather than use a completely new storage system. This talk attempts to describe the high-level details of the system.

More about the event after the break

Posted under Events, Internet, mysql, Technology, Tips & Tricks, Web Development

This post was written by Michael Tougeron on March 28, 2009

Tags: , , ,

Diving into the Yahoo! Open Stack – SF PHP Meetup April 2nd, 2009

When: 6:30pm Thursday April 2nd, 2009
Where: CBS Interactive – 235 2nd Street San Francisco, CA
RSVP: Diving into the Yahoo! Open Stack


Join the SF PHP Meetup for an exploration into the Yahoo! Open Stack and learn how to leverage Yahoo’s platforms and scalability to build your next application. We will examine the components that make up Yahoo’s open stack: developer tools (YUI), data apis (YQL, BOSS), social apis, and the application platform (YAP).

  • Overview
    • What is the Yahoo! Open Stack?
      • Developer Tools – YUI, Tutorials
      • YOS SDK for PHP
    • Data APIs – YQL, BOSS …
    • Social APIs – Profiles, Connections, Updates, …
    • Application Platform
      • OpenSocial Support
  • How does it all work?
    • Development Workflow
  • Building an open application with PHP and YOS
  • Common Problems & Solutions
    • CAJA
    • OpenSocial

More about the event after the break

Posted under Events, Internet, PHP, Technology, Tips & Tricks, Web Development

This post was written by Michael Tougeron on March 11, 2009

Tags: , , , ,

Twitter Updates for 2009-03-08

  • Uploading phptos from this weekend to Flickr #
  • Watching Airwolf from netflix on the xbox #
  • 2minutes is a *really* long time if you have a screaming baby 🙁 #
  • Just wrote a quick blog entry about how I like to use svn hooks: http://tinyurl.com/dk4wfe #
  • I can’t believe I actually did this…but I just purposefully clicked on a ad on a website. That’s the 1st time except in testing @GameSpot. #

Posted under Twitter

This post was written by Michael Tougeron on March 8, 2009

Tags: ,

svn pre-commit & post-commit hooks

I have to say I love subversion (svn) hooks.  I’ve only been playing around with the concept fairly recently and I’m surprised that I lasted this long without them.  I wanted to take a few minutes and share my favorite ways of using them.  As well as a few that I don’t use but I think could be pretty helpful.

Before we keep going, if you don’t know what svn hooks are you can read up on them at http://svnbook.red-bean.com/en/1.1/ch05s02.html.  svnbook.red-bean.com is my reference site of choice whenever I need to look up a svn command.

My #1, all-time, greatest in the world, bestest favorite is a svn pre-commit hook to syntax check my PHP code.  There is nothing I hate more than a developer checking in code with syntax errors and breaking qa or stage or even, god-forbid, production.  It’s as easy as setting up the hook to run a script that calls “/path/to/php -l /path/to/file.php” and on failure reject the commit.  This way code that has syntax errors will never make it into SVN.  Now if only open-source projects like ZendFramework did something like this before releasing their code.  To be fair, the syntax errors are in their documentation not in the /library/ files but that’s still pretty dumb & annoying.

Another nice way to use a post-commit hook is to run something like PHP_CodeSniffer against the commit.  CodeSniffer is an awesome PEAR package that scans your file and flags coding standards violations.  It comes pre-packaged with sniffers for the PHPCS, PEAR, Squiz and  Zend.  Set it up to run and if it finds coding standard violations you can have the hook send out nasty little blame messages.  😉  This isn’t something that I’d want to run on pre-commit just in case you really need to get something into production NOW.  But if I hosted an open source project, I’d consider it.

Having minified JavaScript & CSS has become the standard way of serving JS & CSS.  But who really wants to take the time to minify or combine them?  Once again, enter svn hooks.  One of the developers set this up for GameSpot and it has been working great!  Basically it checks the commits against a defined naming standard, file.src.js or file.src.css, and then runs the YUI! Compressor on it and commits in file.min.(js|css).  He also made it check a global.conf in the file’s directory so it came mash up smaller js files into a global.min.js so we serve one file instead of ten.

Of course, let’s not forget about what is probably the most common svn hook; the post-commit email hook.  I like svnmailer better than mailer.py that comes with svn by default.  I think it’s easier to configure/change and to setup rules for how you want the emails to look or where to go.

If you’re using a ticket tracking system like Bugzilla or Trac that doesn’t have its own way of peeking into svn (like JIRA can with this plugin) you can write a post-commit hook to update the ticket when something is committed.  You can see how Trac recommends you do it at http://trac.edgewall.org/browser/trunk/contrib/trac-post-commit-hook.

To help you on the way to creating your own svn hooks, tigris has a few hooks that are distributed with svn.  They can be quite helpful so you should take a look at them.

Posted under PHP, Tips & Tricks

This post was written by Michael Tougeron on March 7, 2009

Tags: , , , ,

Goodbye Paul Harvey

As you may already know, American icon Paul Harvey died on Saturday. I’ve always said to people, “Why should I care about celebrities?” I never thought I’d care a great deal about the happenings to any of them. Well it turns out I was wrong. It came as a great surprise to me when I actually shed a tear (only one though) when I heard over the weekend that Paul Harvey passed away. It is an even greater surprise that I still feel a bit sad whenever I hear a new tribute or story about the man. As it turns out, I’m really going to miss listening to him on the radio.

I have disagreed with Paul Harvey a lot over the years (in my own thoughts anyway), mostly about the way he reported medical stories without questioning the validity of the data or his reports on the war in Iraq & against terrorism. But even with that, I have always liked his programs. When I used to drive to work I’d try and time my commute so I could catch “The Rest of the Story” (one of my favorite programs) on KGO. I still remember his story about the penny and the Chicago newspaper advertiser who started the 99 is less than 100 trend (because people feel that $9.99 is a lot less than $10) because he wanted to sell his paper for a penny instead of a nickle. I’d even occasionally take lunch around his noon program “News & Comment.”

Being on the radio for over 60 years, even after he turned 90, Paul Harvey became a radio legend. According to his Wikipedia entry, Paul Harvey’s programs were carried on over 1,200 radio stations, 400 Armed Forces Network stations and 300 newspapers. With that sort of exposure, his passing will have a profound impact and noticeable loss to a large and diverse group of people. I’m sure he will be sorely missed. An additional note of merit, Paul Harvey was the recipient of the Presidential Medal of Freedom, America’s highest civilian award.

As you might also know, his son, Paul Harvey Jr, has been filling in for him over the last several years as he dealt illness and the loss of his wife. It is good to hear that Paul Harvey Jr will be continuing in his father’s footsteps and keeping the programs going.

Rest in peace Paul Harvey. I, as well as a nation, mourn your passing. In the words of Paul Harvey himself, “Good day.”

P.S. If you’d like to leave his family your condolence for his family you can do so at http://abcrn.com/harvey/contact/

Posted under News, opinion

This post was written by Michael Tougeron on March 3, 2009

Tags: