Grep My Mind » PHP http://www.grepmymind.com Wonder what’s on my mind? Now you know. Thu, 11 Jun 2009 20:00:55 +0000 en hourly 1 http://wordpress.org/?v=3.0 Diving into the Yahoo! Open Stack – SF PHP Meetup April 2nd, 2009 http://www.grepmymind.com/2009/03/11/diving-into-the-yahoo-open-stack-sf-php-meetup-april-2nd-2009/ http://www.grepmymind.com/2009/03/11/diving-into-the-yahoo-open-stack-sf-php-meetup-april-2nd-2009/#comments Wed, 11 Mar 2009 16:03:35 +0000 Michael Tougeron http://www.grepmymind.com/?p=319 When: 6:30pm Thursday April 2nd, 2009
Where: CBS Interactive – 235 2nd Street San Francisco, CA
RSVP: Diving into the Yahoo! Open Stack

Join the SF PHP Meetup for an exploration into the Yahoo! Open Stack and learn how to leverage Yahoo’s platforms and scalability to build your next application. We will examine the components that make up Yahoo’s open stack: developer tools (YUI), data apis (YQL, BOSS), social apis, and the application platform (YAP).

  • Overview
    • What is the Yahoo! Open Stack?
      • Developer Tools – YUI, Tutorials
      • YOS SDK for PHP
    • Data APIs – YQL, BOSS …
    • Social APIs – Profiles, Connections, Updates, …
    • Application Platform
      • OpenSocial Support
  • How does it all work?
    • Development Workflow
  • Building an open application with PHP and YOS
  • Common Problems & Solutions
    • CAJA
    • OpenSocial

Dustin Whittle is a Technical Yahoo! and Developer Evangelist for Yahoo’s Open Strategy. When he is not evangelizing Yahoo’s Open Stack, he helps develop it through his work on the Yahoo! Developer Network and Yahoo! Application Platform. Prior to his work helping create an open Yahoo!, he has contributed to Yahoo! Answers and Delicious. When not focused on Yahoo!, he is an active open source contributor for the symfony project and is a regular speaker at technical conferences around the world.

Agenda:
6:30 – 7:15 — Doors open/general socializing & food provided by Yahoo!
7:15 – 8:30 — Main presentation by Dustin Whittle on the Yahoo! Open Stack
8:30 – 9:00 — Main presentation Q&A
9:00 – 10:00 — General discussion/Q&A/networking/etc.
10:00ish — end of group

]]> http://www.grepmymind.com/2009/03/11/diving-into-the-yahoo-open-stack-sf-php-meetup-april-2nd-2009/feed/ 0 Validating HTML with tidy http://www.grepmymind.com/2009/01/15/validating-html-with-tidy/ http://www.grepmymind.com/2009/01/15/validating-html-with-tidy/#comments Thu, 15 Jan 2009 20:41:32 +0000 Michael Tougeron http://www.grepmymind.com/?p=42 If you ever have to do HTML validation or parsing in PHP the tidy extension is the way to do it! This extension lets you use the abilities of tidy in some pretty powerful ways. The extension, written by John Coggeshall, has been around for several years now. I can see how if someone just took a quick glance at it they could think it was nice, but not really something they need. How wrong they would be! If you take a few minutes and look under the hood, tidy is an extremely powerful tool. Not only can it format html to standards (what most people use it for), it can also be a powerful parser and validation tool.

When I’m dealing with user inputted data where I want to allow HTML I have two concerns. First, I don’t want to allow XSS (some xml parsers think <p kkk=”></p>” closes the <p> tag). Second, the user frequently enters invalid html (e.g., doesn’t close the <a> tag). Fortunately tidy can easily deal with both. The second issue is the easiest to solve by running tidy->cleanRepair() on the html. The first is taken care of by looping through the tidy nodes and rebuilding the html using a whitelist. More about how to do this after the break.

To start, you’ll need to setup your tidy options. I tend to use the following options. The word-2000 option is great because it strips out all of the annoying html, css, etc. that Microsoft Word inserts for its formatting.

  1. $config = array(‘indent’ => false,
  2.   ‘output-xhtml’ => true,
  3.   ‘wrap’ => 0,
  4.   ‘fix-uri’ => true,
  5.   ‘word-2000′ => true,
  6.   ‘show-body-only’ => true,
  7.   ‘drop-proprietary-attributes’ => true,
  8.   ‘ncr’ => false,
  9.   ‘drop-empty-paras’ => false,
  10.   ‘hide-endtags’ => true,
  11.   ‘lower-literals’ => true,
  12.   ‘markup’ => true,
  13.   ‘quote-ampersand’ => true,
  14.   ‘force-output’ => true);

Add/Remove these options as it works best for you. I’ve found that these work best for the cleaning & repairing of accidentally invalid HTML. Read the tidy docs for what each of these mean and how they affect what tidy does. Please note that in the above options I include “force-output” so that I can display back to the user what the results of trying to fix the HTML is so that they can fix their input.

If you’re parsing user generated content, usually they are just entering the tags and HTML for the specific content they want posted. This means you’ll need to wrap that input in the <!DOCTYPE>, <HTML> and <BODY> tags before processing it through tidy.

  1. // Let’s initialize tidy with our HTML and config options.
  2. $tidy = tidy_parse_string($html, $config);
  3. // Not let’s clean &amp; repair to try and "fix" user errors
  4. $tidy->cleanRepair();
  5. // Start tracking any errors at this point so we can give good feedback to the user
  6. if (tidy_error_count($tidy) &amp;&amp; $tidy->errorBuffer ) {
  7.   $errors = explode("\n", $tidy->errorBuffer);
  8.   foreach ( $errors as $key => $error ) {
  9.     $errors[$key] = htmlentities($error);
  10.   }
  11. }
  12.  
  13. // Once tidy has cleaned &amp; repaired the initial user input
  14. // we need to loop through the object and validate each
  15. // HTML tag block.  We will only need to do that though if
  16. // the $body actually has child tags
  17. $body = $tidy->body();
  18. if ( $body->child ) {
  19.   // we need to validate each HTML tag in the <body>
  20.   // validateTidyNode() will be called recursively for
  21.   // any sub-tag blocks
  22.   foreach ( $body->child as $key => $child ) {
  23.     $error = validateTidyNode($child, $new_html);
  24.     $body->child[$key] = $child;
  25.     // And of course track the errors as they
  26.     // appear to give good feedback to the user
  27.     if ( $error ) {
  28.       $errors = array_merge((array)$errors, (array)$error);
  29.     }
  30.   }
  31. }
  32. return array(‘html’ => $new_html, ‘errors’ => $errors);
  33.  
  34. function validateTidyNode(&amp;$tidy_node, &amp;$html, $allowed_tags) {
  35.   $bad_tag = false;
  36.   // If the node is text, just add the text to the HTML
  37.   if ( $tidy_node->type == TIDY_NODETYPE_TEXT ) {
  38.     $html .= $tidy_node->value;
  39.     return;
  40.   }
  41.   // If the node does not exist in your array of acceptable HTML tags
  42.   // then track and error message and set the $bad_tag flag.
  43.   elseif ( !array_key_exists(strtolower($tidy_node->name), $allowed_tags) ) {
  44.     $errors[] = ‘Tag: ‘ . $tidy_node->name . ‘ is not allowed and has been removed.’;
  45.     $bad_tag = true;
  46.   }
  47.   if ( $tidy_node->child ) {
  48.     $html2 = ”;
  49.     foreach ( $tidy_node->child as $key => $child ) {
  50.       $error = validateTidyNode($child, $html2);
  51.       $tidy_node->child[$key] = $child;
  52.       if ( $error ) {
  53.         $errors = array_merge((array)$errors, (array)$error);
  54.       }
  55.     }
  56.   }
  57.   if ( $bad_tag ) {
  58. // I prefer to "display" the bad HTML tags so that the
  59. // user can see that it was not accepted.  But you
  60. // may prefer to just strip the tag.
  61. //    $html .= ‘ ‘;
  62.     $html .= ‘&amp;lt;’ . $tidy_node->name;
  63.   }
  64.   else {
  65.  
  66.     $html .= ‘<’ . $tidy_node->name;
  67.     if ( $tidy_node->attribute ) {
  68.       unset($found_attribs);
  69.       foreach ( $tidy_node->attribute as $attrib_name => $attrib_value ) {
  70. // verify that the tag is allowed to have the specified attribute.
  71.         if ( !array_key_exists(strtolower($attrib_name), $allowed_tags[$tidy_node->name][‘attribs’]) ) {
  72.           $errors[] = ‘Tag ‘ . $tidy_node->name . ‘ is not allowed to have the attribute ‘ . $attrib_name . ‘ and has been removed.’;
  73.           unset($tidy_node->attribute[$attrib_name]);
  74.         }
  75.         else {
  76. // validate the attribute’s value.  We don’t want invalid values.
  77.           $res = validateHTMLTagAttribute($tidy_node->name, $attrib_name, $attrib_value, $allowed_tags[$tidy_node->name]);
  78.           if ( !$res[‘remove’] ) {
  79.             $html .= ‘ ‘ . $res[‘attrib_name’] . ‘="’ . $res[‘attrib_value’] . ‘"’;
  80.           }
  81.           if ( $res[‘errors’] ) {
  82.             $errors[] = $res[‘errors’];
  83.           }
  84.         }
  85.         if ( trim($attrib_name) ) {
  86.           $found_attribs[] = $attrib_name;
  87.         }
  88.       }
  89.  
  90.     }
  91. // check to make sure that required attributes are set.  e.g. <img> needs to have the src attribute
  92.     if ( $allowed_tags[$tidy_node->name][‘attribs’] ) {
  93.       foreach ( $allowed_tags[$tidy_node->name][‘attribs’] as $attrib_name => $attrib_settings ) {
  94.         if ( $attrib_settings[‘required’] &amp;&amp; (!$found_attribs || !in_array($attrib_name, $found_attribs)) ) {
  95.           $errors[] = ‘Tag ‘ . $tidy_node->name . ‘ is required to have the attribute ‘ . $attrib_name . ‘.’;
  96.         }
  97.       }
  98.     }
  99.  
  100. // some tags require at least one of a set of attributes.  e.g. <a> needs to have either the href or name attribute.
  101.     if ( $allowed_tags[$tidy_node->name][‘required_attribs’] ) {
  102.       $attrib_found = false;
  103.       if ( $found_attribs ) {
  104.         foreach ( $allowed_tags[$tidy_node->name][‘required_attribs’] as $attrib_name ) {
  105.           if ( in_array($attrib_name, $found_attribs) ) {
  106.             $attrib_found = true;
  107.           }
  108.         }
  109.       }
  110.       if ( !$attrib_found ) {
  111.         $errors[] = ‘Tag ‘ . $tidy_node->name . ‘ is required to have one of the following attributes: ‘ . implode(‘, ‘, $allowed_tags[$tidy_node->name][‘required_attribs’]) . ‘.’;
  112.       }
  113.     }
  114.  
  115.   }
  116.   if ( isset($allowed_tags[$tidy_node->name][‘settings’]) &amp;&amp; !$allowed_tags[$tidy_node->name][‘settings’][‘require_close’] ) {
  117.     $html .= ‘ /’;
  118.   }
  119.   if ( $bad_tag ) {
  120. //    $html .= ‘ ‘;
  121.     $html .= ‘&amp;gt;’;
  122.   }
  123.   else {
  124.     $html .= ‘>’;
  125.   }
  126.   $html .= $html2;
  127.   if ( $bad_tag || $allowed_tags[$tidy_node->name][‘settings’][‘require_close’] ) {
  128.     if ( $bad_tag ) {
  129.       $html .= ‘&amp;lt;/’ . $tidy_node->name . ‘&amp;gt;’;
  130.     }
  131.     else {
  132.       $html .= ‘</’ . $tidy_node->name . ‘>’;
  133.     }
  134.   }
  135.   return $errors;
  136. }
  137.  
  138. function validateHTMLTagAttribute( $tag_name, $attrib_name, $attrib_value, $tag_data ) {
  139.   // set to the unmodified version first.  if modified, it will change at the end of the method.
  140.   $res[‘attrib_name’] = $attrib_name;
  141.   $res[‘attrib_value’] = $attrib_value;
  142.  
  143.   if ( !$tag_data) {
  144.     $res[errors] = "$tag_name could not be found.  Invalid tag/attribute.";
  145.     return $res;
  146.   }
  147.  
  148.   if ( $tag_data[‘attribs’][‘unlimited_attribs’] ) {
  149.     $res[‘valid’] = true;
  150.     return $res;
  151.   }
  152.  
  153.   $attrib = $tag_data[‘attribs’][$attrib_name];
  154.   switch ( $attrib[‘type’] ) {
  155. // no restrictions on the value of the attributes (not normally recommended)
  156.     case ‘unrestricted’:
  157.       break;
  158. // the attribute can only have one of the defined values
  159.     case ‘fixed’:
  160.       if ( !in_array(strtolower($attrib_value), $attrib[‘values’]) ) {
  161.         $res[‘errors’] = "$attrib_name is not set to an accepted value for $tag_name tag.";
  162.         return $res;
  163.       }
  164.       break;
  165. // some basic numeric checks
  166.     case ‘numeric’:
  167.     case ‘px’:
  168.     case ‘percent’:
  169.       $attrib_value = intval($attrib_value);
  170.       if ( $attrib_value > $attrib[‘max’] ) {
  171.         $res[‘errors’] = "$attrib_name is greater than the max value (" . $attrib[‘max’] . ") allowed for $tag_name tag.";
  172.         return $res;
  173.       }
  174.       elseif ( $attrib_value < $attrib[‘min’] ) {
  175.         $res[‘errors’] = "$attrib_name is less than the max value (" . $attrib[‘min’] . ") allowed for $tag_name tag.";
  176.         return $res;
  177.       }
  178.       if ( $attrib[‘type’] == ‘percent’ ) {
  179.         $attrib_value = $attrib_value . "%";
  180.       }
  181.       elseif ( $attrib[‘type’] == ‘px’ ) {
  182.         $attrib_value = $attrib_value . "px";
  183.       }
  184.       break;
  185.     case ‘hex’:
  186.       // 6 characters of A-F and 0-9
  187.       // optionally allow a # in front for a total
  188.       // of 7 characters
  189.       break;
  190.     case ‘url’:
  191.       // validate according to your valid URL criteria
  192.       // e.g., only allow for current domain or if the
  193.       // tag is "img" make sure the extension is .jpg or .gif
  194.       break;
  195.     case ‘string’:
  196.       // validate for a string.  I usually consider _-. and similar
  197.       // characters acceptable for a "string" even though they are
  198.       // not truly a string
  199.       break;
  200.     default:
  201.       $res[‘errors’] = "ERROR!  Do not know how to process the tag $tag_name!";
  202.       break;
  203.   }
  204.   $res[‘attrib_name’] = $attrib_name;
  205.   $res[‘attrib_value’] = $attrib_value;
  206.   $res[‘valid’] = true;
  207.   return $res;
  208. }

So how you setup the array for tag validation is pretty much up to you. The format I used looks a lot like this:

  1. $tags = array(
  2.             ‘a’ => array( ‘settings’ => array(‘require_close’ => true),
  3.                             ‘attribs’ => array( ‘href’ => array(‘type’ => ‘url’),
  4.                                                 ‘name’ => array(‘type’ => ‘string’),
  5.                                                 ‘id’ => array( ‘type’ => ‘string’,
  6.                                                                   ‘auth_level’ => AUTH_LEVEL_TRUSTED_USER
  7.                                                                 ),
  8.                                                 ‘class’ => array( ‘type’ => ‘unrestricted’,
  9.                                                                   ‘auth_level’ => AUTH_LEVEL_TRUSTED_USER
  10.                                                                 ),
  11.                                                 ‘onclick’ => array( ‘type’ => ‘unrestricted’,
  12.                                                                   ‘auth_level’ => AUTH_LEVEL_TRUSTED_USER
  13.                                                                 ),
  14.                                                 ‘onmouseover’ => array( ‘type’ => ‘unrestricted’,
  15.                                                                   ‘auth_level’ => AUTH_LEVEL_TRUSTED_USER
  16.                                                                 )
  17.                                               ),
  18.                             ‘required_attribs’ => array( ‘href’, ‘name’ )
  19.                         ),
  20.             ‘b’ => array(‘settings’ => array( ‘require_close’ => true)),
  21.              ‘img’ => array(  ‘settings’ => array(  ‘require_close’ => false),
  22.                                 ‘attribs’ => array(   ‘src’ => array( ‘type’ => ‘url’,
  23.                                                                         ‘required’ => true
  24.                                                                       ),
  25.                                                         ‘alt’ => array(‘type’ => ‘string’),
  26.                                                         ‘border’ => array(  ‘type’ => ‘numeric’,
  27.                                                                             ‘min’ => 0,
  28.                                                                             ‘max’ => 5
  29.                                                                          ),
  30.                                                         ‘align’ => array( ‘type’ => ‘fixed’,
  31.                                                                             ‘values’ => array(  ‘center’,
  32.                                                                                                 ‘left’,
  33.                                                                                                 ‘right’
  34.                                                                                             )
  35.                                                                         ),
  36.                                                         ‘height’ => array(  ‘type’ => ‘numeric’,
  37.                                                                             ‘min’ => 1,
  38.                                                                             ‘max’ => 2048
  39.                                                                          ),
  40.                                                         ‘width’ => array(   ‘type’ => ‘numeric’,
  41.                                                                             ‘min’ => 1,
  42.                                                                             ‘max’ => 2048
  43.                                                                          ),
  44.                                                         ‘title’ => array(‘type’=>‘string’),
  45.                                                         ‘hspace’ => array(  ‘type’ => ‘numeric’,
  46.                                                                             ‘min’ => 0,
  47.                                                                             ‘max’ => 10
  48.                                                                          ),
  49.                                                         ‘vspace’ => array(  ‘type’ => ‘numeric’,
  50.                                                                             ‘min’ => 0,
  51.                                                                             ‘max’ => 10
  52.                                                                          ),
  53.                                                         ‘id’ => array( ‘type’ => ‘string’,
  54.                                                                           ‘auth_level’ => AUTH_LEVEL_TRUSTED_USER
  55.                                                                         ),
  56.                                                         ‘class’ => array( ‘type’ => ‘unrestricted’,
  57.                                                                           ‘auth_level’ => AUTH_LEVEL_TRUSTED_USER
  58.                                                                         ),
  59.                                                         ‘onclick’ => array( ‘type’ => ‘unrestricted’,
  60.                                                                           ‘auth_level’ => AUTH_LEVEL_TRUSTED_USER
  61.                                                                         ),
  62.                                                         ‘onmouseover’ => array( ‘type’ => ‘unrestricted’,
  63.                                                                           ‘auth_level’ => AUTH_LEVEL_TRUSTED_USER
  64.                                                                         )
  65.  
  66.                                                 )
  67.                             ),
  68.  
  69.                   );

The auth_level is a filter that I have in place to only allow users with a certain amount of access, such as internal staff, to use that attribute. As you probably already know, attributes like onmouseover are an easy gateway to XSS exploits. As you add new elements to the tag rule definition you just need to update the validate function in order to validate against it.

You may have noticed that I’m using the deprecated HTML attributes instead of putting all of this in a “style” attribute. I’ve found that most users don’t know CSS styles and are familiar with the old-style HTML attributes. If you force your users to use styles you’ll have whole other validation routine to ensure exploits are not in the CSS.

Note: Please don’t just copy/paste this code into your code and use it. I’m quite positive there are bugs and typos all throughout. Oh, and no, this is not the code used by the sites I work on. It is similar but definitely not the same. :P

]]> http://www.grepmymind.com/2009/01/15/validating-html-with-tidy/feed/ 1 Making frameworks suck less – SF PHP Meetup Nov 2008 http://www.grepmymind.com/2008/10/09/making-frameworks-suck-less-sf-php-meetup-nov-2008/ http://www.grepmymind.com/2008/10/09/making-frameworks-suck-less-sf-php-meetup-nov-2008/#comments Thu, 09 Oct 2008 14:58:35 +0000 Michael Tougeron http://www.grepmymind.com/?p=71 Terry Chay will be returning to speak again!  Due to popular demand Terry has agreed to give his recent “Making frameworks suck less” talk that he gave to the ZendCon08 UnCon this year. Be sure to wear comfortable shoes ’cause Terry will pack the room!

RSVP here: http://php.meetup.com/139/calendar/8911737/

The blurb from ZendCon08:
Terry Chay, the original PHP Terrorist, is well known for his criticism and ridicule of the Ruby on Rails framework. He criticizes, mocks, and f-bombs his enemies into surrender. In this discussion, Terry will share his ideas and thoughts on making frameworks better.

Map to CNET Networks, Inc.
When you arrive, please look for the PHP Meetup sign pointing you to the conference room. Please RSVP if you are attending so that I can prepare enough snacks and beverages.

Look forward to seeing you there!

This event can be found at:
http://php.meetup.com/139/calendar/8911737/
http://www.facebook.com/event.php?eid=39294579201
http://upcoming.yahoo.com/event/1195581/

]]> http://www.grepmymind.com/2008/10/09/making-frameworks-suck-less-sf-php-meetup-nov-2008/feed/ 0 SF PHP Meetup July 2008 – PHP::$unicode->i18n() http://www.grepmymind.com/2008/06/29/sf-php-meetup-july-2008-phpunicode-i18n/ http://www.grepmymind.com/2008/06/29/sf-php-meetup-july-2008-phpunicode-i18n/#comments Sun, 29 Jun 2008 18:15:14 +0000 Michael Tougeron http://www.grepmymind.com/?p=62 Kudos to Mariano for taking the lead while I’m out on paternity leave (I’m going to write about that ones of these days) and setting up July’s PHP Meetup.  Thanks to an introduction from Terry, Mariano asked Andrei Zmievski to talk about the unicode updates to PHP 6.  Due to the 4th of July, we’ve moved the meetup from it’s normal date/time to Thursday July 10th @ 7pm.  It is still at CNET like usual (thanks to GameSpot.com funding the meetup).

Title: PHP::$unicode->i18n()

Description:
PHP 6 brings fully functional and mature Unicode support to the Web world. This talk will cover all the layers of the PHP (bread)/Unicode (butter)/i18n (jam) sandwich. Come and find out how to work with locales, use collation to compare and sort strings, and format numbers, currencies, and dates for any country in the world. Bring your appetite because the toasty goodness is waiting.

When: Thursday, July 10th, 2008 at 7pm
Where: CNET Networks: 235 2nd St, San Francisco, CA  94518

You can RSVP on the SF PHP Meetup’s site or just show up.  I always like it when people RSVP so that I know how much food to buy.  But it isn’t a prereq or anything.

EDIT: I just saw that Terry mentioned PB&J and I like the idea.  So PB&J it is; plus the usual chips, sodas, candy, etc. of course.  :)

]]> http://www.grepmymind.com/2008/06/29/sf-php-meetup-july-2008-phpunicode-i18n/feed/ 0 MySQL Conf 2008 – PHP/DBUnit (Day 2) http://www.grepmymind.com/2008/04/15/mysql-conf-2008-phpdbunit-day-2/ http://www.grepmymind.com/2008/04/15/mysql-conf-2008-phpdbunit-day-2/#comments Tue, 15 Apr 2008 20:50:39 +0000 Michael Tougeron http://www.grepmymind.com/?p=51 I started off the day today listening to the keynotes at the MySQL Conference. Unfortunately I had to work on something so I wasn’t able to really pay attention. :( I hope I didn’t miss anything important.

After the keynotes, I decided to start off with the Testing PHP/MySQL Applications with PHPUnit/DbUnit session by Sebastian Bermann. I know I should already be using PHPUnit, but honestly I’ve never gotten around to it. I’m hoping that this will give me the kick in the ass that I need to start. :P

The first thing that Sebastian wanted us to note is that the PHPUnit listed on SourceForget.net is not the real PHPUnit. I kind of thought that weird and confusing. You’d think by now that he could get that old project de-listed.

Installing PHPUnit is fairly simple. It is not hosted off the standard PEAR channel, but by a quick extra command you can get access to it.
pear channel-discover pear.phpunit.de
pear install phpunit/PHPUnit

By setting up a reusable testing environment with PHPUnit you’ll be able to do automatic execution of your test code. This would be good for things like over night tests of the latest code commits. If a test fails, the developer team can be immediately notified. Be sure to enforce strict separation between production and testing environments. The last thing you want to happen is to have your test cases somehow break production.

Setting up a test case can be as simple or complex as you need it to be. You setup a class for each element/object you need to test. Each test case is a member function of this class where you assert value comparisons. While the documentation examples are best, here is a very simple example:

<?php
class BankAccountTest extends PHPUnit_Framework_TestCase {

    public function testBalanceIsInitiallyZero() {
        $ba = new BankAccount;
        $this->assertEquals(0, $ba->getBalance());
    }
}
?>

Where things can now start to get interesting is when you use the database extension to PHPUnit. The DBUnit extension was ported from jUnit by Michael Lively Jr.

DBUnit uses PDO to talk to a default SQLite database. However, a database it not required as it can emulate a database in memory or via XML. It also works with MySQL, but if the tests are written in a way that is compatible with SQLite, then the tests can be faster by about 20%.

Once of the nice things about how DBUnit works is how it puts the db into a known state between each test run. This means it avoids problems with one test case corrupting the db for other tests. By using its ability to import/export data to/from XML datasets, your db tests will run reliably and accurately.

Like with PHPUnit, DBUnit has assert methods like “assertTableEquals(expected, actual)” and can use PHP arrays for comparison. This can be good for comparing records in the database. e.g., The db gets updated via a PHPunit test and you then do a DBUnit test to verify the action.

A quick example of a DBUnit test case:

<?php
class BankAccountTest extends PHPUnit_Extensions_Database_TestCase {
    protected $pdo;

    public function __construct() {
        $this->pdo = PHPUnit_Util_PDO::factory($dsn);
        BankAccount::createTable($this->pdo);
    }

    protected function getConnection() {
        return $tihs->createDefaultDBConnection($this->pdo, 'mysql');
    }

    protect function getDataSet() {
        return $this->createFlatXMLDataSet('/path/to/seed.xml');
    }
}
?>
]]> http://www.grepmymind.com/2008/04/15/mysql-conf-2008-phpdbunit-day-2/feed/ 0 April SF PHP Meetup – symfony http://www.grepmymind.com/2008/03/28/april-sf-php-meetup-symfony/ http://www.grepmymind.com/2008/03/28/april-sf-php-meetup-symfony/#comments Fri, 28 Mar 2008 18:25:19 +0000 Michael Tougeron http://www.grepmymind.com/2008/03/28/april-sf-php-meetup-symfony/ Wow, I just realized how long it has been since I last posted here. It has been a crazy few weeks. Everything from setting up the baby furniture to fixing my rental condo, taxes to layoffs at CNET and more. I’m beat.

Anway to the topic at hand…

April’s PHP Meetup is looking like it will have another great turn out. The group just keeps growing and growing. Our average attendance for the last two months have been over 70 people! I still have a hard time believing that this many shy, techie geeks like me actually show up. :P

April’s topic will be run by Dustin Whittle from Yahoo! and will be about the symfony framework. He’ll be doing a brief overview of the framework and then walk people through setting up a site. If you’re planning on attending, bring your laptop and code along with us.

]]> http://www.grepmymind.com/2008/03/28/april-sf-php-meetup-symfony/feed/ 1 Feb 2008 PHP Meetup – Terry Chay: OOps! The PHP Fear and Loathing Guide to OO Design http://www.grepmymind.com/2008/01/18/feb-2008-php-meetup-terry-chay-oops-the-php-fear-and-loathing-guide-to-oo-design/ http://www.grepmymind.com/2008/01/18/feb-2008-php-meetup-terry-chay-oops-the-php-fear-and-loathing-guide-to-oo-design/#comments Fri, 18 Jan 2008 23:27:05 +0000 Michael Tougeron http://www.grepmymind.com/2008/01/18/feb-2008-php-meetup-terry-chay-oops-the-php-fear-and-loathing-guide-to-oo-design/ We have a topic now for February’s PHP Meetup thanks to Terry Chay.  He’s going to clean up and update a talk he has on object-orientated PHP programming and give it to the group on Feb 7th.

Terry has had a strong presence in the last couple of meetups; they probably wouldn’t have been successes without him.  This talk should be pretty interesting and should draw a good sized crowd.

On a side note, what’s up with meetup.com lately and their email system?  It used to be I’d email the group and it’d arrive in my inbox a few minutes later.  Still nothing.  Weird.

]]> http://www.grepmymind.com/2008/01/18/feb-2008-php-meetup-terry-chay-oops-the-php-fear-and-loathing-guide-to-oo-design/feed/ 1 last night’s geekSessions 1.3 http://www.grepmymind.com/2008/01/16/last-nights-geeksessions-13/ http://www.grepmymind.com/2008/01/16/last-nights-geeksessions-13/#comments Wed, 16 Jan 2008 19:22:38 +0000 Michael Tougeron http://www.grepmymind.com/2008/01/16/last-nights-geeksessions-13/ Last night’s geekSession as pretty interesting. The panel of PHP superstars knew their stuff (of course) and gave a high level 15 minute talk. For some reason I always enjoy watching when panelist debate back and forth over what the best approach is. Like when Lucas @ Facebook.com was talking about a few milliseconds of speed and Sara Golemon @ Yahoo! said it didn’t matter. :P All in good humor and it highlights that there are many solutions to the same problem. Some may work and be essential for one company and not make a difference for others. Facebook does get a shit load of traffic so they’re always looking to get that extra little bit out of things.

It was nice to meet Cal Henderson, I love his book about site scalability “Building Scalable Web Sites: Building, scaling, and optimizing the next generation of web applications” I highly recommend buying it if you haven’t already. Of course I didn’t want to be a groupie or anything so I didn’t tell him that. Wanted to though. :P

I didn’t get an opportunity to talk with Andrei; I didn’t recognize him (stupid eyes) and it was too noisy to really hear his introduction. I a bit too shy to try to re-introduce myself.

Surprisingly Terry Chay was quite quiet. Didn’t stop the speakers from cussing for him. Really glad he introduced me to Lucas though. That was a big help.

Speaking of Lucas, he’s working on a new talk and wants to try it out before going to the conferences with it. He’s thinking about bringing it to the PHP Meetup in the next couple of months to see how it goes. I hope that happens, bringing more well known PHP names to the group will help take it to the next level.

]]> http://www.grepmymind.com/2008/01/16/last-nights-geeksessions-13/feed/ 0 memcached PHP semaphore & cache expiration handling http://www.grepmymind.com/2008/01/11/memcached-php-semaphore-cache-expiration-handling/ http://www.grepmymind.com/2008/01/11/memcached-php-semaphore-cache-expiration-handling/#comments Fri, 11 Jan 2008 21:01:41 +0000 Michael Tougeron http://www.grepmymind.com/2008/01/11/memcached-php-semaphore-cache-expiration-handling/ There are a lot of different ways that people use memcached and PHP. The most common of which is probably your basic set and get to cache data from your database.

  1. function get_my_data1() {
  2.     $cache_id = "mykey";
  3.     $data = $memcache_obj->get($cache_id);
  4.     if ( !$data ) {
  5.         $data = get_data_from_db_function();
  6.         $memcache_obj->set($cache_id, $data, $sec_to_cache_for);
  7.     }
  8.     return $data;
  9. }

But what if the query that’s going to hit the database is pretty intensive and you don’t want more than one user to hit the db at a time? That’s easily handling via a semaphore lock.

  1. function get_my_data2() {
  2.     $cache_id = "mykey";
  3.     $data = $memcache_obj->get($cache_id);
  4.     if ( !$data ) {
  5.         // check to see if someone has already set the lock
  6.         $data_lock = $memcache_obj->get($cache_id . ‘_qry_lock’);
  7.         if ( $data_lock ) {
  8.             $lock_counter = 0;
  9.             // loop until you find that the lock has been released.  that implies that the query has finished
  10.             do while ( $data_lock ) {
  11.                 // you may only want to wait for a specified period of time.
  12.                 // one second is usually sufficient since your goal is to always have sub-second response time
  13.                 // if you query takes more than 1 second, you should consider "warming" your cached data via a cron job
  14.                 if ( $lock_counter > $max_time_to_wait ) {
  15.                     $lock_failed = true;
  16.                     break;
  17.                 }
  18.                 // you really want this to be a fraction of a second so the user waits as little as possible
  19.                 // for the simplicity of example, I’m using the sleep function.
  20.                 sleep(1);
  21.                 $data_lock = $memcache_obj->get($cache_id . ‘_qry_lock’);
  22.             }
  23.             // if the loop is completed, that either means the user waited for too long
  24.             // or that the lock has been removed.  try to get the cached data again; it should exist now
  25.             $data = $memcache_obj->get($cache_id);
  26.             if ( $data ) {
  27.                 return $data;
  28.             }
  29.         }
  30.         // set a lock for 2 seconds
  31.         $memcache_obj->set($cache_id . ‘_qry_lock’, true, 2);
  32.         $data = get_data_from_db_function();
  33.         $memcache_obj->set($cache_id, $data, $sec_to_cache_for);
  34.         // don’t forget to remove the lock
  35.         $memcache_obj->delete($cache_id . ‘_qry_lock’);
  36.     }
  37.     return $data;
  38. }

More below the break –>

Another good use of the semaphore locking is to set the expire time as part of the cached data and have an extended expire time set in memcache. This allows you to have more control over what happens when the cached data becomes stale. You can make it so that one user repopulates the cache while other users continue to get the existing cache until the first user has finished.

  1. function get_my_data3() {
  2.     $cache_id = "mykey";
  3.     $data = $memcache_obj->get($cache_id);
  4.     // if there is cached data and the expire timestamp has already expired or is within the next 2 minutes
  5.     // then we want the user to freshen up the cached data
  6.     if ( $data &amp;&amp; ($data[‘cache_expires_timestamp’] – time()) < 120 ) {
  7.         // if the semaphore lock has already been set, just return the data like you normally would.
  8.         if ( $memcache_obj->get($cache_id . ‘_expire_lock’) ) {
  9.             return $data;
  10.         }
  11.         // now we want to set the lock and have the user freshen the data.
  12.         $memcache_obj->set($cache_id . ‘_expire_lock’, true, 2);
  13.         // by unsetting the data it will cause the data gather logic below to execute.
  14.         unset($data);
  15.     }
  16.     if ( !$data ) {
  17.         // be sure to include all of the semaphore logic from example 2
  18.         // set the _qry_lock for 2 seconds
  19.         $memcache_obj->set($cache_id . ‘_qry_lock’, true, 2);
  20.         $raw_data = get_data_from_db_function();
  21.         $data[‘cache_expires_timestamp’] = time() + $sec_to_cache_for;
  22.         $data[‘cached_data’] = $raw_data;
  23.         $memcache_obj->set($cache_id, $data, $sec_to_cache_for);
  24.         // remove the _qry_lock
  25.         $memcache_obj->delete($cache_id . ‘_qry_lock’);
  26.         // remove the _expires_lock
  27.         $memcache_obj->delete($cache_id . ‘_expires_lock’);
  28.     }
  29.     return $data;
  30. }

When you mash these functions together, you end up with a system where only one user every freshens the cache and/or hits the database with a specific query at a time. There are a lot of other things that suddenly become available once you start thinking of memcached beyond just saving your db from hits. You have things like session handling, smarty template caching, flags for per-server processing (e.g., clearing local file cache), and even as a temporary database.

]]> http://www.grepmymind.com/2008/01/11/memcached-php-semaphore-cache-expiration-handling/feed/ 7