Grep My Mind » semaphore http://www.grepmymind.com Wonder what’s on my mind? Now you know. Thu, 11 Jun 2009 20:00:55 +0000 en hourly 1 http://wordpress.org/?v=3.0 memcached PHP semaphore & cache expiration handling http://www.grepmymind.com/2008/01/11/memcached-php-semaphore-cache-expiration-handling/ http://www.grepmymind.com/2008/01/11/memcached-php-semaphore-cache-expiration-handling/#comments Fri, 11 Jan 2008 21:01:41 +0000 Michael Tougeron http://www.grepmymind.com/2008/01/11/memcached-php-semaphore-cache-expiration-handling/ There are a lot of different ways that people use memcached and PHP. The most common of which is probably your basic set and get to cache data from your database.

  1. function get_my_data1() {
  2.     $cache_id = "mykey";
  3.     $data = $memcache_obj->get($cache_id);
  4.     if ( !$data ) {
  5.         $data = get_data_from_db_function();
  6.         $memcache_obj->set($cache_id, $data, $sec_to_cache_for);
  7.     }
  8.     return $data;
  9. }

But what if the query that’s going to hit the database is pretty intensive and you don’t want more than one user to hit the db at a time? That’s easily handling via a semaphore lock.

  1. function get_my_data2() {
  2.     $cache_id = "mykey";
  3.     $data = $memcache_obj->get($cache_id);
  4.     if ( !$data ) {
  5.         // check to see if someone has already set the lock
  6.         $data_lock = $memcache_obj->get($cache_id . ‘_qry_lock’);
  7.         if ( $data_lock ) {
  8.             $lock_counter = 0;
  9.             // loop until you find that the lock has been released.  that implies that the query has finished
  10.             do while ( $data_lock ) {
  11.                 // you may only want to wait for a specified period of time.
  12.                 // one second is usually sufficient since your goal is to always have sub-second response time
  13.                 // if you query takes more than 1 second, you should consider "warming" your cached data via a cron job
  14.                 if ( $lock_counter > $max_time_to_wait ) {
  15.                     $lock_failed = true;
  16.                     break;
  17.                 }
  18.                 // you really want this to be a fraction of a second so the user waits as little as possible
  19.                 // for the simplicity of example, I’m using the sleep function.
  20.                 sleep(1);
  21.                 $data_lock = $memcache_obj->get($cache_id . ‘_qry_lock’);
  22.             }
  23.             // if the loop is completed, that either means the user waited for too long
  24.             // or that the lock has been removed.  try to get the cached data again; it should exist now
  25.             $data = $memcache_obj->get($cache_id);
  26.             if ( $data ) {
  27.                 return $data;
  28.             }
  29.         }
  30.         // set a lock for 2 seconds
  31.         $memcache_obj->set($cache_id . ‘_qry_lock’, true, 2);
  32.         $data = get_data_from_db_function();
  33.         $memcache_obj->set($cache_id, $data, $sec_to_cache_for);
  34.         // don’t forget to remove the lock
  35.         $memcache_obj->delete($cache_id . ‘_qry_lock’);
  36.     }
  37.     return $data;
  38. }

More below the break –>

Another good use of the semaphore locking is to set the expire time as part of the cached data and have an extended expire time set in memcache. This allows you to have more control over what happens when the cached data becomes stale. You can make it so that one user repopulates the cache while other users continue to get the existing cache until the first user has finished.

  1. function get_my_data3() {
  2.     $cache_id = "mykey";
  3.     $data = $memcache_obj->get($cache_id);
  4.     // if there is cached data and the expire timestamp has already expired or is within the next 2 minutes
  5.     // then we want the user to freshen up the cached data
  6.     if ( $data &amp;&amp; ($data[‘cache_expires_timestamp’] – time()) < 120 ) {
  7.         // if the semaphore lock has already been set, just return the data like you normally would.
  8.         if ( $memcache_obj->get($cache_id . ‘_expire_lock’) ) {
  9.             return $data;
  10.         }
  11.         // now we want to set the lock and have the user freshen the data.
  12.         $memcache_obj->set($cache_id . ‘_expire_lock’, true, 2);
  13.         // by unsetting the data it will cause the data gather logic below to execute.
  14.         unset($data);
  15.     }
  16.     if ( !$data ) {
  17.         // be sure to include all of the semaphore logic from example 2
  18.         // set the _qry_lock for 2 seconds
  19.         $memcache_obj->set($cache_id . ‘_qry_lock’, true, 2);
  20.         $raw_data = get_data_from_db_function();
  21.         $data[‘cache_expires_timestamp’] = time() + $sec_to_cache_for;
  22.         $data[‘cached_data’] = $raw_data;
  23.         $memcache_obj->set($cache_id, $data, $sec_to_cache_for);
  24.         // remove the _qry_lock
  25.         $memcache_obj->delete($cache_id . ‘_qry_lock’);
  26.         // remove the _expires_lock
  27.         $memcache_obj->delete($cache_id . ‘_expires_lock’);
  28.     }
  29.     return $data;
  30. }

When you mash these functions together, you end up with a system where only one user every freshens the cache and/or hits the database with a specific query at a time. There are a lot of other things that suddenly become available once you start thinking of memcached beyond just saving your db from hits. You have things like session handling, smarty template caching, flags for per-server processing (e.g., clearing local file cache), and even as a temporary database.

]]> http://www.grepmymind.com/2008/01/11/memcached-php-semaphore-cache-expiration-handling/feed/ 7