Category: Programming

This week I had the privilege of doing load testing on an aws setup. The goal of the test was to support 25k concurrent users and was designed to ramp up over a period of a few minutes. The first round of testing failed pretty quick and I only had a select number of large queries cached. So the db spiked and was the bottle neck. During that test we was only able to reach 10k concurrents.

For the second round of testing I decided to just setup output caching before I even loaded the db in a MY_Controller. I set the number of minutes to something relatively high and was ready to commence the load test. Unfortunately this test maxed out the db again and I was honestly scratching my head on just why that would be the case.

Since this didn’t make any sense to me I decided I better figure just what is going on. I got my local system setup the exact same way, enabled mysql query log, and then proceded to test and see what was going on. Queries was being ran even after the cache file was written. At this point it was way late so I decided I better sleep on it and try and think about what I was missing.

The next morning I got to work ran the same tests and got the same results. Starting researching and then realized it was just my ajax calls that was generating the queries. This had me puzzled because the ajax controller extended the MY_Controller so it should be cached as well. Turns out I was wrong and missed the most important note in the user guide:

Warning: Because of the way CodeIgniter stores content for output, caching will only work if you are generating display for your controller with a view.

The more I thought about it the more sense that makes since the Loader ties into the output library but I wasn’t thinking clearly at 3am. Looked at my ajax controller and sure enough I was just returning a json string.

return json_encode($vars);

The quick fix was to change the return to the following:

return $this->output->set_output(json_encode($vars));

My hope is this will help others if they ever run into the same situation. Unfortunately the load testing was only scheduled for two rounds but I feel confident by fixing that it would have passed with flying colors.

So the moral of the story. READ THE MANUAL 🙂

I am happy to announce a simple CodeIgniter library to help you generate slugs for your urls. This is something I use on almost every project and it has been invaluable to me. Hopefully you will find it useful as well.

For those that are not familiar with the term slug WordPress defines it as:

A slug is a few words that describe a post or a page. Slugs are usually a URL friendly version of the title.

Description

What this library does is allow you to pass a string such as a titile and then generate a url friendly string from it. Also and most importantly it checks against fields in your db so that you will not have any duplicates. For example:

"This is my title"

Would become:

"this-is-my-title"

If a duplicate is found then it appends a number. Example:

"this-is-my-title-1"

Example

First load the library with an array of config items:

$config = array(
    'table' => 'mytable,
    'id' => 'id',
    'field' => 'uri',
    'title' => 'title',
    'replacement' => 'dash' // Either dash or underscore
);
$this->load->library('slug', $config);

When creating a uri for a new record in your db you can do it like this:

$data = array(
    'title' => 'My Test',
);
$data['uri'] = $this->slug->create_uri($data);
$this->db->insert('mytable, $data);

Then for when you are editing an existing record: (Notice the create_uri uses the second param to compare against other fields).

$data = array(
    'title' => 'My Test',
);
$data['uri'] = $this->slug->create_uri($data, $id);
$this->db->where('id', $id);
$this->db->update('mytable', $data);

Download

You can clone the repo by running the code below:

$ git clone git://github.com/ericbarnes/CodeIgniter-Slug-Library

Or visit the GitHub Repo.

I am currently building a pretty large CMS system on top of CodeIgniter and I wanted to share some of the best performance optimizations I have found. Although CodeIgniter is optimized out of the box, when you start building large scale applications with large controllers, it is easy to miss things that can add up to your overall performance.

Do Not Save Queries

$this->db->save_queries = FALSE;

By default CodeIgniter keeps an array of all the queries¹ ran so that they can be used in the profiler for debugging. This is great during development but in most production cases that is not needed and just wasted memory. I recommend disabling this in production mode and can be conditionally setup with a simple if statement:

if (ENVIRONMENT == 'production')
{
    $this->db->save_queries = FALSE;
}

Output Cache

CodeIgniter supports several levels of caching and the most extreme is caching is the entire output. That is designed to cache the entire parsed page then just display it the html for the amount of minutes you designate.

By default CodeIgniter writes this cache to files which increases your i/o and it might suit you better to use apc² (example) or memcache for the storage.

The downside to this is if you are needing some form of dynamic data like user information. Then it will not work. If that is a requirement for you then consider caching queries or other areas where your site is slow. Or you can do like getsparks and use ajax calls to dynamically populate the user information.

Consider Nginx + php-fpm + apc

I know everybody uses Apache and it has tons of tutorials and an all around good server. But in my tests straight out of the box Nginx out performs Apache and I am not the only one

I had a standard AWS medium instance running about 25 sites that was averaging 30% CPU usage during the day with Apache + APC. After just switching to Nginx and php-fpm it is now averaging around 10%. That is a huge difference!

Autoloading Can Be Dangerous

CodeIgniter comes with an “Auto-load” feature that permits libraries, helpers, and models to be initialized automatically every time the system runs.

Lets face it autoloading is awesome, but does come with some drawbacks. The biggest is that everything autoloaded is loaded on every request. This means for every page and every ajax call all those items are being loaded. So unless you are, in fact, going to need it for everything then I wouldn’t recommend auto loading. Instead consider base controllers or loading only when needed.

Only Load What You Need

Touching on autoloading is another common issue I have seen. Loading when you don’t need too. Ideally you do not want to load anything unless you are going to use it in the code that is running.

class Welcome extends CI_Controller {
    public function __construct()
    {
        $this->load->library('email');
        $this->load->library('ftp');
    }
    public function index()
    {
        $this->load->view('home_page');
    }
}

As you can see in the example above we are loading two libraries that will never be used when viewing the home page so by including them you are just wasting resources.

Here are some changes I needed to make to my Nginx conf that didn’t seem obvious to me and will hopefully be of help to others:

Random 502 gateway errors

fastcgi_buffers 8 16k;
fastcgi_buffer_size 32k;

413 Request Entity Too Large

This is caused because by default file uploads are limited to 1mb.

server {
    client_max_body_size 20M;

Resources:

• nginx error – 413 Request Entity Too Large
• Fixed the upstream sent too big header error

Currently I am in the process of switching a server from Apache to Nginx and here is a quick tip for those using CodeIgniter or who want to host a bunch of very similar domains. For each server line I do this: 

server {
    server_name .mysite.com;
    root /sites/mysite/www;
    include /sites/nginx/ci_vhost;
}
server {
    server_name .mysecondsite.com;
    root /sites/secondpath/www;
    include /sites/nginx/ci_vhost;
}

Then the ci_vhost includes the common settings: 

index index.html index.php index.htm;

# set expiration of assets to MAX for caching
location ~* .(ico|css|js|gif|jpe?g|png)(?[0-9]+)?$ {
    expires max;
    log_not_found off;
}

location / {
    # Check if a file exists, or route it to index.php.
    try_files $uri $uri/ /index.php;
}

location ~* .php$ {
    fastcgi_pass 127.0.0.1:9000;
    fastcgi_index index.php;
    fastcgi_split_path_info ^(.+.php)(.*)$;
    include fastcgi_params;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
}

This allows you to have all the shared settings in one location and will make future changes much easier. Also this works perfectly for me with CodeIgniter’s default setup. Removing index.php and leaving uri_protocol to AUTO.

The goal of this project is to provide a simple way to minify and combine js and css files inside a CodeIgniter application. Currently other systems exists but I wanted the compression to be part of my build process. So on deployments I compress and minify all the js and css. Then push off to s3 but this could also be useful to write them to a single file.

Visit the CodeIgniter Minify repository.

Usage

Below is an overview of different usages:

Minify JS file

$this->load->driver('minify');
$file = 'test/js/test1.js';
echo $this->minify->js->min($file);

Minify CSS file

$this->load->driver('minify');
$file = 'test/css/test1.css';
echo $this->minify->css->min($file);

Minify String

$this->load->driver('minify');
$content = 'body{ padding:0; margin: 0}';
echo $this->minify->css->min($content);

Minify and combine an array of files. Useful if you need files to be in a certain order.

$this->load->driver('minify');
$files = array('test/css/test2.css', 'test/css/test1.css');
echo $this->minify->combine_files($files, [optionalParams]);

Minify and save a physical file

$this->load->driver('minify');
$file = 'test/css/test1.css';
$contents = $this->minify->css->min($file);
$this->minify->save_file($contents, 'test/css/all.css');

Minify an entire directory. The second param is an array of ignored files.

$this->load->driver('minify');
echo $this->minify->combine_directory('test/css/, array('all.css'), [optionalParams]);

Optional Params

combine_files($files, [type], [compact], [css_charset]);
combine_directory($directory, [ignore], [type], [compact], [css_charset]);

Common:

[type]: string ('css' or 'js')
[compact] : bool (TRUE, FALSE). TRUE Compact/compress output, FALSE doesn't compress output (only aggregation)
[css_charset] : string (default 'utf-8'). If CSS you can force a starting single charset declaration (when aggregate files)
               due to the charset pre-removal (for stantdars compliance and Webkit bugfix prevention)
               set to null or leave empty if JS.

Combine dir:

[ignore] : array with files to ignore

Now updated for MAMP 2.0.1

I like using MAMP Pro on my mac because it is so easy to setup and the included virtual hosts are very nice. Plus it is easy to install for designers and those that don’t like mucking in the file system.

One gotcha though is installing php, pear, and then phpunit so it will run through terminal.  I know that there are tons of tutorials on this but from my recent research finding the exact steps hasn’t been very simple. (Plus some of them are just strange ways of doing it imo) So I decided I should write out a post describing all the steps to get this setup and have this for future reference
After you have mamp pro installed open terminal and follow these steps…

1. Backup old files. (just incase)

sudo mv /usr/bin/php /usr/bin/php_bak
sudo mv /usr/bin/pear /usr/bin/pear_bak
sudo mv /usr/bin/phpunit /usr/bin/phpunit_bak

2. Symlink the mamp “php” and “pear” file

sudo ln -s /Applications/MAMP/bin/php/php5.3.6/bin/php /usr/bin/php
sudo ln -s /Applications/MAMP/bin/php/php5.3.6/bin/pear /usr/bin/pear

3. Update pear info

sudo pear config-set  doc_dir /Applications/MAMP/bin/php/php5.3.6/lib/php/doc
sudo pear config-set  ext_dir /Applications/MAMP/bin/php/php5.3.6/lib/php/extensions
sudo pear config-set  php_dir /Applications/MAMP/bin/php/php5.3.6/lib/php/
sudo pear config-set  php_bin /Applications/MAMP/bin/php/php5.3.6/bin/php
sudo pear config-set  php_ini /Applications/MAMP/bin/php/php5.3.6/conf/php.ini
sudo pear config-set  sig_keydir /Applications/MAMP/bin/php/php5.3.6/pearkeys
# This fixes a bug in pear
sudo rm /Applications/MAMP/bin/php/php5.3.6/conf/pear.conf 

4. Upgrade pear

sudo pear channel-update pear.php.net
sudo pear upgrade pear
pear -V
PEAR Version: 1.9.4
PHP Version: 5.3.6

5. Install phpUnit

sudo pear channel-discover pear.phpunit.de
sudo pear channel-discover components.ez.no
sudo pear channel-discover pear.symfony-project.com
sudo pear install --alldeps phpunit/PHPUnit

6. Move “phpunit” to /usr/bin

sudo ln -s  /Applications/MAMP/bin/php/php5.3.6/bin/phpunit /usr/bin/phpunit</pre>

7. Optionally Install vfStream:

sudo pear channel-discover pear.php-tools.net
sudo pear install pat/vfsStream-alpha

That should be it. At this point php, pear, and phpunit should all work via terminal.

Errors

If you get this error: Notice: unserialize(): Error at offset 276 of 1133 bytes in Config.php on line 1050 ERROR: The default config file is not a valid config file or is corrupted.

Try removing this file:

sudo rm /Applications/MAMP/bin/php/php5.3.6/conf/pear.conf

A common issue I have seen lately is people using the new CSRF protection inside CodeIgniter 2.0.  This is a great new feature but it affects all post data no matter if it comes from a form or from an ajax call.  The ajax part is what seems to be tripping people up.  So I decided to write a short overview on how to accomplish sending jQuery ajax posts with CSRF enabled.

In this tutorial I will be using the jQuery cookie plugin for simplicity.

Here is the whole javascript to handle the ajax:

$("#selector").click(function () {
    e.preventDefault();
    var form_data = {
        cat_name: $('#something').val(),
        csrf_token_name: $.cookie("csrf_cookie_name")
    };

    $.ajax({
        type: "POST",
        url: SITE_URL + "controller",
        data: form_data,
        success: function(data) {
            alert('it worked');
        }
    });
});

As you can see the only thing I had to add was the following line:

csrf_token_name: $.cookie("csrf_cookie_name")

Hopefully this will help some of you folks out.