Archive for the ‘General’ Category
Node.js and nginx
This took me some time to figure out, and I didn’t see any detailed posts or bug reports on how to fix this. Nginx doesn’t support HTTP 1.1 on proxy pass, meaning, when you place Node.JS behind a proxy (for load balancing purposes, or you just have multiple endpoints on port 80), websockets will not work properly, since HTTP 1.1 is a core requirement. You’ll know, when you get errors similar to this:
|
1 2 |
Error during WebSocket handshake: 'Connection' header value is not 'Upgrade'
XMLHttpRequest cannot load ***. Origin *** is not allowed by Access-Control-Allow-Origin. |
I’m running nginx 0.6.8, with nginx 1.0.11. To fix this, you need to upgrade to a later version of nginx (a development version), which supports HTTP 1.1 (albeit, experimentally), and then enable the proxy_http_version 1.1 parameter in your vhost configuration.
I’m doing this on Ubuntu.
First, let’s compile nginx:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
cd /usr/src
wget http://nginx.org/download/nginx-1.1.13.tar.gz
sudo tar xzvf nginx-1.1.13.tar.gz
cd nginx-1.1.13
sudo ./configure --prefix=/etc/nginx --sbin-path=/usr/sbin --with-http_ssl_module
sudo make; sudo make install
/usr/sbin/nginx -V
# Something like this should show:
nginx version: nginx/1.1.13
built by gcc 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3)
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin --with-http_ssl_module |
Next, we setup our vhost:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
server {
listen 80;
server_name node.domain.com;
location / {
proxy_pass http://localhost:8000;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_redirect off;
proxy_http_version 1.1;
}
} |
And now no more errors, and nodejs is working properly. Do note that this is a “bleeding edge” version of nginx, and could come with its own share of issues – so keep an eye out and test thoroughly!
Edit: If you’re still running into some problems, you can enable only xhr-polling/jsonp-polling in your Node.JS/socket.io configuration:
|
1 2 3 4 |
io.set('transports', [
'xhr-polling',
'jsonp-polling'
]); |
Domain updated!
I’m now using “nabeelio.com” – since it’s been my long-time nickname, and nsslive just doesn’t mean anything anymore. All links have been updated, the old site will just redirect here. Cheers!
Installing Redmine on Ubuntu 11.04 w/nginx and mongrel
This one took me a few hours, but I’ve got my handy-dandy notes. I’m going to assume you’re got nginx installed, whether there are vhosts or not…
I’m also installing to /var/www/redmine
|
1 2 3 4 5 6 7 |
sudo apt-get install mongrel ruby gems
cd /var/www
wget http://rubyforge.org/frs/download.php/75097/redmine-1.2.1.tar.gz
tar xzvf redmine-1.2.1.tar.gz
mv redmine-1.2.1 redmine
sudo chown www-data: redmine -R
sudo chmod 775 redmine -R |
Next, we are going to patch Redmine, to work with Mongrel
|
1 2 3 4 5 |
cd /var/www/redmine/config/initializers/
wget http://www.redmine.org/attachments/6146/rails_6440_patch.rb
wget https://gist.github.com/raw/826692/cb0dcf784c30e6a6d00c631f350de99ab99e389d/mongrel.rb
sudo chmod 775 . -R
sudo chown www-data: -R |
Next, setup the right versions of Rails, etc
|
1 2 3 |
gem install -v=2.3.14 rails
gem install rack -v=1.1.1
gem install rake -v0.8.7 |
Now setup MySQL:
|
1 2 3 4 |
mysql -uroot -p
create user 'redmine'@'localhost' identified by 'password';
grant all privileges on `redmine%` . * to 'redmine'@'localhost';
create database redmine character set utf8; |
And next, we configure and run the installer for Redmine. We are going to edit the database.yml, set it to match your above settings
|
1 2 3 4 5 6 7 |
cd /var/www/redmine/config
mv database.yml.production database.yml
nano database.yml
cd /var/www/redmine
rake generate_session_store
RAILS_ENV=production rake db:migrate
RAILS_ENV=production rake redmine:load_default_data |
Next, we start the server
|
1 |
mongrel_rails start -e production -p 9001 -d |
Next, create the nginx vhost, I created it as /etc/nginx/sites-enabled/redmine
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
server {
listen 80;
server_name YOUR_HOSTNAME_HERE;
root /var/www/redmine/public;
#error_log /var/log/nginx/redmine.log debug;
expires epoch;
location / {
expires epoch;
alias /var/www/redmine/public/;
try_files $uri/index.html $uri.html $uri @mongrel;
}
location @mongrel {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_pass_header Set-Cookie;
proxy_pass http://127.0.0.1:9001;
}
} |
MySQL Diff Tool
After searching for a while, I haven’t been able to find a tool which will show the diffs between two MySQL Databases. There are plenty to handle migrations, but migrations are tough when you’re writing an app which is install by an end-user. So I wrote a tool/class which will take the XML of a proper database (that file can be distributed in your package), and then will compare the XML schema against the schema in the current database.
Generate a MySQL Dump file:
|
1 |
mysqldump --xml --no-data testuser -utestuser -ptest1 > structure.xml |
Then call the command line script (diffgen):
|
1 2 3 4 5 6 7 8 |
diffgen -utestuser -ptest1 -dtestuser -hlocalhost -fstructure.xml -tshow
-u Database User
-p Database Password
-d Database Name
-h Database Host
-f Dump File Path
-t "show" or "run" - show will output the SQL, "run" will run the SQL |
There’s also a class file (which it is all from), which you can use to integrate into your own custom scripts (as-is the case with phpVMS, which is distributed with the structure.xml that is generated by my Phing build process, and it “shapes” the database on the remote server properly in an update script).
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
include 'MySQLDiff.class.php';
$params = ;
try {
$diff = new MySQLDiff(array(
'dbuser' => 'testuser',
'dbpass' => 'test1',
'dbname' => 'testuser',
'dbhost' => 'localhost',
'dumpxml' => 'structure.xml',
));
} catch(Exception $e) {
echo $e->getMessage(); exit;
}
# This returns an array of what's missing in the database
try {
$diff_lines = $diff->getDiffs();
var_dump($diff_lines);
catch(Exception $e) {
echo $e->getMessage(); exit;
}
# This returns SQL queries which can be run to fix the database
try {
$diff_lines = $diff->getSQLDiffs();
var_dump($diff_lines);
} catch(Exception $e) {
echo $e->getMessage(); exit;
}
# This generates the SQL and actually runs all of them
try {
$diff_lines = $diff->runSQLDiff();
var_dump($diff_lines);
} catch(Exception $e) {
echo $e->getMessage(); exit;
} |
The script can be downloaded from https://github.com/nshahzad/MySQLDiff
Amazon PHP API
I couldn’t find a working PHP class for the Amazon API, which had Exception handling and some versatility. So I wrote one up. It’s a work-in-progress at the moment, but it’s available on GitHub:
http://github.com/nshahzad/AmazonAPI
The included README has detailed instructions. The class uses the __call() functionality, and you just pass the required parameters as a dictionary array. Requires some reading of the Amazon docs, but much more flexible.
|
1 2 |
include 'amazon.php';
$amz = new AmazonProductLookup('YOUR AWS KEY', 'YOUR SECRET KEY'); |
PHP class for Google Geocoder API
For a project I’ve been working on, I needed to access Google’s Geocoder API. I search for names (schools in this case), and return as much info as I can that Google has about it.
I’ve posted the class up on Github, it’s straightforward, and might help some people out:
http://github.com/nshahzad/Google-Geocoder
The usage is in the readme/displayed right on the github page. It uses cURL and JSON to keep the traffic transfered low. That also means you need the json_decode() function, which is in PHP 5.2 and up.
Happy 4th!
Building php-fpm against Ubuntu PHP Packages
This is how I’ve been building php-fpm against the Debian PHP packages. It’ll be useful for when Ubuntu 10.04 (Lucid Lynx) Comes out with PHP 5.3. I do this from my home directory. It will download the package souce from Ubuntu, then compile php-fpm standalone against that.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
sudo apt-get install php5-cli php5-cgi php5-common \
php5-curl php5-dev php5-gd php5-mcrypt \
php5-memcache php5-mysql php5-suhosin
# Get from the Debian repo
sudo apt-get build-dep php5-common
sudo apt-get -b source php5-common
# Change this to the PHP version being used!
# Look on http://launchpad.net/php-fpm/master to match it
export PHP_VER=5.2.10
wget "http://launchpad.net/php-fpm/master/0.6/+download/php-fpm-0.6-$PHP_VER.tar.gz"
tar -zxvf "php-fpm-0.6-$PHP_VER.tar.gz"
cd "php-fpm-0.6-$PHP_VER"
mkdir fpm-build && cd fpm-build
sudo ../configure --srcdir=../ \
--with-php-src="../../php5-5.2.10.dfsg.1" \
--with-php-build="../../php5-5.2.10.dfsg.1/cgi-build" \
--with-fpm-conf="/etc/php5/fpm/php-fpm.conf" \
--with-libevent="/usr/lib"
sudo make
sudo make install
sudo update-rc.d php-fpm defaults
# Resulting in:
Installing PHP FPM binary: /usr/local/bin/php-fpm
Installing PHP FPM config: /etc/php5/fpm/php-fpm.conf
Installing PHP FPM man page: /usr/local/man/man1/php-fpm.1
Installing PHP FPM init script: /etc/init.d/php-fpm
*** FPM Installation complete. ***
run:
`update-rc.d php-fpm defaults; invoke-rc.d php-fpm start` |
Then, configure the php-fpm file. That should be it 
A better way for nginx PHP config
Doing some reconfiguration on my webserver (nginx) to make it easier to administer. My first goal was to get rid of this nastiness:
|
1 2 3 4 5 6 7 8 9 10 |
server {
...
location ~ \.php$ {
include /etc/nginx/conf/fastcgi_params;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /var/path/to/$fastcgi_script_name;
}
} |
It’s too verbose to copy/paste into each virtual host file. Instead, you can just combine the file into the /etc/nginx/conf/fastcgi_params file. I renamed it to php_params, and this is what it’s got:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
location ~ \.php(.*)$ {
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_param DOCUMENT_URI $document_uri;
fastcgi_param DOCUMENT_ROOT $document_root;
fastcgi_param SERVER_PROTOCOL $server_protocol;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param GATEWAY_INTERFACE CGI/1.1;
fastcgi_param SERVER_SOFTWARE nginx/$nginx_version;
fastcgi_param REMOTE_ADDR $remote_addr;
fastcgi_param REMOTE_PORT $remote_port;
fastcgi_param SERVER_ADDR $server_addr;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_NAME $server_name;
# PHP only, required if PHP was built with --enable-force-cgi-redirect
fastcgi_param REDIRECT_STATUS 200;
} |
Now I don’t have to change it everywhere. So, instead, now I do:
|
1 2 3 4 5 |
server {
...
# Include the PHP Fast-CGI Params
include /etc/nginx/conf/php_params;
} |
Bam! 6 lines down to one, and much easier to administer. I like, I like.
PHP Resources
I put a list together for a friend of some good PHP resources, thought I’d stick it up here as well:
Of course, the best resource, the official docs:
http://www.php.net
Another great (official) place:
http://talks.php.net/
The talks are by the creators of PHP. Any talks by Rasmus Lerfdorf are excellent, he stresses simplicity over complexity. He’s also the creator of PHP. Derick Rethans is also an excellent presenter, he focuses a lot on security and debugging. Definitely watch the presentations in the “Security” section of the talks, but overall, any talk in there has information you can use to your advantage.
Other sites:
http://www.smashingmagazine.com/2009/03/24/10-useful-php-tips-revisited/
http://php.about.com/od/advancedphp/Advanced_PHP.htm
Sitepoint is where I first started learning HTML and CSS many years ago:
http://www.sitepoint.com/subcat/php-tutorials
MVC tutorials (how apps should be coded; obviously there’s some contention between OO and procedural styles, but you need knowledge of both to be able to make an educated judgment about what a good balance between the two is)
http://www.phpro.org/tutorials/Model-View-Controller-MVC.html
Good to go through to understand MVC completely
CakePHP has a good introduction:
http://book.cakephp.org/view/10/Understanding-Model-View-Controller
Which brings me to CakePHP itself. It’s an excellent MVC framework; after trying out CodeIgniter, Zend, Yii, Kohana, I’ve settled on Cake.
http://cakephp.org/
SQL resources – the best is the manual. Learning the concepts behind joins is essential and important. A good tutorial:
http://www.codinghorror.com/blog/archives/000976.html
Also database design:
http://www.simple-talk.com/sql/database-administration/ten-common-database-design-mistakes/
http://woork.blogspot.com/2008/09/10-useful-articles-about-database.html
Following that up are good ORM, which you may want to use as your database layer. CakePHP has ORM built-in, but sometimes all you need is just a DB layer.
http://www.doctrine-project.org/
For conventions, I tend to follow the CakePHP model (since that’s the framework I use the most):
http://book.cakephp.org/view/24/Model-and-Database-Conventions
http://bakery.cakephp.org/articles/view/database-design-and-cakephp
http://book.cakephp.org/view/22/CakePHP-Conventions
And then rounding it out, some general knowledge information:
http://articles.sitepoint.com/category/html
http://articles.sitepoint.com/category/javascript
http://articles.sitepoint.com/category/cssh
http://www.jquery.com
CakePHP Models – multiple columns to the same table
This one took me a few to figure out. On VACentral, there are schedules, which have an arrival and departure point. These points are all stored in one table, so one row in schedule refers to multiple entries in the airports table. It looks something like (ok, not something like, but exactly):
|
1 2 3 4 5 |
Schedules:
id | departure_icao | arrival_icao
Airports
id | icao |
So two ICAO columns in routes map to one same column in airports. The ICAO is a unique 4 character identifier, which is assigned to an airport. It’s quite simple actually, but took me a while to figure it it. First the Airports model:
|
1 2 3 4 5 6 |
class Airport extends AppModel
{
public $name = 'Airport';
public $primaryKey = 'id';
public $actAs = array('Containable');
} |
And then our Schedules model:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
class Schedule extends AppModel {
public $name = 'Schedule';
public $primaryKey = 'id';
public $actsAs = array('Containable');
public $belongsTo = array(
'DepartureAirport' => array(
'className' => 'Airport',
'foreignKey' => false,
'conditions' => 'DepartureAirport.icao = Schedule.departure_icao',
'fields' => '',
'order' => ''
),
'ArrivalAirport' => array(
'className' => 'Airport',
'foreignKey' => false,
'conditions' => 'ArrivalAirport.icao = Schedule.arrival_icao',
'fields' => '',
'order' => ''
)
);
} |
So we used the $belongTo relationship, and we will define two relationships – “DepartureAirport” and “ArrivalAirport”. We also select the class we will use (which IMO, should really be called “modelName” or “useModel”, that really tripped me up, but I digress). Next, we define the conditions – we’ll use the relationship name (DepartureAirport or ArrivalAirport), and the column name, along with the column name on the current table it should join on. And that’s pretty much it. You don’t really need a relationship on the “receiving” end (the Airports table), unless you will be querying airports, and finding out what schedules go there. I’ll leave that upto you 
And then for the query itself:
|
1 2 3 4 5 6 |
$this->Schedule->contain('DepartureAirport', 'ArrivalAirport');
$schedule = $this->Schedule->find('first');
// Our Airport specific data will be contained in:
$schedule['DepartureAirport']
$schedule['ArrivalAirport'] |
Which will now return something like (etc fields ommitted):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
Array
(
[Schedule] => Array
(
[schedule_id] => 4178
[airline_id] => 2
[code] => AEA
[flightnum] => 6371
[depicao] => CYUL
[arricao] => KJFK
)
[DepartureAirport] => Array
(
[airport_id] => 597
[iata] => YUL
[icao] => CYUL
[name] => Montreal / Pierre Elliot Trudeau International Airport, Quebec
[timezone] => US/Eastern
[location] => Montreal QC Canada
[lat] => 45.470556
[lng] => -73.740833
)
[ArrivalAirport] => Array
(
[airport_id] => 268
[iata] => JFK
[icao] => KJFK
[name] => JFK Airport
[timezone] => US/Eastern
[location] => New York-Kennedy NY
[lat] => 40.6398262
[lng] => -73.7787443
)
) |
Note how it’s using the Containable behavior; this is so it doesn’t pull every relationship you’ve defined with that table (the schedules table above has many more relationships, but for brevity, I only pulled the relevant ones). Not specifying Containable() is REALLY expensive, especially when you don’t need all those relationships to be included in every time! To speed it up even more, you should specify the actual field names to pull (the SQL * operator is expensive).
