We previously talked about the importance of building fast websites and easy ways to achieve that by enabling gzip compression and using CSS sprites.

That’s all well and good, but how do you know if the changes you made actually translate to faster website load times for your visitors? An easy way to measure this is to use Google Analytics, an extremely popular analytics tool. Having performance data for your website is very nice in that it allows you to measure:

• Content – which landing pages are slowest?
• Visitor – how does latency compare by visitor type or geographic region?
• Technology – does browser, operating system or screen resolution impact latency metrics?
• Performance – what is the most common load time, or the longest?

This metric is not tracked by default, so you’ll have to slightly modify your tracking code by adding a “_trackPageLoadTime” parameter. For example:

<script type="text/javascript">// <![CDATA[
  var _gaq = _gaq || [];
  _gaq.push(['_setAccount', 'UA-XXX-1']);
  _gaq.push(['_trackPageview']);
  _gaq.push(['_trackPageLoadTime']);

  (function() {
    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
  })();
// ]]></script>

Once implemented, you’ll be able to observe your website’s performance from the “Site speed” section in Google Analytics.

This week is a good week for web developers with Google announcing a slew of new services at Google I/O 2010 conference, and an even better one for Python web developers because the new version of Django has arrived!

Django 1.2 was released just a few days ago with many major features such as improved CSRF protection, Model validation, multiple database support and many more, please read the full release notes for details.

There is one more thing that the release notes doesn’t say, though – and it’s really important for web developers in Indonesia. Django 1.2, the latest version of the world’s best web application framework now ships with Indonesian translation and localization!

The translation was contributed by us and we sent the translations to Django for inclusion only a few days before Django 1.2 was released. This marks our first contribution to the core Django framework (with more coming) with the hope that this contribution will boost Indonesia’s Django and Python communities. So give Django a spin and don’t forget to test out Indonesia’s localization format (‘id’ is the code) – if you find translations that can be better worded, by all means let us know and we will submit translation patches to further improve it.

And in case you haven’t heard, Google also announced a few incredibly useful services for developers. I am personally most excited about Google Font API and Google Prediction API.

Google Font API lets you easily use non standard font in websites only by adding one line of code:

You can read more about what was being talked about at Google I/O here, here and here.

Building test cases for your Django project is a good practice that can catch a lot of bugs during development but unfortunately, running a large number of test cases can take quite some time.

One of our Django projects which has 215 tests takes 460.117s to complete with MySQL’s MyISAM storage engine on a relatively modern Core 2 Duo box with 4GB of RAM. But we later found out that we can run the tests much faster if we run the tests using SQLite database. To illustrate this, the test that took 460.117s to complete in MySQL only took 6.919s in SQLite. That is very huge improvement (roughly 65x) and will save your time. According to Django’s documentation, the reason for the huge speed boost is because when used to run tests, SQLite will create databases in memory, bypassing the filesystem entirely so SQLite is definitely a huge time saver when used to run tests.

Switching to SQLite engine is easy, all you need to do is change your database settings (in settings.py) from:

DATABASE_ENGINE = 'mysql'
DATABASE_NAME = 'db_name'
DATABASE_USER = 'username'
DATABASE_PASSWORD = 'password'

to

DATABASE_ENGINE = 'sqlite3'
DATABASE_NAME = 'test.db'
DATABASE_USER = ''
DATABASE_PASSWORD = ''

But it’s changing your database settings every time you want to run tests is very annoying. One solution to avoid this hassle is to append http://seanhayes.name/2010/01/09/test-database-django/a few lines of code to the end of settings.py (you have to add this below the main database configuration so it properly overwrites your default configuration).

...
if 'test' in sys.argv:
    DATABASE_ENGINE = 'sqlite3'
    DATABASE_NAME = 'test.db'
...

Now, every time we call manage.py test in our project, it will automatically use SQLite engine to run.

We can even improve this further by modifying Django’s setup and tear down methods to use database transaction. Instead of tearing down the tables every time a test is made, we use SQLite’s rollback function to revert our test table’s state to when we first set it up. The code below shows how to do exactly this (taken from Yummy Apple Pie’s blog):

from django.db import transaction
from django.core import mail
from django.test import TestCase as DjangoTestCase
from django.core.management import call_command
from django.conf import settings
from django.core.urlresolvers import clear_url_caches

class TestCase(DjangoTestCase):

    def _pre_setup(self):
        transaction.enter_transaction_management()
        transaction.managed(True)
        if hasattr(self, 'fixtures'):
            # We have to use this slightly awkward syntax due to the fact
            # that we're using *args and **kwargs together.
            call_command('loaddata', *self.fixtures, **{'verbosity': 0})
        if hasattr(self, 'urls'):
            self._old_root_urlconf = settings.ROOT_URLCONF
            settings.ROOT_URLCONF = self.urls
            clear_url_caches()
        mail.outbox = []

    def _post_teardown(self):
        transaction.rollback()
        super(TestCase, self)._post_teardown()

Happy coding and hope this post encourages you to use Django’s excellent testing framework more extensively!

Credits:

1. http://yummyapplepie.wordpress.com/2008/07/14/speed-up-your-functional-django-tests/
2. http://seanhayes.name/2010/01/09/test-database-django/

Creating a generic and reusable application needs a lot of time and effort but thankfully Django framework provides us with many convenient tools. This time I will write about how to filter a model with Django’s contenttypes framework. The contenttypes framework is useful, especially when you want to create a generic model that can reference any other model listed in your application such as historical data of an object.

For example, I have these models in my application :

class History(models.Model):
    content_type = models.ForeignKey(ContentType)
    object_id = models.PositiveIntegerField()
    content_object = generic.GenericForeignKey('content_type', 'object_id')
    action_time = models.DateTimeField(auto_now=True)
    message = models.CharField(max_length=100)
    action_type = models.PositiveSmallIntegerField()

class SalesOrder(models.Model):
    subject = models.CharField(max_length=100, db_index=True)
    client = models.CharField(max_length=100)
    value = models.PositiveIntegerField()

class PurchaseOrder(models.Model):
    subject = models.CharField(max_length=100, db_index=True)
    vendor = models.ForeignKey(Contact, db_index=True)
    value = models.PositiveIntegerField()

Sometimes we want to filter the history model by instance, but unfortunately Django’s filter method does not allow us to directly filter the model by content_object i.e. (history_list = History.objects.filter(content_object=SalesOrder.objects.get(pk = 1)) <– this won’t do).

There are a few ways to get around this problem, but I’ll try to explain how to do it in the simplest yet reliable way (since I’m new in python-django programming, there might be another way to solve this case more efficiently).

From the model declarations above we can assume that a history instance can refer to both SalesOrder’s or PurchaseOrder’s instances. So how do I filter for all history instances referring to a specific SalesOrder? Here is how:

#We first get the SalesOrder object :
salesorder = SalesOrder.objects.get(pk=1)

#Then we need to get the SalesOrder's content type:
content_type = ContentType.objects.get_for_model(salesorder)
#ContentType is a model in django that store information about the models installed in your project, and new instances of ContentTypes are automatically created whenever new models are installed.
#and the final step :
history_list = History.objects.filter(content_type=content_type, object_id = salesorder.id)

We can also do it without first getting the object instance to save an extra database hit, assuming we already know the primary key of the object we want to filter for. Here is how we do it if we want to filter for a PurchaseOrder instance

content_type = ContentType.objects.get(model=PurchaseOrder._meta.module_name)
#and do filtering as in the first example given :
history_list = History.objects.filter(content_type=content_type, object_id = purchaseorder_id)

Easy isn’t it? Happy trying and don’t give up too fast ^^.

bible : http://docs.djangoproject.com/en/1.1/ref/contrib/contenttypes/#ref-contrib-contenttypes

Django, our web development framework of choice provides many useful functions, send_mail() being one of them which we frequently use.

Though convenient, send_mail() in Django is a synchronous operation – meaning Django will wait for the mail sending process to finish before continuing other tasks. This is not usually that big of a deal if the mail server Django is configured to use is located in the same site (and has low latency), but in cases where the mail server is not located in the same cloud, this could give the impression that your website is slow, especially in sites with relatively slow internet connectivity (like Indonesia).

Consider a really common scenario where an application automatically sends an email verification message to a newly registered user. If configured to use a third party SMTP server like Gmail, the email sending process takes close to half a second if our app is located in Jakarta. This means that we are adding close a half a second of latency before rendering a response to the user, making our app look sluggish (in reality the user creation process itself takes less than 50ms).

So one of our engineers Gilang wrote a simple wrapper around Django’s send_mail() to perform the task asynchronously using Python’s threading.

class EmailThread(threading.Thread):
    def __init__(self, subject, body, from_email, recipient_list, fail_silently, html):
        self.subject = subject
        self.body = body
        self.recipient_list = recipient_list
        self.from_email = from_email
        self.fail_silently = fail_silently
        self.html = html
        threading.Thread.__init__(self)

    def run (self):
        msg = EmailMultiAlternatives(self.subject, self.body, self.from_email, self.recipient_list)
        if self.html:
            msg.attach_alternative(self.html, "text/html")
        msg.send(self.fail_silently)

def send_mail(subject, body, from_email, recipient_list, fail_silently=False, html=None, *args, **kwargs):
    EmailThread(subject, body, from_email, recipient_list, fail_silently, html).start()

To use this small wrapper simply put django_asynchronous_send_mail to your Python path and import it as send_mail():

try:
    from django_asynchronous_send_mail import send_mail
except:
    from django.core.mail import send_mail

The syntax is the same as Django’s built in send_mail() so you don’t need to change anything. It also supports HTML email:

send_mail('Subject here', 'Here is the message.', 'from@example.com', ['to@example.com'], fail_silently=False, html = '')

The full source code is available at GitHub. Obviously, this is a very simple wrapper aimed only to make Django send_mail() behave asynchronously, if you want more advanced featuers like mail queuing or scheduling you should look at other alternatives such as Django Mailer.