Category Archives: web development

“Website Obesity Crisis”

Interesting talk by Maciej Cegłowski: Website Obesity Crisis.

What do I mean by a website obesity crisis?

Here’s an article on GigaOm from 2012 titled “The Growing Epidemic of Page Bloat”. It warns that the average web page is over a megabyte in size.

The article itself is 1.8 megabytes long.


Here’s an almost identical article from the same website two years later, called “The Overweight Web”. This article warns that average page size is approaching 2 megabytes.

That article is 3 megabytes long.

If present trends continue, there is the real chance that articles warning about page bloat could exceed 5 megabytes in size by 2020.

The problem with picking any particular size as a threshold is that it encourages us to define deviancy down. Today’s egregiously bloated site becomes tomorrow’s typical page, and next year’s elegantly slim design.

I would like to anchor the discussion in something more timeless.


To repeat a suggestion I made on Twitter, I contend that text-based websites should not exceed in size the major works of Russian literature.

This is a generous yardstick. I could have picked French literature, full of slim little books, but I intentionally went with Russian novels and their reputation for ponderousness.

In Goncharov’s Oblomov, for example, the title character spends the first hundred pages just getting out of bed.

This seems good to keep in mind. It’s important, given recent developments such as Accelerated Mobile Pages project. How much cruft should there be on a page? When’s the last time you measured the loading size+time of your page? When’s the last time you did something useful about it?

Maybe this site is something to look at and think about?

Switched to nginx

I’ve switched the server to nginx. I don’t have too many htaccess-based rules, and overall the Apache2 configuration was very simple to begin with. I installed php5-fpm, followed tutorials on configuring nginx (moving what’ll be common among 15ish virtual hosts into a separate file), configured proxying for XMPP BOSH and finally improving SSL security.

Apache2 now sleeps quietly and the RAM usage is down a bit. Hopefully the server won’t suffer any more hard crashes due to lack of RAM like it did immediately after I moved to the new host.

Overall, nginx seems like a much lighter and easier to understand solution for my very simple needs; if you’re running a simple server, I recommend it (as long as you read a bit about its security, if you intend to use it for serving PHP!).

Ajax Animator

A few months ago I wrote about a necessity for HTML5’s success: a design tool similar to Flash.

Well, here it is: Ajax Animator. Test it:

Which XMPP technologies to use?

I’ve been writing a piece of software in order to perform a major upgrade of ZATEMAS, a specialized web app suite I’m a co-author on. Since there is a small community of users on it that might interact better if they had an opportunity to do so, I came to an idea to provide them with an instant messaging client built right into the browser, similar to Facebook Chat, Meebo Bar and Gmail Chat. I’ve picked XMPP as the protocol; better known as Jabber, this is the same protocol Google Talk uses.

Examining various options took a while. I’ve looked at Meebo Bar, and concluded that it doesn’t fit my use case, since I want users to be automatically logged in on a local XMPP server.

I’ve examined several clients. In the process, I’ve learned a bit about BOSH (the standardized method for using XMPP over HTTP), about Apache’s mod_redirect, and a bit about rules for doing cross-domain xmlHttpRequests in Javascript.

The only serious contender for the throne of the best open easily deployable web-based XMPP client is JWChat. This is a venerable old client which creates a popup and behaves much like a typical desktop IM client. This means it was unsuitable; there is no easy way to embed it in a page, yet preserve its state upon page switching.

This is actually the biggest problem I came upon. There seems to be no XMPP client nor library written in Javascript that can trivially handle page switching (which means destroying entire Javascript context); you cannot trivially serialize their state. Which is why I’m proud to announce a custom-made client, Z-XMPP. It’s already available for preview on https://bitbucket.org/ivucica/zxmpp, and will soon be available for preview on ZATEMAS. Its license is currently a custom one, hand-written, and most definitely not satisfying any open source/free software definition. That will change soon, as soon as I pick the right license.

That brings us to the other part of the XMPP stack. Which server to pick? Which BOSH connection manager to pick?

There are, really, two contestants in my arena when it comes to a server. First one is a veteran of XMPP, ejabberd. ejabberd is written in Erlang, and is massively scalable. It’s trivial to install and configure in Debian, and it supports a lot of cool features out of the box. It supports something called “shared rosters”, which basically means you can create groups in people’s contact lists that you, as the admin, can enforce to contain whomever you want. You can force people, for example, to see everyone else working in the company. This is a critical feature for ZATEMAS, just as is so-called “external authentication”. What I’m missing here is: external auth does not support fetching any attribute apart from basic operations with passwords (is it correct? please change it!), and vCard cannot be created from command line, only updated. This means I cannot trivially set people’s real names during an update run.

Obviously, ejabberd has flaws, and I cannot easily update it since Erlang is a language fundamentally different from any other I worked with.

So the second contender is a server I discovered only tonight. It’s Prosody, an extremely lightweight XMPP server written in Lua by a bunch of very friendly folks. I really like the attitude and personal approach the principal author of Prosody has, but that’s not all. Server’s source code looks extremely well organized, the server is quite featureful, and most importantly, it’s written in a language that mere mortals can understand. I’m not a big fan of Lua, but I can read it, and I can update it, especially when it’s so well written as Prosody seems to be.

I’m currently not very familiar with Prosody, but the fact that I managed to set it up very quickly and that it starts up momentarily… well, I think that we can hack a ZATEMAS-based external authentication module into it! Also, I think I might be able to better add my own debug functions, to easily see what I did wrong while developing my client.

Both ejabberd and Prosody come with a BOSH connection manager (the thing that translates HTTP requests into a continuous XMPP stream; a continuous XMPP TCP stream is something you cannot achieve from the web). So why another one?

Well, perhaps you want to log into Google Talk!

Yep, folks, that’s what PunJab allows: have your BOSH-based client log into any XMPP server. I must say I like PunJab; it’s written in Python. Despite that, its internals seem a bit less clear than Prosody’s, yet still manageable. PunJab does its job and does it extremely well.

So there you have it. Perhaps we’ll soon have an opportunity to talk about how to install Z-XMPP instead of just talking about why and how I’m working on an IM service :)

Until next time!

After MySQL upgrade on OS X, all tables are missing

When upgrading MySQL on OS X (anno domini 2010),  I learned the hard way that the “data” folder containing your databases and tables does not get moved. Look into /usr/local; you will probably have the old mysql-* folder there containing just the “data” folder. /usr/local/mysql is a symlink to the current MySQL installation, so move its data folder elsewhere, and put the old one in there.

PHP on Mac no longer able to connect to MySQL

It’s been a while since I did web development on my Mac, and MySQL ceased to work. Could be related to that one time that I accidentally removed a bunch of dot-files from my home directory, could be related to MySQL upgrade. In any case, it no longer worked.
What I figured out was that the /var/mysql/mysql.sock UNIX domain socket was no longer generated. The /var/mysql folder was missing. When just creating it and assigning it to user _mysql (chown _mysql /var/mysql) did not help, I knew something was wrong with configuration.

MySQL tries to read configuration from /etc/my.cnf. To get its UNIX domain socket where PHP expects it, this needs to be contained in this configuration file:

[client]
port = 3306
socket = /var/mysql/mysql.sock


[mysqld]
port = 3306
basedir = /usr/local/mysql/
socket = /var/mysql/mysql.sock
; datadir = /servers/raiddrive/databases/

Please read the manual for details, and this is not administration advice; I’m not sure if this opens security holes, but I only use MySQL for development purposes. (Because you should know better than letting programmers near mission critical production servers.)