Scale(1), Scale(2), ... Scale(n)
Date : 01 26 2008 Category : Web
O'Reilly Network Articles and WeblogsTed Neward attempted to pull apart some of the silliness in the debate over scalability with Can Dynamic Languages Scale?. In particular, one of the most important insights is:
There’s an implicit problem with using the word “scale” here, in that we can think of a language scaling in one of two very orthogonal directions:
Size of project, as in lines-of-code (LOC) Capacity handling, as in “it needs to scale to 100,000 requests per second”While I felt the need to quibble with Ted on an unnecessary swipe at Perl at the very end of the piece, I agree that scale(1) and scale(2) are important but very different factors. They’re also not the only questions of scale.
If size-of-project is a good metric (and this should count both maximum and minimum size of project), then an important project management scalability question is “Can I hire or train enough good developers in this language or framework or platform to meet the project’s goals?” It’s a subtler question than it appears, though. Even setting aside questions of language productivity (which matters greatly), the expertise of the developers with the language, platform, libraries, and tools as well as in the domain of the problem affects your answer. It doesn’t matter if there are a million Blub programmers available if you want to do something in Blub that only three existing Blub programmers know how to do.
Downward scalability is likewise important. My head still hurts when I remember sitting in a meeting with a Sun system administrator who nearly demanded that Ward Cunningham find a wiki with an Oracle backend because, and I quote, “Flat files don’t scale”. (Ward had drawn a classic horizontal architecture with a cluster of read-only servers and a single write-then-sync-outward server.)
One of my favorite moments of tech schadenfreude was watching Java developers gradually react in horror as they saw exactly how little code and work was necessary to deploy the (admittedly stupidly simple) Ruby on Rails demo weblog application. Yes, it was stupid. Yes, it was simple. Yet it took a handful of seconds to make a change — you could build a very small project in an hour. Languages, libraries, platforms, and tools that scale down to small projects have advantages as well, particularly for encouraging energetic and dedicated novices to teach themselves how to use a technology.
(I spent a while installing a SAP server and development environment the past couple of weeks. The SAP server took a couple of hours to install and takes ten minutes to start, every time you want to start or re-start it. I realize it does a lot more than a simple Jifty or Catalyst or Maypole or CGI::Application project running on a SQLite backend, but guess what I’ll tell my nephew to use when he wants to build a web project?)
A final important type of scalability is from novice to experienced practitioner. How well does the ecosystem encourage people to become productive and to develop good taste in using the language’s idioms, patterns, and strengths effectively? This is not just learnability or the “intuitiveness” of the language’s syntax at first glance to complete novices.
I’ve rarely seen people address this question directly. In the Perl world, related questions often are “How can we overwhelm all of the lousy examples of Perl code in the world with good examples that use the strengths of Perl 5.10, not Perl 4?”, “How can we find the 90% of the million Perl programmers in the world who’ve never even heard of the Perl Mongers or PerlMonks or even Perl.com and encourage them to become part of the community for mutual benefit?”
This scale(5) is also subtle, however. Not all languages or ecosystems optimize for learnability and expertise in the same way. As a hoary example, Python deliberately eschews some advanced features in favor of learnability for programming novices. Idiomatic C++ favors pervasive template metaprogramming and genericity which isn’t exactly the first lesson I’d start in on after getting past syntax. Java recommends (requires?) the use of powerful tooling to promote a style of work that resembles directed code-generation.
These are all deliberate language choices or, at least, first-order results of deliberate language choices. Their effects on scale(5) are measurable, however.
One point often overlooked in academic language-bashing discussions of scale (and note the lack of a disambiguating number) is that scalability depends on what you need to do now and expect to do in the future. Disambiguating orthogonal axes of scale may help disentangle that discussion.