Rackspace cloudserver complaints

The control panel is slow… Guys, just because it’s ajax doesn’t mean it’s fast. Get some more responsive servers, or at least add a spinner or some other UI element to let me know that something is happening. UI 101. Slicehost has a much better panel. The rackspace panel is almost as slow as Dreamhost’s, which is saying something.

“Fanatical Support” is still staffed by non-native english speakers. “I appreciate your patiences”? wtf.

Live chat doesn’t work in Chrome.

Why are DNS records artificially associated with each individual VPS? I have to select the VPS I want before I can modify the DNS records – what if I have a DNS record that points to multiple servers? That’s a bit of a weird dichotomy. DNS is a separate concept from individual servers, and should stay that way.

GSOC 2010 – I’m updating the Django Test Suite

I’ve been accepted to work on Django’s Test Suite during the Google Summer of Code this year. I’ll be working to improve the runtime and uniformity of the test suite, mainly by converting existing doc tests to unit tests. This isn’t particularly glamorous, but it should be a chance to get intimately familiar with the Django code base, and it is something that really should be done.

Unit tests will make it quicker to pinpoint exactly what is causing a test to fail. Moving each test case into its own function may reveal bugs that were concealed or glossed over as a side effect of the existing test case environment in the doctest. As I’m working through the code, I’ll be looking out for tests which are doing inefficient things. Developers run the tests regularly, and runtimes can vary from 10 minutes to over an hour depending on hardware and database configuration. By speeding them up, I’ll be saving countless future developer-hours.

Copying large drives with DD

I needed to do an exact copy of the byte-level contents of one Seagate Barracuda 7200.11 1.5 TB (ST31500341AS) to another identical drive. dd is the obvious tool for the job, but the default settings copy very slowly. This is because the default block copy size is only 512 bytes. After a bit of experimentation, I settled on a block size of 64k as providing the best performance.

dd if=/dev/disk1 of=/dev/disk2 conv=noerror,sync bs=64k

(obviously you will need to specify the correct disk devices for your setup)

Using the larger block size resulted in a speed increase from 7MB/s to an average transfer speed of 37MB/s. It makes a big difference when you’re copying a full 1.5 TB!

Upgrading from Ubuntu 9.04 to 9.10 – memcached problems, postgresql upgrades, and svn

I recently upgraded a couple of servers over at slicehost from Ubuntu 9.04 to 9.10 via the command line upgrade. There were only a couple problems.

During the install, I kept my php and svn config files, since the diff showed no major changes to the ones in the package (other than my configurations, of course).

The first problem was that the Postgresql 8.3 databases needed to be updated to 8.4, and the installer doesn’t do that. I made sure to run these before the installer removed the 8.3 package (it asks before doing so at the end of the install script), though I don’t know if that is strictly necessary. The upgrade was relatively simple:

#see what clusters you currently have
pg_lscluster

#drop the empty new 8.4 cluster the installer created
sudo pg_dropcluster 8.4 main

#upgrade your existing cluster
sudo pg_upgradecluster 8.3 main /var/lib/postgresql/8.4/main

The upgrade command SHOULD take care of copying the data, upgrading it to the new format, stopping the old cluster, and starting the new cluster on the same port. On one of my machines the process of stopping the old cluster failed, but doing it manually with pg_ctlcluster worked.

Once I upgraded the Postgres databases, I allowed the script to remove the old packages and reboot the machine. Then I selected the kernel upgrade in the slicehost control panel (which rebooted the machine again) and let it come up. Almost everything worked perfectly.

The other issue I encountered was that after the upgrade, memcached no longer started on reboot. After a bit of poking around, I discovered that the upgraded package that comes with 9.10 comes with a new file in /etc/default/memcached which defaults to having memcached disabled. To enable it, set ENABLE_MEMCACHED=yes and memcached starts when your server boots. This does have a launchpad bug, but nobody seems to be taking responsibility for fixing it.

Another gotcha turned out to be the fact that memcached must already be running when Django starts. I only resolved the issues with memcached after my Django processes had started on the other server, and it took me a few minutes to remember that I needed to restart them to fix caching.

The only other minor issue was that (as expected) the upgrade overwrote my script that gets invoked whenever svnserve is called. I have this in place so that I can svn+ssh into the server and check files out without specifying their absolute path in the file system. My script looks like this:

#!/bin/sh
umask 002
exec /usr/bin/svnserve.bin -t -r [path to svn repo] "$@"

I keep the actual svnserve binary (default installed to /usr/bin/svnserve) in svnserve.bin and the script in svnserve.ssh. I then link /usr/bin/svnserve to svnserve.ssh, so when svn gets upgraded, I just recopy the binary and re-link.

Dreamhost facebook app dev note

Dreamhost has deprecated php4 and doesn’t serve it using apache anymore. Nonetheless, the php command is still hardlinked to the php4 binary. This means that when you try to execute your php code to see what’s going wrong, facebook’s code (which only works with php5) throws a lovely error:

Parse error: syntax error, unexpected T_STRING, expecting T_OLD_FUNCTION or T_FUNCTION or T_VAR or '}' in /home//facebook.php on line 38

To fix this, add alias php='/usr/local/php5/bin/php' to your ~/.bash_profile and re-login. Now when you run php from the command line, it will use version 5, which you can verify by running php -v.

Creativity in Education

My friend Volty recently started blogging with a focus towards intellectual development. His post on Sir Ken Robinson’s claim that creativity is as important as literacy in education encouraged me to consider my own responses.

The Oxford English Dictionary defines literate as “Acquainted with letters or literature; educated, instructed, learned.”

I do not take well to being told how to behave. I never have. I take even less well to being told how to think. How did this affect my schooling?

I will repeat what others have said, and assert that I’ve managed to prevent my schooling from getting in the way of my education. I’ve been supported in this attitude by friends, family, and even teachers for as long as I can remember.

Did I go to a “struggle to pass the state-mandated standardized test” public school? No. Did my schooling kill my creativity, or discourage my natural curiosity? Most profoundly, no. The dichotomy is certainly revelant — private schools can be assumed to have more resources devoted to tasks other than passing standardized tests in core subjects. Does my outcome represent my innate personality, or a difference in educational method from the one that Robinson is criticizing? I tend to lean towards the former.

I can only speak to the schools I have attended. In these, literacy was clearly held as a higher priority than creativity. There were creative classes, and creative activities within academically focused classes, but the primary goal was academic, not creative, proficiency.

In retrospect, I feel that this was appropriate. I disagree with Robinson’s (most likely intentionally) inflammatory claim that creativity must be the equal of literacy in education.

On the contrary, literacy is absolutely fundamental to true creativity. It provides the foundation for meaningful contributions. Without a firm grasp of the existing subject matter, how can one hope to make a new contribution to a creative field? Academic knowledge should not constrain a creative individual to follow an existing path. Instead, it shows possible paths which may optionally be followed for a while, before diverging into uncharted territory. Without the palette, there can be no masterpiece.

Literacy allows creativity without the painful steps of rediscovering that which could be taught by instruction. There is nothing shameful or innately wrong about organized instruction in a topic which allows creativity. In fact, the example Robinson himself cites — the dance school — is precisely this.

Perhaps it does boil down to talking about different kinds of education after all. In my experience, school has always been about learning how to ask questions, thinking of things that are “wrong” or unconventional, and generally exploring the topic at hand through whatever means available. I have yet to experience a topic which truly consisted of little but rote memorization, and if confronted with such, would most likely argue that it had no place in “education.” Whether this is a matter of personal character, or just going to the right schools, I cannot say. I can say that the education system he seems to presuppose is not one in which I would willingly participate.

I should stop before I again begin to diverge into unrestrained metaphor. Next time, I hope to consider intellectualism. Should everyone be explicitly trained in this discipline? Is everyone capable of it? How does “natural temperament” play a role?

As a footnote, I do really appreciate the written transcripts which accompany each TED Talk Video. They make the content far more accessible for review.

WordPress and music on android

Posting this via wptogo on my android. The app seems to work pretty well even if typing on a touchscreen does suck a bit.

I think I’ve been being overprecise in my typing. I can actually go much faster than I have been.

Since I got it, I’ve loved my ipod shuffle for being small, light, and just what I need without too many extras. I’ll listen on it literally all day. However, it IS an extra thing to carry around, keep charged, and lose.

Therefore I’m going to experimentally try using my phone for this for the next while. In the past I have been dissatisfied with this setup for several reasons, the largest of which is battery life. It is not ok to run my phone out of battery because I was listening to music. I need at least a day and a half of battery life (I get a bit more than two right now, depending on usage). I’m not happy about the need for a headphone adaptor. HTC needs to fix this serious flaw in otherwise nice hardware. I’m also a bit worried about sound quality – the amps in the last phone I had sucked horribly. In any case, it should be interesting.

Composing posts on this keyboard sucks. Coffee makes it worse.

IBM x346 Power Connector

For the record, IBM P/N: 90P4632 (FRU P/N: 25R5163) is made up of 2 end-to-end molex part number 43025-1800 connectors with female terminals.

Molex connector info:

http://www.marvin3m.com/connect/

Thoughts on Android, fixing os slowdown

I recently started using an android-powered Google Ion (also known as the HTC Magic or T-Mobile myTouch) courtesy of the Google I/O conference.

I really like it, though I do miss the hard keyboard of the G1 I borrowed for several weeks. I feel that the smaller size is a fair trade, since it now fits comfortably in my pocket, where the G1 felt quite bulky. I had assumed that I would generally use the horizontally oriented keyboard since the keys are larger in that format, and typing should be faster. In fact, I find that I usually use the vertical orientation and type with my thumbs. The reason for this is that I can see more of the content I’m replying to, and don’t have to toggle the keyboard on and off to switch between reading a conversation and replying to it. Overall, the soft keyboard works pretty well, though I’ve found it works better with my thumbs than my fingers (I recently broke both my wrists, and my thumbs are immobilized, so I’m forced to type with the fingers).

When I first got the phone, the interface was snappy, and everything responded quickly. Over the past few weeks of use, things seemed to slow down to the point that menus would take several seconds to load, and pulling down the notification tray was sometimes not possible. Needless to say, I was extremely unhappy about this turn of affairs. I tried rebooting the phone, but that only caused a minor improvement. I checked to make sure there weren’t spurious background processes, and that the phone’s memory wasn’t overflowing. Finally today, I discovered the fix quite by accident. I am notoriously bad about checking my voicemail, and have had messages in my inbox for nearly the entire time I’ve had the phone. Amazingly, when I checked my voicemail and cleared out the inbox, the phone instantly returned to it’s old snappy self. My best guess is that clearing the notification tray entirely was what fixed the problem.

This may be related to issue 3453 in the android source, but my resolution seems to be different enough that it might be a separate bug.

5DmkII video – Playback on PS3, Editing in Premiere

The Canon 5D Mark II shoots beautiful video, but it is unfortunately encapsulated in the .mov container, which the Playstation 3 will not play natively. After much trial and error, I’ve found that H.264 encoded video and Dolby AC3 audio muxed into a .m2ts file works well.

If you simply want to watch your unedited video on the PS3, follow the following steps:

  • Install YAMB and select “Editing” and “Click to extract streams…”
  • Select your input .MOV and select “Extract to Raw Format” and then click “Extract”
  • Use BeSweet to convert the audio file (it’s a PCM wave) to AC3
  • Install tsMuxeR, and add the extracted video file and converted audio file to the input file list
  • Select “M2TS muxing” and specify your output filename
  • Click “Start muxing”, and when it’s finished, copy the file to the PS3 and play it

The above steps work well, but don’t allow for much editing to take place. I use Adobe Premiere Pro CS4 to edit video, but none of the native H.264 output formats that include both audio and video seem to play back correctly on the PS3.

This is the workflow I use for editing and rendering these videos:

  • Create a new project using the correct settings. Here’s my Adobe Premiere New Sequence Preset preset for the Canon 5D Mark II (It goes in the “Documents and Settings\<username>\Application Data\Adobe\Premiere Pro\4.0\Settings\Custom” folder on Windows XP)
  • Import your .MOV files and edit them as usual
  • Export your project using my Canon 5D Mark II Video Export Preset for PS3 or use these settings:
    • Format: H.264
    • Export Video and Audio checked
    • Multiplexer set to None
    • Video set to NTSC, 1920×1080 resolution, 30fps, None(Progressive) Field Order, Widescreen pixel aspect ratio, High profile. 20-30 Mbps Bitrate
    • Audio Format set to Dolby Digital
  • Open tsMuxeR and import your separate rendered video and audio files
  • Select “M2TS muxing” and specify your output filename
  • Click “Start muxing”, and when it’s finished, copy the file to the PS3 and play it

This produces full-resolution video output that does not drop frames, stutter, or display excessive transcoding artifacts.

This is what has worked for me. If you’ve got suggestions, or have problems with these methods, let me know and I’ll see if I can help you.