Spark 1.1 and standalone Python scripts

Spark 1.1 was released a few weeks ago, and I've been curious about using it from a standalone Python script. While Spark has Java and Scala bindings, the Python one appeals to me most--I'd really like to be able to use Spark from within a standalone Python script, and then step through it in PyCharm, which is the IDE I really enjoy. Here's how to do that on Mac OS X 10.9

more ...


Provisioning with Vagrant, Ansible, and Packer

One big issue in using servers on cloud providers suck as Amazon Web Services, Rackspace, and Google Compute Engine is dealing with their provisioning and deployment--in other words, how to build them out to begin with. Sure, you can write lots of Bash scripts, invest in Puppet and Chef training ...

more ...


First Post!

Today I moved off an old blogging platform onto a new one--a static blog using Pelican and Markdown, and hosted on Github.

more ...