Things I Want To Do This Year (2015)

This will be my second year documenting things that I want to do this year and giving myself a mark on the results. I found it to be a valuable exercise last year, though my priorities did shift over the course of the year.

Continued / Extended goals

  • Read 2 books a month at least 50% non-fiction (This is now based on my velocity from last year)
  • Collect another large dataset
  • Obtain a healthy BMI
  • 3D print and build a 3D printer
  • Publish usable libraries in Elixir and Golang

New goals

  • Speak publicly about technology at least twice
  • Write at least 12 blog posts for the year
  • Write a library for use in ML or AI
  • Write an automated trading system that uses AI
  • Fly a drone in an automated way
  • Create a full game using VR
  • Learn Italian
  • Learn how to make a quantum

Scorecard for 2014

Wow! 2015 already. 2014 certainly went by in a flash. At the start of the year I documented everything that I wanted to get done in 2014. Now I’d like to do a quick retrospective on those items.

I’ll start by going through each of the items listed and giving myself a mark out of 10 and then I’ll document why I gave myself that mark.

Mark Item Comments
5/10 Create a computer game of some type (mobile or desktop) While I did a bunch of work in Unity (and for Oculus Rift), I did not launch a game to market
5/10 Commit code to an open source project every day. If you forget, just begin again (my streak so far is 50 days) My streak was about 170 days by the time I got sick of it.
0/10 Take those rollerblades you’ve had in the cupboard for 6 years and actually use them They’re still in the cupboard
0/10 Finish the personal training app you were working on last year This has been shelved indefinitely
10/10 Read at least 1 book per month In total I finished 25 books in 2014. There was a good mix of novels and non-fiction
7/10 Learn another programming language till you can use it proficiently (I’m working on learning golang) I have dabbled with both Golang and Elixir this year, but have not produced anything substantial in either
0/10 Get the RHoK project FGIS out into some fire trucks for testing Perhaps one of the saddest of my non-completes for 2014, this failed to progress both from lack of interest on the problem owners side and also lack of pushing on my part
10/10 Design something from scratch and 3d print it I have designed several items from scratch and 3d printed them this year
Silly Walk 100km in one go (as of last year I can comfortably walk about 20km in one day, I want to improve on this I walked consistently for most of the year, but I’m ruling this a silly goal and not scoring it
0/10 Build muscle (Last year I lost about 15km, going from 97kg-83kg) Weight is back, and I haven’t put on any muscle
7/10 Collect and publish a large dataset under creative commons I have collected the dataset but have not yet published it
10*/10 Build a quadcopter (carried over from last year) Ended being smart and buying an almost completed quadcopter. I have been flying it a lot and introducing other people to it. Have managed to get the ball rolling on my employer investing in the technology as well
10/10 Visit a country you haven’t been too before Visited Italy for work, and went for a day trip to Switzerland while there. Also spent a bunch of time in San Francisco and LA, which while I’d been to the US before (NY) were both new as well
10/10 Meet/befriend 100 new people (more than just friending on twitter) Can’t say that I collected 100 business cards and that I’ll stay in touch with them all, but I have met and got to know a LOT of people this year

Final Score: 74 / 130

Thoughts

A passing grade but not as good as it could have been if we count the things I intended to do at the start of the year and didn’t end up doing.

On the other hand there are many many things that I accomplished this year that are not on the list.

The only one that I’m particularly unhappy about is my lack of progress on the fitness side. This is truly and area where I need to do better.

Convert an AWS Route53 Resource Record Set to point to an Alias Target

Occasionally, you’ll want to change a Route53 Resource Record Set from one type to another. In the case of going from a traditonal ‘A’ record to an Alias Target the following code (with the aws-sdk rubygem) should do the trick.

The gotcha with this process is the need to unset ttl and resource_records as a part of the update.

zone = AWS::Route53.new.hosted_zones.select { |z| z.name == 'example.com.' }.first # find zone by name
rrset = zone.rrsets.create('jedi.example.com.', 'A', { :ttl => 60, :resource_records => [{ :value => '127.0.0.1' }] })

# Now lets update it to be an alias target
stack = AWS::CloudFormation.new.stacks['jedi-stack'] # assuming the stack exists

lb_resources = stack.resources.select {|x| x.resource_type == 'AWS::ElasticLoadBalancing::LoadBalancer' }
lb_resource = lb_resources.first

lb = AWS::ELB.new.load_balancers[lb_resource.physical_resource_id]

rrset.alias_target = { dns_name: lb.canonical_hosted_zone_name, hosted_zone_id: lb.canonical_hosted_zone_name_id, evaluate_target_health: false }
rrset.ttl=nil if rrset.ttl
rrset.resource_records=nil if rrset.resource_records
rrset.update

You should now have a RRSet pointed at your ELB.

Mount an S3 bucket using IAM roles

After much searching around I discovered that newer versions of s3fs support IAM roles (I’m using 1.77). Unfortunately, this functionality is barely documented.

I ended up using the following line to mount my directory:

s3fs bucketname /mnt/bucket -o use_cache=/tmp,allow_other,iam_role=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/`

NB: s3fs does not need s3://

If the role you’re trying to use does not have access you’ll get something like this:

$ ls /mnt
total 20
drwxr-xr-x 5 root root 4096 Jun 16 23:39 .
drwxr-xr-x 7 root root 4096 Jun 16 23:39 ..
d????????? ? ?   ?       ?            ? bucket

The other error message I’ve seen is:

touch: cannot touch ‘/mnt/bucket/test.txt’: Transport endpoint is not connected

This happened when I specified the role incorrectly (it does not need to be the full arn).

Awesome Links #10

I’ve promised myself that I’m going to do a lot of reading this year. To date I’ve done quite well, and the books have been amazing.

“How to Create a Mind” by Ray Kurzweil was an excellent read. It seemed a great idea to follow up with more AI books, I read two William Hertling novels, Avogadro Corp and A.I. Apocalypse. They were both great, but I enjoyed the second more.

Not that I have a heap of time for it, but reading “How to Create a Mind” by Ray Kurzweil got me thinking about Machine Learning again. Today I stumbled upon (Machine Learning in Javascript)[http://burakkanber.com/blog/machine-learning-in-other-languages-introduction/], which is an interesting series of posts.

Things I Want To Do This Year

While I was on my flight back from visting family in Sydney for Xmas, I started writing a list of everything I want to get done this year.

It’s quite a long list, and it will be a very full year if I manage to do half of it. But it’s worth writing down:

  • Create a computer game of some type (mobile or desktop)
  • Commit code to an open source project every day. If you forget, just begin again (my streak so far is 50 days)
  • Take those rollerblades you’ve had in the cupboard for 6 years and actually use them.
  • Finish the personal training app you were working on last year
  • Read at least 1 book per month
  • Learn another programming language till you can use it proficiently (I’m working on learning golang)
  • Get the RHoK project FGIS out into some fire trucks for testing
  • Design something from scratch and 3d print it
  • Walk 100km in one go (as of last year I can comfortably walk about 20km in one day, I want to improve on this)
  • Build muscle (Last year I lost about 15kgm, going from 97kg-83kg)
  • Collect and publish a large dataset under creative commons
  • Build a quadcopter (carried over from last year)
  • Build a for the REA hackday in Feb
  • Visit a country you haven’t been too before
  • Meet/befriend 100 new people (more than just friending on twitter)

As you can see. I have my work cut out.

Awesome Links #9

Now that the end of year is upon us, it seems like an opportunity to get some reading done & work on a few small projects.

I just finished The Last Firewall by William Hertling, which was a very good read. I haven’t read his previous two books, but I’ll have to pick them up now. I partically liked the (brief) reference to a Puppen (half dog, half cat).

I’ve been dabbling a bit with Golang. It’s a fantastic language and I’ll be spending more time learning it over the break.

An update on my AWS bill

The last 48 hours have been surreal. When I typed up the experience up yesterday morning, I was not expecting it to get more than a handful of votes on Hacker News.

As I watched the discussion yesterday I was warmed that people both cared about my plight and found it useful to have a reminder.

All that while, the case with Amazon slowly progressed. They first answered and asked me terminate the stopped instances. They indicated that once this was done they would put in a request to waive the fees (subject to approval).

Overnight (I’m in Australia), this has been approved. I still can’t see it from the Billing Console, which I’m told is normal, but I’ve been informed in the support case that Amazon will be crediting the entire bill. Yay!

There was part of me, when I saw this bill, that knew what Amazon’s response would be. I don’t deserve that kind of relief after leaking my key, but Amazon has always been a cut above.

In my interactions with Amazon Account Managers and Architects (work account), Amazon has consistently cared about people being efficient in their instance usage. They have gone so far as to suggest changes that would wipe thousands off a monthly bill.

With this experience they have reinforced, to me a least, that they value me as a customer for the long term. I will one day be paying Amazon $3000 dollars a month, but I hope that it will be the bill for, an as yet non-existent, startup that is skyrocketing to success :D

This is all without mentioning that Werner Vogels stopped by the Hacker News thread (The ticket was already on it’s way to being dealt with by that point, but it was amazing).

Finally, thanks again to all the supportive people over the last 2 days.

Discuss it on Hacker News

My run in with Unauthorised Litecoin mining on AWS

Update: You can read an update to this story here

Normally I’m a big advocate of open sourcing projects both current (and old) on GitHub. Today though, I wish that I wasn’t.

On sunday night I received an email from Amazon saying that they’d detected my Amazon key on one of my repositories. This was a little bit of a surprise, because I’m usually so diligent about not saving credentials into repositories.

After a brief search I found the key buried in an old project that I’d just decided didn’t need to be private.

That wasn’t the end of the matter, I was in for a rude shock when I logged into my Amazon account to check for unauthorised usage. $3000+ in pending charges. Woah!

Billing Dashboard

It didn’t take long to find the source of the billing. Twenty cc2.8xlarge instances humming along in the us-east region for the last two days.

By this stage I’d already revoked the key (as suggested in the email). So I quickly shut the instances down, while I would have liked to preserve them for forensics, I just couldn’t afford to leave them running while waiting for Amazon support (I do not pay for support, since this is just my private account that I dabble with).

After taking stock for a few moments, I detached one of the volumes and attached it to another instance. Having a poke around confirmed what I had already guessed. The unauthorised user had been mining litecoin with the mining pool pool-x.eu.

I’ve emailed pool-x.eu asking them to suspend the account, but I’ve yet to receive a reply.

What have I learned from this experience?

Enable billing alerts

Given I spend about $60-80 a month with Amazon usually, I could have been warned MUCH earlier. Needless to say, now that the horse has bolted I’ve enabled the horse bolting detector.

Check GitHub

It’s not really that hard to do a regular search of GitHub for keys and passwords in your repositories. Check your friends repositories as well…many eyes.

Audit code before open sourcing

Always a good rule, but be especially careful flicking the switch on repositories that you’ve had as private for a long time.

Update: @joneaves suggested either using something like checkstyle (java) and/or a pre-commit hook. Good advice.

Use IAM Keys

Quite a few people have pointed out on twitter and hacker news that the other thing you should be doing is using restricted IAM keys.

More tips on Amazon

A friend pointed out that Amazon has a good security blog post that deals with this and other risks to your account.

Discuss it on Hacker News

Awesome Christmas Gifts 2013

Pebble Smartwatch
> Pebble Smartwatch

In a special edition of ‘Awesome Links’ I share what you should be buying your geeky friends for Christmas (assuming they haven’t already bought it for themselves)

We may all use smartphones for everything now, but if you asked the futurists of the 1980s what we would be using now, they probably have described something like Pebble. This Smartwatch took Kickstarter by storm, and continues to impress. At USD$150 it may fit into your budget for a loved one.

For all those geeks with kids, the Leap Motion is the perfect gift. There is nothing quite like watching a kid discover magic. And at USD$79 it has a much better time to entertainment ration than the movie gift voucher you were thinking of giving them.

If you’re looking for something with a smaller budget the Raspberry Pi ($35) or Beaglebone Black ($45) may fit the bill. One of the benefits to giving one of these as a gift, is even if they already have one, a second or third is always useful (they tend to get used in projects). The Raspberry Pi tends to be a bit friendlier for getting started, and the Beaglebone Black fits into the slightly more advanced category.

Need more ideas? Hacker Things has an impressive list of awesome gadgets. Also browsing Adafruit, Sparkfun and Tindie should help you find that perfect geek gift.

Update: opensource.com also has an awesome gift guide.