Builders

My dad is a builder.

When I was young, my dad would take my brother and I to visit real estate. Show homes and open houses, we'd even occasionally sneak into places under construction to have a look. My brother and I would run between rooms imagining which ones we could claim as bedrooms, but dad would inspecting the layout, looking for the details in construction, and critiquing the use of space. He taught us to estimate the dimensions of the rooms with our feet: one foot length stopped in front of the other.

For us, it was an adventure. For him, it was an opportunity to consider the features of each design, and visualizing whether they'd help or hinder the owners, pointing out the obvious-to-him issues that construction firm had missed. Money was tight growing up so a new house was out of the question, but that didn't stop him from looking and planning for a day in the future.

This weekend, I visited my parents new home. They've retired now, and finally have the time and resources to build a new one. My dad walked me around the house, pointing out things that, the contractors missed, changes in the design he insisted on, and mistakes that slipped through anyway. The place is beatiful, spacious, and thoughtfully organized. And still, I know that same familiar combination of pride and frustration.

I'm a builder too.

As we walked around, we talked about how he'd do it differently if he did it again. The lessons he'd apply, the subcontractors he'd call in (or blacklist). I immediately fell into it, brainstorming ways to solve the problems the remained; how to properly wire the house or evaluate the true color of paint in the light of the room. In these situations, my software brain spins to life but I also found myself thinking about building well-designed houses too. And though I haven't ever had any inclination to the construction industry, I was immedately excited about the option.

I've been building for nearly all my life, but walking around with my Dad reminded me where I received that spark.

And I can't wait to start building again.

Unstructured days

Week 2 and I've been completely unsuccessful in hitting all four of my targets every day. My best is three, but median is two of four.

I blame this on poor time management.

I've spent the last few years as a slave to my calendar. PM roles tend to involve an embarassing number of meetings. Though good PMs will stay constantly vigilant for useless ones, some amount remains.

My first week was heavily calendared as well, but mostly with hour long coffees, obstensibly to reach my "Talk" goals. Turns out that coffees aren't progressing me toward my goals and they are costing a lot in time (including transit) and money (including transit).

• They're too easy too set up
• They move around too easily
• They're even harder to keep structured and valuable (particularly for both people)

I've only had one day so far that ended up being completely free, but only after a few things canceled and rescheduled. And I ended up wasting it by heading home and then cooking, which triggered couch sitting and distractions. Despite the newly found free time, I wasn't able to pull out of the productivity nose dive.

What I need to fix:

• I'm not making progress on exercise and reflection goals each day. I need to schedule time for them
• I also haven't spent any time in other startup offices because its hard to find large chunks of time
• Avoid rescheduling meetings impacting the rest of my day
• Neee a predetermined fallback plan when things get canceled
• Keep the time of day I feel most productive (9am-12pm or so) free for productivity

I'm taking as a given that I won't be able to abandon my calendar. So instead, I'm proposing a default structure for my day:

8am - 12pm

• Exercise
• Meditation
• Code/write

12pm-1pm

• Lunch
• (Transition time)

1pm - 6pm

• Hang out in an office space
• Code/write

3pm - 6pm (Optional)

• Coffees and meetings

6pm onward

• Hanging out with girlfriend, friends
• Code
• Read

On the surface, it seems really natural and attainable. It also appears to be a schedule I could put into place for my day-to-day as a PM. So it's somewhat surprising I haven't run into other people preaching a more structured default day model. I'm interested to see how it works in practice!

The plan

As of May 1st, I'm on a two month sabbatical. I'm grateful for Dropbox being down with me taking time off and giving me the opportunity to reflect and explore.

I'm trying to figure out what's next "work"-wise. ie how I want to spend my working time for the next few years of my life. Of course, the non-work aspects of life are just as important and will weave into this process regardless. But I'm a worker, I love the work I do and the people I work with. Getting this side of the house in order is fundamental.

(I'll tackle the elephant in the room: there's an assumption that this means I'm leaving Dropbox. That's not how I'm looking at it. There's a whole world of possible answers to the question of what's next, and Dropbox is the best place in a world for a whole lot of them. It's possible that it's not for some, but I'll cross that bridge if I get there.)

Two months is both a huge amount of free time and a period that will blink by in a moment if I'm not careful. And my biggest fear is that I'll end up at the end of the two months and not have made any progress in my thinking. So I've been trying to be really structured with that the time.

I started putting together a TODO list, but after a few years at TapEngage and Dropbox, I'm accutely aware that laying out a plan in uncertainty isn't useful any further than a few weeks. So I boiled down my TODO into a few themes that, in theory, I can aim to repeat every day. And, as a initial goal, I propose doing each of these things for at least 30 minutes each day.

Sean's really overwrought sabbatical framework:

• Talk with potential users/customers
• Write thoughts and reflections (like this!)
• Code and build things
• Exercise

So, how am I doing so far?

tl;dr - Not a single day where I did all four, and right now, averaging only two per day. On my own version of this table, I've added a fifth column: "What actually happened?". Everything on that list is important (so far), but it does limit the 'Yep's in the table.

It's only four days in, but I already figured a few things I need to improve in order to get to a Yep in each category. Follow along and I'll report back every week until the end of my time. Hopefully I'll have some interesting lessons as a result.

Service and perks included with American Express Platinum

I recently signed up for an Amex Platinum card after being a Blue Cash customer for six years. I was interested in taking it for a spin to understand the perks associated with the card. Unfortunately, it doesn’t seem like American Express does a very good job capturing or keeping this list up to date.

This is a list of services I’ve had to discover and set up myself. I’ll add to it as I find more.

Centurion lounge access
Nothing to set up, just show your card. Only four airports available so far.

Priority Pass lounge access
Sign in with your American Express card account at https://rewards.americanexpress.com/olet/splash?campaignId=plpp0401&offerType=platinumprioritypass

This is only a Priority Pass Select account which excludes access to United lounges.

2x Membership Rewards on Uber
Instructions at https://get.uber.com/amex/

Free Boingo wifi access
Sign up at http://www.boingo.com/americanexpress

Parsing JSON in Go

I got to hang out with the team at Sendwithus at their Battlesnake competition. While the actual competitors scrambled to build the mightiest of snake AIs, I took the opportunity to learn a bit more about Golang by attempting to write a snake in Go. Coming from a Python background, I never expected that JSON parsing would be where I spent most of my time. Today, I’ll cover the different approaches you can take when you start Parsing JSON in Go.

All the code and various methods here are sampled from my Golang Json Experiment project if you want to cut to the chase.

There’s really three general approaches to pick from:

• Define structs for all the JSON you’ll need to parse. This is the most Go-idiomatic way to parse JSON in Go. It’s also the furthest away from using Python dicts if you’re coming from that world and frustrating if the JSON doesn’t have a clearly defined spec.
  • There’s even a few tools that use the explicit definition to pre-generate parsers for better performance.
• Use empty interfaces to parse the JSON, then convert each field to the type you need. This is promising on the surface but quickly becomes complex.
• Use one of a few wrapper packages that use an empty interface underneath but make the type conversion more natural.

Parsing JSON using struts

This is the standard path for Go and, cutting to the chase, the option that felt most natural in the end. Though it may seem frustrating if you’re coming from a dynamically typed world, fighting against this will be frustrating in different ways.

My server needed to receive a JSON-encode request and response back with more JSON. I used the standard library’s net/http server (walkthrough if you need it). There’s a bit of boilerplate required to make it behave properly (throw 400 and 500 errors, set the right content-type), so I’ll include that at the bottom of the post.

Your first step is to define your struct. You have a lot of options at your disposal and Eager wrote a great blog post on how to leverage structs to parse JSON. Here’s what a basic one might look like:

type MoveRequest struct {
    GameID string  `json:"game_id"`
    Snakes []struct {
        Coords  [][]int `json:"coords"`
        Name    string  `json:"name"`
        Taunt  *string  `json:"taunt"`
    } `json:"snakes"`
    Turn int `json:"turn"`
}

A few things to note here:

• the json:"game_id" fields on the right are called tags and are used by the standard JSON package to map JSON fields into the right field in a struct. They’re not always required (Go will try to match up names ignoring capitalization for example), but it’s helpful documentation
• You can include arrays and nest structs in your definition.
• One place I got caught: If your JSON can contain nulls then you need to do a bit more work. It turns out that Go variables can’t be nil, only pointers can be nil so you need to use *string as the type for a field that might be null, and refrence it appropriately in your code. You can see an example of this with taunt in the above struct.

Now, creating a structure for an arbitrary JSON API can be a daunting task, but thankfully there’s a few projects that will help you do this automatically just by providing a sample of the JSON. I tested the four I found (listed in the repo) and prefered the output from
gojson the most as it nailed the formatting and naming, even getting capitalization right. If you’re in a hurry, json-to-go will do the conversion in your browser. Keep in mind, it’s automated. So take a look at the output and make any changes you need.

With your structure, you can now parse your JSON with the built in encoding/json library.

func MoveHandler(w http.ResponseWriter, r *http.Request) {
    request := MoveRequest{}
    err := json.NewDecoder(r.Body).Decode(&request)
    if err != nil { 
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    ...

You may see other examples that use Unmarshal, you really shouldn’t use that unless the data is already in memory (see Datadog’s love letter to io.Reader).

Once that’s done, you can access all the fields in a very natural way:

    fmt.Printf("%+v\n", request) // The + in %+v adds field names when printing structs
    fmt.Printf("Turn: %d\n", request.Turn)
    fmt.Printf("Snake name: %s\n",request.Snakes[0].Name)

Generating a JSON response follows pretty much the same model:

    ...

    // You'll define this struct beforehand
    response := MoveResponse{Move: "down"}

    jsonResponse, err := json.Marshal(response)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    w.Header().Set("Content-Type", "application/json")
    w.Write(jsonResponse)
}

Nothing revolutionary here, but you can start to add some nice things. Go’s function model lets you hang functions off any struct. For example, I can add a String() function to the MoveRequest that will automatically get used whenever I try and print it, making logging and debugging easier:

func (m MoveRequest) String() string {
    return fmt.Sprintf("Game [%s] is on turn %d", m.GameID, m.Turn)
}

You can use this feature with the built in JSON decoder to change the behavior by implementing the func type UnmarshalJSON(b []byte) error interface on your struct. Let’s change Coords from a two dimensional array of ints to an array of X,Y Coord type.

type Coord struct {
    X int
    Y int
}

type MoveRequest struct {
    GameID string  `json:"game_id"`
    Snakes []struct {
        Coords  []Coord `json:"coords"`
        Name    *string  `json:"name"`
    } `json:"snakes"`
    Turn int `json:"turn"`
}

func (c *Coord) UnmarshalJSON(b []byte) error {
    var tmp []int
    if err := json.Unmarshal(b, &tmp); err != nil {
        return err
    }

    if len(tmp) != 2 {
        return errors.New("Coord only accepts a length two array")
    }

    c.X, c.Y = tmp[0], tmp[1]
    return nil
}

Then accessing the values is straightforward:

    snake = request.Snakes[0].Coords[0]
    fmt.Printf("Snake position: %d,%d\n", snake.x, snake.y)

Using empty interfaces instead

But what if you don’t want to do all that struct work ahead of time? I sure didn’t, so I kept looking. I quickly found the empty interface but it took me a lot longer to realize the problems.

Here’s the theory: Every variable in Go must be typed. But a type can be an interface and an interface can have zero functions. This means that you can catch any variable of any type by looking for interface{} which every possible type satisfies. So instead of doing the hard work defining a struct, you can just import your JSON into a map of strings -> empty interfaces like so.

var request map[string]interface{}
err := json.NewDecoder(r.Body).Decode(&request); 

Well that sure looks easy! By default, golang looks at the input and tries to unmarshal to the right type, then you just have to cast to the variable you need. Turns out, that’s much easier said that done.

turn := request["turn"] // returns a interface{} typed value

So I need to type assert it using .(int) syntax. This immediately breaks. It turns out that go marshals ints into float64 because it can’t confidently move between JavaScript’s int and Go ints. There’s a long discussion about how the community thinks its a bug and Go creators effectively telling themselves to Go &^#@ themselves. What this means is you actually need to do this:

turn := int(request["turn"].(float64))

Well that’s ugly. What about this JSON.UseNumber thing? That seems to inspire some confidence. Nope, just as ugly:

turn, err := request["turn"].(json.Number).Int64()

And accessing nested values get even crazier:

taunt := request["snakes"].([]interface{})[0].(map[string]interface{})["taunt"]

At this point, I quickly wrote this off. But I couldn’t be the only person looking for a more flexible way to parse JSON. Turns out I wasn’t.

Wrapping empty interface maps in syntactic sugar

There’s at least three projects out there that wrap importing into an empty interface to make accessing the values easier. None are perfect, but they might be a good choice depending on your use case.

typed

This was my quick favorite because of the terse syntax. Even digging multiple levels is relatively straightforward:

request.Objects("snakes")[0].String("taunt")

Unfortunately, ran into a few issues:
– No support for multi-dimensional arrays (no .Arrays() import)
– No io stream initializer
– No ability to create JSON, though there is a sister project that offers this ability

go-simplejson

It’s syntax is a bit more verbose, requiring you to switch between gets and type assertions:

taunt, _ := request.Get("snakes").GetIndex(0).Get("taunt").String()

On the plus side, it includes the ability to create JSON built in. Though it also struggles with multidimensional arrays, it doesn’t make it impossible.

jason

This syntax is also verbose, and because each step returns a possible error, it’s hard to chain. So the one liners above become:

snakes, _ := request.GetObjectArray("snakes")
snake := snakes[0]
taunt,_ := snake.GetString("taunt")

It’s also impossible to handle multi-dimensional arrays and no JSON creation, and this one seems to be missing a lot of the sugar.

End of the day

Despite all my attempts to break away from the default, I couldn’t find a solution that felt right. I think typed might be the closest to handling arbitrary JSON, but it stills falls short for this case. I’ll keep an eye on it.

For now, the best “feeling” solution for what is now a very familiar API is to stick to structs. I’m still concerned about having to do all that work upfront before testing any API, but maybe that’s just what idiomatic feels like.