Up All Night

“Knew we would crash at the speed that we were going
Didn’t care if the explosion ruined me…”
– Charlie Puth, “Dangerously”

The quote above is from a song that I’ve been listening to a fair bit recently, and I’ve picked up on those two lines although in a different context (as you might expect, the original song is concerned with a reaction to a breakup). I’ve been thinking about how my work practices could work in the longer term and what would be sustainable. Nonetheless, hearing those two lines makes me think of deep surges; some of the most short-term of these could perhaps take the form of all-nighters.

I’ve been fairly lucky in that I haven’t had to pull many all-nighters for quite some time. I think I only did this once for MCMAS-Dynamic (during the report-writing stage; generally given the technical complexity of the work I don’t think it would have made sense), and I don’t think I did one during the third year group project. I also remember having executed one during second year when revising for the exams, though that was thankfully well before said exam period. There have been several hackathons, of course, as well as other occasional personal surges but generally I find that I perform best if I have adequate sleep, and even in the relatively short run I’d be better off doing three say 15-hour days, punctuated by relatively normal sleep (well, as normal as that can be given such a schedule) than plugging away in a continuous stretch.

Anyway, besides the Charlie Puth song I’m also writing about this now because I voluntarily did one this week, though for a rather different reason: watching the US presidential election. I had a couple tabs open with various election newsfeeds and a couple watching market futures and GBPUSD. On hindsight I’m not sure exactly why I did it since it was pretty apparent midway through (I think around 2-3 am in London time) that things were going Trump’s way, and I wasn’t trading through the night (by the time markets opened in the morning there wasn’t too much of a cheap-buying opportunity). That’s a subject for another post, though.

I think the negative effects of sleep deprivation are well-documented; I’m not sure exactly why I pulled the all-nighter for the MCMAS-Dynamic report (probably wanted to rush something out for a supervisor meeting the next day), but I do distinctly remember that the two or three pages that I cranked out, while probably not bad per se fell particularly far short of my quality standards in a later proofread. The problem I’m trying to address with an all-nighter involves not having enough time to deal with a short-run (typically next-day) requirement, and in less extreme cases it’s not the only solution; where possible, I’d also like to try an alternative of waking up abnormally early to work on the issue. Understandably, there are risks that one might fail to actually wake up early, though I think this can be mitigated with suitable (read: loud and highly dissonant) alarms.

However, there are cases where I find this to be the best solution anyway. Some of this might involve external time constraints (for example, if it involves live following of current events – the aforementioned US election is one, or the recent World Series if one’s so inclined; examples from software engineering could include firing off long-running performance or integration tests, or meeting sudden customer requirements). Also, for suitably short time spans this is likely to be an optimal or near-optimal solution (even then, a 1.5 hour nap could potentially be useful in such cases). I think another useful factor to bear in mind would be the activities planned for next day (an exam or interview would be very bad, for instance).

Once the decision to forego sleep has been made, I usually don’t find the direct implementation of all-nighters to be too bad, perhaps because for things to have reached that point there must have been a compelling reason. Typically, by then the outcome-oriented side of me takes over and decides that it would be a night of crushing things (though it doesn’t always calculate the costs appropriately).

I think for me at least the most challenging part of this is managing its costs the next day. I personally don’t perform well if I haven’t had enough sleep, and there’s also a risk of overcorrection (that is, sleeping too early, which messes with the sleep schedule for the next few days). I guess caffeine can be deployed to some extent to address this, though I’ve been on the wrong side of that as well. I find that removing access to a bed at least until only a few hours before one’s normal bedtime can help as well – in fact, staying outside is probably even better (I can sleep on a chair if I’m at home).

In summary, it’s a very useful tool in my experience, and there are circumstances where it might be necessary or optimal, but generally speaking where possible this should be avoided.

Tracking Times at Imperial

Graph of my work time at Imperial

The graph you see above reflects the number of hours I spent on “work” each week, from the week starting 5th October 2015 (I finished my internship at Palantir on the 2nd of October) up to the week starting 5th September 2016 (I started full-time at Palantir on the 12th). Obviously, this includes all time spent explicitly on academic work, at Palantir (social events do not count) as well as time spent tutoring (inclusive of marking and preparing the problem sheets). There’s more to it, though – I place work in quotation marks, as there are quite a number of activities that people might not classify as work that I do count towards the total, such as having 1:1 syncs with people and personal reviews.

I’ve annotated the significant peaks and troughs on the graph with some of the events that had taken place around then that contributed to why I worked so much (or so little). You’ll see that I’ve shaded the part of the graph above 70 on the y-axis in red; for me at least, I think I instinctively start feeling some degree of push-back at that point (and I’ve been cautioned that 70 is already pretty far on this).

Typically, when I look at a graph, I try and identify things that I find to stand out as unusual, and then seek explanation for them. Initially, what does stand out to me is the relative lack of height of the peak labelled (6), the weeks leading up to the end of the Final Project; I would have expected something quite a bit more. I’d attribute this to the sheer cognitive difficulty of the final stages of said project; I remember finding that I would be drained very, very quickly when working on it. I guess for the final project I worked at it pretty consistently over the year, so there was no need for a massive surge at the end as well.

I notice extremely sharp drop-offs (A) and (B) after the end of term 1 and 2, yet no such drop-off exists after term 3. Perhaps, this is a signal that the 58.95 or 61.25 averages in those terms are too harsh (summer term was a relatively tame 52.33), and this does already factor in exam week or week 11, which tends to be less intense as I need to conserve my energy for the examinations themselves. I tend to think of week 8 or 9 as the busiest week in each term, owing to exam revision, and this pattern is reflected in peak (5), but seems absent in term 1 which, in fact, exhibits a convex sequence. This might feed back into the earlier point about considering 70 hours a week as a dangerous point to be insufficiently prudent; there is a crash even after a series of weeks in the 60s.

Although I recompiled this graph recently, I first performed the labelling in September just before I started at Palantir. Nonetheless, looking back at it about two months later, one of the labels stands out to me, that being (A) perhaps because in and of itself it does not seem to give a proper explanation of why the trough (or peak, in the other cases) was there. I did return to Singapore to spend time with my family and a few friends, but it wasn’t really the case that I did very much on that front – in fact, a fair chunk of my time in Singapore went towards MCMAS* (which explains that mid-50 spike, which is actually the week starting 28th December) and Fallout 4 (which, of course, did not count).

It’s heartening for me at least to see that I have a fair degree of intrinsic motivation, as shown by the red line. Over the roughly two months, I managed to work on MCMAS-Dynamic, program extensions for Keep Talking and Nobody Explodes (and, in doing so, revisit programming in C#), set up this website, complete a full retrospective round of the first and second year examinations and learn more about personal finance and investment.

Clearly, the data may be analysed as a time series; in fact, I have bucketed this graph into weekly aggregates, but I do have data down to a daily granularity. An alternative way of handling the daily data could have been to compute a simple or exponential moving average of the data, though I don’t really like doing this at a daily granularity is because of very clear seasonality (in particular, I tend to work the most on Tuesdays, or Wednesdays part-time at Palantir; I work the least on Thursdays and Sundays).

I only did this in year 4; it would be interesting to see how the time profile would have looked like in previous years and how similar it might have been, though the different structure of each academic year would probably have made the series look somewhat different (for example, the beginning of the third term in year 3, very near the (B) trough, would have been a very high peak as I aggressively ramped up for the industrial placement at Palantir).

This was a rather interesting exercise, even without looking into the distribution of time across modules and/or activities. If that is factored in, there are other interesting patterns; for example, the amount of time spent on MCMAS, which involved considerable surges after each set of exams was dealt with, and the time spent on 1:1s and syncs, which was generally a few hours per week but had a 12.5 blip in the final week of the summer term (before people dispersed) and a 10 somewhere later on (maximum likelihood guess would be meeting three of the guys I regularly sync with nowadays for meals on three distinct occasions).

I haven’t been doing this as rigorously ever since I started at Palantir, mainly because the initial impetus behind this initiative was actually understanding which modules I was spending a disproportionate amount of time on (Computing for Optimal Decisions, Software Reliability), and also because I had ready access to my personal email accounts (I tracked the data using Google Calendar); also, my work is far more reactive to changes that may happen because some high-priority issue appeared. Perhaps using it for spare time could be useful, but then the administrative overhead is much larger relative to the time actually being tracked – to a point where I’m not sure it’s worthwhile.

Charting Courses

I had lunch at Nando’s with a friend 2 days ago, and I mentioned that I had been doing my planning for the month of November before said lunch. We had a brief discussion concerning the scope and methodology behind this, and I thought it might be interesting to share how I approach it. I’ve been iterating on this approach since starting fourth year at Imperial. I figured explicit planning was important for two reasons:

  1. I had a lot more flexibility as to how I could schedule my time, with projects spanning multiple terms, and
  2. I had more on my plate as well, having to manage 2 part-time jobs and my own personal development in parallel.

Now that I’ve graduated from Imperial, I find this is arguably even more important:

  1. There isn’t even a “term” unit now. I might have to answer questions, do emergency investigations/dev work, and even be an SME about things I implemented on previous projects. (Though that’s validation that my code’s still in production!)
  2. Personal development is still an issue, and there might be even less of an obvious driver or reminder that it’s important (the part-time jobs introduced weekly quotas and scheduling, in a way).

I have tried to be careful to avoid excessive planning for quite a fair bit of time, mainly because I want to maintain some degree of flexibility and also because it’s difficult for me to measure how long certain tasks will take (especially creative tasks; cue Hofstadter’s law). I thus don’t view the plans as absolutely prescriptive and dynamically redraft them as needed.

Typically, I have an annual planning exercise, which largely involves developing objectives and key requirements for the year across various domains, both professional and otherwise. This has been in place since year 2 at Imperial, actually, and was typically written at the start of the academic year, though I plan to realign it with the calendar year starting from 2017. Here’s an excerpt from what I wrote at the beginning of year 4 (note that there were more objectives as well as more subgoals within each objective):

  1. To be an accomplished student for my final year at Imperial.
    1. Obtain a solid overall average for the fourth year at Imperial. The score is measured by linear interpolation with 84 as 0.0, and 94 as 1.0. (So 0.7 is 91.)
    2. Complete a strong final individual project. 85 is 0.0, and 100 is 1.0. (So 0.7 is 95.5.)
    3. Successfully train a group of PMT students to develop aptitude and, hopefully, interest for the subject. Measurement involves giving the students a feedback form at the end of term 2, asking them to rate the sessions for quality on a 7-point scale. 3.5 is 0.0 and 6.5 is 1.0.
    4. Challenge said group of PMT students. The survey should ask how difficult the sessions were, again on a 7-point scale. 4 is not good (the sessions should challenge them). In fact I’d target 5.25 as an average, for 1.0. 4.0/6.5 would be 0.0.
  2. To manage one’s finances in a coordinated and efficient way.
    1. Max out my Stocks and Shares ISA for the 2017 tax year. 9,000 is 0.0, and 15,240 is 1.0 (obviously). This is allowed to wait for my start at Palantir.
    2. Earn a sufficient income from part-time work over the course of the academic year. 2,000 is 0.0, and 10,000 is 1.0.
    3. Read books concerning finance and financial management from (a reading list I defined at that time), with extensions allowed should I find other quality texts along the way. 0 would be 0; 9 would be 1.0.

The next finer-grained level of planning would be academic terms, which corresponds to quarters in the post-Imperial world. I think I started this from year 3. (I treat the summer holiday as Term 4/Q4, and would do planning for what I wanted to achieve during those as well.) This would include slightly more information about what I planned to do, though it would still be at a very high level (no/minor concrete action planned). I normally didn’t do any planning beyond that.

For monthly planning, there is some degree of executive function that’s involved. I block out time for certain time-inflexible important events or things I need to do, where feasible (for example, making sure to attend the BCS award presentation ceremony, or the graduation at Imperial). It’s also a good time to pencil in anything that should be done monthly, even if it’s not necessarily the case that it’s time to do it. For example, in the absence of shock events I typically have a look at my investment portfolio once a month on the day after payday – this is used to make potential movements and have a quick look as to whether there’s anything in particular I need to position myself for (for example, getting a chunk ready to toss at the market in the wake of the Brexit referendum; similarly, I’m keeping an eye out for November 8th this month).

I also review the quotas that I’ve set – that is, things which I set out to do N times every time interval T (thanks to John Sonmez’s book for the word). Some of these involve meeting or talking to certain people whom I value highly and wouldn’t get to see otherwise (>50% of the guys I regularly talk to are not or no longer based in London or the UK for that matter); others involve financial planning, or personal learning and development objectives.

It’s also a good time to reflect on how things have gone in the previous month, and whether my OKRs and things I’ve wanted to get done for the quarter are tracking or not – and possibly to react to these should things not be going well and/or be overheating (as tempting as it is for me to try and run my winners, in some sense). In any case, I find allocating a decent chunk of time to plan this out (usually around 90 minutes) useful for helping me to decide what to allocate the rest of my time for the month towards, without imposing an excessive overhead (that I suspect would otherwise be amortised over the course of the month anyway). I’ve found it to be a useful exercise, though of course its applicability will depend on the individual.

New Home

I thought I’d establish a little corner of the Internet to document some of my work, as well as my individual profile. The site’s still under construction – watch this space for updates.