egrefen

Honest answer? A nap, and maybe a few hours to play Cyberpunk 2077 on PS5? I bought it and haven't really touched it (or any other games) in like a year, aside from 5-10 minutes of playtime gleaned here and there.

OBVIOUSLY THE RIGHT ANSWER HERE WAS HAPPINESS FOR MY FAMILY AND WORLD PEACE, BUT I'M SELFISH LIKE THAT.

Large Language Models. I'm not only saying this because of my role at Cohere. In fact, my belief in this is what led me to my role at Cohere, when I was happily hacking away at Reinforcement Learning and Open Ended Learning research up until 2021 (an agenda I still pursue via my PhD students at UCL).

Language is not just a means of communication, but is also a tool by which we interact with each other, negotiate, transact, collaborate, etc. We also use this prima facia external tool internally to reason, plan, and help with cognitive processes like memorization. It seems almost obvious that giving computers something like the ability to generate language pragmatically, to do something like understanding language (or a close enough functional equivalent) has the immediate potential to positively disrupt the tools we build, use, and the way we work and operate as a society.

With the ability to zero-shot or few-shot adapt large language models to a surprising number of downstream cases, and further specialize them via fine-tuning (further training), I believe this class of technologies is at the point where it is on the cusp of being practically applicable and commercially beneficial, and I'm excited to be part of the effort to make both of those things happen.

There is a spectrum of sorts when it comes to fears about AI, spanning practical concerns to existential ones. I do not want to dismiss the latter end of the spectrum, although I have little time for the whole killer AI story line (humans are already experts at destroying each other) or the whole longtermism debate, and I'm more interested and concerned by the practical risk that rapid technological advance will disrupt the economy in a way which is so rapid individual and professional fields don't have time to adapt rapidly enough. We saw this (not directly, mind you) with the industrial revolution, as machines replaced manual labour, and the same could happen again. I don't have any easy answers to this, but when it comes to building products, services, and new ways of working and producing economic value on top of the technology we are building, I can only hope developers and inventors alike will prioritise building tools that work symbiotically with humans, that assist their work and simplify it, rather than seek to automate away human jobs (at least in the short term), giving society and the economy time to adapt.

As this is not my specific area of bleeding edge expertise, I've asked people on my team who have a more learned opinion on the matter (delegation!!). My colleague Eddie Kim writes:

The SOTA for explicit, reproducible, configurable data pipelining has advanced a ton in the past ~5y, and this has been tightly coupled with the rise of MLOps and the fact that ML vastly increases the amount of statefulness you must manage in a system or product due to datasets, data-dependent models and artifacts, and incorporating user feedback.

It's kind of you to say I've achieved a lot, although from my perspective that is thanks to have been fortunate enough to work with people who've achieved a lot. I always feel I could do more, and feel stimulated by chasing the opportunity to innovate, be it scientifically, through entrepreneurship, or the intersection of my technical interests and entrepreneurship as I am currently doing. At the same time, I have a family and young kids who want to spend time with me (for now!) and a lovely partner who wants to have a life of her own and time to focus on her career, so I'm learning to balance the need to focus on my own need for excitement and stimulus, and the responsibility to ensure others in that unit are also kept happy and stimulated in their own way. It's hard and, in its own way, a stimulating challenge in itself :)