As I sit here in my living room, it’s the first of 2024, it’s 9pm. I finally realise that this blog is a parallel to me. It is neglected, flabby, ugly, and needs some work.
I finished last year by becoming a doctor (my viva went well, thanks for asking), at great sacrifice to my waistline, blog, and mental health. This year I intend on blossoming in my personal projects, self-care and self-improvement.
I have begun reading to re-sharpen my machine learning knowledge, and will soon commit to code, to try and get some research done. I am primarily interested in fundamental model structure, as things like transformers, while amazing, are almost certainly not the optimal solution. I predict there is a vast ocean out there of more efficient solutions, and hope to explore it, and (potentially) publish in the field one day.
I have picked running back up with the hope to train for distance this time, my goal is to be able to do a half marathon by August. I log my exercise on Strava to keep myself accountable, however I may sometimes need to take extended breaks for medical reasons (I have haemophilia).
I have also begun work on a new open source software project. The new project I’ve started has been nicknamed ‘Sonnet’, and it intends to solve the problem of people misusing bookmarks. Expect a post soon with more details, with a brief manifesto about what I think is going wrong.
There are three papers left from my DPhil that I am yet to submit for review, and my minor corrections on my thesis are due very soon. With the conclusion of this package of work, my seven years at Oxford comes to an end. It is sad to finally close the chapter on what has been some of the best years of my life, but I am excited for the future.
Once these publications have concluded, I want to commit my spare time towards the following pledge:
Finally, an aspirational goal for 2024 is to create a video for the platform YouTube.
I figured it might be worthwhile throwing some of my thoughts out there (it’s mostly going to be my friends who read my blog… right?) in case anyone fancies a read of my shouting into the void. This is the first time I’ve really had a go at this idea, it might fail or be unpopular, but I might as well have a try.
In this site’s history, I generally have kept my personal views out of it and stuck to nothing but facts. I think this is due a change, this is my space and I should have somewhere to record my viewpoints, even though I generally dislike opinion-column journalism.
Therefore, the budding reader should expect more of my personal views embedded in this website, these opinions are reasonably representative of what I mean. I suspect there is value in me documenting my thoughts in public, we should have more challenging discussions, and proactively call out things that are bad.
I do worry that papers (in ML) are tragically becoming a vessel for silicon valley bull$hit. It appears that collectively businesses finally figured out that people actually read papers, and have therefore decided that they are an appropriate vector of advertising. This new breed of publication generally materialise as low effort arXiv dumps, for example this paper, and should not be given the time of day. Unfortunately, many of them gain undue fame and frequently end up on the top of hacker news.
We need to take a step back and remember that while it is cool, arXiv is not peer reviewed, and in this age of machine learning hype where there is (I’m sure) immense pressure to publish, bad papers will bleed through. As researchers in general we should take care in how we publish, and not mindlessly stampede into this new regime, as everybody is fallible. I fear that the field of ML may end up having its own replication crisis-esque moment as a result, but who can really tell, maybe I’m wrong.
Silicon valley, regrettably, has a pattern of enshittification. I hope that academic publishing does not fall victim to this too, however I would argue that a reasonable portion of of machine learning research is already there.
I used to enjoy keeping a tab open on Hacker News, to keep up to date with the latest happenings in the space, that has rather unfortunately changed recently for me. As you may know, I just finished my DPhil in Optical Wireless Communications (OWC). I happened upon a post in hacker news about OWC, and was excited to jump in, however the discussion thread was nothing but uninformed non-experts getting basal facts wrong. My voice, an actual expert, was lost among a gaggle of software engineers frothing at the mouth to get their uninformed opinion across. This realisation was where I decided the comments no longer provide value to me, if the comments get the topic I know as an expert so flatly wrong, why should I treat them as credible for things I’m not an expert in?
From my viewpoint:
For this reason, I’m off looking for greener pastures for information. I find myself typically reading more important information direct from the source (be it papers on huggingface, IEEE etc). I do sometimes still use hacker news to see if anything I may have missed materialised, but it’s no longer a tab always open on my desktop I relentlessly check - I’d be happy if I found another news site so I can rid myself of it forever.