Leftover Links, 2021 edition

Whenever I skim my RSS reader (I use Netnewswire, in case you are wondering), I make vigorous use of its “starring” feature and then forget about most of the things that I marked as “starred”. Then, every couple of months I feel entirely overwhelmed by all those “starred” articles in my feed reader and I start vigorously pruning them, but there are always a few links that are stuck because I feel I should post about them here. And then, I move on and never post them.

To remedy this unfortunate situation and to help me clean out my “starred” folder, I’ll start putting those links where I always intended to put them, namely on this weblog.

Here are the five leftover links from my starred folder from 2021:

Hopefully I’ll get around to posting the 2022 batch of leftover links soonish.

New Video: Beyond the Desktop – Designing Future Interactions

I recently published a new lecture recording on Youtube: Beyond the Desktop – Designing Future Interactions.

This one is a bit special to me as it is probably my favorite among the dozens of lectures I’ve prepared and presented over the years. It is about the history of desktop GUIs, the promise of ubiquitous computing, the rise of mobile, technological innovation, technology adoption and finally, it is also about science fiction as a tool for prototyping and thinking about the future. It covers a lot of ground but I like to think that I managed to pull it all together in a rather elegant and coherent fashion.

The first version of this lecture was presented in 2016 and it gave me an opportunity to collect and assemble a lot of my thinking and preoccupations back at the time. The published video is from 2020, recorded during the great pandemic with its mandated distance learning formats and live-streamed video lectures, with some modest editing and expansion in subsequent years. Looking at it now it becomes painfully apparent that it doesn’t live up to the technological Zeitgeist – there isn’t enough talk of mixed reality, the metaverse and artificial intelligence for that – yet at the same time I’m happy that this archival version of possibly my favorite lecture exists and that it is finally publicly available.

I should mention that the production of this video was only possible thanks to Descript, a new AI-enhanced video editing tool that allows you to edit a video by editing the text transcript of that video’s audio track, allowing for precise edits down to cutting out single words and utterances. Previous versions of this video were edited in iMovie and none of those were fit for publication, but Descript enabled a level of editing precision and efficiency unlike anything I had experienced before. So if you’re editing text-heavy videos such as lectures, how-to guides, reviews, podcasts and the likes, I can highly recommend Descript.

PSA: I’m now on YouTube

One of the more interesting side effects of the pandemic and lock downs was that as a researcher and lecturer it became almost a necessity to also be a Youtuber and video producer. At first I very much resisted this development, but over time I came to embrace it and to my great surprise, I’ve even come to enjoy it.

So while I have pretty much stopped posting on this weblog and most of social media over the past few years, I have started a small Youtube channel. Originally it was mostly old conference presentations, but now I’ve also started to archive some of my lectures over there. I’m particularly fond of my Usability Engineering lecture archive. I originally started presenting this lecture in 2009 and the past summer term of 2023 was probably the last time the lecture was held due to upcoming changes in the computer science curriculum at TU Wien. As such, the lecture archive might be considered the final, definitive version of this lecture after 15 years of iterative updates, expansions and refinements. It’s only available in German, but if you are so inclined, maybe check it out.

Musings on text classification and LLMs

Over the past few months I have been fascinated by machine learning and have spent quite some time trying to understand it better: figuring out what it can do, but also its limitations. I believe that ML is the most exciting thing in IT and CompSci right now and that it will have a truly transformative effect on how we work. It remains utterly compelling and its capabilities regularly surprise me still.

Recently I was looking into text classification for a master’s thesis that I’m advising on. In earlier days, the conventional approach might have entailed creating a specialized framework, perhaps designed specifically for tasks like sentiment analysis. While others could have used the same framework for their individual sentiment analysis endeavors, you wouldn’t have been able to use it for much of anything else.

Now with ML, I expected it should be possible to do general text classifications – e.g. train a pre-existing model on some example cases and then it will figure out the rest. That’s pretty much what models like BERT do, and they are marvelous: load the model from Hugging Face in Tensorflow, provide a CSV file with your text classification data, and it will do the rest. Once you figure out the tooling (in my opinion the hardest part in contemporary IT), it is surprisingly simple.

However, during my research I also came across zero shot classification using large language models, which is basically classification with no prior training. You just show the model some data and ask for its opinion, and with modern LLMs you can even ask in natural language, e.g. “Does the following sentence convey a positive or negative sentiment, answer in one word: I am happy”.

That such an approach yields useful results is truly astounding to me, because it is so remarkably simple. You don’t need to develop a framework, you don’t need to train a model, you don’t need to learn the syntax of some obscure library or API, you just ask. As I mentioned before, tooling is perhaps the hardest part in IT, and this kind of approach gets rid of that completely.

Bye Twitter, it was nice not really knowing you

So like any sane person it was time for me to give up on Twitter, née X, a couple weeks ago. My account had been set to private for years but now it was time to go through my timeline and delete all the old tweets. While I expected the task to be cumbersome, it turned into a pleasant stroll down memory lane, with some genuinely curious and delightful things to be rediscovered. I probably hit my peak Twitter activity between 2011 and 2013. Looking back, it is fascinating how little some things have changed, while others have completely transformed.

I never quite developed a proper Twitter habit aside from that short stretch about ten years ago, so I won’t really miss it. Nonetheless, I felt a pang of regret when deleting my Twitter archives, until I realized that luckily, I have syndicated copies of all my tweets as Twitter Digests here on this site – lucky me!

Anyways, abandoning Twitter I immediately signed up for a Mastodon account like everyone else, only to be baffled what I should be writing over there – I guess this sort of Microblogging just isn’t really my thing. However, after weeks of indecisiveness I have decided that it might be time to give this old weblog here another shot. Sometimes it would be nice to have a place to jot down half-formed thoughts and brain farts in a semi-anonymous, public space.

We’ll see how it goes.

Straight2Spam

Straight2Spam generates invisible text that you can paste into your e-mails so that they (hopefully) get caught in the recipient’s spam folder. I can see how this could be useful.

Google Slides is Actually Hilarious

Perhaps like you, I naively started out thinking that Google Slides was just a poorly maintained product suffering from some questionable foundational decisions made ages ago that worshipped at the shrine of PowerPoint and which have never since been revisited, but now, after having had to use it so much in the past year, I believe that Google Slides is actually just trolling me.

Google Slides is Actually Hilarious | by Laura Javier | Medium

Death by a thousand cuts.

A Meta prototype lets you build virtual worlds by describing them – The Verge

Meta is testing an artificial intelligence system that lets people build parts of virtual worlds by describing them, and CEO Mark Zuckerberg showed off a prototype at a live event today. Proof of the concept, called Builder Bot, could eventually draw more people into Meta’s Horizon “metaverse” virtual reality experiences. It could also advance creative AI tech that powers machine-generated art.

A Meta prototype lets you build virtual worlds by describing them – The Verge

Copy & Paste the Real World into Cyberspace

A very clever and impressive tech demo by Cyril Diagne allows you to copy & paste objects from your real-world surroundings into Photoshop using your smartphone:

The code is available on GitHub and it’s probably worth pointing out that BASNet, the machine learning smarts responsible for object recognition, are available on GitHub as well.