Twitter’s Open Source Algorithms Are More Complex Than Musk Suggests

Of all the great ideas Elon Musk has for Twitter, the one he has presented most fervently is making the platform’s algorithms open source.

The Tesla tycoon proposed the plan before its takeover bid was released, it reiterated it on the day its bid was revealed and launched it once more after the deal was confirmed.

Musk outlined his proposal in a TED Conference on April 14:

It is very important that people have the reality and the perception that they can speak freely within the limits of the law. So one of the things I believe Twitter should do is make the algorithm open source.

Musk argued that disclosing what amplifies or demeans tweets would reduce the risk of “behind-the-scenes manipulation.”

The approach has gained support from some transparency advocates and critics of Twitter’s content moderation. They argue that the measure will reveal how Twitter determines what you see on your timeline — and what you don’t.

“It has the potential to turn Twitter into a truly trusted platform where users would understand why certain tweets appear at the top of the list, and all concerns about secrecy or behind-the-scenes bias would be removed,” said Marc Linster, CTO of open source database company EDB.

“These concerns have been rampant with Google and Facebook. This open source move could be a game-changer for social media in general.”

Skeptics, however, questioned the plan’s feasibility. They note that Twitter is made up of multiple feeds, from the trending section to its home timeline, each controlled by a complex mix of recommender systems and human decisions.

These processes produce results that not even their developers fully understand. Some of them supposedly mocked Musk adding a public repository (now removed) on the company’s GitHub platform — with zero code.

Another issue is that algorithms alone offer limited insights.

There are several other factors behind ranking a tweet. They include the content that enters the platform, the profile of each user, the algorithm training data, the moderation rules and the code that trained the models.

These constitute a huge dataset, which would be difficult to sift through and expensive to disseminate.

“You can’t just open code an ML [machine learning] like a bubble-like implementation,” said Steve Teixeira, Twitter’s vice president of product.

Other complexities arise from the mutability of the system.

“Typically, recommendation models are retrained quite frequently and keep changing over time,” Bindu Reddy, CEO and co-founder of Abacus.AI, an artificial intelligence startup, told TNW.

“While it is possible to release all the trained models continuously, it will also not be very useful unless you understand exactly what inputs and outputs go into the model for predictions.”

There are also the potential dangers of the open source proposal.

Information can be copied by competitors, provide a tempting target for cybercriminals, and violate user privacy. It could also get in the way of another of Musk’s ambitions: “defeating spambots.”

On the other hand, open source offers new opportunities to find vulnerabilities and flaws.

Reddy is optimistic about the potential benefits. She argues that the open source classification algorithm will be useful for researching and evaluating any biases.

She also hopes to find more insights into the infrastructure components that influence what gets flagged and filtered in feeds.

“Open getting these algorithms – and more importantly, these models – will help a lot in terms of transparency,” she said.

Another prominent proponent of the approach is Twitter co-founder Jack Dorsey.

The company’s former CEO suggested allowing users to choose which – if any – algorithm they use.

Dorsey envisions creating an open market for algorithms.

Users would choose whatever best suited their desires, from prioritizing subtle conversations to bringing up a constant stream of thirst traps.

It looks potentially idyllic – especially if it can keep my feed from constantly showing nasty Elon Musk tweets.

Leave a Reply

Your email address will not be published.