Liveblog: Open-sourcing Twitter's algorithm

02:02pm

We are signing off for now. Check out the following:

• Source code for Twitter's Recommendation Algorithm
• Request access to Cody

01:43pm

Government requests for intervention on Twitter must have been so pervasive Twitter Engineers even have a class for it in the Twitter Algorithm pic.twitter.com/F05sD5h9Lk— Alec Sears (@alec_sears) March 31, 2023

Link to code

=======

01:19pm PDT

Twitter just released source code for "the algorithm" Oh, what file is this? Predicates for tweets on the home timeline? Oh what is that 2nd image? pic.twitter.com/UE3dU8e3Os— Ólafur Waage (@olafurw) March 31, 2023

view the code

What is this? ( "author_is_elon", candidate => candidate .getOrElse(AuthorIdFeature, None).contains(candidate.getOrElse(DDGStatsElonFeature, 0L))),https://t.co/mLdjWWYHrF— David Mander (@davmander) March 31, 2023

view the code

12:43pm PDT

The 4 types of Twitter posters, according to the just open-sourced algorithm 😯https://t.co/xTLX77vJ75 pic.twitter.com/SaQN03P9eK— Amjad Masad ⠕ (@amasad) March 31, 2023

Link to code

A quick search in Twitter's Recommendation Algorithm for Ukraine. 🇺🇦 topic is on the same list as: Do not amplify, do not public publish, medical misinformation, NSFW, and violence. What do you think it means? 🤔 pic.twitter.com/PYqm8pZjI4— Mykhailo (@mxpoliakov) March 31, 2023

Link to code

12:24pm PDT

• Precise code navigation is now on! Example
• Cody codebase exploration
  • Using Cody to explore the codebase; it pretty quickly found the search indexer, which handles about half of the tweets

12:14pm PDT: LOC

12:04am PDT: communications

Twitter recommendation source code now available to all on GitHub https://t.co/9ozsyZANwa— Elon Musk (@elonmusk) March 31, 2023

The real magic of Twitter is in our recommendations algorithm, which powers the hit Tweets you see in your For You timeline. We broke down how it all works here: https://t.co/2s5Hk57JPe— Twitter Engineering (@TwitterEng) March 31, 2023

Blog post TL;DR (thank you Cody)

• Twitter is releasing source code for parts of its platform, including its recommendations algorithm
• The source code is being released on GitHub in two repositories: main repo and ml repo
• The release aims for maximum transparency while excluding code that could compromise safety/privacy or enable bad actors
• Training data and model weights for the recommendations algorithm are not being released at this time
• This is Twitter's first step towards more transparency and they plan to release more code in the future that does not pose significant risks
• The community is invited to submit GitHub issues and pull requests to suggest improvements to the recommendations algorithm
• Twitter is working on tools to manage community suggestions and sync changes to internal repositories
• Security concerns or issues should be reported through Twitter's official bug bounty program on HackerOne
• Twitter hopes the global community can help identify issues and suggest improvements to lead to a better Twitter
• Twitter is doing this to increase transparency and build trust with users, customers, and the public

11:50am PDT: code pushed

The code is now live: https://sourcegraph.com/github.com/twitter/the-algorithm

We are digging in!

2:15am PDT: start the countdown

@sqs: A little under 10 hours to go until it's open source. We'll be back closer to 12pm PDT (unless Twitter unexpectedly releases it early, which might happen!). If you want to start exploring the rest of Twitter's open-source code in the meantime, here's a starting point.