MIT scientists develop algorithm to predict Twitter trending topics

MIT
MIT researchers say they have developed an algorithm that can predict which topics will trend on Twitter an average of an hour and a half in advance.

The predictions have a 95 percent accuracy rate, they say, and sometimes can be made four or five hours in advance. Basically, it sounds like the algorithm learns to recognize the low-level data patterns that precede a topic making “the jump” into a conversational trend.

Pretty impressive stuff. Though, if you think further about it, a popular application of this technology would probably self-destruct.

If Doc Brown and Marty McFly taught us anything, it’s that “no one should know too much about their destiny.” Knowing what’s going to happen in the future will cause us to take actions that end up altering that future.

But seriously, a scientific principle called the “observer effect” describes how the act of observation causes changes in the effect being observed. If this algorithm tells too many people what is currently on track to be trending in an hour, we would all start tweeting about it now. Those ripple effects will make it trend sooner, or in some other way change what actually happens at the end of that hour.

If you want to geek out over that and learn more about the algorithm, Associate Professor Devavrat Shah and his student, Stanislav Nikolov, will talk about it Friday, Nov. 9 at noon on MIT’s campus.

We have made it easy to comment on posts, however we require civility and encourage full names to that end (first initial, last name is OK). Please read our guidelines here before commenting.