MIT scientists develop algorithm to predict Twitter trending topics

MIT researchers say they have developed an algorithm that can predict which topics will trend on Twitter an average of an hour and a half in advance.

The predictions have a 95 percent accuracy rate, they say, and sometimes can be made four or five hours in advance. Basically, it sounds like the algorithm learns to recognize the low-level data patterns that precede a topic making "the jump" into a conversational trend.

Pretty impressive stuff. Though, if you think further about it, a popular application of this technology would probably self-destruct.

If Doc Brown and Marty McFly taught us anything, it's that "no one should know too much about their destiny." Knowing what's going to happen in the future will cause us to take actions that end up altering that future.

But seriously, a scientific principle called the "observer effect" describes how the act of observation causes changes in the effect being observed. If this algorithm tells too many people what is currently on track to be trending in an hour, we would all start tweeting about it now. Those ripple effects will make it trend sooner, or in some other way change what actually happens at the end of that hour.

If you want to geek out over that and learn more about the algorithm, Associate Professor Devavrat Shah and his student, Stanislav Nikolov, will talk about it Friday, Nov. 9 at noon on MIT's campus.

  • Profile picture for user jsonderman

    Jeff Sonderman

    Jeff Sonderman is the deputy director of the American Press Institute, helping to lead its use of research, tools, events, and strategic insights to advance and sustain journalism.


Related News

Email IconGroup 3Facebook IconLinkedIn IconsearchGroupTwitter IconGroup 2YouTube Icon