Last week Klout released a major update to our score model and a preview of our new moments feature that showcases your most influential social media activity. Internally the codename for this project was “Maxwell”, for the Scottish physicist James Clerk Maxwell, whose discoveries helped pave the way for modern physics.
The Maxwell project brought together a team of engineers, designers and product managers across Klout in a months-long effort to evolve the Klout Score and rethink how people can benefit from the insights Klout enables. We’ve gathered stories from a handful of the members of the Maxwell project for you here.
Big Data, Big Challenges
Andras Benke: I’m the technical leader of the Maxwell data and science team. We built up a big data pipeline based on a brand new architecture and completely replaced our scoring system.
Adithya Rao: I was responsible for building the scoring models for each network individually, and combining them together into the final Klout score. This also included extracting information from all the networks, as well as real world sources and using them meaningfully to count towards the score. Continuously tuning the models to be accurate, while meeting users’ expectations, was probably the most challenging problem I faced during the project.
Nemanja Spasojevic: I worked on the backend data pipeline. We kept up with increasing demands for data quality as the project progressed, and with a high pace of iterations given the size of data. Processing and re-processing big data to make sure we kept up the development pace required, while still maintaining the legacy pipeline, required a lot of nerves from the team.
Girish Lingappa: During development and testing, we had to dig through millions of users’ scores and billions of messages to debug issues. Sometimes it felt like looking for a needle in a haystack.
David Ross: I am responsible for the data serving infrastructure for the Score and moments. including designing fast, scalable, and reliable API endpoints. Moments is the most challenging serving problem Klout has ever tackled. If you think about the amount of data that needs to get served to millions of users, moments is quite the scalability challenge. After many whiteboard sessions and iterations, we are making it happen.
Jerome Banks: I built out most of the data pipeline. Technically we had to deal with large datasets while still remaining agile.
Designing the Vision
Matt Sperling: The biggest challenge from a design standpoint was shifting the focus from the score to the story behind the score. Our solution was to show people the social interactions impacting their score on the dashboard, while giving them a showcase of their best moments on their public facing profile. Hence the design of interaction and moments. Alongside this we did a pretty substantial redesign of the site.
Mark Azevedo: I was responsible for building the original moments user interface prototypes. Now that we are no longer in the prototype phase, we will continue iterating on moments as a platform for revealing influence in your everyday social interactions. The Klout.com team who worked on the site redesign made it possible to support moments as a product.
Maxwell in the Real World
Adithya Rao: I am proud to be part of a team which was able to crunch such huge amounts of data and create a product that is hopefully going to redefine how people consume social data on the internet.
Nemanja Spasojevic: In the end, its the users that really matter. Being able to deliver on such an ambitious goal is the dream of any engineer.
Matt Sperling: I’m thrilled that we are giving Klout users a profile they can be proud of. It’s something they can show off no matter what their score.
Andras Benke: We are processing an amazing amount of data every day. Creating a stable system which is able to do this is pretty challenging even with today’s big data technologies.
What’s Next
The launch of the updated Klout score and moments is just one step on the road to helping everyone discover their influence. We’re excited about this release but we’ve already begun work on more great new features to come.
In the past few months Klout has undergone some serious changes, not least of which is a complete site overhaul and a huge brand transformation. This is not a small feat, given the timeframe. Time Magazine has even selected us as one of their top 50 websites! It’s really a testament to the hard work of everyone here at Klout: the engineers, product managers, designers, and the executive team for steering the ship. So when I was asked to sit down and really think about what the Klout brand is and what it should look like, the core of that concept was pretty obvious: The Klout Score.
