Mooning memes

Thanks for all the feedback the last days! Now I know what to fix first ;)

I'm giving a small update of what has changed the last week, and I'm planning to to this periodically once in a while.

Something is wróng with the data.

Last week has been one crazy ride price-wise in crypto world and post-wise at reddit. Although in times of ATHs, memes tend to take over the popular messageboards, some of you shared my intuition that something was óff with the data presented at sprout. Very óff. Looking at reddit, some posts were very popular, but didn't show up here. Crazily enough, memes didn't seem to be affected. Sprout seemed to be the ultimate meme collector...

Digging into post scores

In sprout, a post has a score, which is the sum of its upvotes, and the upvotes of al its replies and shares, normalized over time. What does this mean?\ Well, a post (a tweet, a link, a reddit post or reply) has upvotes on its own. It is being shared, or it has replies, and those posts have upvotes on their own. Over time, the post and all the discussion it started, will gather a total amount of upvotes. By looking at this total, popular posts will score well, as well as posts with interesting discussions. After around two days, the post has become old news and people move on; the post has achieved its maximum total upvotes. This is what we use to normalize post scores: a recent post doesn't have that many upvotes or that much discussion, but we can expect what its total score will be by looking what it already collected during the time it was online. So in short, a posts score is the expected total amount of upvotes of the post, and all the discussion around it, after two days.

Bugs, bugs, bugs

In theory, this works well. However, in order to keep sprout fast not too slow, these scores can't be calculated on the fly: they have to be computed as new posts come in so we can query them. This is where stuff gets complicated: optimized code has already resulted in a decent spaghetti with a few bugs:

• Some scores for reddit posts linking to an external site were not
  updated after the first insertion. → fixed
• Because of an update of our database adapter a while ago, stronger
  typesetting introduced an error not calculating the score at all for
  some posts. → fixed
• In some cases, the upvotes of the post itself were not counted
  towards the total upvotes. → fixed

These bugs were initially not there, but introduced in more quiet times, when they didn't affect the total output that much. However, in times of heavy traffic, together they tend to favour posts having an external link, without much discussion. In other words, memes all day!

Timeframes

After fixing these bugs and recalculating all scores, the short-term (h and 4h) timelines for all asset pairs seem to have returned to sane levels. The daily and weekly timelines still suffer from too much spam, but this is due to other issues and to be fixed with more rigourous changes down the line.

Other changes

• Added per-user config options. If you want to see non-listed asset
  pairs, let me know. (No, not possible to add them yourself
  (yet...!))
• Added calendar and daily ledger for portfolio enabled accounts.
  Contact me if you're interested in this feature.
• Post score calculation done every 5 minutes instead of 1 hour.
• Fix truncated long tweets (>140 characters).
• Lots of small bug fixes and layout tweaks.

tl;dr

Some hard to discover bugs made sprout prefer memes all day long. With recent high activity, they became more obvious and were smashed with the anti-meme-hammer. Many more fixes and tweaks to make your experience more worthwhile.

That's it for now! Keep them feedback flowing!

Image by John Baker