Improved news selections

Time to improve the selection of news events. To recall: all events get a score, calculated by how many likes, shares, upvotes, downvotes, favorites, comments, retweets, links, etc it gathers over a certain period. This all depends on comments and discussions being linked correctly to the right events. Over the last weeks, many improvements have been made towards this end. A list of changes:

• Resolve external links to Reddit posts and comments
• Resolve external links to Tweets
• Make better selection of what Twitter discussions to fetch
• Fix badly formatted external links to Reddit and Twitter
• Merge events by removing many parameters from external links
• Better support for reddit crossposts
• Recreate discussion streams of merged events
• Allow admins to manually link events
• Improve selection of relevant events for Total Crypto Market
• Fix non-updated scores for discussionless Tweets
• Fix events with unresolved links being invisible
• Fix invisible posts from r/CryptoCurrency

Overall, the results are very satisfying: newsworthy events get pushed to the top a lot easier than spam, the algorithm does a better job for charts over larger timeframes and discussions held over multiple platforms will get correct scores. Also, scores have been updated for all existing data, so the quality of historical charts got a big boost as well.

There is a caveaty side effect though: the improved scores uncovered some disadvantages of our relevance algorithm: big events that are irrelevant to a certain coin or even crypto in general still get selected, pushing relevant stuff off the charts. You can be sure fixes are underway!