TheMuffinMan you can find the explanation of the scoring here: https://docs.botto.com/details/voting-mechanism

tl;dr is that pieces are given a score out of 100 based on voting, 80% of the time higher scores are shown and 20% lower scores are. new fragments are given a default score of 100 until they are viewed 100 times to ensure they are seen and have a fair sample from which to be given a fair score.

What this unfortunately means is that our leaderboard isn’t actually a selection of the most desirable pieces but rather a representation of which pieces were lucky enough to be shown more frequently to Bottonians that spend more votes during the first portion of the voting cycles.

This ^ isn't quite right. Based on the scoring mechanism, new pieces are actually guaranteed to get viewed 100 times at the beginning of the round. But, it is true the early voters have significant influence on that critical early momentum, which actually conflicts with your reasoning that people don't have enough incentive to vote early in the round.

Culling formula
I disagree with removing fragments that haven't made it to the leaderboard as this goes against your own reasoning of the leaderboards not currently being representative.

@choobie's proposal to remove by VP I think makes more sense to me, but does need some mitigation so as to normalize across rounds.

Instead can retroactively remove bottom 75% based on votes cast just in the last round.
We could adapt this if the feeling is that the voting is already too skewed and we'd lose some favorable pieces (though this leaves 1600+ fragments, so feel like there's plenty). For instance, apply the 75% cut to round 10, then from there only have remove bottom 75% of fragments created that round up to present round.

Then going forward can remove bottom 350 of each individual round (equates to _bottom 20%)

Scoring formula
As for adapting the scoring to determine what gets shown, I think it's too early to say. The views are too divergent here: one rationale is that there's a lot of junk in the new set that get in the way of promising pieces from previous rounds, the other is that newer is presumably better. I tend to agree with there being a lot of random stuff in each new set (along with promising stuff) and it wouldn't be bad for them to be sorted more quickly so that other pieces have a chance at catching early momentum in the round. But there needs more discussion and doesn't need to be included in this proposal.

Generally agree that more voting is going to help everything, and could help draw attention to the influence of voting earlier in the round.

I want to dive straight in to where it seems like our opinions drastically diverge- "but this is not only because of the sheer quantity of fragments in the voting pool at any given stage. It also has to do with the quantity of active voters voting earlier on in the week."

Firstly, regarding the quantity of active voters - this is not my opinion, this is based on data from the admin panel. We take too long to sort through the fragments early on in the week because they take too long to score. They take too long to score (meaning they retain a temporary score of 100) because they never hit 100 unique views until it's leaderboard time. This has happened frequently over the course of the past few weeks. And this is why the phase 2 the team proposed is crucial in the short-term while we make voting more attractive earlier in the week, which is something all of us want to address! For one, I would suggest we change the speed in which a fragment gets its true 'Score'. This should either act as a precursor to any culling and/or be embedded into your proposal.

The Fragment Pool should have a heavy focus on newer pieces since they are created from the most advanced/highly trained version of Botto.

Secondly, I actually think this is the key divergence point between what the team wants to do and your proposal. The fragment pool already has a heavy focus on newer pieces - because of the scoring system and the 80/20 priority (80% of the time, top scoring fragments appear). This 'priority' in newer pieces has already been accommodated for through the implementation of scoring new pieces at 100 until they reach 100 views.

Based on your proposal, by preserving 3 weeks worth of the latest 350 fragments in the voting pool:

1. There will still be 350 new pieces per week that appear in the voting pool, just like before. This has no effect on focusing on the newer pieces. The voting pool will still force users to first comb through the latest 350 pieces to assign them a score, and we will still see a lot of these pieces appear in the leaderboard because they don't attain their true score in time (due to little voting activity).

2. Assuming the community successfully does comb through these 350 pieces and attribute them a score (say, because of an uptick of voters earlier in the week), then ii) 80% of the time, top performing fragments from previous rounds (the 240+15 that make it to the leaderboard each week) will be paired against one another over and over again, with the other 20% of the time lower scoring ones attempting to make a resurgence. This is not speculating user behavior - this is what will happen based on Botto's current system.

While this means we'll likely vote for fragments that might have just narrowly missed out on a mint (this is speculation/an educated guess of what will happen). But it also means that we'll likely be voting on the same or similar pairs over and over again at the tail end of the week, in perpetuity or until the period ends (assuming we make it past the 100/100 score of the latest round fragments).

I'm not sure that I fully understand your true score/100 unique views, however it seems like with a smaller fragment pool and motivation to vote earlier in the week, the true score will be determined faster.

Hopefully this is clarified above - the tl;dr here is that no, likely none of your proposal in its current form will directly affect how fast we achieve that true score value. We might see an uptick in how many users vote just to 'try out' the new implementation, but I surmise that this will fall off quite quickly after the first week or two.

3. This proposal disregards the majority of data we have available about each fragment across all weeks in favour of offering 'protection' to the latest 1050 fragments. The data I am referring to here are scores of each fragment as well as VP expenditure on each, which I strongly believe should be taken into consideration for any removal of fragment from the voting pool. It beckons the question: Why did we, as a community, vote for 20+ weeks if we don't use our votes to directly cull the tail-end of the fragment pool? And while yes, leaderboard data is based on round votes, it's been heavily skewed to newer fragments because of the [lack of] score attribution. We have the ability to cull based on existing voting data, i.e. based on Score or all time VP expenditure on a specific fragment.

4. This proposal does not take into consideration that a leaderboard did not exist in the first few weeks of the project.

A (small!) drawback here is also developer time, this is not as straightforward relative to other options at our disposal.

I suggest OP updates their proposal with the following considerations in mind:

• There is duality to this proposal - instead of discussing removing historically poor performing fragments, we are talking about introducing a protectionist mechanism for the 3 latest rounds worth of fragments.
• Existing data is available on every fragment, with these datapoints signalling historical voter preference
• The scoring of latest round fragments is crucial to understanding how this proposed system plays out, and whether any proposed system plays out all

Based on your current proposal, we could put it to a vote as I do see this proposal is committed to time protection. In case you're willing to reconsider elements of your proposal, I'm going to mention a few things on the latest implementation specifications:

The Initial Cull:
a. Remove all fragments from the pool that are more than 3 weeks old and have a True Score in the bottom 95%.

If we take score as the method for culling, preserving 3 of the latest rounds of fragments doesn't make sense - they've achieved their true score at the end of each week. All this does is bias culling to preserve recent rounds worth of fragments in the to-be fragment pool, rather than actually using the scoring system as intended.

Ongoing Cull:
a. At the end of each voting period, remove the 350 fragments with the lowest True Score that have been around for more than 3 weeks.

If we have a fragment pool of +-1350 fragments, we're in actuality committing to culling the oldest fragments. This is because we'd be 'time protecting' the latest 3 rounds worth of fragments (1050 pieces, or 3x350 - you state 'more than 3 weeks old'). Have I misunderstood something here? Clarification would be highly appreciated so I don't misunderstand what's being proposed. Irrespective of this, culling (be it initial/ongoing) by true score makes sense and is consistent with current system implementation.

In terms of culling 95% of fragments, we'd be looking at an even smaller pool than initially stipulated in your proposal - I believe your original proposal stated 80%. The current pool size (including the round that's just started) is 6808. If we deem the latest round (Round 21, 03 March 2021) of fragments untouchable, we'd be looking at (0.2*6458)+350 ~= 1642 fragments. Ultimately I'm not sure whether this difference is negligible but I'd err on the side of caution with culling too much! While I believe the 'sweet spot' for pool size is ultimately an arbitrary number, cutting it too aggressively might negatively impact voting experience in terms of frequency of pieces (we already experienced this in the earlier weeks of Botto, even after a bugfix). We should also be mindful that this isn't a simple case of voters cycling through fragments until they've seen every piece.

Combined with the 3 round time protection element, we'd also be retroactively culling many fragments that already have normalized true scores (because there's ample voting data on them) for the sake of protecting fragments that have lower, more volatile true scores. The current trend with more recent round fragments is that their (current) true scores will further normalize at lower values, as seen with older fragments from earlier rounds. A simple reason for this would be that some 'get tired' of seeing the old crowd favorites, so they vote on newer fragments. Between the lines, that means we'll be preserving newer fragments that are already 'deemed' ugly by the community.

Alternatively put, older fragments that we can dub 'crowd favorites' have normalized scores at lower values relative to newer fragments, meaning newer fragments with fresher true high scores are prioritized in being shown 80% of the time. This means older fragments have their 'backs against the wall', while newer fragments have initially higher true scores that are volatile and more susceptible to normalizing at lower values anyway (unless they are deemed incredible fragments and have a very high appreciation rate over an extended period of time).

Looking at 3 week/round time protection in isolation: This discredits the scoring system we've used for 20+ weeks where the clear noticeable issue is that we are taking too long to score the most recent fragments, not that the system doesn't work. The current system already has time protection baked into it: Newer fragments from each week are scored by default at 100, meaning their visibility is high prioritized at 80% throughout the earlier stages of the week. This in itself is time protection: we want to give newer fragments a 'fair' chance at being reviewed by the community in and through their votes. Even then, when true scores of the latest fragments are eventually attributed, they are likely to be higher relative to older fragments (see previous argument). This means that without the time protection you've proposed, the system will cull an old fragment anyway - unless there is clearly something worse from the newer batch of fragments to take its place.

I apologize if this post comes across as convoluted, I'm happy to try clarify what I've written (also wrote this pretty quickly so I hope it doesn't read as gibberish). My TL;DR here is we should not commit to other measures of time protection as the system does accommodate for it already.