BIP-29: Fourth Period

hudsonsims

quimp personally like the idea of having a default cadence that is assumed to continue rather than relying on a BIP to "renew" each period.

I would keep a 1-2 week buffer however, if for anything than to give the periods some breathing room. there's also time in there to set up/implement changes, wrap the period with a kind of public retrospective, build up the next period, and fit in a collab.

The main risk I see is that it removes a cadence of considering how Botto can improve. It's not ideal we cram so much into a BIP for each period, but I think it's more indicative of needing a deadline to make decisions on certain improvements. Given the team's size, it also helps to segment out maintenance vs. major updates for a period.

Having said that, it's well within the community's ability to be active on different proposals and initiatives throughout each period. I think this it would ultimately be a lot healthier to be working towards actual decisions when discussing improvements during the period. And there would still be the new periods functioning as soft deadlines to get a decision passed in time to implement.

Finally, I would consider shortening to an 8 week cadence if we make it automatic. I think 12 gets a bit long and the speed of generative AI development is rapid.

Ben

I think a period buffer is fine so long as minting cadence is maintained such that one new 1/1 is minted per week, no more, no less. I mentioned a potential protocol here for that here. Probably would make sense as a separate BIP that I could draft at some point.

Ben

What's the thinking behind maintaining SD1.5 while SD2 is introduced? Also, is it possible to introduce a newer version?

See here:

On the day after the publication of this article, Stable Diffusion 2.1 was released. 2.1 was intended to address many of the relative shortcomings of 2.0 compared to 1.5.

refri

Agree to @Quasimondo that too many issues addressed her. I am fine with drop retroactive rewards to make things easier to understand, also I see the problem with playing the game (which might become even easier if the rewards system is somewhat simpler) without giving much value to the project. This alone needs some discussion before, but if I understand it right this isn't a proposal how it should be modified but only about that it should be, so I am fine with it.

Regarding the comment Ben I agree that it's not quite clear why SD1.5 and SD2 are used both, and even more I would oppose against the rates (imo 44,5 + 25 makes SD too dominant, the sum should be definitely max. 50%). Another way could be to have different collections, whereby one is only to let SD1.5 and SD 2 compete against each other.

Regarding the negative vp I am also not too sure if it will improve much. At least when looking at my own voting behaviour it wont have much impact, cause I don't see how much vp a fragment has already received (and thereby wouldn't know it's necessary to down-vote it).

choobie

Cadence

quimp We need a better cadence.

We're breaking Botto's predictability by making periods into BIPs. The extra delay takes away from the "One 1/1 per week, forever" narrative, which is one of our greatest assets.

This is the biggest takeaway for me based on feedback in this post, and I wholeheartedly agree.

There is also a possibility that a Period BIP fails to pass, causing extra delays and hurting trust in Botto.
I agree. For this BIP, I will separate the economy aspect from the rest of BIP-29, and we can run a concurrent vote. This means that there will be a BIP-29 and BIP-30 to be voted on by end of the week.

I would suggest that 1) periods run automatically

I'd like to spawn a separate BIP once the fourth period has started outlining the de-facto standard that the DAO would operate on, after the Fourth Period has started. This would specify when theme curation would take place, length, models, etc (similar to this BIP), so that moving forward, theme curation, and the Fifth Period would occur uninterrupted and without another lengthy BIP. Whether this includes an interim period (of say, 1 week), I'm unsure - but to your point, this change should significantly improve cadence.

and sequentially without delay, and that 2) improvement BIPs are separated.

After fourth period, this should be the case (see points above).

New Models & Cutoff

Ben What's the thinking behind maintaining SD1.5 while SD2 is introduced? Also, is it possible to introduce a newer version?

Yes, 2.1 will be introduced. My fault - it seems I put the wrong one in the BIP. I've updated this to reflect the change. Apologies for any confusion

Ben If that’s the case I’m not sure giving VQGAN only 5.5% allocation makes sense. Let’s say there are even just two or three super promising VQGAN fragments. I wouldn’t want to lose those just because VQGAN didn’t outcompete on what seems like an unlevel playing field (get 10% of votes when only comprising 4.5% of the pool).

The formula applied tried to respect the current % distribution between SD 1.5 and VQGAN, while simultaneously accommodating two new models into Botto's Art Engine. In this scenario, the two newer models were allocated larger % distributions to ensure they were given a fairer chance in competing, see full reasoning in §3.2. While VQGAN has a much higher chance to lose in this scenario, we have seen VQGAN drop from 30.47% of outputs down to 11% in one period. Over the past 24 mints (Fragmentation & Paradox), a VQGAN artwork was minted once, only after the DAO rallied together to vote through A Life in the Suburbs back in Fragmentation. The past 12 mints did not feature a VQGAN artwork.

Finally, one unmentioned reason would be server costs. VQGAN runs independently on its own instance (as does each model, now). There will come a time where we need to consider cost of inclusion for each model. The 10% cutoff attempted to preemptively tackle this. The only recommendation I can make to accommodate your point and refri's point would be to hold off on the 10% cutoff. This would ensure we are not acting too hastily.

Holding off on the 10% cutoff would also allow for a more gentle introduction of Negative VP Expenditure, addressing these concerns:

Ben For example, a model may be unfairly disqualified because one of its fragments receives a large negative expenditure.

refri Regarding the negative vp I am also not too sure if it will improve much. At least when looking at my own voting behaviour it wont have much impact, cause I don't see how much vp a fragment has already received (and thereby wouldn't know it's necessary to down-vote it).

Retroactive Rewards

To address Retroactive Rewards concerns voiced by Encryptus (and perhaps others) on Discord:

But tbh it was quite surprising to find the elimination of retroactive rewards all of a sudden, packed in a BIP of wide scope, without grounded and compelling arguments beyond cost.

I think the ability to sybil retroactive rewards is the most important reason to get rid of retroactive rewards.

What is 100% clear is that we are being prompted to make a blind decision

We didn't even reach any sort of agreement on some guidance on what good voting behavior is (just remember something Choobie said about there not being anything as too much voting atm). The fact that there isn't any standard/guidance, and that dao members have ZERO insights on what's going on with voting, is the perfect breeding ground for missing opportunities on improving the project (and losing resources)

And then we would have some evidence (or hints) of grinding and many other useful variables. It would be a big plus for everyone, and attractive to new investors/members, who also care about what's going on in terms of activity, given that sales can be subsidized/faked/sustained...artificially for some time

I disagree with this - we have gone through the retroactive rewards allocation process once for Fragmentation, and we know what to expect. Data was also made available for the 2nd period of retroactive rewards. We need to do a better job at making data consistently available - but again, said data is ultimately not relevant insofar as retroactive rewards can be sybilled, which is IMO the main point. Here are the reward categories that were presented in Fragmentation, for reference.

I'm suggesting to invest on the first step required to make informed decisions: information

Agree - but what more information is required here to make an informed choice on removing retroactive rewards? As far as I understood from your comments on #governance-discussion, we are interested in seeing whether 'grinding' has been harmful and we should partially base our decision off of that. Whether there was grinding or not - I think the main issue here is that I can split all my $BOTTO and automate a 'grind' to qualify for as many possible retroactive scenarios as possible, while benefitting from active rewards as well. Again, I'm not contesting that its important we consistently have data readily available. I am just not sure whether having more data is what stops us from making an informed decision here, especially when retroactive rewards is sybillable by current design.

choobie We should be spending this time to either isolate preferred engagement variables and find meaningful ways to reward them, while at the same time keeping current incentive mechanisms straightforward and focusing on the idea of rewarding voters.

How DAO members vote is entirely up to them, we know that more voting data is better for the art engine. We also know that there is core alignment to voting in general: Botto mints something bad and we should expect less reward. We also know that voting behavior earlier in the round affects curation significantly as it has an effect on what others see in the voting pool (this has been communicated multiple times). The latter is a good starting point for isolating a preferred engagement variable. We have had up to +-30 weeks to come up with viable alternatives to retroactive rewards. I do not see this as a "surrender", rather a modification that aligns with existing DAO feedback and simplifies a reward structure.

If the problem is complexity only, I have another proposal: eliminate retroactive and send that 25% to treasury

I will include this as an alternative to the separated BIP-30 that proposes the economy changes. I see this is as a useful alternative for those with a more conservative stance.

TL;DR

Remove 10% Cutoff for the timebeing
Isolate Economy / Retroactive Rewards removal, create BIP-30
BIP-30 voted on concurrently
BIP-30 proposes an alternative allocation method to retroactive rewards (25% to treasury instead)
Upcoming BIP after fourth period starts, improving cadence of Botto

Ben

Am I understanding it correctly that if a model receives less than 10% of VP in a given round it is removed from the pool altogether?

If that’s the case I’m not sure giving VQGAN only 5.5% allocation makes sense. Let’s say there are even just two or three super promising VQGAN fragments. I wouldn’t want to lose those just because VQGAN didn’t outcompete on what seems like an unlevel playing field (get 10% of votes when only comprising 4.5% of the pool).

More generally, I wonder with the presence of four models if 10% cutoff is too demanding..maybe something more like 2-5%? Like one mega VP allocation toward a single fragment in one pool could put other pools in jeopardy. I wonder if a cutoff is even needed? Seems like it creates risk of losing quality fragments with no real upside.

Ben

I feel like introducing new models and a new vp function (negative expenditure) at the same time may be too much change at once for the art engine.

I’m concerned about unintended consequences and making it hard to track what’s working or not working.

Negative VP expenditure on top of having four models all of which must hit a 10% quota to survive may create risk. For example, a model may be unfairly disqualified because one of its fragments receives a large negative expenditure.

I’m not convinced that negative VP expenditure is constructive. It may be, and I do think it's worth experimenting with. I just think that experiment should happen at a time when other major changes are not being made to the art engine.

Ben

choobie this sounds better to me.

Do we feel strongly about still including SD1.5 given the addition of SD2.1? Are they sufficiently different or is their sufficient uncertainty whether 2.1 is better? Are the additional server costs the same in that case as VQGAN? And if SD1.5 and SD2.1 are included, do we think they are different enough such that they should collectively comprise ₇₀% of the pool starting off? Like would it make sense instead to do something like 5.5% VQGAN, 25% SD1.5, 25% 2.1, 44.5% Kandinsky?

With significant changes like negative VP expenditure seems it would be useful to have a staging site and "mock round" where power users could beta test for a week and share any feedback/issues before pushing live. Not sure if this is possible this time but for future updates.

juanje

choobie

To my understanding, one of the reasons to make retroactive rewards "retroactive" was to be able to analyze ex-post whether bad behavior took place, and adjust the rewarding scheme accordingly. Sybil attacks may happen, but under the current design they can be detected and rendered useless beforehand, which itself discourages sybil attacks. So while I get your worry, I miss the evidence in the sense that we don´t know whether sybil attacks happened in Fragmentation and Paradox (given the incentives above mentioned). If they didn´t in fact happen up to a material extent, what would lead us to conclude that (caeteris paribus) now they will?
*Note: can actually be done in a way that is profitable given the 2k requirements, gas fees, low sales, etc.?

I´m increasingly wondering whether we could effectively be crowding out people who are not willing/able to commit more than some hundred bucks into the project. It does not pay off for them in any way unless they fell in love with the idea.

Data is relevant for any decision, for this one too. For instance, it would be interesting to see how many small holders actively vote and how many whales consistently don´t. It would also be significantly easier to defend the project and grow conviction if we took clear steps towards showing that our decision-making is increasingly factual. Something in this regard (but related to financial transparency) was mentioned by Hashdad.eth and by Mazzi in two important separate discussions that are going kind of overlooked. What worries me the most is the image we project to an (in this case small) observer who might be considering joining.

Solutions for rewards? Not a simple thing. I ignore how far the team went about onboarding rewards tied to some sort of verification via Discord. Another way could be running cohorts of 10/20 of people per period and reward generously those who are active in voting and participating in other matters. Can´t be automated, will be burdensome, but would bring some engaged individuals...

That won´t solve the problem of vote dumping, though, which, as I said in the past, could also be discouraged in a negative way (i.e: negating active rewards to those who don´t show a minimum viable activity that would have tbd). An option like this has not resonated in the past, either.

Is the team atm building something related to making data available? Is there any way we could contribute in terms of ideation?

quimp

This is great @choobie. Before writing a BIP, can we bring these ideas to Discord for discussion?

As an example, I like to review raw ideas when they are 30% complete so we can adjust direction, and then at 90% so we can iron out details. To help decentralize Botto, most significant BIPs should start as a Discord forum conversation. I feel like too many BIPs are posted at 90% completion, and we might miss important details.

choobie

quimp Yep, I think the follow up to this BIP is actively discussing the idea of cadence on Discord. In general, yes, we should introduce contents of any BIP earlier in discussion threads.

Ben Perhaps weighing negative expenditure more heavily in training than in minting could help here.

I think this is an avenue to explore. So far, you have highlighted two potential concerns:

(Negative) VP Expenditure on one fragment may result in deprioritization of a model as a whole
(Negative) VP Expenditure may result in minted artworks that have less overall conviction

Out of these two concerns, I'd be wary of the first: negative VP expenditure conflates the idea of training and curation, as you have alluded to. I don't see this as an immediate issue as we've omitted the 10% cutoff for the time being. At the same time, our existing voting model has done the very same, just it hasn't accommodated for negative sentiment. I am uncertain whether we would need to address this in the future. For now, I think simply monitoring VP expenditure for the fourth period and understanding voter behavior should be our focus.

hudsonsims

Ben SD1.5 is indeed different from 2.1. The primary difference is that 2.0 was not trained on any NSFW images, which is believed to have significantly reduced the quality of the data set. There were also a number of filters added to 2.0, many of which were removed in 2.1. You'll find that the open source community has continued to prefer 1.5 over 2.0/.1, despite 2.1 being a larger model. I think this adds reason to have them compete with each other, reflecting the differences of opinions about the two models.

Staging site would be great though so that people can see for themselves.

quimp

Especially since down votes are a nice idea in theory, but in practice the social medias that tried them reverted the decision.

I can see why we may not want to train Botto on nudity and mint a sensitive fragment, but then why not if the community elects to?

(Another reason to split BIPs)

Ben

quimp I can see how it could be a problem in social media where the recipients of downvotes are humans rather than AI. That's less of an issue with Botto. My concern in our case is that I think we want the fragments to go to mint that are most loved (by some) even if they are also hated (by others). Not the fragments necessarily that are most liked. Perhaps weighing negative expenditure more heavily in training than in minting could help here. In any case, I do think negative expenditure is worth experimenting with.

hudsonsims

juanje To my understanding, one of the reasons to make retroactive rewards "retroactive" was to be able to analyze ex-post whether bad behavior took place, and adjust the rewarding scheme accordingly. Sybil attacks may happen, but under the current design they can be detected and rendered useless beforehand, which itself discourages sybil attacks. So while I get your worry, I miss the evidence in the sense that we don´t know whether sybil attacks happened in Fragmentation and Paradox (given the incentives above mentioned). If they didn´t in fact happen up to a material extent, what would lead us to conclude that (caeteris paribus) now they will?

Yes, the original thinking was that we could catch bad behaviour and not reward it, but it would still leave the problem of having that vulnerability sitting there for when the incentives to become ripe for it and those bad inputs would still have been injected by Botto.

juanje Is the team atm building something related to making data available?

Yes, it's actively in the works, though would look to @choobie to give a timeline. We're also looking at making bottoms up ideation for features and priorities much more robust as a big take away from this BIP.

juanje Another way could be running cohorts of 10/20 of people per period and reward generously those who are active in voting and participating in other matters. Can´t be automated, will be burdensome, but would bring some engaged individuals...

I think we'll should look into a number of ideas here. I tend to think most will necessarily be burdensome as qualitative assessments that can't be botted, but I do believe the willingness to engage there from some current members for the sake of attracting more engaged individuals is there.

juanje negating active rewards to those who don´t show a minimum viable activity that would have tbd

I find this very interesting. Kind of like how there's a minimum 2k $BOTTO stake, there could also be a minimum voting participation to get full rewards and wouldn't be vulnerable to sybil. Some more thinking to do there on how to set the thresholds so that it's reasonable and passable by BIP, but my initial thought is it's very promising. Would welcome more thoughts there, whether for the BIP-30 on the economy or after!