Pricing Engine

[owocki]

Right now, as a hypothetical user, I'm just making up values for the tips / bounties I give out, which is fine for when this project is a novelty, but doesn't scale as the project 'grows up'. What factors should go into deciding how much to price a bounty?

Some thoughts from my side:

For a feature or bug bounty:

• Value of the submission to the repo, it's users, the ecosystem.
• Skills involved, and how niche they are.
• Average hourly rate of a person with above skills.
• Competition: Do market forces like supply/demand play a role?
• For security bounty, how bad is the exploit? how big is the attack surface area?
• Liquidity and incentives matter. The value of the submission will differ depending upon whether the value is in a token that represents ownership in part of a illiquid network or a liquid platform with scale (like ETH)

For tips:

• How much you'd like to build a relationship / show appreciation for, the counter party.

UI

I think it'd be great to tell the user, right in the bounty submission page, what the odds are that their task will be turned around. Kind of like ether gas station but for bounties. Here's a quick mockup:

[owocki]

@mbeylin what do you think?

@bransbury @gamwe6 @pipermerriam @gasolin @mateodelnorte @anglinb you have each received tips or bounties on the system; i'd love to hear your feedback if you have 2 minutes to type something quick up. thanks 🙏

[bransbury]

I think it would be cool to have two options (maybe as two tabs):

1. Enter amount (functionality as is) - provides ultimate flexibility
2. Calculate/recommend an amount - this could involve selecting a value (1->5 or 1->10 scale) for some of the above (i.e. impact of change, skill level required to deliver, effort/time to complete) and a calculated amount could be returned. These values could also then be used as filters for the bounties. "Experience level" and "project length" already effectively map to "skill level" and "effort/time" above.

[mateodelnorte]

I'll go by different steps of the process:

Received tip:

On mobile, the giant gitcoin banner makes all the other content illegible. You can probably just put a small logo image up top. Also the colors of the logo have a lot of dissonance with the rest of the design. Dark blue and purple backing vs crisp text on white. Consider just the text logo.

Justify text. Put more space between paragraphs. That should clean things up a bit and look more profesh.

[mateodelnorte]

Accepting tip:

Super easy. UI looks simple and well done, with one exception: I wasn't able to snag a screenshot of it in time, but there is a spinner above some text saying "Loading..." while the txn is waiting to be verified. When it's done it says: "SUCCESS 🚀!" and "Status: Processed". I don't think you're actually "Loading", here. I think you're waiting for the transaction to process. There's probably better verbiage to use.

Also overlay1.png didn't load, but high five for the sneaky developer text.

[mateodelnorte]

Tip value: 0.08 ETH (~$25) seems like a pretty generous tip for the size of my project. Happy to have it. ;)

[anglinb]

As someone submitting a bounty it would be awesome to have some feel for what to charge. I think it could be cool to leverage the already open nature of the work being done. For example, if you went through the steps, maybe we could show the submitter some fulfilled bounties at around the same price with similar tags.

[mbeylin]

My idea for a bounty pricing engine consisted of:
Presenting users with a set of various sliders which will have an effect on the price, those sliders being:

• deadline: I need it done in X days
• difficulty/ skill level required
• effort/time required

[mateodelnorte]

My recommendation would be to base the feature off the simplest possible trustworthy algorithm first. Then, build internal analytics for all sorts of data points in order to surface what a better algorithm will be in the long-run. Your data points are going to be so low here, that it may have wild, un-trustworthy swings. You're better collecting for internal use so you can have quicker iterations of your product.

…

On Mon, Oct 2, 2017 at 11:49 AM, Mark Beylin ***@***.***> wrote: My idea for a bounty pricing engine consisted of: Presenting users with a set of various sliders which will have an effect on the price, those sliders being: - deadline: I need it done in X days - difficulty/ skill level required - effort/time required — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#21 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAh3X3nA-ZtUbnrCn-4rxdEJ0eEaXWaDks5soQYLgaJpZM4PqfmS> .

[jwaggs]

I agree, determining how to get the bounty system to scale is going to be the biggest hurdle for gitcoin in my opinion. I agree with your comments on what things should be considered to determine the value of the bounty, however, actually grabbing the metrics to be used is going to be a difficult process. Figuring out how to quantify the difficulty isn't going to be easy without human input. And it's the human input that needs to be gotten rid of for it to truly scale. My two cents is this though, ignoring how you come to an initial bounty price, it should be true that in a large enough community, bounties should get swiped up the moment the payout is more than worth the actual time invested. Therefor, a variable system that increases payout over time for bounties based upon them not being completed, may be worth considering.

For example: let's say you start all bounties at .1 gitcoins (for the sake of argument). Something that takes no time for someone to complete may get finished quickly, but as the bounty sits there, if it gains bounty payout value over time, eventually it's going to cross the threshold in which a person determines its worth their time to complete. Therefor, rather than using traditional human logic to determine difficulty and henceforth value, maybe using a dynamic time based scale can help give more clear insight into the actual worth of a given bounty.

To expand on this ever so slightly: here is a summary of my thoughts. Things like usefulness or difficulty level are things that are useful for you to use when a person sits down and determines the value of a bounty, but a nightmare to quantify without human input. Instead, use a time based system and possibly even a bidding system to redefine how you determine the value of a bounty. Maybe every bounty has a bidding period where people input what they would be willing to accept to complete the task. And then after a period of time passes, if the bounty isn't fulfilled, another round of bidding occurs with an offset multiplier of sorts that is applied based upon how many bidding rounds the bounty has survived without being completed.

[mbeylin]

@jwaggs I agree strongly with your vision. Based on the current implementation of the standard bounties, we allow for the payout amount to be increased even when the bounty is Active, so long as it has the sufficient balance to do so.

I believe a combo of pre-bidding on draft-stage bounties, with this slow increase in bounty payout amounts will be very fruitful, however only if the time required to administer it isn't too high.

[gasolin]

similar to @mbeylin 's idea, we can put BOUNTY DETAILS in main submit form (instead of hiding it in the advance field). then show the suggested minimum price based on different combination of experience level/project length/Bounty Type.

The price could be adjusted manually or automatically by analyze. And warning(this price is lower than average, it might take longer to get issue solved) could be shown at the time Bounty submitter adjust bounty to lower than average price.

A slider seems more intuitive than dropdown menu(but should double check when mobile UI is in concern).

As a friendly(open) ecosystem like github, current Bounty + Tip could fulfill most cases. People can track bounty submitter's previous record to see if the bounty submitter does appreciate the contribution and decide if they want to take the bounty.

[gamwe6]

Trial and error or A/B testing. But that's not the simplest solution.
As few data as now, it will be difficult to automate this completely without human intervention.
I would split "Value of the submission" into two sub factors. "The comments" and "the reactions" (👍 👎 😄 🎉 😕 ❤️).

[owocki]

an idea for first iteration of the engine => #26

[gamwe6]

To add to what @bransbury said, maybe not two tabs but suggest a price as Airbnb do.

[jwaggs]

Instead of going with the current status quo of crypto coin generation, lets develop a pricing engine that can reliably enforce a predetermined growth rate for the coin while creating acceptable bounties. This would turn GitCoin’s bounties for open source work into what is effectively its own flavor of mining for the coin.

This system would need to be rolled out in phases as the community grows obviously. Right now we are at a place where we can focus on better determining an appropriate price for a bounty based on input from the submitter like what is mentioned in issue #26 - for now lets build a simple engine that accepts the metrics we have discussed and returns a bounty value. That should be simple enough. But I think we should keep in the back of our minds that with a large enough community, we will need an engine that is resistant to abuse from people who would manipulate the bounty system for personal gain by not fairly assessing the value of the bounty they submit.

A lot of the actual math of converting the metrics we care about into a GitCoin bounty value is also going to be heavily dependent upon the rate at which we want to be throwing new GitCoins out into the world, which I don’t remember seeing in the white paper. What has been determined about the creation rate and cap for the coin? @owocki?

Also - if we are building the pricing engine, we may as well discuss sooner rather than later how we want to bounty out our own work on this engine.

[owocki]

for now lets build a simple engine that accepts the metrics we have discussed and returns a bounty value.

great idea! thats sort of what #26 is meant to do.

But I think we should keep in the back of our minds that with a large enough community, we will need an engine that is resistant to abuse from people who would manipulate the bounty system for personal gain by not fairly assessing the value of the bounty they submit.

My thought is that the pricing engine would simply suggest the lowest price for which a bounty would actually get turned around. Market forces will dictate if the bounty is actually turned around.

I think the system is fairly resistent to abuse in that if someone submits a bounty that's underpriced, market forces dictate that it won't be turned around. But I'm happy to hear from others who can point out specific attack vectors.

A lot of the actual math of converting the metrics we care about into a GitCoin bounty value is also going to be heavily dependent upon the rate at which we want to be throwing new GitCoins out into the world, which I don't remember seeing in the white paper. What has been determined about the creation rate and cap for the coin? @owocki?

I think it's important to note that the project has not committed to a token distribution event or token distribution model, and what is in the alpha whitepaper is a tentative plan, subject to the project gaining traction and proving out that there is fundamental value in pushing open source repos forward. Given all of the consternation about the legal aspects of doing a token distribution event, I am intentionally going against the grain and trying to be deliberate and thoughtful about not doing a token distribution event (at least, not until the product is proven). I'd feel much more comfortable if we focus on providing value to the community via ETH funded issues and ERC20 token funded issues (which is a huge market in itself).

Your idea to " turn GitCoin's bounties for open source work into what is effectively its own flavor of mining for the coin" is a good one. Its an idea that's being kicked around, but the creation of a model for the coin needs to be subject to proving out the value proposition of the rest of the system.

Also - if we are building the pricing engine, we may as well discuss sooner rather than later how we want to bounty out our own work on this engine.

For now, I'm happy to just put up ETH bounties in order to grease the wheels on a distributed build of the platform. Take a look at the issue explorer and the open issues on this repo, and let me know if there's any that interest you.

[jwaggs]

My thought is that the pricing engine would simply suggest the lowest price for which a bounty would actually get turned around. Market forces will dictate if the bounty is actually turned around.

I completely agree that market forces will dictate the turn around rate. But rather than letting a bounty die if its set too low, lets slowly increase its value over time until it passes that threshold to get it turned around.

I think the system is fairly resistent to abuse in that if someone submits a bounty that's underpriced, market forces dictate that it won't be turned around. But I'm happy to hear from others who can point out specific attack vectors.

I'm not worried about underpriced bounties, but rather over-priced bounties. When this starts to grow to the point of it being difficult to closely monitor bounties, people could easily be setting up overpriced bounties for themselves. This is preventable... I simply was suggesting that we should put things in place to prevent/minimize this. But that aspect only applies to the mining of GitCoins and does not apply to ETH and other bounties.

I think it's important to note that the project has not committed to a token distribution event or token distribution model, and what is in the alpha whitepaper is a tentative plan, subject to the project gaining traction and proving out that there is fundamental value in pushing open source repos forward. Given all of the consternation about the legal aspects of doing a token distribution event, I am intentionally going against the grain and trying to be deliberate and thoughtful about not doing a token distribution event (at least, not until the product is proven). I'd feel much more comfortable if we focus on providing value to the community via ETH funded issues and ERC20 token funded issues (which is a huge market in itself).

I completely agree with all of that - I'm even excited to see that you are diligent enough to take this approach because we all know ICOs can get hairy legally speaking. Plus, I'm not looking for a pump and dump scheme myself. I'm genuinely interested in the underlying innovation.

(at least, not until the product is proven).

Starting to look at how to implement 'open source mining' for GitCoin is how I feel like you could prove its value long before you ever went to market with it (whether or not you actually do). I agree with you that in short term speak, ironing out the payment system with other coins is the right call. But I also think that since GitCoin is technically a coin itself, it might not be a bad idea to actually get it circulating a bit more among developers. This doesn't have to be dependent upon going to market. But I've also not been a part of GitCoin Core's discussions, so I'm coming at that from an outside perspective with a fresh take on things.

Take a look at the issue explorer and the open issues on this repo, and let me know if there's any that interest you.

I will continue to do so and will surely pursue some things in the repo. My comment about working out bounties on this engine was probably a little misplaced since we haven't decided to go the route I am speaking of. I was more so speaking of bounty-ing out developing the core algorithm for the mining of GitCoin, which is a much larger task than what we have agreed to even pursue at this point... so that comment of mine can take a back seat for a while.

[gasolin]

FYR I saw kickico do an interesting way to mine their coin: After ico, they only mine new coin when new ICO is founded on their site. When people participate, the issuer and the participents will get certain % of KICKCOIN back to their wallet.

I'd like to get some gitcoin as a bonus when I complete a bounty or tip/issue a bounty.

[owocki]

two sided auction where people can bid / ask on issues.

[jwaggs]

@gasolin I like the perspective that brings to the table. Not only focusing the mining on getting people in on the open source work - but also using it to incentivize the adoption and use of the coin itself makes a lot of sense to me.

And I agree a bid / ask solution makes a lot of sense here @owocki

[merryt]

Bringing up an old topic. Does the pricing engine allow two bounty posters to one issue?

Use case:
I really think x is important but the bounty for it is so low that it will never get finished/worked on.

[mbeylin]

@merryt Gitcoin is currently working towards integration with StandardBounties, which already has the functionality for 3rd party contributors to add funds to the bounty

[owocki]

I think though the way it currently is would only attract maybe new devs looking to learn, or people that want to contrib to open source, and maybe make some extra money. The bounties just aren’t high enough I think in most cases. It makes sense for like easy tasks (design something from scratch, change some CSS, etc). But like in the case of the stuff I’m doing for ethereum/web3.py, you need to invest a fair amount of time learning the whole library, API, sorting through existing bugs, before even starting work on the bounty. (edited)

feedback above on higher priced / higher risk tasks

[tra38]

What I think could work is a two-tiered system:

1. Short-term gigs, like what we currently have
2. Developer sponsorships, like this proposal for Bountysource...for more involved projects like etherum/web3.py. Instead of paying per gig, you're paying per month.

There are benefits for supporting both tiers. Sometimes, you may just have a minor issue and don't want the overhead of paying someone for a month to work on your open source project. Other times, you have a very involved project and are willing to pay the overhead so that you incentivize the right candidates. Let the repo owner choose what approach would be right for them.

[owocki]

just put together some WIP code for this http://bits.owocki.com/0U2C0M2h0G3j/Screen%20Recording%202018-04-25%20at%2010.52%20AM.mov