Granular Pricing Engine: Store estimated hours to complete work in the funder form in the Gitcoin data model

[frankchen07]

User Story

As a Gitcoin admin, I'd like data on the estimated hours to complete work in the funder form updated on the Gitcoin data model.

Why Is this Needed

Currently, funders can price the issue based on the total price, but that doesn't give any other information on whether or not that hourly rate is sufficient for their issue to get completed. Adding an input field for hours estimation allows for an hourly rate calculation, which translates to a predicted bounty success rate. This data will be populated in our data model so we can calculate hourly rates in the future, and hopefully improve our pricing mechanism via analytics.

Current Behavior

#2158 has been pushed. Estimated hours of Work is present on the funder form, but the data model has not yet been updated to capture hours that have been inputted.

Definition of Done

1. Add estimated hours of work to the data model.
2. Ensure that the data is persisted in the data model to Postgres.

Future Discussions (NOT IN SCOPE)

Previous strategic discussions here.

Eventually, we want to move away from modular pricing to more market-based pricing, something like a variable system that increases payout over time for bounties (e.g a "count-up" timer) that increases the incentive at a steady rate until the market finds the incentive attractive or it hits an upper cap limit. There would be additional scoping, discussion, and discovery of attack vectors to see this through.

1. What happens to the incentive if someone repeatedly starts work and then stops to stall the issue?

2. A user can also start and stop work using multiple user names to stall the issue until the incentive increases.

3. Bounty hunters might start the work prematurely but wait longer to actually hit "start work" in order to collect a higher payout. Alternatively, they can also start work, actually do the work, pretend like they can’t finish, and have a friend or another alias submit the work at a later time for greater incentive.

Appendix

Based on Gitcoin data, an analysis was done that yielded a relationship:

(0.002 x Hourly Rate) + 0.65 = Predicted Success Rate

This gives us the ability to estimate success percentages by the hourly rate, at the granularity of a dollar (for every $ increase, we have a 0.2% percentage point increase in success rate).

The success rate curve should be made more granular but would suggest that what pops up is a change in success rate given different input values of issue payout and hours estimation for the task, not a change in hourly rates at defined points.

Of course, we need data to back this up. I converged on three main metrics to look at:

1. Success rate (for bounties priced at this range, what is the success rate? Higher is better).

2. Median time to start bounty (for bounties priced at this range, what is the time elapsed before the bounty is picked up? Shorter is better).

3. Median time to complete bounty (for bounties priced at this range, what is the time it takes to finish the bounty? Shorter is better).

Explorations on 2 and 3 yielded no significant correlations, even after segmentation by experience level, bounty types, and deadlines. I'm not comfortable here making any suggestions on pricing related to these two metrics.

Success rate yielded a weak positive correlation with regression, which gives us a more granular scale on how success rates change with increasing or decreasing hourly rates:

(0.002 x Hourly Rate) + 0.65 = Success Rate

For success rates, when categorizing them into buckets, our data becomes increasingly sparse, so finding a smoothed out curve in the upper echelons has not yielded super promising results. There fore, near the higher hourly rates, this analysis breaks down due to small sample size and high variance. The relationship was established post-eliminating low sample size and extreme outlier data, so caveat emptor.

It does, however, align with @owocki's Medium analysis. Starting success rate for bounties ~ 65%, with every increase of a $/hour, we see a 0.002 percentage point increase in success rate. If we calculate success rates at the pre-defined points:

$20 ~ 70%
$40 ~ 75%
$80 ~ 81%
$120 ~ 90%

and the ability for success percentages in between, at the granularity of a dollar.

[owocki]

exciting stuff..

i wonder if there is an 80/20 way to implement this that doesnt have to do a ton of complicated regression-inline... @mbeacom might have some thoughts here.

cc @PixelantDesign for the UI design (for example, i'm not sure "num hours" should go inline with price on the form)

[PixelantDesign]

Met with @frankchen07 today to sync on next steps!

[mbeacom]

Introducing a new endpoint to consume estimated hours and denomination/amount performing the calculation should be enough.

If we're storing the estimated hourly rate on the Bounty model (DoD #2 - Valid inputs from USD and estimated hours are stored in the database once the issue is funded.), is this something we intend to pin at the current ethereum market value or should we include a background job to update these calculations when currency market prices have been updated? Another option is to display the hourly estimation in bounty details dynamically and cache it unless ConversionRate changes, but this might prove heavy regardless.

[frankchen07]

Rethinking some comments here - some updated definition of done comments:

Introducing a new endpoint to consume estimated hours and denomination/amount performing the calculation should be enough.

👍

If we're storing the estimated hourly rate on the Bounty model (DoD #2 - Valid inputs from USD and estimated hours are stored in the database once the issue is funded.), is this something we intend to pin at the current ethereum market value or should we include a background job to update these calculations when currency market prices have been updated? Another option is to display the hourly estimation in bounty details dynamically and cache it unless ConversionRate changes, but this might prove heavy regardless.

USD amounts are already stored when an issue is funded. To keep it simple, we additionally store the estimated hours to complete work on the funder form, and leave it like that. We don't have to worry about dynamically updating the hourly rate since that information won't be surfaced to the funder again. Basically, we store estimated hours to complete work in the model, and not have to worry about dynamically changing anything. With the estimated hours, we have enough data on the backend to calculate hourly rates.

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

This issue now has a funding of 0.4 ETH (45.26 USD @ $113.15/ETH) attached to it as part of the Gitcoin fund.

• If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $35,295.81 more funded OSS Work available on the Gitcoin Issue Explorer

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work has been started.

These users each claimed they can complete the work by 3 years ago.
Please review their action plans below:

1) kn has started work.

1. Add a "estimated_hours" field in Bounty model.
2. Add a migration.
3. Add a test case.

Learn more on the Gitcoin Issue Details page.

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work for 0.4 ETH (43.85 USD @ $109.63/ETH) has been submitted by:

1. @kn

@frankchen07 please take a look at the submitted work:

• PR by @kn

---

• Learn more on the Gitcoin Issue Details page
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $35,032.66 more funded OSS Work available on the Gitcoin Issue Explorer

[kn]

FYI: The change has been merged into the master branch #2956. Please let me know if there's anything else I can do for the issue. Thanks!

[owocki]

@frankchen07 ^^

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

The funding of 0.4 ETH (59.31 USD @ $148.26/ETH) attached to this issue has been approved & issued to @kn.

• Learn more on the Gitcoin Issue Details page
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $46,008.58 more funded OSS Work available on the Gitcoin Issue Explorer