Add management command to track Kudos purchases #4198

santteegt · 2019-04-13T05:11:43Z

Description

Gitcoin team members want to track kudos purchases by recording revenue in the database. So, a management command to track Kudos purchases is implemented to track ETH and token transaction from a specified account address, while a cron job is setup to run the command once an hour.

Checklist

Read and conforms to the contributor guidelines
Read and conforms to the style guidelines
Linter status: 100% pass
Added relevant tests
Tests pass and code coverage has not decreased
Changes don't break existing behavior
Commit message follows commit guidelines
Code documentation
Completed: Code review by core team and any requested changes

Affected core subsystem(s)

revenue

Refers/Fixes

Refs: #4122

Testing and Sign-off

Before testing new features please create an Etherscan API Key on https://etherscan.io/apis and set its value on ETHERSCAN_API_KEY env variable.

Contributor

Read and followed the Contributor Guidelines
Tested all changes locally
Verified existing functionality
Ran make test and everything passed!

Reviewer

Affirm contributor guidelines have been followed and requested changes made
CI tests and linting pass
No conflicts (migrations, files, etc)
Regression tested against staging or local deployment

Funder

Validated requested changes were made to specification
Bounty payout released to the contributor

codecov · 2019-04-13T05:16:56Z

Codecov Report

Merging #4198 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master   #4198   +/-   ##
======================================
  Coverage    30.3%   30.3%           
======================================
  Files         249     249           
  Lines       21470   21470           
  Branches     3115    3115           
======================================
  Hits         6506    6506           
  Misses      14677   14677           
  Partials      287     287

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c2558f7...5e919f3. Read the comment docs.

owocki · 2019-04-15T03:24:31Z

scripts/crontab

@@ -19,6 +19,7 @@ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/us
 ## KUDOS
 */15 3,10,17 * * * cd gitcoin/coin; bash scripts/run_management_command_if_not_already_running.bash sync_kudos mainnet opensea --catchup >> /var/log/gitcoin/sync_kudos_catchup.log 2>&1
 */10 4,11,18 * * * cd gitcoin/coin; bash scripts/run_management_command_if_not_already_running.bash sync_kudos mainnet opensea --start 1 >> /var/log/gitcoin/sync_kudos_all.log 2>&1
+0 */1 * * * cd gitcoin/coin; bash scripts/run_management_command_if_not_already_running.bash kudos_revenue mainnet --account-address 0xdb282cee382244e05dd226c8809d2405b76fbdc9


*1 == * right?

https://crontab.guru/#0_/1_**

app/revenue/management/commands/kudos_revenue.py

owocki · 2019-04-15T03:27:09Z

app/revenue/management/commands/kudos_revenue.py

+            logger.info(f"ETH Tx Records were found. Continuing from block {params['startblock']}")
+
+        params['action'] = 'txlist'
+        call_etherscan_api(network, params)


why do we need to call call_etherscan_api() twice? why not just once

Etherscan API uses two separate methods to retrieve ETH transactions (txlist) and ERC-20 token transactions (tokentx)

owocki · 2019-05-29T02:49:05Z

this LGTM. @danlipert look ok to you?

PixelantDesign · 2019-06-08T03:10:59Z

bumping this @danlipert ^^

app/revenue/management/commands/kudos_revenue.py

danlipert · 2019-06-12T09:15:07Z

@santteegt Can you add some tests for this? Thanks!

santteegt · 2019-06-14T06:06:54Z

Hi @danlipert,

I've just updated my PR to consider your review comments

danlipert · 2019-06-19T12:31:43Z

app/app/settings.py

+ETHERSCAN_API_KEY = env('ETHERSCAN_API_KEY', default='')
+
+# Kudos revenue account
+KUDOS_REVENUE_ACCOUNT_ADDRESS = env('KUDOS_REVENUE_ACCOUNT_ADDRESS', default='0xdb282cee382244e05dd226c8809d2405b76fbdc9')


@owocki is this the correct address (and is it the same on rinkeby, or is there even a rinkeby equivalent?)

no rinkeby equiv; this is the right addr yes

danlipert · 2019-06-19T12:34:27Z

@santteegt looking great - I'm wondering if you can test the management command directly and actually inspect the resulting DGP objects that are created after running to ensure they match whats on the blockchain. Also, how are the tests returning data properly even though no API key is set? 🤔

santteegt · 2019-09-09T16:01:19Z

Hi @danlipert,

I've finally added a test case that directly performs a call to the management command. Regarding the empty API key during tests, Etherscan has a limited quota to call their API for free without an API_KEY

danlipert · 2019-11-06T14:02:38Z

@owocki how is this looking to you?

owocki

@danlipert LGTM; well have to get an etherscan api key b4 deploy tho!

owocki · 2019-11-06T21:49:44Z

app/app/settings.py

+ETHERSCAN_API_KEY = env('ETHERSCAN_API_KEY', default='')
+
+# Kudos revenue account
+KUDOS_REVENUE_ACCOUNT_ADDRESS = env('KUDOS_REVENUE_ACCOUNT_ADDRESS', default='0xdb282cee382244e05dd226c8809d2405b76fbdc9')


no rinkeby equiv; this is the right addr yes

danlipert · 2019-11-12T13:32:04Z

API Key is in place for when this gets merged 👍

danlipert · 2019-11-27T13:58:26Z

@santteegt we are almost able to get this in, but the automated tests are failing because of the reliance on Etherscan. Can you mock those out so that our automated tests don't depend on Etherscan's availability or whether they block the free requests in the future? Thanks!

santteegt · 2019-11-27T15:09:55Z

@danlipert sure! I'll try to get this done ASAP

thelostone-mc · 2019-12-16T13:54:03Z

@santteegt any update on this . ?

Add management command to track Kudos purchases

santteegt · 2019-12-16T21:30:44Z

Hi @thelostone-mc,

Sorry for the late PR submission. I just updated it for you to review

gitcoinbot mentioned this pull request Apr 13, 2019

As a Gitcoin team member, I want kudos purchases tracked, so I can book that revenue in the database. #4122

Closed

owocki reviewed Apr 15, 2019

View reviewed changes

app/revenue/management/commands/kudos_revenue.py Outdated Show resolved Hide resolved

owocki reviewed Apr 15, 2019

View reviewed changes

app/revenue/management/commands/kudos_revenue.py Outdated Show resolved Hide resolved

owocki reviewed Apr 15, 2019

View reviewed changes

santteegt force-pushed the track-kudos-purchases branch from 9608d37 to 3ecfb9c Compare April 15, 2019 04:33

owocki assigned danlipert and thelostone-mc May 29, 2019

danlipert suggested changes Jun 12, 2019

View reviewed changes

app/revenue/management/commands/kudos_revenue.py Outdated Show resolved Hide resolved

santteegt force-pushed the track-kudos-purchases branch from 3ecfb9c to c06be7d Compare June 14, 2019 06:06

danlipert reviewed Jun 19, 2019

View reviewed changes

santteegt force-pushed the track-kudos-purchases branch from c06be7d to 3f08221 Compare September 9, 2019 15:57

santteegt requested review from octavioamu and thelostone-mc as code owners September 9, 2019 15:57

danlipert approved these changes Sep 9, 2019

View reviewed changes

octavioamu approved these changes Nov 6, 2019

View reviewed changes

owocki approved these changes Nov 6, 2019

View reviewed changes

thelostone-mc approved these changes Nov 27, 2019

View reviewed changes

Add management command to track Kudos purchases

c2558f7

Add management command to track Kudos purchases

santteegt force-pushed the track-kudos-purchases branch from 2c4cb73 to c2558f7 Compare December 16, 2019 21:28

Merge branch 'master' into track-kudos-purchases

5e919f3

thelostone-mc merged commit ac45c7c into gitcoinco:master Mar 4, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add management command to track Kudos purchases #4198

Add management command to track Kudos purchases #4198

santteegt commented Apr 13, 2019 •

edited

Loading

codecov bot commented Apr 13, 2019 •

edited

Loading

owocki Apr 15, 2019

santteegt Apr 15, 2019 •

edited

Loading

owocki Apr 15, 2019

santteegt Apr 15, 2019

owocki commented May 29, 2019

PixelantDesign commented Jun 8, 2019

danlipert commented Jun 12, 2019

santteegt commented Jun 14, 2019

danlipert Jun 19, 2019

owocki Nov 6, 2019

danlipert commented Jun 19, 2019

santteegt commented Sep 9, 2019

danlipert commented Nov 6, 2019

owocki left a comment

owocki Nov 6, 2019

danlipert commented Nov 12, 2019

danlipert commented Nov 27, 2019

santteegt commented Nov 27, 2019

thelostone-mc commented Dec 16, 2019

santteegt commented Dec 16, 2019

Add management command to track Kudos purchases #4198

Add management command to track Kudos purchases #4198

Conversation

santteegt commented Apr 13, 2019 • edited Loading

Description

Checklist

Affected core subsystem(s)

Refers/Fixes

Testing and Sign-off

Contributor

Reviewer

Funder

codecov bot commented Apr 13, 2019 • edited Loading

Codecov Report

owocki Apr 15, 2019

Choose a reason for hiding this comment

santteegt Apr 15, 2019 • edited Loading

Choose a reason for hiding this comment

owocki Apr 15, 2019

Choose a reason for hiding this comment

santteegt Apr 15, 2019

Choose a reason for hiding this comment

owocki commented May 29, 2019

PixelantDesign commented Jun 8, 2019

danlipert commented Jun 12, 2019

santteegt commented Jun 14, 2019

danlipert Jun 19, 2019

Choose a reason for hiding this comment

owocki Nov 6, 2019

Choose a reason for hiding this comment

danlipert commented Jun 19, 2019

santteegt commented Sep 9, 2019

danlipert commented Nov 6, 2019

owocki left a comment

Choose a reason for hiding this comment

owocki Nov 6, 2019

Choose a reason for hiding this comment

danlipert commented Nov 12, 2019

danlipert commented Nov 27, 2019

santteegt commented Nov 27, 2019

thelostone-mc commented Dec 16, 2019

santteegt commented Dec 16, 2019

santteegt commented Apr 13, 2019 •

edited

Loading

codecov bot commented Apr 13, 2019 •

edited

Loading

santteegt Apr 15, 2019 •

edited

Loading