Fix data model error in 3Box integration #5955
Conversation
Merge latest change from gitcoinco/web
think-in-universe
requested review from
danlipert,
octavioamu and
thelostone-mc
as code owners
Codecov Report
@@ Coverage Diff @@
## master #5955 +/- ##
==========================================
- Coverage 30.22% 28.94% -1.29%
==========================================
Files 248 271 +23
Lines 21186 23678 +2492
Branches 3068 3441 +373
==========================================
+ Hits 6403 6853 +450
- Misses 14507 16552 +2045
+ Partials 276 273 -3
Continue to review full report at Codecov.
|
|
nice, worked on my local |
|
Does this still require testing? If so, I can do it. |
|
@octavioamu cool. @loganbek thanks~ I'd prefer we do more local testing if possible, especially to confirm the I'm still trying to create some bounties in my local env... |
|
I'll do some local testing and get back to you. |
|
@loganbek awesome. thanks may need to change Lines 2432 to 2436 in 460c466 to below if it doesn't work web/app/dashboard/views.py active_work = Bounty.objects.none()
interests = profile.active_bounties
for interest in interests:
active_work = active_work | Bounty.objects.filter(interested=interest, current_bounty=True) |
|
@think-in-universe heads up, I'm hitting pip BUIDL issue will retry local test tom. |
|
@loganbek sure. no problem. not sure why pip build fails. I re-setup the environment yesterday and the pip build completed in a few minutes. |
|
@loganbek are you using docker? |
|
@think-in-universe not sure how the data should look but here is my data, check please if that is right https://share.vidyard.com/watch/hCn7h5myiyNupHvYSGSPK8? |
|
@octavioamu cool. thanks! I have watched the recorded video and I think most of the data are correct. But I noticed that I have been trying locally for creating bounties but were blocked at connecting to IPFS for quite some time, so haven't been able to proceed yet.... |
|
@octavioamu maybe the command in the doc is helpful for creating bounties locally but it failed for me again for IPFS connection issues: https://docs.gitcoin.co/mk_setup/#optional-import-bounty-data-from-web3-to-your-database |
|
@octavioamu I'm able to create bounties locally now after reconfigure IPFS daemon locally, but not able to complete bounties due to some unknown contract error ... The |
|
Seems is not working again now is flagged on production as alpha so you will able to see if you are an alpha tester https://gitcoin.co/settings/account?cb=alphatester |
|
from our internal debuggin thread the issue is the size of the data i think |
|
My guesses, is the data on production bigger and the serializer is failing. Or as it is bigger is taking to long to serialize |
|
we received the 500 error when exporting data: so it's probably still a serialization / data exporting error in production. could anyone please send me the error message on server and I'll take a look first? so far I don't think it's related to size of data at least in my test. If we want to move that into a celery job, we may need a python binding client library or call that 3box sync process with 3box.js via node.js on server, which could be done in next step. I discussed with 3box team on this earlier. |
|
and it might be faster to test, and accelerate the feedback loop, if we could efficiently mock or import the profile data in local dev env. I think we still didn't test all cases locally without high quality test data. |
|
please find me on Discord with robertyan#8312 if more instant communication is needed to resolve this issue faster. and if anyone could please send me the error message on Django server, we should be able to fix the issue in production sooner. |
|
oh sorry, I didn't check the screenshot https://bits.owocki.com/E0uq5rqZ from @owocki. According to the screenshot, it would probably be data size issue for these active users when exporting data from database. We could either move the 3box backup process into a celery job, or split the call to And in the meanwhile, there're other serialization issues in my profile that we may need to fix. |
|
here are the sentry logs i see for the 500 in prod
https://bits.owocki.com/7KuRBgA6
…On Tue, Feb 11, 2020 at 9:12 PM think-in-universe ***@***.***> wrote:
oh sorry, I didn't check the screenshot https://bits.owocki.com/E0uq5rqZ
from @owocki <https://github.com/owocki>. According to the screenshot, it
would probably be data size issue for these active users when exporting
data from database.
And in the meanwhile, there're other serialization issues in my profile
that we may need to fix.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#5955?email_source=notifications&email_token=AAD5PCJFG75AOE4H7EBPS6DRCNZK7A5CNFSM4KQ5UDW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELPK57Y#issuecomment-585019135>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAD5PCMC3AZH6VFLIDJCQWDRCNZK7ANCNFSM4KQ5UDWQ>
.
_________________________________________
gitcoin is live and has generated over $3.5mm for Open Source Software - see
our results <https://gitcoin.co/results>
|
|
@think-in-universe check the new Owocki screenshot https://bits.owocki.com/7KuRBgA6 |
|
thanks @octavioamu @owocki with the help of the screenshot, I submitted another PR for fixing: #5981 but the data size issue is not tackled yet. it would be helpful if we could collect more detailed data for the query timeout. for the solutions, an easy one might be split the request/response and execute that in parallel, but it's still possible the serialization process might takes too long for a certain data field in this case. |
Description
Fix the testing failure in #5947, when backing up profile data to 3Box.
Refers/Fixes
Fix the testing failure in #5947
Testing
Tested locally with GitCoin development environment.
However, I failed to
fund issuesin the local environment and need extra testing from anyone could mock the data in their dev environment, especially for theactive_workandportfoliofields in the backup profile data.