Release (#295)

* Fabo/fix gas (#250) * fix gas estimate * linted * fixed test * do not keep data sources (#251) * track failing transactions in Sentry (#249) * correctly set the tx schema for a failing tx (#248) * Fabo/remove per block caching as not working (#247) * remove per block caching as not working * fix memoized results Co-authored-by: Ana G. <[email protected]> * delete perblockcachedatasource (#253) * Ana/fix balances in actionmodal (#255) * fix action modal available balance * include regen * use dictionary for denomlookup * use correct events for received txs (#257) * enable account creation for some networks (#252) * network update time metric added (#256) * network update time metric added * added missing dep Co-authored-by: Fabian <[email protected]> * Fix proposal deposit (#261) * Remove denom handling from getDeposit() * Revert undesired change * delete package-lock.json * localtestnet config change (#265) * Ana/handle "address not from this network" error (#263) * add check address function for all queries * apply suggestions * Ana/add fiatvalue to balances query (e-Money) (#262) * preparation * more preparation * add fiatvalue field to balances query * fix get account info * apply suggestions * apply one last suggestion * suggestions+ Co-authored-by: Fabian <[email protected]> * Ana/emoney fix expected returns with inflation and totalbacked (#243) * fix expected returns with inflation and supply * minor fixes. dictionary * query exchange rates from emoney api * fix infinite expected returns * convert api url to const * add eur value to totalbackedvalue. totalngm gains * add important comment * finish calculation * lint * catch errors with sentry Co-authored-by: Fabian <[email protected]> * readd coin conversion (#268) * delete amount field (#274) * Fabo/increase gas again (#271) * icrease gas again * fixed test * Fabo/load all txs (even if more then first page in response) (#270) * load all txs (even if more then first page in response) * improved handling of txs * missing renaming * fixed paginated load * add pagination fix also to cosmosV0-source Co-authored-by: iambeone <[email protected]> Co-authored-by: Ana G. <[email protected]> * fixing issue with multiple senders in one event (#273) * fixing issue with multiple senders in one event * Update lib/source/cosmosV2-source.js Co-authored-by: Fabian <[email protected]> * Fabo/allow signing for terra + emoney (#267) * allow signing for terra * readd coin conversion * enable actions for terra * fix correct terra testnet url * comments and guards * enabled more txs for emoney and fixed broadcasting * added a catch for wrongly formatted broadcast urls * recover default field. change some network titles (#277) * Fabo/add network data to API (#278) * non desctructive introduction of better address prefix wording * added address creator to API * adjusted test * added ledger app to networks config * add icon property to schema (#281) * add icon property to schema * fix network schema validation Co-authored-by: Ana G. <[email protected]> * filter out validator specific txs (#279) * Ana/balances coinreducer good fix (#269) * balances coinreducer good fix * refactored fiat value logic Co-authored-by: Fabian <[email protected]> * Create network_integration.md * Update network_integration.md * Update network_integration.md * Fabo/avoid 500 errors (#288) * avoid using the latest query * cleanup * Ana/filter validator tx cross network and add txvalue reducer (#285) * filter validators cross network * add value reducer. necessary for multi claim txs * add validator txs filter also for cosmosv0 source * filter and make array only claim rewards msg value * filter txs by whitelist * change length in multi claim reward reducer * add withdrawvalidators * replace dictionary for set * refactor transaction snippet. avoid repetition * Ana/emoney upgrade (mergeable) (#282) * update emoney api_url * fix denom. add default fiat currency * fix rpc endpoint * fix value (my bad) (#293) * fix value (my bad) * trigger another ci flow * erase space * set correct new chain id (#294) Co-authored-by: Ana G. <[email protected]> Co-authored-by: Aleksey Rudometov <[email protected]> Co-authored-by: Mario Pino <[email protected]> Co-authored-by: Jordan Bibla <[email protected]>
luniehq · Feb 4, 2020 · d6a7ef3 · d6a7ef3
1 parent cb66913
commit d6a7ef3
Show file tree

Hide file tree

Showing 12 changed files with 128 additions and 80 deletions.
diff --git a/.github/ISSUE_TEMPLATE/network_integration.md b/.github/ISSUE_TEMPLATE/network_integration.md
@@ -0,0 +1,58 @@
+---
+name: Network Integration
+about: Details for new network integration
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+### What is the network's name?
+
+
+### Is there a description for the network?
+
+
+#### Network Phase?
+- [ ] Mainnet
+- [ ] Testnet
+
+### Please share a link to this project's website.
+
+
+### Please share a link to this project's Github.
+
+
+### Please share the full node endpoints we can use.
+
+
+### What features need to be implemented?
+All functionality listed below will work with the appropriate combination of Lunie web, Lunie iOS and Android, Lunie browser extension, and Ledger Nano S and X with a cable (no bluetooth yet) and theCosmos Ledger app.
+
+**Wallet functionality**
+- [ ] Create address
+- [ ] Restore from seed phrase
+- [ ] Send tokens
+- [ ] Check balance and rewards
+- [ ] View transaction history
+
+**Staking functionality**
+- [ ] View validator list in Lunie
+- [ ] Ability to view network portfolio
+- [ ] Ability to stake tokens
+- [ ] Ability to unstake, restake tokens
+- [ ] Ability to claim rewards
+
+**Governance functionality**
+- [ ] Create a proposal
+- [ ] Deposit on a proposal
+- [ ] Vote on a proposal
+- [ ] View current and past proposals
+
+**Block explorer**
+- [ ] View user generated transactions per block
+
+### Additional context
+Add any other context or screenshots about the network integration here.
+
+### Delivery Date
diff --git a/data/networks.json b/data/networks.json
@@ -134,9 +134,9 @@
   {
     "id": "emoney-testnet",
     "title": "e-Money Testnet",
-    "chain_id": "lilmermaid-4",
-    "api_url": "https://emoney-lilmermaid-4.lunie.io",
-    "rpc_url": "wss://emoney-lilmermaid-4.lunie.io/websocket",
+    "chain_id": "lilmermaid-5",
+    "api_url": "http://lilmermaid.validator.network/light",
+    "rpc_url": "wss://lilmermaid.validator.network/websocket",
     "bech32_prefix": "emoney",
     "address_prefix": "emoney",
     "address_creator": "cosmos",

diff --git a/lib/block-listeners/cosmos-node-subscription.js b/lib/block-listeners/cosmos-node-subscription.js
@@ -46,7 +46,8 @@ class CosmosNodeSubscription {
         })
       })
 
-    this.newBlockHandler()
+    // disabled immediate validator query as, as the `latest` query is buggy
+    // this.newBlockHandler()
   }
 
   // For each block event, we fetch the block information and publish a message.
@@ -62,16 +63,16 @@ class CosmosNodeSubscription {
     const block = await this.cosmosAPI.getBlockByHeight({
       blockHeight: height
     })
-    const validatorMap = await this.getLatestBlockData()
+    // in the case of height being undefined to query for latest
+    // eslint-disable-next-line require-atomic-updates
+    height = block.height
+
+    const validatorMap = await this.getValidatorMap(height)
     this.store.update({ height, block, validators: validatorMap })
     publishBlockAdded(this.network.id, block)
     // TODO remove, only for demo purposes
     // publishEvent(this.network.id, 'block', '', block)
 
-    // in the case of height being undefined to query for latest
-    // eslint-disable-next-line require-atomic-updates
-    height = block.height
-
     // For each transaction listed in a block we extract the relevant addresses. This is published to the network.
     // A GraphQL resolver is listening for these messages and sends the
     // transaction to each subscribed user.
@@ -98,8 +99,8 @@ class CosmosNodeSubscription {
     this.cosmosAPI.memoizedResults.clear()
   }
 
-  async getLatestBlockData() {
-    const validators = await this.cosmosAPI.getAllValidators()
+  async getValidatorMap(height) {
+    const validators = await this.cosmosAPI.getAllValidators(height)
     const validatorMap = _.keyBy(validators, 'operatorAddress')
     return validatorMap
   }

diff --git a/lib/message-types.js b/lib/message-types.js
@@ -15,4 +15,16 @@ const cosmosMessageType = {
   MULTI_SEND: 'cosmos-sdk/MsgMultiSend'
 }
 
-module.exports = { cosmosMessageType }
+const cosmosWhitelistedMessageTypes = new Set([
+  `MsgSend`,
+  `MsgDelegate`,
+  `MsgBeginRedelegate`,
+  `MsgUndelegate`,
+  `MsgVote`,
+  `MsgDeposit`,
+  `MsgWithdrawDelegationReward`,
+  `MsgSubmitProposal`,
+  `MsgMultiSend`
+])
+
+module.exports = { cosmosMessageType, cosmosWhitelistedMessageTypes }
diff --git a/lib/reducers/cosmosV0-reducers.js b/lib/reducers/cosmosV0-reducers.js
@@ -1,4 +1,6 @@
+const { uniqWith, sortBy, reverse } = require('lodash')
 const { cosmosMessageType } = require('../message-types')
+const { cosmosWhitelistedMessageTypes } = require('../../lib/message-types')
 const BigNumber = require('bignumber.js')
 const _ = require('lodash')
 const Sentry = require('@sentry/node')
@@ -372,6 +374,28 @@ function undelegationEndTimeReducer(transaction) {
   }
 }
 
+function formatTransactionsReducer(txs, reducers) {
+  const duplicateFreeTxs = uniqWith(txs, (a, b) => a.txhash === b.txhash)
+  const sortedTxs = sortBy(duplicateFreeTxs, ['timestamp'])
+  const reversedTxs = reverse(sortedTxs)
+  // here we filter out all transactions related to validators
+  reversedTxs.forEach(tx => {
+    tx.tx.value.msg = tx.tx.value.msg.filter(msg =>
+      cosmosWhitelistedMessageTypes.has(msg.type.split('/')[1])
+    )
+  })
+  return reversedTxs.map(tx => transactionReducer(tx, reducers))
+}
+
+// to be able to catch all validators from a multi-claim reward tx, we need to capture
+// more than just the first value message.
+function txMultiClaimRewardReducer(txMessages) {
+  const filteredMessages = txMessages.filter(
+    msg => msg.type.split('/')[1] === `MsgWithdrawDelegationReward`
+  )
+  return filteredMessages.length > 0 ? filteredMessages : null
+}
+
 function transactionReducer(transaction, reducers) {
   try {
     let fee = coinReducer(false)
@@ -398,6 +422,9 @@ function transactionReducer(transaction, reducers) {
       signature: transaction.tx.value.signatures[0].signature,
       value: JSON.stringify(transaction.tx.value.msg[0].value),
       raw: transaction,
+      withdrawValidators: JSON.stringify(
+        txMultiClaimRewardReducer(transaction.tx.value.msg)
+      ),
       undelegationEndTime: reducers.undelegationEndTimeReducer(transaction)
     }
 
@@ -429,6 +456,7 @@ module.exports = {
   accountInfoReducer,
   calculateTokens,
   undelegationEndTimeReducer,
+  formatTransactionsReducer,
 
   atoms,
   proposalBeginTime,

diff --git a/lib/reducers/emoneyV0-reducers.js b/lib/reducers/emoneyV0-reducers.js
@@ -8,8 +8,8 @@ const Sentry = require('@sentry/node')
 
 async function totalBackedValueReducer(totalBackedValue) {
   const exchangeRates = await fetchTokenExchangeRates()
-  const token = `e`.concat(totalBackedValue.denom.substring(2))
-  const ticker = totalBackedValue.denom.substring(2).toUpperCase()
+  const token = `e`.concat(totalBackedValue.denom.substring(1))
+  const ticker = totalBackedValue.denom.substring(1).toUpperCase()
   // First we calculate the fiat net value of the token's total supply in it counterpart
   // fiat currency
   // TODO Not all the tokens have yet their rate value in the e-Money API. So the calculation

diff --git a/lib/schema.js b/lib/schema.js
@@ -150,6 +150,7 @@ const typeDefs = gql`
     fee: Coin
     signature: String!
     value: String!
+    withdrawValidators: String
     undelegationEndTime: String
   }
 

diff --git a/lib/source/cosmosV0-source.js b/lib/source/cosmosV0-source.js
@@ -1,7 +1,6 @@
 const { RESTDataSource } = require('apollo-datasource-rest')
 const BigNumber = require('bignumber.js')
 const _ = require('lodash')
-const { uniqWith, sortBy, reverse } = require('lodash')
 const { encodeB32, decodeB32, pubkeyToAddress } = require('../tools')
 const { UserInputError } = require('apollo-server')
 
@@ -115,8 +114,8 @@ class CosmosV0API extends RESTDataSource {
     }
   }
 
-  async getAllValidatorSets() {
-    const response = await this.query(`validatorsets/latest`)
+  async getAllValidatorSets(height = 'latest') {
+    const response = await this.query(`validatorsets/${height}`)
     return response
   }
 
@@ -148,7 +147,7 @@ class CosmosV0API extends RESTDataSource {
     return this.reducers.delegationReducer(selfDelegation, validator).amount
   }
 
-  async getAllValidators() {
+  async getAllValidators(height) {
     let [
       validators,
       annualProvision,
@@ -161,7 +160,7 @@ class CosmosV0API extends RESTDataSource {
         this.query(`staking/validators?status=unbonded`)
       ]).then(validatorGroups => [].concat(...validatorGroups)),
       this.getAnnualProvision(),
-      this.getAllValidatorSets(),
+      this.getAllValidatorSets(height),
       this.getSignedBlockWindow()
     ])
 
@@ -494,33 +493,7 @@ class CosmosV0API extends RESTDataSource {
         `/txs?action=withdraw_validator_rewards_all&source-validator=${address}`
       )
     ]).then(transactionGroups => [].concat(...transactionGroups))
-
-    const duplicateFreeTxs = uniqWith(txs, (a, b) => a.txhash === b.txhash)
-    const sortedTxs = sortBy(duplicateFreeTxs, ['timestamp'])
-    const reversedTxs = reverse(sortedTxs)
-    return reversedTxs.map(tx =>
-      this.reducers.transactionReducer(tx, this.reducers)
-    )
-  }
-
-  async awaitUp() {
-    const start = new Date().getTime()
-    // we need to wait until the stargate API is up and has the expected shema
-    let up = false
-    while (!up) {
-      if (new Date().getTime() - start > 90000) {
-        throw new Error('Timed out waiting for stargarte API to be up.')
-      }
-      try {
-        // test if the stargate API has the expected schema by probing one setup table
-        await this.get('/blocks/latest')
-        up = true
-      } catch (err) {
-        console.log(err)
-        await new Promise(resolve => setTimeout(resolve, 1000))
-        console.log('Waiting for stargate API to be up')
-      }
-    }
+    return this.reducers.formatTransactionsReducer(txs, this.reducers)
   }
 }
 

diff --git a/lib/source/cosmosV2-source.js b/lib/source/cosmosV2-source.js
@@ -1,5 +1,4 @@
 const CosmosV0API = require('./cosmosV0-source')
-const { uniqWith, sortBy, reverse } = require('lodash')
 
 class CosmosV2API extends CosmosV0API {
   setReducers() {
@@ -41,21 +40,7 @@ class CosmosV2API extends CosmosV0API {
       this.loadPaginatedTxs(`/txs?transfer.recipient=${address}`)
     ]).then(([senderTxs, recipientTxs]) => [].concat(senderTxs, recipientTxs))
 
-    const dupFreeTxs = uniqWith(txs, (a, b) => a.txhash === b.txhash)
-    const sortedTxs = sortBy(dupFreeTxs, ['timestamp'])
-    const reversedTxs = reverse(sortedTxs)
-    // here we filter out all transactions related to validators
-    const filteredTxs = reversedTxs.filter(
-      tx =>
-        tx.tx.value.msg[0].type !==
-          `cosmos-sdk/MsgWithdrawValidatorCommission` &&
-        tx.tx.value.msg[0].type !== `cosmos-sdk/MsgSetWithdrawAddress` &&
-        tx.tx.value.msg[0].type !== `cosmos-sdk/MsgEditValidator` &&
-        tx.tx.value.msg[0].type !== `cosmos-sdk/MsgUnjail`
-    )
-    return filteredTxs.map(tx =>
-      this.reducers.transactionReducer(tx, this.reducers)
-    )
+    return this.reducers.formatTransactionsReducer(txs, this.reducers)
   }
 
   extractInvolvedAddresses(transaction) {

diff --git a/lib/source/emoneyV0-source.js b/lib/source/emoneyV0-source.js
@@ -2,7 +2,6 @@ const TerraV3API = require('./terraV3-source')
 const BigNumber = require('bignumber.js')
 const fetch = require('node-fetch')
 const apiURL = `https://api.exchangeratesapi.io/latest?`
-const { UserInputError } = require('apollo-server')
 
 class EMoneyV0API extends TerraV3API {
   setReducers() {
@@ -47,18 +46,16 @@ class EMoneyV0API extends TerraV3API {
 
   // Currently this function only works for e-Money and is very e-Money centered.
   // But soon will also be enabled for other similar multiple-tokens networks like Terra.
-  async calculateFiatValue(balance, selectedFiatCurrency) {
-    if (!selectedFiatCurrency) {
-      throw new UserInputError(
-        `No fiatCurrency selected. Please, enter the fiat currency you'd like the fiat value to be displayed in`
-      )
-    }
+
+  // We just default to EUR if there is no fiat currency included in the query
+  // This is because the standard price for NGM tokens given by e-Money team is 0.50€
+  async calculateFiatValue(balance, selectedFiatCurrency = `EUR`) {
     // When e-Money goes live they will count with a trading platform where the value
     // for the different backed tokens will be changing slightly.
     // They will provide with an API for us to query these values.
     // For now we will assume a 1:1 ratio and treat each token like it were the real
     // fiat currency it represents.
-    const denom = balance.denom.substring(2).toUpperCase()
+    const denom = balance.denom.substring(1).toUpperCase()
 
     // To handle the NGM balance, first we convert to EUR value and then to the selected
     // fiat currency value

diff --git a/lib/source/regenV0-source.js b/lib/source/regenV0-source.js
@@ -1,6 +1,4 @@
 const CosmosV0API = require('./cosmosV0-source')
-const { transactionReducer } = require('../reducers/cosmosV0-reducers')
-const { uniqWith, sortBy, reverse } = require('lodash')
 
 class RegenV0API extends CosmosV0API {
   constructor(network) {
@@ -31,12 +29,7 @@ class RegenV0API extends CosmosV0API {
       this.get(`/txs?recipient=${address}${pagination}`).then(({ txs }) => txs)
     ]).then(([senderTxs, recipientTxs]) => [].concat(senderTxs, recipientTxs))
 
-    const dupFreeTxs = uniqWith(txs, (a, b) => a.txhash === b.txhash)
-    const sortedTxs = sortBy(dupFreeTxs, ['timestamp'])
-    const reversedTxs = reverse(sortedTxs)
-    return reversedTxs.map(transaction =>
-      transactionReducer(transaction, this.reducers)
-    )
+    return this.reducers.formatTransactionsReducer(txs, this.reducers)
   }
 }
 

diff --git a/lib/source/terraV3-source.js b/lib/source/terraV3-source.js
@@ -11,14 +11,14 @@ class TerraV3API extends CosmosV2API {
     this.reducers = require('../reducers/terraV3-reducers')
   }
 
-  async getAllValidators() {
+  async getAllValidators(height) {
     let [validators, validatorSet, signedBlocksWindow] = await Promise.all([
       Promise.all([
         this.query(`staking/validators?status=unbonding`),
         this.query(`staking/validators?status=bonded`),
         this.query(`staking/validators?status=unbonded`)
       ]).then(validatorGroups => [].concat(...validatorGroups)),
-      this.getAllValidatorSets(),
+      this.getAllValidatorSets(height),
       this.getSignedBlockWindow()
     ])