Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement CockroachDB preset #95

Merged
merged 3 commits into from
Jan 30, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,17 @@ jobs:
name: Test server
command: go test -race -cover -v ./internal/gnomockd -run TestK3s

test-cockroachdb:
machine: true
steps:
- setup-for-go-test
- run:
name: Test preset
command: go test -race -cover -v ./preset/cockroachdb/...
- run:
name: Test server
command: go test -race -cover -v ./internal/gnomockd -run TestCockroachDB

### preset tests go here

workflows:
Expand All @@ -236,4 +247,5 @@ workflows:
- test-mssql
- test-mongo
- test-k3s
- test-cockroachdb
### circleci jobs go here
9 changes: 9 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
**/.*
**/*.md
**/*.MD
**/*.yaml
**/*.yml
**/sdktest/**
**/testdata/**
**/*_test.go
**/Dockerfile
48 changes: 30 additions & 18 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -60,25 +60,14 @@ jobs:
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v1
- name: Get dependencies
run: |
cd sdktest/python
pip3 install -r requirements.txt
- name: Generate python sdk
run: swagger/bin/generate-python.sh
- name: Build docker image
run: docker build --tag gnomock .
- name: Run gnomock
run: |
docker run -itd \
-p 23042:23042 \
-v /var/run/docker.sock:/var/run/docker.sock \
-v `pwd`:`pwd` \
gnomock
- name: Install Taskfile
uses: Arduino/actions/setup-taskfile@master
with:
version: '3.0.0'
- name: Generate
run: task generate:sdk:python
- name: Test
run: |
cd sdktest/python
python3 -m pytest -n 3 -v -k TestSDK
run: task test:sdk:python

test-localstack:
name: "[preset] localstack"
Expand Down Expand Up @@ -379,4 +368,27 @@ jobs:
cat preset-cover.txt server-cover.txt > coverage.txt
bash <(curl -s https://codecov.io/bash)

test-cockroachdb:
name: "[preset] cockroachdb"
runs-on: ubuntu-latest
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
steps:
- name: Set up Go 1.15
uses: actions/setup-go@v1
with:
go-version: 1.15
- name: Check out code into the Go module directory
uses: actions/checkout@v1
- name: Get dependencies
run: go get -v -t -d ./...
- name: Test preset
run: go test -race -cover -coverprofile=preset-cover.txt -coverpkg=./... -v ./preset/cockroachdb/...
- name: Test server
run: go test -race -cover -coverprofile=server-cover.txt -coverpkg=./... -v ./internal/gnomockd -run TestCockroachDB
- name: Report coverage
run: |
cat preset-cover.txt server-cover.txt > coverage.txt
bash <(curl -s https://codecov.io/bash)

### preset tests go here
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,7 @@ RabbitMQ | [Go package](https://github.com/orlangure/gnomock/tree/master/preset/
Kafka | [Go package](https://github.com/orlangure/gnomock/tree/master/preset/kafka) | [Docs](https://app.swaggerhub.com/apis/orlangure/gnomock/1.4.6#/presets/startKafka) | [Reference](https://pkg.go.dev/github.com/orlangure/gnomock/preset/kafka?tab=doc) | `2.5.1-L0`
Elasticsearch | [Go package](https://github.com/orlangure/gnomock/tree/master/preset/elastic) | [Docs](https://app.swaggerhub.com/apis/orlangure/gnomock/1.4.6#/presets/startElastic) | [Reference](https://pkg.go.dev/github.com/orlangure/gnomock/preset/elastic?tab=doc) | `5.6`, `6.8.13`, `7.9.3`
Kubernetes | [Go package](https://github.com/orlangure/gnomock/tree/master/preset/k3s) | [Docs](https://app.swaggerhub.com/apis/orlangure/gnomock/1.4.6#/presets/startKubernetes) | [Reference](https://pkg.go.dev/github.com/orlangure/gnomock/preset/k3s?tab=doc) | `v1.19.3`
CockroachDB | [Go package](https://github.com/orlangure/gnomock/tree/master/preset/cockroachdb) | [Docs](https://app.swaggerhub.com/apis/orlangure/gnomock/1.4.6#/presets/startCockroachDB) | [Reference](https://pkg.go.dev/github.com/orlangure/gnomock/preset/cockroachdb?tab=doc) | `latest`
<!-- new presets go here -->

It is possible to use Gnomock [directly from
Expand Down
116 changes: 116 additions & 0 deletions Taskfile.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
version: '3'

tasks:
release:sdk:python:
cmds:
- task: swagger:update-version
- task: git:clone:python
- task: generate:sdk:python
- task: test:sdk:python
- task: git:commit:python
- rm -rf ./dist
- python3 setup.py sdist bdist_wheel
- python3 -m twine upload dist/*
dir: ./sdktest/python/gnomock

swagger:update-version:
cmds:
- |
sed -Ei '' 's/^ version: (.*)$/ version: {{.version}}/' swagger/swagger.yaml
- |
sed -Ei '' 's/^packageVersion: (.*)$/packageVersion: {{.version}}/' swagger/config/python.yaml
- |
sed -Ei '' 's/apis\/orlangure\/gnomock\/(.*)#/apis\/orlangure\/gnomock\/{{.version}}#/' README.md
- git add README.md swagger/config/python.yaml swagger/swagger.yaml
- git commit -m 'Update version to {{.version}}'
silent: true
dir: .
var:
CURRENT_BRANCH:
sh: git branch --show-current
preconditions:
- sh: test $version
msg: version is not set
- sh: test "{{.CURRENT_BRANCH}}" != "master"
msg: version changes cannot happen on master branch

git:clone:python:
cmds:
- rm -rf gnomock
- git clone --depth 1 https://github.com/orlangure/gnomock-python-sdk gnomock
dir: ./sdktest/python

generate:sdk:python:
cmds:
- |
docker run --rm -v $PWD:/local \
openapitools/openapi-generator-cli:v4.3.1 generate \
-i /local/swagger/swagger.yaml \
-g python \
-o /local/sdktest/python/gnomock \
-c /local/swagger/config/python.yaml \
--git-user-id orlangure \
--git-repo-id gnomock-python-sdk
silent: true
dir: .

git:commit:python:
cmds:
- rm .travis.yml
- git add .
- git commit -m 'Bump version to {{.version}}'
- git push
dir: ./sdktest/python/gnomock
preconditions:
- sh: test $version
msg: version is not set

test:sdk:python:
cmds:
- task: gnomock:run
- |
{{ if not .CI }}
python3 -m venv venv
source venv/bin/activate
{{ end }}
pip3 install -r requirements.txt
python3 -m pytest -n 3 -v -k TestSDK
- task: gnomock:stop
dir: ./sdktest/python

gnomock:run:
cmds:
- task: gnomock:stop
- task: gnomock:build:server
- task: gnomock:build:cleaner
- |
docker run -itd --rm \
-p 23042:23042 \
-v /var/run/docker.sock:/var/run/docker.sock \
-v $PWD:$PWD \
--name gnomockd_test \
gnomock
dir: .

gnomock:stop:
cmds:
- docker container stop gnomockd_test
vars:
ID:
sh: docker container ls -q -f name=gnomockd_test
status:
- test -z "{{.ID}}"

gnomock:build:server:
env:
DOCKER_BUILDKIT: "1"
cmds:
- docker build --progress plain --tag gnomock .
dir: .

gnomock:build:cleaner:
env:
DOCKER_BUILDKIT: "1"
cmds:
- docker build --progress plain --tag orlangure/gnomock-cleaner .
dir: ./cmd/cleaner
17 changes: 14 additions & 3 deletions cmd/generator/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import (
"log"
"os"
"path"
"regexp"
"strings"
"text/template"
)
Expand All @@ -30,8 +31,20 @@ const (
`
)

var matchFirstCap = regexp.MustCompile("(.)([A-Z][a-z]+)")
var matchAllCap = regexp.MustCompile("([a-z0-9])([A-Z])")

var fMap = template.FuncMap{
"lower": strings.ToLower,
"title": func(s string) string {
return strings.Title(strings.ToLower(s))
},
"snake": func(s string) string {
snake := matchFirstCap.ReplaceAllString(s, "${1}_${2}")
snake = matchAllCap.ReplaceAllString(snake, "${1}_${2}")

return strings.ToLower(snake)
},
}

type presetParams struct {
Expand Down Expand Up @@ -168,9 +181,7 @@ func gnomockdPkg(params presetParams) error {
"preset": map[string]interface{}{
"version": "latest",
},
"options": map[string]interface{}{
"debug": true,
},
"options": map[string]interface{}{},
}); err != nil {
return fmt.Errorf("can't write into preset file: %w", err)
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
def test_{{ lower .Name }}(self):
# TODO: make sure names below match the code
options = gnomock.Options(debug=True)
preset = gnomock.{{ .Name }}(version="latest")
{{ lower .Name }}_request = gnomock.{{ .Name }}Request(options=options,
preset = gnomock.{{ title .Name }}(version="latest")
{{ lower .Name }}_request = gnomock.{{ title .Name }}Request(options=options,
preset=preset)
id = ""

try:
response = self.api.start_{{ lower .Name }}({{ lower .Name }}_request)
response = self.api.start_{{ snake .Name }}({{ lower .Name }}_request)
id = response.id
self.assertEqual("127.0.0.1", response.host)

Expand Down
1 change: 1 addition & 0 deletions cmd/server/presets.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package main
// all known presets should go right here so that they are available when
// requested over HTTP:
import (
_ "github.com/orlangure/gnomock/preset/cockroachdb"
_ "github.com/orlangure/gnomock/preset/elastic"
_ "github.com/orlangure/gnomock/preset/k3s"
_ "github.com/orlangure/gnomock/preset/kafka"
Expand Down
9 changes: 9 additions & 0 deletions codecov.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,12 @@ ignore:
# part of Gnomock
- cmd/generator/**/*
- internal/testutil/**/*

coverage:
status:
project:
target: 80%
threshold: 0%
patch:
target: 70%
threshold: 0%
76 changes: 76 additions & 0 deletions internal/gnomockd/cockroachdb_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
package gnomockd_test

import (
"bytes"
"database/sql"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"net/http/httptest"
"testing"

"github.com/orlangure/gnomock"
"github.com/orlangure/gnomock/internal/gnomockd"
_ "github.com/orlangure/gnomock/preset/cockroachdb"
"github.com/stretchr/testify/require"
)

func TestCockroachDB(t *testing.T) {
t.Parallel()

h := gnomockd.Handler()
bs, err := ioutil.ReadFile("./testdata/cockroachdb.json")
require.NoError(t, err)

buf := bytes.NewBuffer(bs)
w, r := httptest.NewRecorder(), httptest.NewRequest(http.MethodPost, "/start/cockroachdb", buf)
h.ServeHTTP(w, r)

res := w.Result()

defer func() { require.NoError(t, res.Body.Close()) }()

body, err := ioutil.ReadAll(res.Body)
require.NoError(t, err)

require.Equalf(t, http.StatusOK, res.StatusCode, string(body))

var c *gnomock.Container

err = json.Unmarshal(body, &c)
require.NoError(t, err)

connStr := fmt.Sprintf(
"host=%s port=%d user=root dbname=%s sslmode=disable",
c.Host, c.DefaultPort(), "gnomockd_db",
)

db, err := sql.Open("postgres", connStr)
require.NoError(t, err)

row := db.QueryRow(`select count(distinct ip_address) from customers`)
count := 0
require.NoError(t, row.Scan(&count))
require.Equal(t, 1000, count)

row = db.QueryRow(`select a from tbl`)
value := 0
require.NoError(t, row.Scan(&value))
require.Equal(t, 42, value)

row = db.QueryRow(`select max(foo) from things limit 1`)
value = 0
require.NoError(t, row.Scan(&value))
require.Equal(t, 3, value)

bs, err = json.Marshal(c)
require.NoError(t, err)

buf = bytes.NewBuffer(bs)
w, r = httptest.NewRecorder(), httptest.NewRequest(http.MethodPost, "/stop", buf)
h.ServeHTTP(w, r)

res = w.Result()
require.Equal(t, http.StatusOK, res.StatusCode)
}
1 change: 1 addition & 0 deletions internal/gnomockd/start.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ func startHandler() http.HandlerFunc {
p,
gnomock.WithOptions(&sr.Options),
gnomock.WithLogWriter(logWriter),
gnomock.WithContext(r.Context()),
)

close(started)
Expand Down
16 changes: 16 additions & 0 deletions internal/gnomockd/testdata/cockroachdb.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"preset": {
"queries_files": [
"./testdata/cockroachdb/customers.sql",
"./testdata/cockroachdb/things.sql"
],
"queries": [
"create table tbl(a int)",
"insert into tbl(a) values (42)"
],
"db": "gnomockd_db",
"version": "v19.2.11"
},
"options": {
}
}
Loading