Venom - manage and run your integration tests with efficiency
source link: https://www.tuicool.com/articles/hit/nARBJv3
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
:snake: Venom
Venom run executors (script, HTTP Request, etc. ) and assertions. It can also output xUnit results files.
Command Line
Install with:
$ go get github.com/ovh/venom $ go install github.com/ovh/venom/cli/venom
$ venom run -h Run Tests Usage: venom run [flags] Flags: --env Inject environment variables. export FOO=BAR -> you can use {{.FOO}} in your tests (default true) --exclude strings --exclude filaA.yaml --exclude filaB.yaml --exclude fileC*.yaml --format string --format:yaml, json, xml, tap (default "xml") -h, --help help for run --log string Log Level : debug, info or warn (default "warn") --no-check-variables Don't check variables before run --output-dir string Output Directory: create tests results file inside this directory --parallel int --parallel=2 : launches 2 Test Suites in parallel (default 1) --profiling Enable Mem / CPU Profile with pprof --stop-on-failure Stop running Test Suite on first Test Case failure --strict Exit with an error code if one test fails --var strings --var cds='cds -f config.json' --var cds2='cds -f config.json' --var-from-file string --var-from-file filename.yaml : hcl|json|yaml, must contains map[string]string'
Executors
- dbfixtures : https://github.com/ovh/venom/tree/master/executors/dbfixtures
- exec : https://github.com/ovh/venom/tree/master/executors/exec
exec
is the default type for a step - http : https://github.com/ovh/venom/tree/master/executors/http
- imap : https://github.com/ovh/venom/tree/master/executors/imap
- kafka https://github.com/ovh/venom/tree/master/executors/kafka
- ovhapi : https://github.com/ovh/venom/tree/master/executors/ovhapi
- readfile : https://github.com/ovh/venom/tree/master/executors/readfile
- redis : https://github.com/ovh/venom/tree/master/executors/redis
- smtp : https://github.com/ovh/venom/tree/master/executors/smtp
- ssh : https://github.com/ovh/venom/tree/master/executors/ssh
- web : https://github.com/ovh/venom/tree/master/executors/web
- grpc : https://github.com/ovh/venom/tree/master/executors/grpc
TestSuite files
- Run
venom template
- Examples: https://github.com/ovh/cds/tree/master/tests
Example:
name: Title of TestSuite testcases: - name: TestCase with default value, exec cmd. Check if exit code != 1 steps: - script: echo 'foo' type: exec - name: Title of First TestCase steps: - script: echo 'foo' assertions: - result.code ShouldEqual 0 - script: echo 'bar' assertions: - result.systemout ShouldNotContainSubstring foo - result.timeseconds ShouldBeLessThan 1 - name: GET http testcase, with 5 seconds timeout steps: - type: http method: GET url: https://eu.api.ovh.com/1.0/ timeout: 5 assertions: - result.body ShouldContainSubstring /dedicated/server - result.body ShouldContainSubstring /ipLoadbalancing - result.statuscode ShouldEqual 200 - result.timeseconds ShouldBeLessThan 1 - name: Test with retries and delay in seconds between each try steps: - type: http method: GET url: https://eu.api.ovh.com/1.0/ retry: 3 delay: 2 assertions: - result.statuscode ShouldEqual 200
Using variables and reuse results
name: MyTestSuiteTmpl vars: api.foo: 'http://api/foo' second: 'venomWithTmpl' testcases: - name: testA steps: - type: exec script: echo '{{.api.foo}}' assertions: - result.code ShouldEqual 0 - result.systemout ShouldEqual http://api/foo - name: testB steps: - type: exec script: echo 'XXX{{.testA.result.systemout}}YYY' assertions: - result.code ShouldEqual 0 - result.systemout ShouldEqual XXXhttp://api/fooYYY
Extract variable from results and reuse it in step after
name: MyTestSuite testcases: - name: testA steps: - type: exec script: echo 'foo with a bar here' extracts: result.systemout: foo with a {{myvariable=[a-z]+}} here - name: testB steps: - type: exec script: echo {{.testA.myvariable}} assertions: - result.code ShouldEqual 0 - result.systemout ShouldContainSubstring bar
Builtin venom variables
name: MyTestSuite testcases: - name: testA steps: - type: exec script: echo '{{.venom.testsuite}} {{.venom.testsuite.filename}} {{.venom.testcase}} {{.venom.teststep.number}} {{.venom.datetime}} {{.venom.timestamp}}' # will display something as: MyTestSuite MyTestSuiteWithVenomBuiltinVar.yml testA 0 2018-08-05T21:38:24+02:00 1533497904
Builtin variables:
- {{.venom.testsuite}}
- {{.venom.testsuite.filename}}
- {{.venom.testcase}}
- {{.venom.teststep.number}}
- {{.venom.datetime}}
- {{.venom.timestamp}}
Testsuite Versions
Version 2
On this new version, venom use the testsuite folder as the basepath instead of location of venom execution.
Considering this workspace:
tests/ testsuiteA/ testsuite.yml testa.json
On version 1
name: TestSuite Read File testcases: - name: TestCase Read File steps: - type: readfile path: testa.json assertions: - result.contentjson.foo ShouldEqual bar
If you execute venom run *
into tests/ folder, venom will try to find testa.json on tests/testa.json and will failed. You must execute venom on the testsuite dir.
On version 2, venom use as basepath the testsuite file. So no matter where you execute venom command, testa.json will be found.
To specify the version 2, add version property on the testsuite:
version: "2" name: TestSuite Read File testcases: - name: TestCase Read File steps: - type: readfile path: testa.json assertions: - result.contentjson.foo ShouldEqual bar
RUN Venom locally on CDS Integration Tests
cd $GOPATH/src/github.com/ovh/cds/tests venom run --var cdsro='cds -f $HOME/.cds/it.user.ro.json' --var cds='cds -f $HOME/.cds/it.user.rw.json' --parallel=5
RUN Venom with file var
vars.yaml :
cdsro: 'cds -f $HOME/.cds/it.user.ro.json' cds: 'cds -f $HOME/.cds/it.user.rw.json'
cd $GOPATH/src/github.com/ovh/cds/tests venom run --var-from-file vars.yaml --parallel=5
RUN Venom, with an export xUnit
venom run --format=xml --output-dir="."
Assertion
Keywords
- ShouldEqual
- ShouldNotEqual
- ShouldAlmostEqual
- ShouldNotAlmostEqual
- ShouldResemble
- ShouldNotResemble
- ShouldPointTo
- ShouldNotPointTo
- ShouldBeNil
- ShouldNotBeNil
- ShouldBeTrue
- ShouldBeFalse
- ShouldBeZeroValue
- ShouldBeGreaterThan
- ShouldBeGreaterThanOrEqualTo
- ShouldBeLessThan
- ShouldBeLessThanOrEqualTo
- ShouldBeBetween
- ShouldNotBeBetween
- ShouldBeBetweenOrEqual
- ShouldNotBeBetweenOrEqual
- ShouldContain
- ShouldNotContain
- ShouldContainKey
- ShouldNotContainKey
- ShouldBeIn
- ShouldNotBeIn
- ShouldBeEmpty
- ShouldNotBeEmpty
- ShouldHaveLength
- ShouldStartWith
- ShouldNotStartWith
- ShouldEndWith
- ShouldNotEndWith
- ShouldBeBlank
- ShouldNotBeBlank
- ShouldContainSubstring
- ShouldNotContainSubstring
- ShouldEqualWithout
- ShouldEqualTrimSpace
- ShouldHappenBefore
- ShouldHappenOnOrBefore
- ShouldHappenAfter
- ShouldHappenOnOrAfter
- ShouldHappenBetween
- ShouldHappenOnOrBetween
- ShouldNotHappenOnOrBetween
- ShouldHappenWithin
- ShouldNotHappenWithin
- ShouldBeChronological
Most assertion keywords documentation can be found on smartystreets/assertions README .
Write your executor
An executor have to implement this interface
// Executor execute a testStep. type Executor interface { // Run run a Test Step Run(ctx context.Content, venom.Logger, TestStep) (ExecutorResult, error) }
Example
// Name of executor const Name = "myexecutor" // New returns a new Executor func New() venom.Executor { return &Executor{} } // Executor struct type Executor struct { Command string `json:"command,omitempty" yaml:"command,omitempty"` } // Result represents a step result type Result struct { Code int `json:"code,omitempty" yaml:"code,omitempty"` Command string `json:"command,omitempty" yaml:"command,omitempty"` Systemout string `json:"systemout,omitempty" yaml:"systemout,omitempty"` // put in testcase.Systemout by venom if present Systemerr string `json:"systemerr,omitempty" yaml:"systemerr,omitempty"` // put in testcase.Systemerr by venom if present Executor Executor `json:"executor,omitempty" yaml:"executor,omitempty"` } // GetDefaultAssertions return default assertions for this executor // Optional func (Executor) GetDefaultAssertions() venom.StepAssertions { return venom.StepAssertions{Assertions: []string{"result.code ShouldEqual 0"}} } // Run execute TestStep func (Executor) Run(ctx context.Context, l venom.Logger, step venom.TestStep) (venom.ExecutorResult, error) { // transform step to Executor Instance var e Executor if err := mapstructure.Decode(step, &e); err != nil { return nil, err } // to something with e.Command here... //... systemout := "foo" ouputCode := 0 // prepare result r := Result{ Code: ouputCode, // return Output Code Command: e.Command, // return Command executed Systemout: systemout, // return Output string Executor: e, // return executor, useful for display Executor context in failure } return dump.ToMap(r) }
Feel free to open a Pull Request with your executors.
TestCase Context
TestCase Context allows you to inject data in all steps.
Define a context is optional, but can be useful to keep data between test steps on a testcase.
Write your TestCase Context
A TestCase Context has to implement this interface
type TestCaseContext interface { Init() error Close() error SetTestCase(tc TestCase) GetName() string }
Example
// Context Type name const Name = "default" // New returns a new TestCaseContext func New() venom.TestCaseContext { ctx := &DefaultTestCaseContext{} ctx.Name = Name return ctx } // DefaultTestCaseContext represents the context of a testcase type DefaultTestCaseContext struct { venom.CommonTestCaseContext datas map[string]interface{} } // Init Initialize the context func (tcc *DefaultTestCaseContext) Init() error { return nil } // Close the context func (tcc *DefaultTestCaseContext) Close() error { return nil }
Methods SetTestCase and GetName are implemented by CommonTestCaseContext
Dependencies
Individual packages were updated using the rough procedure:
dep ensure dep ensure -update ${PACKAGE} dep prune go build
Hacking
You've developed a new cool feature? Fixed an annoying bug? We'd be happy to hear from you! Make sure to read CONTRIBUTING.md before.
License
This work is under the BSD license, see the LICENSE file for details.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK