Skip to content
Snippets Groups Projects
Select Git revision
  • 84f555bfd8aefb1b0e12d14b56949bbfbef78d5f
  • master default protected
  • arm-aarch-platform
  • arm-platform
  • vjpai-patch-3
  • vjpai-patch-1
  • v1.27.x
  • jtattermusch-patch-2
  • jtattermusch-patch-1
  • update-java-worker-example-in-performance-docs
  • revert-21805-revert-21797-reintroduce_21527
  • revert-21804-tls-credentials-1
  • zhen_cleanup_namecheck
  • revert-21806-revert-21767-revert-21725-revert-21680-cq_ordering
  • vjpai-patch-2
  • revert-21766-tls-credentials-1
  • revert-21640-change_local_tcp_security_level
  • revert-21680-cq_ordering
  • revert-21527-unify_boringssl_deps2
  • revert-20803-grpclb_stabilization
  • fix-kokoro-rvm-key
  • v1.27.0
  • v1.27.0-pre2
  • v1.27.0-pre1
  • v1.26.0
  • v1.26.0-pre1
  • v1.25.0
  • v1.25.0-pre1
  • v1.24.3
  • v1.24.2
  • v1.24.1
  • v1.23.1
  • v1.24.0
  • v1.24.0-pre2
  • v1.24.0-pre1
  • v1.22.1
  • v1.23.0
  • v1.23.0-pre1
  • v1.22.0
  • v1.22.0-pre1
  • v1.21.4
41 results

interop_matrix

user avatar
Add v1.22.3 and v1.23.1 release of grpc-go to interop matrix.
Easwar Swaminathan authored 
v1.23.0 seems to have missed out since the
[PR](https://github.com/grpc/grpc/pull/19928) to add that had a merge
conflict and was never merged.
e9802ca1
History
user avatar e9802ca1
History
Name Last commit Last update
..
patches
testcases
README.md
client_matrix.py
create_matrix_images.py
create_testcases.sh
run_interop_matrix_tests.py
README.md

Overview

This directory contains scripts that facilitate building and running gRPC interoperability tests for combinations of language/runtimes (known as matrix).

The setup builds gRPC docker images for each language/runtime and upload it to Google Container Registry (GCR). These images, encapsulating gRPC stack from specific releases/tag, are used to test version compatibility between gRPC release versions.

Step-by-step instructions for adding a GCR image for a new release for compatibility test

We have continuous nightly test setup to test gRPC backward compatibility between old clients and latest server. When a gRPC developer creates a new gRPC release, s/he is also responsible to add the just-released gRPC client to the nightly test. The steps are:

  • Add (or update) an entry in ./client_matrix.py file to reference the github tag for the release.
  • Build new client docker image(s). For example, for C and wrapper languages release v1.9.9, do
    • tools/interop_matrix/create_matrix_images.py --git_checkout --release=v1.9.9 --upload_images --language cxx csharp python ruby php
  • Verify that the new docker image was built successfully and uploaded to GCR. For example,
  • Verify the just-created docker client image would pass backward compatibility test (it should). For example,
    • gcloud docker -- pull gcr.io/grpc-testing/grpc_interop_java:v1.9.9 followed by
    • docker_image=gcr.io/grpc-testing/grpc_interop_java:v1.9.9 tools/interop_matrix/testcases/java__master
  • git commit the change and merge it to upstream/master.
  • (Optional) clean up the tmp directory to where grpc source is cloned at /export/hda3/tmp/grpc_matrix/. For more details on each step, refer to sections below.

Instructions for adding new language/runtimes

  • Create new Dockerfile.template, build_interop.sh.template for the language/runtime under template/tools/dockerfile/.
  • Run tools/buildgen/generate_projects.sh to create corresponding files under tools/dockerfile/.
  • Add language/runtimes to client_matrix.py following existing language/runtimes examples.
  • Run tools/interop_matrix/create_matrix_images.py which will build (and upload) images to GCR.

Instructions for creating new test cases

  • Create test cases by running LANG=<lang> [RELEASE=<release>] ./create_testcases.sh. For example,
    • LANG=go ./create_testcases.sh will generate ./testcases/go__master, which is also a functional bash script.
    • LANG=go KEEP_IMAGE=1 ./create_testcases.sh will generate ./testcases/go__master and keep the local docker image so it can be invoked simply via ./testcases/go__master. Note: remove local docker images manually afterwards with docker rmi <image_id>.
  • Stage and commit the generated test case file ./testcases/<lang>__<release>.

Instructions for running test cases against GCR images

  • Run tools/interop_matrix/run_interop_matrix_tests.py. Useful options:
    • --release specifies a git release tag. Defaults to --release=all. Make sure the GCR images with the tag have been created using create_matrix_images.py above.
    • --language specifies a language. Defaults to --language=all. For example, To test all languages for all gRPC releases across all runtimes, do tools/interop_matrix/run_interop_matrix_test.py --release=all.
  • The output for all the test cases is recorded in a junit style xml file (defaults to 'report.xml').

Instructions for running test cases against a GCR image manually

  • Download docker image from GCR. For example: gcloud docker -- pull gcr.io/grpc-testing/grpc_interop_go1.8:v1.16.0.
  • Run test cases by specifying docker_image variable inline with the test case script created above. For example:
    • docker_image=gcr.io/grpc-testing/grpc_interop_go1.8:v1.16.0 ./testcases/go__master will run go__master test cases against go1.8 with gRPC release v1.16.0 docker image in GCR.

Note:

  • File path starting with tools/ or template/ are relative to the grpc repo root dir. File path starting with ./ are relative to current directory (tools/interop_matrix).
  • Creating and referencing images in GCR require read and write permission to Google Container Registry path gcr.io/grpc-testing.