Skip to content

Commit 0511a75

Browse files
committed
DRA E2E: fix data race in test driver
UnprepareResourceClaims was not locking the mutex and thus raced with GetPreparedResources, as found when data race detection was enabled in kubernetes-kind-dra-all. The locking gets added at the same level as in PrepareResourceClaims, i.e. in the underlying implementation right before doing the actual work. WARNING: DATA RACE Write at 0x00c0047e03c0 by goroutine 9325: runtime.mapdelete() runtime/map_swiss.go:139 +0x0 k8s.io/kubernetes/test/e2e/dra/test-driver/app.(*ExamplePlugin).nodeUnprepareResource() k8s.io/kubernetes/test/e2e/dra/test-driver/app/kubeletplugin.go:483 +0x391 k8s.io/kubernetes/test/e2e/dra/test-driver/app.(*ExamplePlugin).UnprepareResourceClaims() k8s.io/kubernetes/test/e2e/dra/test-driver/app/kubeletplugin.go:496 +0x19d k8s.io/dynamic-resource-allocation/kubeletplugin.(*nodePluginImplementation).NodeUnprepareResources() k8s.io/dynamic-resource-allocation/kubeletplugin/draplugin.go:941 +0x4c1 k8s.io/kubelet/pkg/apis/dra/v1._DRAPlugin_NodeUnprepareResources_Handler.func1() k8s.io/kubelet/pkg/apis/dra/v1/api_grpc.pb.go:181 +0xbe k8s.io/kubernetes/test/e2e/dra/test-driver/app.(*ExamplePlugin).recordGRPCCall() k8s.io/kubernetes/test/e2e/dra/test-driver/app/kubeletplugin.go:523 +0x2a6 k8s.io/kubernetes/test/e2e/dra/test-driver/app.(*ExamplePlugin).recordGRPCCall-fm() <autogenerated>:1 +0x8f google.golang.org/grpc.getChainUnaryHandler.func1() google.golang.org/[email protected]/server.go:1241 +0x23c k8s.io/kubernetes/test/e2e/dra/utils.(*Driver).interceptor() k8s.io/kubernetes/test/e2e/dra/utils/deploy.go:1045 +0x37d k8s.io/kubernetes/test/e2e/dra/utils.(*Driver).SetUp.func8() k8s.io/kubernetes/test/e2e/dra/utils/deploy.go:657 +0xb2 google.golang.org/grpc.getChainUnaryHandler.func1.getChainUnaryHandler.1() google.golang.org/[email protected]/server.go:1241 +0x11e k8s.io/dynamic-resource-allocation/kubeletplugin.(*grpcServer).interceptor() k8s.io/dynamic-resource-allocation/kubeletplugin/nonblockinggrpcserver.go:157 +0x545 k8s.io/dynamic-resource-allocation/kubeletplugin.(*grpcServer).interceptor-fm() <autogenerated>:1 +0x8f google.golang.org/grpc.getChainUnaryHandler.func1() google.golang.org/[email protected]/server.go:1241 +0x23c k8s.io/dynamic-resource-allocation/kubeletplugin.startGRPCServer.unaryContextInterceptor.func2() k8s.io/dynamic-resource-allocation/kubeletplugin/nonblockinggrpcserver.go:99 +0x7e google.golang.org/grpc.NewServer.chainUnaryServerInterceptors.chainUnaryInterceptors.func1() google.golang.org/[email protected]/server.go:1232 +0xe7 k8s.io/kubelet/pkg/apis/dra/v1._DRAPlugin_NodeUnprepareResources_Handler() k8s.io/kubelet/pkg/apis/dra/v1/api_grpc.pb.go:183 +0x1e6 google.golang.org/grpc.(*Server).processUnaryRPC() google.golang.org/[email protected]/server.go:1428 +0x1a69 google.golang.org/grpc.(*Server).handleStream() google.golang.org/[email protected]/server.go:1832 +0x185b google.golang.org/grpc.(*Server).serveStreams.func2.1() google.golang.org/[email protected]/server.go:1063 +0x149 Previous read at 0x00c0047e03c0 by goroutine 9257: runtime.mapIterStart() runtime/map_swiss.go:160 +0x0 k8s.io/kubernetes/test/e2e/dra/test-driver/app.(*ExamplePlugin).GetPreparedResources() k8s.io/kubernetes/test/e2e/dra/test-driver/app/kubeletplugin.go:506 +0x112 k8s.io/kubernetes/test/e2e/dra/test-driver/app.(*ExamplePlugin).GetPreparedResources-fm() <autogenerated>:1 +0x33 runtime.call16() runtime/asm_amd64.s:774 +0x42 reflect.Value.Call() reflect/value.go:365 +0xb5 github.com/onsi/gomega/internal.(*AsyncAssertion).buildActualPoller.func3() github.com/onsi/[email protected]/internal/async_assertion.go:337 +0x244 github.com/onsi/gomega/internal.(*AsyncAssertion).match() github.com/onsi/[email protected]/internal/async_assertion.go:560 +0xe01 github.com/onsi/gomega/internal.(*AsyncAssertion).Should() github.com/onsi/[email protected]/internal/async_assertion.go:145 +0xc4 k8s.io/kubernetes/test/e2e/dra.init.func1.2.4() k8s.io/kubernetes/test/e2e/dra/dra.go:314 +0x4fc k8s.io/kubernetes/test/e2e/dra.init.func1.2.4() k8s.io/kubernetes/test/e2e/dra/dra.go:304 +0x204 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2() github.com/onsi/ginkgo/[email protected]/internal/node.go:517 +0x5d github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3() github.com/onsi/ginkgo/[email protected]/internal/suite.go:945 +0x6ed
1 parent 59cdded commit 0511a75

1 file changed

Lines changed: 2 additions & 0 deletions

File tree

test/e2e/dra/test-driver/app/kubeletplugin.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -462,6 +462,8 @@ func (ex *ExamplePlugin) nodeUnprepareResource(ctx context.Context, claimRef kub
462462

463463
logger := klog.FromContext(ctx)
464464

465+
ex.mutex.Lock()
466+
defer ex.mutex.Unlock()
465467
claimID := ClaimID{Name: claimRef.Name, UID: claimRef.UID}
466468
devices, ok := ex.prepared[claimID]
467469
if !ok {

0 commit comments

Comments
 (0)